Projects

Overview: This project provides a JAX implementation of a diffusion-based controller (DM-AC) for robust quadrotor dynamics learning. The controller is designed to handle disturbances such as payload variations and wind gusts, ensuring reliable trajectory tracking. The repository includes a clear, modular, and extensible codebase suitable for rapid prototyping of diffusion-based system identification for real-time deployment. Key features: - JAX implementation - Robust trajectory tracking under disturbances - Configurable training and inference - Integration with PX4 SITL via Gazebo and ROS 1/MAVROS for real-time control Tech stack: Python Why it matters: This project aims to provide a clear, modular, and extensible codebase for rapid prototyping of diffusion-based system identification for real-time deployment, making it suitable for various robotic platforms.

### Overview: This repository is a research fork of `antirez/ds4` designed to run the DeepSeek V4 Flash model on a Linux laptop with 7.7 GiB of RAM and no GPU, using demand paging backed by NVMe storage. The GGUF (Generalized Graphical Universal Format) size is 78.62 GiB, and the logical parameters are 284.33B. The model is CPU-only and uses Linux faults to page demand-loaded weights into memory. ### Key Features: - CPU-only execution of the DeepSeek V4 Flash GGUF - NVMe-backed demand paging - 78.62 GiB GGUF size - 7.7 GiB of physical RAM - No GPU support ### Tech Stack: C ### Why it Matters: This repository demonstrates the execution of a large model size (78.62 GiB) on a constrained system (7.7 GiB RAM) using demand paging backed by NVMe storage, showcasing the model's ability to handle extreme memory pressure and demonstrate the performance under a severe model-size-to-RAM mismatch scenario.

### Overview: InfantAgent is a multimodal agent designed to interact with its own PC in a multimodal manner, enabling autonomous mouse operations and other PC tasks without relying solely on browser analysis. ### Key Features: - **Multimodal Interaction**: The agent can interact with the PC using both mouse clicks and keyboard inputs. - **Autonomous Mouse Operations**: The agent can autonomously operate the mouse and click anywhere on the screen. - **Reinforcement Learning Training**: The agent system will be used for subsequent reinforcement learning training of agents. ### Tech Stack: - Python ### Why It Matters: InfantAgent represents a significant advancement in the field of multimodal interaction and agent autonomy, offering a new approach to interacting with computers in a more natural and efficient manner.

Overview: Text-to-SQL Copilot is a tool designed to assist users in extracting actionable insights from SQL databases by converting natural language questions into SQL statements. Key features: - Converts natural language questions into SQL statements - Runs the generated SQL query on a database - Analyzes the results and provides insights Tech stack: Python, Jupyter Notebook, HuggingFace Inference API Why it matters: This application simplifies the process of querying SQL databases by leveraging natural language processing and machine learning models, making complex data analysis accessible to a broader audience.

Overview: Jev Account Filter is a project designed to screen video-platform accounts and channels for datasets, focusing on sign language, news, piano/instrument performance, and makeup tutorials. It uses decision models and account metadata to efficiently identify relevant creators without requiring full video downloads or watching. Key features: - Account screening and channel classification with decision models - Efficiently finds sign language creators, news channels, instrument performers, and makeup tutorial creators - Fast, cheap, accurate, flexible, and promising results Tech stack: Python Why it matters: Jev Account Filter is the first project to screen video-platform accounts without downloading or watching full videos, making it a significant advancement in dataset collection and research.

Overview: This plugin is designed to assist developers in creating personalized datasets for text generation models, making the process of finetuning models more efficient and streamlined. It can help researchers and practitioners train models to generate more detailed, detailed, and well-formatted text. Key features: - Personalized dataset creation for text generation models - Easy integration with Obsidian for seamless workflow - Support for both prompt-based and completion-based dataset generation Tech stack: TypeScript Why it matters: This plugin simplifies the process of creating datasets for text generation models, allowing for faster and more accurate training, ultimately leading to better results in text generation tasks.

Overview: A simple GUI application designed to record audio dictated from text prompts, ideal for speech recognition or synthesis training. Key features: - Record audio from text prompts - Randomly select and order prompts - Save audio and metadata to `.wav` and `recorder.tsv` files - Play, delete, or re-record previous recordings Tech stack: Python Why it matters: This tool simplifies the process of training speech recognition or synthesis models by providing a straightforward way to record and analyze speech data. Repository: daanzu/speech-training-recorder Primary language: Python GitHub stars: 40 GitHub description: Simple GUI application to help record audio dictated from given text prompts, for use with training speech recognition or speech synthesis. README excerpt: # speech-training-recorder > Simple GUI application to help record audio dictated from given text prompts, for use with training speech recognition or speech synthesis. Given a text file containing prompts, this app will choose a random selection and ordering of them, display them to be dictated by the user, and record the dictation audio and metadata to a `.wav` file and `recorder.tsv` file respectively. You can select a previous recording to play it back, delete it, and/or re-record it.  **Requirements:** * Python 3 * See [`requirements.txt`](requirements.txt) for required packages * Cross platform: Windows, Linux, MacOS ## Getting Started ``` git clone https://github.com/daanzu/s

Overview: This repository contains Jupyter notebooks for training neural audio and latent diffusion models on Kaggle, including RAVE, VSCHAOS2, MSPrior, AFTER, RAVE-Latent Diffusion, Stable Audio Tools, and underfit, LoRA finetune for Stable Audio 3. Key features: - Training neural audio and latent diffusion models on Kaggle - Includes RAVE, VSCHAOS2, MSPrior, AFTER, RAVE-Latent Diffusion, Stable Audio Tools, and underfit, LoRA finetune for Stable Audio 3 Tech stack: Jupyter Notebook Why it matters: This repository provides a comprehensive set of Jupyter notebooks for training various neural audio and latent diffusion models on Kaggle, making it an invaluable resource for developers and researchers looking to experiment with these models.

Overview: This repository contains papers and tools related to Automatic Speech Recognition (ASR) and Natural Language Processing (NLP). Key features: - Papers on ASR and NLP - Tools for ASR and NLP Tech stack: Python, TensorFlow, Keras, PyTorch Why it matters: This repository provides essential resources and tools for researchers and developers working in the fields of ASR and NLP.

Overview: RAGPulse is an open-source project designed to trace and optimize RAG serving systems, focusing on a university-wide Q&A service scenario. Key features: - Real-world workload trace - Contains over 134K records - Includes trace, system prompt, passages, chat history, and web search logs Tech stack: Python, VLLM, Qwen2.5-14B-Instruct Why it matters: This project provides valuable insights into real-world RAG serving systems, helping developers optimize their own systems and improve performance.

Overview: This project fine-tunes BLOOM, utilizing Stanford Alpaca for training and evaluation. Key features: - Fine-tuning BLOOM - Support for Google Colab - Progress includes adding a Colab example, testing BLOOM-560m, BLOOM-1b7, and BLOOM-7b1 Tech stack: Jupyter Notebook Why it matters: This project provides a comprehensive solution for fine-tuning BLOOM, making it accessible for developers and researchers alike. Repository: hyintell/BLOOM-fine-tuning GitHub stars: 40

### Overview: Inst-It: Boosting Multimodal Instance Understanding via Explicit Visual Prompt Instruction Tuning is an official repository for a NeurIPS 2025 project that aims to enhance multimodal instance understanding through explicit visual prompt instruction tuning. This project provides a fine-grained multimodal benchmark and dataset for evaluating language-modality models (LMMs) at the instance level. ### Key Features: - **Inst-It Bench:** A fine-grained multimodal benchmark for evaluating LMMs at the instance-level, including ~1,000 image and video questions. - **Inst-It Dataset:** A dataset of 21k videos, 51k images, 21k video-level descriptions, and 207k frame-level descriptions, providing fine-grained annotations for specific instances. - **GitHub Repository:** The repository is hosted on GitHub, with 40 GitHub stars, making it a valuable resource for researchers and practitioners in the field of multimodal instance understanding. ### Tech Stack: - **Python:** The project is implemented using Python, leveraging its flexibility and ease of integration with various machine learning libraries. ### Why It Matters: - **Benchmarking:** Inst-It Bench provides a standardized benchmark for evaluating LMMs, ensuring that researchers can compare their models effectively. - **Dataset:** The Inst-It Dataset offers a rich and diverse dataset for training and testing models, particularly focusing on fine-grained instance-level understanding. - **Community Impact:** As a well-documented project, Inst-It has the potential to influence the development of

### Overview: This repository is dedicated to the implementation of the "Learning the Power of No" paper, which explores the use of foundation models with negations. The code, checkpoints, and data are provided for the *ViT-L/14, ViT-B/16, and ViT-B/32* backbones of CLIP, offering a practical guide for researchers and developers interested in understanding and applying negation techniques in their projects. ### Key Features: - Official code and checkpoints for the "Learning the Power of No" paper. - Preprocessed dataset CC-Neg for benchmarking CLIP-like VLMs. - Detailed instructions on loading and using the CoN-CLIP model. - Comprehensive documentation and links to the repository, including the paper and dataset access. ### Tech Stack: Python ### Why it Matters: This repository provides a practical solution for researchers and developers looking to incorporate negation techniques into their foundation models, offering a valuable resource for advancing the field of natural language processing and computer vision. ### Repository: jaisidhsingh/CoN-CLIP ### GitHub Stars: 40 ### GitHub Description: Implementation of the "Learning the Power of No" paper.

Overview: LoRA-CLIP is an easy-to-use Python wrapper for inserting LoRA layers into CLIP models, making fine-tuning and adaptation more accessible. Key features: 1. Dynamic LoRA-layer insertion across image or text encoder, or both. 2. Safely loads pre-trained OpenAI CLIP weights. 3. Loads fine-tuning ready models by correctly un-freezing `positional_embedding`, `logit_scale`, and projections. Tech stack: Python Why it matters: LoRA-CLIP simplifies the process of fine-tuning and adapting CLIP models, making it easier for developers to experiment with different architectures and fine-tune their models on various datasets.

Overview: The Groq Chrome Extension is a tool designed to enhance your web browsing experience by summarizing content from websites using Groq. It allows you to quickly understand entire web pages and specific text, and even engage in conversations with the content. Key features: - Summarize Entire Web Pages - Summarize Selected Text - Chat with Content Tech stack: TypeScript Why it matters: This extension is particularly useful for users who need to quickly comprehend or interact with online content, such as researchers, students, or professionals. Repository: jose-mdz/groq-chrome-ext Primary language: TypeScript GitHub stars: 40 GitHub description: Chrome extension that interacts with content using Groq

Overview: This project introduces a graph learning-based multiview clustering algorithm that addresses the limitations of existing graph-based clustering methods by learning initial graphs from data points across different views and optimizing them with a rank constraint on the Laplacian matrix. The resulting global graph is then integrated to improve clustering performance without the need for graph cuts, achieving direct cluster indicators and k-means clustering. Key features: - Graph learning for multiview clustering - Learning initial graphs from different view data - Optimization with rank constraint on Laplacian matrix - Integration of optimized graphs into a global graph - Direct cluster indicators and k-means clustering Tech stack: Matlab Why it matters: This algorithm significantly enhances multiview clustering performance by improving graph quality and directly obtaining cluster indicators without the need for graph cuts, making it more efficient and effective compared to state-of-the-art methods.

### Overview: This project implements a basic Multi-Layer Perceptron (MLP) for Hidden Markov Model (HMM)-Deep Neural Network (DNN) speech recognition. The MLP is trained using PyTorch, while feature extraction, alignments, and decoding are performed with Kaldi. The current implementation supports dropout and batch normalization. An example for phoneme recognition using the standard TIMIT dataset is provided. ### Key features: - PyTorch-based MLP training - Kaldi for feature extraction, alignments, and decoding - Dropout and batch normalization support - TIMIT dataset for phoneme recognition ### Tech stack: - Python 2.7 - PyTorch 0.3 - CUDA 9.1 - Kaldi 1.15 ### Why it matters: This project provides a practical example of integrating PyTorch with Kaldi for speech recognition, showcasing how to train a basic MLP for HMM-DNN speech recognition. The example includes feature extraction, alignment, and decoding, making it useful for understanding the pipeline and implementing similar systems.

Overview: MindBridge MCP Server is an AI orchestration platform that enables seamless communication between various LLMs, allowing developers to easily switch between models and leverage their unique strengths for improved workflows and decision-making. Key features: - Multi-LLM Support: Connects to OpenAI, Anthropic, DeepSeek, and Ollama, as well as local models. - Reasoning Engine Aware: Routes models to those best suited for deep reasoning. - getSecondOpinion Tool: Compares responses from multiple models to provide a more comprehensive understanding. - OpenAI-Compatible API Layer: Integrates with tools expecting OpenAI endpoints. - Auto-Detects Providers: Configures with minimal setup. - Flexible as Hell: Configured via environment variables, MCP config, or JSON. Tech stack: TypeScript Why it matters: MindBridge MCP Server simplifies the integration of multiple LLMs, making it easier for developers to create more intelligent and versatile AI-driven applications. Repository: pinkpixel-dev/mindbridge-mcp GitHub stars: 40 GitHub description: A comprehensive solution for AI orchestration, enabling seamless communication and collaboration between various LLMs.

Overview: This repository provides a boilerplate for async bulk data ingestion and querying from various document, graph, and vector databases via their Python clients. It aims to streamline the process by utilizing async APIs and exposing query endpoints via FastAPI for reproducible setups. Key features: - Neo4j - Elasticsearch - Meilisearch - Qdrant - Weaviate - LanceDB Tech stack: Python Why it matters: This repo offers a clean, efficient, and reproducible workflow for experimenting with a variety of databases and APIs, making it useful for both data ingestion and querying. The integration of async APIs and the provision of FastAPI endpoints enable seamless integration and testing of advanced use cases, such as contextual querying with large language models.

### Overview: A vLLM fork with RelayAttention implemented. This project details the implementation and usage of RelayAttention for efficient large language model serving with long system prompts, as detailed in the paper [RelayAttention for Efficient Large Language Model Serving with Long System Prompts](https://arxiv.org/abs/2402.14808). ### Key features: - Implemented RelayAttention for efficient LLM serving. - Includes scripts for data collection and plotting. - Forked from vLLM v0.2.6. - Used to produce tables and figures in the paper. ### Tech stack: Python ### Why it matters: This project provides a practical implementation and detailed documentation for RelayAttention, which can significantly enhance the efficiency and performance of large language models in serving systems with long prompts.