Projects

### Overview: FedRAG is an open-source framework designed for fine-tuning Retrieval-Augmented Generation (RAG) systems, supporting both centralized and federated architectures. ### Key features: - Comprehensive support for state-of-the-art RAG fine-tuning methods. - Seamless integration with popular frameworks like HuggingFace, LlamaIndex, and LangChain. - Clean, intuitive abstractions for simplified RAG fine-tuning. - Easy federated learning and integration with existing tools. ### Tech stack: Python ### Why it matters: FedRAG provides a robust and flexible framework for fine-tuning RAG systems, making it an essential tool for researchers and developers looking to enhance their models with retrieval-based techniques.

### Overview: This repository contains a curated collection of datasets designed for Large Language Models (LLMs). It covers a wide range of domains including medical AI, natural language processing (NLP), multimodal learning, instruction tuning, reasoning, code generation, and evaluation benchmarks. The datasets are sourced from reputable sources and are licensed under MIT, CC BY, and other open-source licenses, ensuring accessibility and flexibility. ### Key Features: - **MedQA (USMLE)**: A benchmark dataset for Medical Question Answering, focusing on USMLE exams. - **MedMCQA**: A medical multiple-choice question dataset, suitable for Indian licensing exams. - **PubMedQA**: A biomedical question answering dataset, focusing on biomedical QA. - **BioASQ**: A biomedical question answering dataset with semantic indexing capabilities. - **MASH-QA**: A healthcare question answering dataset with multi-span annotations. - **MedQuAD**: A medical question answering dataset, designed for various medical tasks. ### Tech Stack: Python, PyTorch, TensorFlow, and other relevant tools. ### Why it Matters: These datasets are crucial for researchers and developers working on large language models, providing a rich set of examples and challenges that can help improve the performance and accuracy of models across various domains. They are essential for evaluating the effectiveness of different techniques and models, ensuring that advancements in AI and LLMs are applied effectively and ethically.

### Overview: This project is an AI API aggregator companion that provides a machine-readable catalog of 300+ models behind one key and one base URL, offering routing patterns, cost accounting, and OpenAI SDK and curl examples. ### Key features: - One key, entire catalog. - $1 minimum, pay as you go. - Charge comes back in the response. - Async by design. ### Tech stack: Python ### Why it matters: Teams can route through APIMart for a single key, base URL, and billing surface, accessing many models with one API call. The price is reported in the response, making budgeting straightforward.

Overview: This repository contains a PyTorch implementation of the Factorized TDNN (TDNN-F) model, originally introduced in the paper "Semi-Orthogonal Low-Rank Matrix Factorization for Deep Neural Networks" and further developed in Kaldi. The model is used for deep neural network architectures and is known as TDNN-F in nnet3 of Kaldi. Key features: - Factorized TDNN (TDNN-F) implementation - PyTorch implementation - Semi-orthogonal constraint for first two convolutions - Orthogonal error measurement for debugging - Factorized TDNN x-vector architecture (FTDNN) up to the embedding layer Tech stack: Python Why it matters: This model is significant for deep neural network architectures as it addresses the orthogonality problem in traditional TDNNs, leading to improved performance and more efficient training. GitHub stars: 150

### Overview: CodeJury is a terminal-first, knowledge-grounded multi-agent software delivery pipeline designed to streamline the review process. It separates the reviewer from the developer, using two models to review changes independently, ensuring a less correlated review and reducing the likelihood of bad changes being merged. ### Key Features: - **Two Reviewers**: Two models, one from each provider, review changes independently to catch bad changes that a single reviewer might overlook. - **Persistent Code Graph**: Indexes a repository once and looks up changes instead of rediscovering them, reducing the cost. - **No Foreperson**: No need for an arbiter, as the two reviewers are opinionated in different directions, catching issues that a single reviewer might miss. ### Tech Stack: - Python ### Why It Matters: CodeJury reduces the likelihood of bad changes being merged by using two independent reviewers, ensuring a less correlated review process. This approach is particularly effective in reducing the number of bad changes that make it through the pipeline, making it a valuable tool for software development teams looking to improve their quality assurance and review processes.

### Overview: LLMFlex is a Python package designed for developers to work with different large language models (LLMs) and perform prompt engineering. It provides a simple interface for creating local and private AI-powered applications using free and local resources instead of relying on paid APIs. ### Key features: - **Multiple LLMs with different generation configurations** - Supports multiple LLMs with different temperature and max new tokens using the `LlmFactory` class. - **Langchain compatibility** - All LLMs created with `LlmFactory` are langchain compatible, making them easily integrated into existing Langchain code. - **Multiple model formats support** - Supports PyTorch, AWQ, GPTQ (via transformers), GGUF (via llama-cpp-python), OpenAI API, and EXL2. - **Custom tools** - Provides a base class `BaseTool` and an example `BrowserTool` powered by DuckDuckGo. - **LLM Agents** - An `Agent` class for initializing agents with tools and LLMs. - **Embedding Toolkits** - Bundled classes for using embedding models with a text splitter. - **Vector database** - Utilizes FAISS to store and search texts for RAG tasks. - **Chat memories** - Supports `BaseChatMemory` and `LongShortTermChatMemory` for storing chat memory. ### Tech stack: Python ### Why it matters: LLMFlex simplifies the process of integrating and using different LLMs for AI applications, making it easier to develop local and private

### Overview: Kern is a model-agnostic GPU runtime designed to ship models as verified programs, enabling fast and safe inference for any workload and GPU. ### Key Features: - **Model Agnostic Runtime**: Runs models as verified programs, ensuring correctness and performance. - **GPU Optimization**: Fast and safe inference on multiple GPUs, including NVL72, HGX B300, and EP4. - **Model Manifests**: Models are packaged as JSON manifests, containing buffers, states, and parallelism details. - **Kernels and Weights**: Models are compiled into device code, weights are stored separately, and parallelism is managed. - **Agent-Driven Development**: Engines are split from models, allowing agents to manage and optimize them independently. ### Tech Stack: Rust ### Why It Matters: Kern simplifies the complex world of GPU inference by separating models from their engines, making it easier and safer for agents to optimize and deploy models on various GPUs and workloads. ### Repository: pegainfer-project/kern ### Primary Language: Rust ### GitHub Stars: 145

### Overview: This project is a Model Context Protocol (MCP) server designed to facilitate reading and writing to a Pinecone index. It serves as a rudimentary Retrieval-Augmented Generation (RAG) solution, allowing developers to easily integrate Pinecone into their Claude Desktop applications. ### Key Features: - **Model Context Protocol (MCP) Server**: Enables seamless communication between the MCP client and the Pinecone index. - **Semantic Search**: Facilitates searching through the Pinecone index for relevant records. - **Document Operations**: Lists all documents in the Pinecone index and processes them for upserting into the index. - **Embedding and Chunking**: Generates embeddings and chunks documents for efficient indexing. ### Tech Stack: - **Python**: The primary programming language used for developing the MCP server. ### Why It Matters: This server simplifies the integration of Pinecone into Claude Desktop applications, making it easier for developers to leverage Pinecone's capabilities without extensive customization. It provides a basic framework for implementing RAG solutions, which can be further enhanced with additional tools and features. ### Repository: - **Repository**: `sirmews/mcp-pinecone` - **GitHub Stars**: 150 - **GitHub Description**: Model Context Protocol server to allow for reading and writing from Pinecone. Rudimentary RAG ### Quickstart: - **Smithery Installation**: To install the server automatically via Smithery, use the command: ```bash npx -y @smithery/cli install mcp-pinecone --client claude

### Overview: This repository is a curated list of awesome papers, methods, and resources related to Large MultiModal Model Hallucination (LMM hallucination). It focuses on the occasional instances where Large Language Models (LLMs) generate content that appears plausible but deviates from or conflicts with the provided image. The repository aims to provide insights into detecting, evaluating, and mitigating hallucinations in LLMs. ### Key Features: - **Detecting Hallucination**: Methods for detecting and preventing hallucinations in Large Vision Language Models. - **Evaluating Hallucination**: Tools and frameworks for evaluating hallucinations in LLMs. - **Mitigating Hallucination**: Techniques for mitigating hallucinations in LLMs. ### Tech Stack: - **Python** - **JavaScript** - **C++** ### Why It Matters: This repository serves as a valuable resource for researchers and practitioners in the field of Large Language Models, providing essential tools and methods to understand, detect, and mitigate hallucinations in LLMs. It is crucial for advancing the development of more reliable and accurate LLMs. ### Repository: xieyuquanxx/awesome-Large-MultiModal-Hallucination ### GitHub Stars: 150 ### GitHub Description: 😎 curated list of awesome LMM hallucinations papers, methods & resources.

Overview: This repository contains the official PyTorch implementation of a paper titled "MLLM Can See? Dynamic Correction Decoding for Hallucination Mitigation". The paper explores the reasons behind hallucination phenomena in multimodal large language models (MLLMs) and proposes a novel method called DeCo (Dynamic Correction Decoding) to mitigate these issues. Key features: - DeCo (Dynamic Correction Decoding) - Adaptively selects the appropriate preceding layers - Integrates knowledge proportionally into the final layer - Seamlessly incorporates with various classic decoding strategies - Applied to different MLLMs Tech stack: Python Why it matters: This method aims to reduce hallucination rates in MLLMs, which are commonly observed but poorly understood. By identifying the root cause of hallucinations, DeCo provides a practical solution to mitigate these issues, potentially improving the overall quality and reliability of language models. Repository: zjunlp/Deco GitHub stars: 150

Overview: EasyRec is a PyTorch implementation of a language model designed for recommendation tasks. It was proposed in the paper "EasyRec: Simple yet Effective Language Models for Recommendation" at EMNLP 2025, authored by Xubin Ren and Chao Huang. Key features: - EasyRec is a language model specifically designed for recommendation tasks. - It leverages collaborative information from multiple recommendation datasets. - It employs novel contrastive learning objectives to encode user/item profiles into high-quality semantic embeddings suitable for recommendation. Tech stack: EasyRec is built using PyTorch, with additional dependencies on transformers, sentencepiece, and sentence-transformers. Why it matters: EasyRec demonstrates strong performance in text-based zero-shot recommendations and text-enhanced collaborative filtering scenarios, making it a valuable tool for recommendation systems.

### Overview: RustyRAG is a production-grade RAG API built in Rust, utilizing cutting-edge techniques such as HNSW dense vectors, BM25 sparse matching, cross-encoder reranking, and Docling for layout-aware document extraction. Powered by Cerebras, Groq, Milvus, and Jina AI, it offers 94.5% accuracy on Open RAG Bench and is capable of processing 3,045 questions in under 200 milliseconds. ### Key Features: - **Production-Grade RAG API**: Built in Rust, leveraging advanced search and retrieval techniques. - **Cross-Encoder Reranking**: Enhances the accuracy of search results. - **Layout-Aware Document Extraction**: Uses Docling for optimal document extraction. - **High Performance**: Benchmarked on 3,045 questions from Open RAG Bench, achieving 94.5% accuracy with a time-to-first-token (TTFT) of 279ms. ### Tech Stack: - **Rust**: Written in Rust, leveraging Rust for performance and reliability. - **Actix-web**: Used for building the API. - **Docker**: Containerized the application for easy deployment. - **Swagger**: Provides API documentation. ### Why It Matters: RustyRAG is a game-changer in the field of RAG APIs, offering unparalleled performance and accuracy, making it a must-have tool for developers looking to integrate advanced search capabilities into their applications.

Overview: Apowerb is an open-source platform designed to build, orchestrate, and govern production AI agents. It includes features such as multi-agent orchestration, RAG, Text-to-SQL, MCP servers, and built-in business tools like email and schedule triggers, token quotas, and session audit. Key features: - Build agents from the web interface or the REST API - Connect agents to systems via MCP servers - Provide built-in tools for Google Workspace, Microsoft 365, GitHub, and H - Orchestrate and govern AI agents in production Tech stack: Python Why it matters: Apowerb offers a comprehensive solution for building, deploying, and managing AI agents, making it ideal for developers and businesses looking to integrate AI into their workflows efficiently and securely.

Overview: Pebblo is a tool designed to enable developers to safely load data and promote their Gen AI app to deployment without worrying about compliance and security requirements. It identifies semantic topics and entities found in the loaded data and summarizes them on the UI or a PDF report. Key features: - Safely loads data and promotes Gen AI apps to deployment - Identifies semantic topics and entities found in the loaded data - Generates summaries on the UI or a PDF report Tech stack: Python Why it matters: Pebblo simplifies the process of integrating Gen AI applications into organizations, ensuring compliance and security without the developer having to worry about these aspects. Repository: daxa-ai/pebblo GitHub stars: 149

### Overview: A Pure MLX port of LTX-2 for Apple Silicon, featuring three-package monorepo with native Metal support for inference, pipelines, and training. ### Key features: - **LTX-2.5**: Pipeline coverage on 2.5 packs, including `--distilled`, `--two-stage`, `--two-stages-hq`, `--one-stage`, `--dfr`, `a2v`, `retake`, `extend`, `ic-lora`, `hdr-ic-lora`, and `auto-predicted duration`. - **Text-to-Video**: Generate video from a text prompt. - **Image-to-Video**: Animate a reference image. - **Audio-to-Video**: Generate video conditioned on an audio track. - **Retake / Extend**: Edit existing videos. - **Interpolation between images**: Generate video with each image as a guide. - **IC-LoRA**: Reference video conditioning. - **HDR IC-LoRA**: Single-stage ACEScct SDR-to-HDR. - **LipDub**: Lip-dub a reference video by re-syncing visuals to the source audio. - **Two-stage generation**: Half-res → neural upscale → refine. - **HQ generation**: res_2s second-order sampler + CFG/STG guidance. - **Prompt Relay**: Sequence local prompts over time within one generation. - **Negative prompt**: Custom CFG negative on every pipeline. - **Prompt enhancement**: Gemma 3 12B rewrites short prompts into detailed

### Overview: This curated list provides a comprehensive survey of foundation models, datasets, and tools specifically tailored for biosignals. The field of biosignal foundation models is rapidly evolving, with dedicated models and tools emerging to analyze and interpret vast amounts of sensor readings. ### Key features: - Foundation models (FMs) - Datasets - Benchmarks - Preprocessing toolboxes - Large language models (LLMs) for biosignal analysis ### Tech stack: Python, TensorFlow, PyTorch, and other machine learning frameworks. ### Why it matters: Foundation models and tools are essential for unlocking patterns and insights from biosignals, enabling better health monitoring and diagnostics. The curated list provides a valuable resource for researchers and practitioners looking to advance biosignal analysis.

### Overview: MiniMax H3 Director Cut Studio is a Premiere-inspired PySide6 director studio designed for the MiniMax H3 Ref2VA system. It offers advanced features such as character swap, semantic media enrichment, timeline prompt reconciliation, and long-video rendering through ComfyUI. The project is built on Python and utilizes AI technologies like Qwen3-TTS and VoxCPM2 for seamless integration with Adobe Premiere Pro. ### Key Features: - Character Swap with SAM3.1 dual contour joint synthesis - Full coverage of old and new bodies using `V1 Mask ∪ Generated Mask` - Automatic marking of dirty cache for old syntheses - Timeline prompt reconciliation for better shot planning - Shot-aware long-video rendering with ComfyUI ### Tech Stack: Python ### Why It Matters: This project combines the functionalities of a traditional director studio with advanced AI tools, making it a powerful tool for filmmakers and content creators looking to enhance their video production workflow. The integration with ComfyUI ensures smooth collaboration and rendering, making it ideal for creating high-quality, AI-driven content. ### Repository: Repository: karuvanan/MiniMax-H3-Director-Cut-Studio Primary Language: Python GitHub Stars: 148 ### Deployment: - **Prerequisites**: Ensure all computers have completed the Step 1 deployment. - **Deployment Steps**: 1. Deploy Studio Runtime, Qwen3-TTS, and VoxCPM2 2. Deploy Music Workbench Host/Client 3. Deploy Singer Host/Client

### Overview: TypeGPT is a Python application that allows seamless integration of various AI models and LLMs into your operating system. It supports multiple models such as ChatGPT, Google Gemini, Claude, and Llama3, and provides features like global accessibility, multiple AI model support, a GUI manager for key management, and clipboard integration for larger text inputs and image pasting. ### Key Features: - **Global Accessibility**: Invoke AI models from any text input field across your system. - **Multiple AI Models**: Support for ChatGPT, Google Gemini, Claude, and Llama3. - **GUI Manager**: Easy-to-use interface for managing API keys and program status. - **Clipboard Integration**: Utilize the clipboard for larger text inputs and image pasting. - **Screenshot Capability**: Capture and include screenshots in your queries. - **Model-Specific Features**: Support for ChatGPT, Google Gemini, Claude, and Llama3 with specific model-specific features. - **Input Modes**: Line Mode, Screenshot Mode, and Clipboard Support. - **Keyboard Controls**: Send Query, Cancel Input, Paste Content. - **Real-time Program Status Monitoring**: Easy API key management. - **Start/Stop Controls**: Start/Stop controls for TypeGPT. - **Automatic Key File Creation and Management**: Key file creation and management. - **Visual Feedback for Program State**: Automatic key file creation and management. ### Tech Stack: Python ### Why It Matters: TypeGPT is designed to make it easy for developers and users to integrate AI models into their operating systems, enhancing

Overview: WhisperVideo is a tool designed to track who speaks in long videos and align their speech with the faces on screen, providing clean and consistent speaker labels. Key features: - Visually grounded speaker attribution - Long-video friendly - Identity memory and clean speaker labels - Panel view and subtitles for review Tech stack: SAM3, WhisperX, TalkNet, Pyannote, CUDA, FFmpeg Why it matters: It simplifies the process of understanding and tracking conversations in videos, making it easier to follow and analyze discussions in various applications.

Overview: This project focuses on evaluating video generation models using a method called Content-Debiased FVD, which addresses the bias towards individual frame quality over realistic motion in the Fréchet Video Distance metric. Key features: - Provides a simple interface to compute FVD scores between two sets of videos. - Includes precomputed statistics for UCF101, Sky, Taichi, Kinetics, and FFS datasets. - Offers a way to mitigate bias in FVD by using features from large-scale unsupervised models. Tech stack: Python Why it matters: This project addresses a significant issue in video generation evaluation by providing a method that is less biased towards individual frame quality, making it more reliable for comparing different models.