Projects

### Overview: AutoAct is an automatic agent learning framework designed for QA tasks. It leverages self-planning to automatically synthesize and differentiate based on limited data and a tool library, achieving better performance compared to strong baselines. ### Key Features: - Automatic tool selection - Trajectories synthesis - Self-differentiation - Group planning ### Tech Stack: Python ### Why it Matters: AutoAct simplifies the learning process by eliminating the need for large-scale annotated data and closed-source models, making it more accessible and cost-effective. This framework enables the development of language agents that can perform multiple functions effectively, without the limitations of single-model performance. Repository: zjunlp/AutoAct GitHub Stars: 241 GitHub Description: [ACL 2024] AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planning

### Overview: J-Wash is a powerful tool for local editing and exporting large language models, allowing users to reshape a model's identity and behavior without the need for training or fine-tuning. It uses the Jacobian lens from Anthropic to visualize and edit the model's J-space, then exports the changes as a pure-weights checkpoint, enabling seamless integration into existing systems. ### Key features: - **Interactive Chat UI**: Real-time visualization of the Jacobian lens. - **Token Editor**: Converts lens directions into persistent, composable edits. - **Export Pipeline**: Bakes edits into a pure-weights checkpoint for use in any Hugging Face model. ### Tech stack: Python ### Why it matters: J-Wash simplifies the process of aligning large language models, making it accessible for non-experts. By allowing direct manipulation of the model's J-space, users can quickly modify a model's identity or behavior without the need for complex training or fine-tuning. This tool is particularly useful for developers and researchers looking to explore and experiment with model modifications in a controlled environment.

### Overview: MISAKA is a multi-agent research system designed by Luciole Studio for the humanities and social sciences. It leverages graph algorithms to manage research branches and generate a research article that is fully traceable with every citation. ### Key Features: - **Graph Algorithms for Research Branch Management**: Graph algorithms handle the graph of research branches, ensuring a structured and comprehensive approach. - **Multi-Agent Coordination**: Last Order, the coordinator, pairs with specialized agents (Sisters) who report, liaise, and consult. Their reports are read and written, and the conclusion is defended before reviewers. - **Symptomatic Reading**: The system reads Last Order's reasoning to uncover what is overlooked or what the problematic does not permit to be seen, distinguishing between what is overlooked and what premises are occluded. - **Directed Acyclic Graph (DAG) of Possibilities**: Each new direction forks from Last Order's session, inheriting all prior reasoning, and becomes a node with its own team and red team. The graph unfolds level by level, opening questions and integrating answers. ### Tech Stack: - **Python 3.12+** - **macOS, Linux, and Windows** ### Why It Matters: MISAKA is designed to pair a Directed Acyclic Graph (DAG) with symptomatic reading to unearth plural narratives. It is a powerful tool for researchers in the humanities and social sciences, offering a structured and comprehensive approach to research that is fully traceable with every citation.

### Overview: NodeDB is a memory and storage engine designed for AI agents, providing durable, queryable, and access-guarded memory that runs where the agent runs. It combines vector search, graph traversal, full-text ranking, document storage, and timeseries functionality in a single engine, eliminating network hops and offering a PostgreSQL-compatible wire protocol. ### Key Features: - **Memory that runs where the agent runs:** The same engines embed on device and in the browser (NodeDB-Lite, WASM), work offline, and sync conflict-free to the server (CRDT). NodeDB-Lite is available for Node.js and browsers, while NodeDB Origin supports a distributed cluster with CRDT sync. - **All of an agent's memory in one query:** Vector search, graph traversal, full-text ranking, document storage, timeseries, and key-value share one storage core, one identity space, and one planner, allowing a query to flow seamlessly from semantic search into graph traversal filtered by full-text, in one process with no network hops. - **PostgreSQL wire protocol:** Connects with `psql` or any PostgreSQL client library, offering standard SQL with engine-specific extensions where SQL can't express the operation. - **One engine instead of a polyglot stack:** When you outgrow the edge, the same engine folds a server-side sprawl of separate systems into a single deployment, reducing services to run, eliminating cross-system sync pipelines, and avoiding schema drift. - **Serious about performance:** NodeDB is optimized with a thread-per-Core data plane, io_uring,

Overview: Backprop simplifies the use, fine-tuning, and deployment of state-of-the-art machine learning models, making it accessible for developers without extensive expertise. Key features: - Solves a variety of tasks with pre-trained models or finetuning in one line for custom use cases. - Supports conversational question answering, text classification, image classification, text vectorization, image vectorization, summarization, emotion detection, and text generation. - Offers pre-built tasks and models for easy integration and deployment. Tech stack: Python Why it matters: Backprop provides a straightforward way to leverage advanced machine learning models, making it accessible for developers and enabling them to focus on application development rather than deep model optimization.

Overview: This repository is a curated list of some awesome public SNN (Spiking Neural Network) projects. Key features: - Frameworks - Datasets - Applications Tech stack: Python, PyTorch Why it matters: This repository serves as a valuable resource for developers interested in exploring and utilizing Spiking Neural Network projects, providing a comprehensive overview of various frameworks, datasets, and applications in the field.

### Overview: FlexRAG is an open-source framework designed to streamline the rapid reproduction, development, and evaluation of Retrieval-Augmented Generation (RAG) systems. It provides comprehensive support for various RAG scenarios, including text-based, multimodal, and web-accessible applications. With an end-to-end pipeline covering everything from data preparation to system evaluation, FlexRAG empowers researchers to efficiently share their work with the community and swiftly develop demonstrative prototypes based on their algorithms. ### Key Features: - **End-to-End Pipeline**: FlexRAG covers the entire process from data preparation to system evaluation, making it highly versatile. - **Comprehensive Support**: It supports text-based, multimodal, and web-accessible applications, ensuring flexibility. - **RAG Framework**: FlexRAG is built on a retrieval augmented generation framework, enhancing the capabilities of RAG systems. - **Metrics and Benchmarks**: It includes various information retrieval metrics and benchmarks for evaluating RAG systems. ### Tech Stack: Python ### Why It Matters: FlexRAG is an innovative open-source framework that simplifies the process of developing and evaluating RAG systems. It provides a comprehensive solution for researchers and developers looking to streamline their workflows and share their work efficiently. ### GitHub Stars: 241 ### Repository: - Repository: ictnlp/FlexRAG - Primary language: Python - GitHub stars: 241 ### Table of Contents: - 📖 Table of Contents - ✨ Key Features - 📢 News - 🚀 Getting Started

### Overview: NaviLLM is a generalist model designed for embodied navigation, achieving state-of-the-art performance on CVDN, SOON, and ScanQA. It adapts LLMs to embody navigation by introducing schema-based instruction, enabling integration of diverse data sources and demonstrating strong generalizability. ### Key Features: - **Schema-Based Instruction**: Flexibly casts various tasks into generation problems to unify a wide range of tasks. - **Unified Model Training**: Integrates diverse data sources from various datasets, equipping NaviLLM with a wide range of capabilities required by embodied navigation. - **State-of-the-Art Performance**: Achieves significant improvements over previous methods on CVDN, SOON, and ScanQA. ### Tech Stack: Python ### Why It Matters: Embodied navigation is crucial for agents to interact with the world effectively. By proposing NaviLLM, we address the limitations of task-specific agents and provide a versatile solution that can handle a broad range of tasks, making it a significant advancement in the field of embodied navigation.

Overview: AM (Advanced Mathematics) Chat is a large language model designed to integrate advanced mathematical knowledge and exercises in higher mathematics, providing solutions to these problems. Key features: - Integrates advanced mathematical knowledge and exercises in higher mathematics. - Offers solutions to various mathematical problems. - Supports multiple languages including English, Chinese, and Japanese. Tech stack: Python Why it matters: This model is useful for students and educators looking to enhance their understanding of advanced mathematics through interactive and practical problem-solving. Repository: AXYZdong/AMchat Primary language: Python GitHub stars: 238

Overview: Uniflow is a Python-based solution designed to extract and transform unstructured data from PDFs, HTMLs, and TXTs into a format suitable for LLM training, aiming to simplify the process of generating privacy-preserving datasets for machine learning models. Key features: - Extracts documents from PDFs, HTMLs, and TXTs. - Supports various LLMs including OpenAI, Google Gemini, BedRock, Huggingface, and Azure OpenAI models. - Addresses challenges in extracting legacy documents and preparing data for training LLMs. Tech stack: Python Why it matters: Uniflow simplifies the process of preparing LLM training data, making it more accessible and efficient for researchers and data scientists, thereby democratizing the use of LLMs in various applications.

Overview: 《大模型项目实战：多领域智能应用开发》配套资源，包括大语言模型的基础知识、应用架构、工作模式、安装与微调、量化、多模态模型应用、编程大模型应用、VS Code插件开发、检索增强生成应用、翻译应用、智能代理应用、语音模型应用、数字人应用、模型训练应用、AI小镇应用等多方面的详细教程和示例代码。 Key features: - 大语言模型应用架构 - 大语言模型应用工作模式 - 大语言模型微调 - 大语言模型量化 - 多模态模型应用 - 编程大模型应用 - VS Code插件开发 - 检索增强生成应用 - 翻译应用 - 智能代理应用 - 语音模型应用 - 数字人应用 - 模型训练应用 - AI小镇应用 Tech stack: JavaScript Why it matters: 本资源是《大模型项目实战：多领域智能应用开发》的配套教程，为读者提供了从基础到实战的全面指导，帮助读者快速掌握大语言模型的开发和应用。

### Overview: FaceTalk is a novel generative approach designed for synthesizing high-fidelity 3D motion sequences of talking human heads from input audio signal. It proposes a new latent diffusion model operating in the expression space of neural parametric head models to create audio-driven realistic head sequences. ### Key features: - Audio-driven motion diffusion - Neural parametric head models - High-fidelity, temporally coherent motion sequences - Realistic and high-quality motion synthesis of volumetric human heads ### Tech stack: Shell ### Why it matters: FaceTalk stands out in its ability to generate plausible motion sequences that can produce high-fidelity head animation coupled with the NPHM shape space, representing a significant advancement in the field of audio-driven 3D animation.

### Overview: Agentic RAG is an advanced reasoning system designed to mimic human-like reasoning, enhancing the ability to understand and solve complex problems through a combination of structured knowledge, expert tools, and adaptive reasoning. ### Key Features: - **Rich Knowledge Base:** Parses documents carefully and uses an LLM to add summaries and keywords, creating multiple layers of understanding. - **Specialist Agents:** Relys on expert tools: a Librarian for documents, an Analyst for databases, and a Scout for live web data. - **Gatekeeper:** Checks the question before acting, verifying if the query is clear and specific. - **Planner:** Breaks the request into step-by-step tool calls, keeping the process structured. - **Auditor:** Verifies results, each tool's output is reviewed for quality and consistency. - **Strategist:** Connects the dots, turning raw data into deeper insights. - **Adversarial Testing:** Challenges the system with tricky, misleading, or biased questions to ensure robustness. - **Cognitive Memory:** Learns from past interactions, Watchtower monitors important events, and Oracle interprets visual data. - **Evolution:** Continuously improves with new capabilities like Cognitive Memory, Watchtower, and Oracle. ### Tech Stack: Jupyter Notebook ### Why it Matters: Agentic RAG is designed to enhance the ability to understand and solve complex problems, making it feel less like a search tool and more like an expert. It mimics human-like reasoning, adding steps that mimic how humans solve problems, and is designed to be

### Overview: 本项目是一款基于图RAG技术的AI美食推荐助手，旨在通过图数据库和向量检索系统为用户提供个性化的美食推荐和详细的烹饪指导。它集成了Neo4j图数据库、Milvus向量检索系统以及智能对话系统，为用户提供了便捷的美食探索体验。 ### Key features: - **智能推荐**: 利用图RAG技术结合向量检索，为用户推荐个性化美食。 - **详细指导**: 提供分步骤的烹饪指南，即使是新手也能轻松上手。 - **对话交互**: 拥有ChatGPT风格的自然语言交互功能。 - **现代界面**: 玻璃质感的响应式设计。 - **跨平台**: 支持桌面和移动设备访问。 ### Tech stack: - **前端**: Next.js 14, Tailwind CSS, Zustand, Framer Motion - **后端**: Python 3.11 + Flask, Neo4j, Milvus, 图RAG + 混合检索 - **部署**: Docker + Docker Compose, Nginx (反向代理) ### Why it matters: 本项目展示了如何将图RAG技术应用于实际场景中，通过智能推荐、详细指导和自然语言交互，为用户提供便捷、个性化的美食体验。它不仅提高了用户体验，还为开发者提供了丰富的实战案例和学习资源。 ### GitHub stars: GitHub stars: 237 ### Repository: Repository: FutureUnreal/What-to-eat-today

Overview: DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation aims to enhance the generation of audio-video content by leveraging distribution matching and adversarial distillation techniques. The repository includes the 4-step MiniMax-H3 students, optimized for both consumer and professional GPUs, achieving high-quality video generation with native stereo audio. Key features: - 4-step DMAD students for joint audio-video generation - Optimized for both consumer and professional GPUs - Native stereo audio with 1344x768 video output - Code for training and inference Tech stack: Python Why it matters: This project offers a significant improvement in the generation of audio-video content by utilizing distribution matching and adversarial distillation, resulting in high-quality video generation with native stereo audio.

### Overview: HouseDiffusion is an implementation of "HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoising," published on arXiv. This project uses a diffusion model to generate vectorized floorplans, providing a novel approach to floorplan generation. ### Key features: - **Vector Floorplan Generation**: HouseDiffusion generates floorplans as vectors, offering a compact and efficient representation. - **Diffusion Model**: Utilizes a diffusion model for both discrete and continuous denoising, enhancing the model's ability to handle noise and generate realistic floorplans. - **Discrete and Continuous Denoising**: The model incorporates both discrete and continuous denoising techniques to improve the quality of generated floorplans. ### Tech stack: Python ### Why it matters: This project introduces a novel method for generating floorplans, offering a significant improvement over traditional approaches by providing a compact and efficient representation. The use of diffusion models for both discrete and continuous denoising enhances the model's ability to handle noise and generate high-quality floorplans, making it a valuable contribution to the field of floorplan generation.

### Overview: The official implementation of our series of works on **Cross Modal Generalization (CMG)** and **Multimodal Unified Representations**. ### Key features: - **ICCV 2025 · Open-set Cross Modal Generalization (OSCMG)**: Implements an architecture for open-set cross-modal generalization using multimodal unified representation. - **ACL 2025 (Findings) · Feature Disentangling & Training-Free Optimization**: Enhances multimodal unified representations for cross-modal generalization through feature disentangling and training-free optimization. - **NeurIPS 2023 · Foundational CMG Framework**: Provides a foundational framework for achieving cross-modal generalization with multimodal unified representation. ### Tech stack: Python ### Why it matters: This project is essential for researchers and developers interested in cross-modal generalization and multimodal unified representations, offering practical implementations and theoretical foundations for improving cross-modal understanding and generalization.

### Overview: Diff-HierVC is an official PyTorch implementation of a hierarchical voice conversion (VC) system that addresses the challenges of accurate pitch generation and robust speaker adaptation in zero-shot scenarios. The model, introduced at the Interspeech 2023 conference, leverages two diffusion models to improve pitch generation and uses a masked prior in diffusion models to enhance speaker adaptation. ### Key Features: 1. **DiffPitch**: Generates accurate pitch for the target voice style. 2. **DiffVoice**: Converts speech with a target voice style using the generated pitch. 3. **Data-Driven Prior**: Uses the converted Mel-spectrogram as a data-driven prior in the Mel generator to improve voice style transfer. 4. **Masked Prior**: Improves speaker adaptation quality by using a masked prior in diffusion models. ### Tech Stack: - **Python**: The primary programming language used for the implementation. ### Why It Matters: - **Pitch Generation**: The model excels in generating accurate pitch, which is crucial for voice conversion. - **Speaker Adaptation**: It achieves a high speaker adaptation quality, reducing the Common Error Rate (CER) to 0.83% and the Equal Error Rate (EER) to 3.29% in zero-shot VC scenarios. - **Hierarchical Architecture**: The hierarchical structure allows for better handling of complex voice conversion tasks. - **Robust Pitch Generation**: The model addresses the issue of inaccurate pitch in existing methods, making it more suitable for real-world applications. ### Repository: - **GitHub Repository**: [hayeong

### Overview: Perfusion-pytorch is an implementation of Key-Locked Rank One Editing, a paper from Nvidia AI. This project aims to achieve extremely low extra parameters per added concept, down to 100KB, by applying the Rank-1 editing technique from a memory editing paper specifically for Large Language Models (LLMs). The method is designed to be applicable to any text-to-`<insert modality>` network using cross attention conditioning, making it a versatile tool for researchers. ### Key Features: - **Key-Locked Rank One Editing**: Implements the cutting-edge Rank-1 editing technique from a memory editing paper, optimized for LLMs. - **Low Extra Parameters**: Achieves 100KB per added concept, significantly reducing the computational overhead. - **Superclass Concept Handling**: Identifies keys and values to determine the "where" and "what" of new concepts, respectively. - **Cross Attention Conditioning**: Works seamlessly with any text-to-modality network using cross attention conditioning. ### Tech Stack: Python ### Why It Matters: This project is particularly valuable for researchers and developers looking to improve the efficiency and performance of text-to-modality models. By reducing the extra parameters required for each concept, it can lead to more effective and faster model training, ultimately benefiting applications such as text-to-image generation, text-to-speech, and more. ### Install: ```bash $ pip install perfusion-pytorch ``` ### Usage: ```python import torch from torch import nn from perfusion_pytorch import

Overview: Pie is a programmable system designed for high-performance model serving, allowing inference engines to run directly alongside models in WebAssembly, enabling direct access to GPU kernels for efficient decoding, sampling, caching, and agent loops. Key features: - Direct WebAssembly (Wasm) execution for models - Inference engines that own their state and compile custom epilogues for GPU kernels - Models written in multiple languages: Python, JavaScript, and Rust - Binary ship as a single 60 MB file, running on CUDA, ROCm, Metal, Vulkan, XLA (TPUs), and WebGPU - Supports Hugging Face models and can be imported from the Hugging Face Hub Tech stack: Rust Why it matters: Pie simplifies the process of deploying models by allowing inference engines to run directly alongside models in WebAssembly, eliminating the need for Python or PyTorch, and enabling efficient GPU processing for tasks like decoding, sampling, and caching. This approach is particularly beneficial for high-performance applications where model inference is a bottleneck. Repository: pie-project/pie GitHub stars: 238 GitHub description: A programmable system for high-performance model serving