Projects

### Overview: 基于DeepSpeed，本项目旨在突破硬件限制，实现大模型高效训练。 ### Key features: - 收录到[PytorchNetHub](https://github.com/bobo0810/PytorchNetHub) - 分布式数据并行DDP的训练pipeline - MoE用法 - 学习率调度器的配置 - ZeRO零冗余优化器的配置 - 流水并行的训练pipeline - 流水模型的保存、加载、指标评估 - TensorBoard可视化 ### Tech stack: Python ### Why it matters: 通过使用DeepSpeed，开发者可以轻松地实现分布式训练、模型并行、零冗余优化器等高级功能，从而显著提高大模型训练的效率和效果。

Overview: pdfLLM is a proof-of-concept Retrieval-Augmented Generation (RAG) microservice designed for processing, embedding, and querying documents. It integrates document parsing, semantic search, knowledge graph construction, and LLM-based generation to provide advanced document intelligence. Key features: - Semantic Search using OpenAI models - Graph-Based Search using Dgraph - LLM-Powered Responses using OpenAI chat models - Hybrid Retrieval combining vector similarity and entity relationships Tech stack: Python Why it matters: pdfLLM is a highly scalable and cost-effective solution for document processing and retrieval, leveraging cutting-edge AI technologies to provide advanced document intelligence.

### Overview: Penny Royal is an optimized SGLang runtime designed for NVIDIA RTX PRO 6000 Blackwell, featuring persistent conversation caching, HiCache, NIXL, and FlashInfer verification. It provides 524K context, HiCache, and NIXL for long-running sessions, and supports Flash-Next with FR-Spec and DFlash2 for advanced features. ### Key Features: - Persistent conversation caching - HiCache and NIXL for long-running sessions - FlashInfer verification - Flash-Next with FR-Spec as the maintained default profile - Flash-Next DFlash2 with a separate draft model - FlashInfer draft-token verification (experimental / alpha) - 524K context, HiCache, and NIXL ### Tech Stack: Python ### Why it Matters: Penny Royal optimizes Qwen3.8-27B FP8 performance on NVIDIA RTX PRO 6000 Blackwell, providing high context retention and improved long-term conversation continuity.

### Overview: DiffThinker is an official repository for "DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models," presenting a comprehensive framework for generative multimodal reasoning using diffusion models. This project aims to advance the field of generative models by enabling more complex and realistic multimodal reasoning tasks. ### Key Features: - **Model Training:** Utilizes diffusion models to generate high-quality images and text. - **Data Preparation:** Provides tools for generating and preparing datasets for training. - **Inference and Parsing:** Facilitates inference and parsing of generated images and text for evaluation. - **Evaluation:** Offers evaluation scripts to compare generated outputs with ground-truth data. ### Tech Stack: Python ### Why it Matters: This project represents a significant advancement in the field of generative models, particularly in the realm of multimodal reasoning. By enabling more realistic and complex multimodal reasoning tasks, it opens up new possibilities for applications in areas such as natural language processing, computer vision, and more. ### Repository: lcqysl/DiffThinker - **Stars:** 187 - **Description:** Official repo for "DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models." - **Language:** Python - **GitHub Description:** [ICML 2026] Official repo for "DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models" - **GitHub Stars:** 187

### Overview: Pokémon Champion is an expert-level minimax language agent designed for battle simulations. The project aims to showcase the capabilities of language models in competitive environments. ### Key Features: - **Language Model Agent (LLM Player)**: Implements a minimax language agent using various backend models like OpenAI GPT, Meta LLaMA, and Google Gemini. - **Battle Engine (poke_env)**: Independent of the LLM backend, ensuring flexibility and modularity. - **Bayesian Prediction System (bayesian)**: Supports Bayesian prediction for team and live battle outcomes. - **Battle Execution Scripts (scripts)**: Includes scripts for local 1v1 battles and evaluation tools. - **Training Dataset Processing (training)**: Handles the preprocessing of battle data for training. ### Tech Stack: - Python ### Why It Matters: PokéChamp demonstrates the potential of language models in competitive environments, showcasing their ability to strategize and make decisions effectively. The project aims to push the boundaries of what language models can achieve in complex, real-time simulations. ### Repository: sethkarten/pokechamp ### GitHub Stars: 188 ### GitHub Description: Official repository of the spotlight ICML 2025 paper, PokeChamp: an Expert-level Minimax Language Agent.

Overview: MCP Image Generator is a server-side tool designed for AI image generation and editing, optimized for prompt optimization and quality presets. It supports Nano Banana (Gemini), OpenAI GPT Image, and BytePlus Seedream, making it versatile for various image generation needs. Key features: - Automatic prompt optimization - Supports multiple image providers: Nano Banana (Gemini), OpenAI GPT Image, and BytePlus Seedream - Enhances image quality and composition based on the provided details Tech stack: Node.js 22 or later, TypeScript Why it matters: MCP Image Generator streamlines the image generation process by enhancing prompts, ensuring high-quality output, and supports multiple providers, making it a valuable tool for developers and creators looking to automate their image generation workflows. Repository: shinpr/mcp-image Primary language: TypeScript GitHub stars: 185 GitHub description: MCP server for AI image generation and editing with automatic prompt optimization and quality presets. Supports Nano Banana (Gemini), OpenAI GPT Image, and BytePlus Seedream.

### Overview: This survey provides a comprehensive review of research on efficient diffusion models, focusing on three main categories: algorithm, system, and framework perspectives. It aims to help researchers and practitioners gain a systematic understanding of the field and inspire contributions to this exciting and practical area. ### Key Features: - **Algorithm-Level Optimization:** - **Latent Diffusion:** A method for improving the efficiency of diffusion models by optimizing the latent space. - **Loss Formulation:** Techniques for refining the loss function to reduce computational costs. - **Training Tricks:** Strategies for accelerating the training process. - **Low-Rank Adaptation:** Methods for reducing the dimensionality of the model for faster inference. - **Adapter:** Techniques for adapting models to specific tasks. - **ControlNet:** A tool for controlling the diffusion process. - **Solver:** Algorithms for efficient sampling. - **Sampling Scheduling:** Scheduling techniques to optimize sampling. - **Truncated Sampling:** Methods to reduce the computational cost of sampling. - **Knowledge Distillation:** Techniques for transferring knowledge from large models to smaller ones. - **System-Level Optimization:** - **Compression:** Techniques for reducing the size of diffusion models without significant loss in performance. ### Tech Stack: The survey is based on the following technologies: - Latent Diffusion - Loss Formulation - Training Tricks - Low-Rank Adaptation - Adapter - ControlNet - Solver - Sampling Scheduling - Truncated Sampling - Knowledge Distillation ### Why It Matters: Eff

Overview: This project provides a RAG-based chatbot integrated with Confluence data, allowing users to interact with a streamlit interface for creating and answering questions. Key features: - Streamlit UI for creating a Question Answering bot - Integration with Confluence for data retrieval - Evaluation of the RAG model using a provided dataset Tech stack: Python, Jupyter Notebook Why it matters: This project offers a practical solution for automating knowledge management and enhancing user interaction through natural language processing and AI integration with Confluence. Repository: BastinFlorian/RAG-Chatbot-with-Confluence Primary language: Jupyter Notebook GitHub stars: 187 GitHub description: RAG Chatbot with Confluence README excerpt: This project includes a chatbot built with RAG and streamlit, integrated with Confluence data for efficient question answering. It also provides a guide on how to set up and use the chatbot, including data loading and evaluation.

### Overview: This repository contains tutorials, step-by-step guides, and scripts for implementing diffusion models, specifically focusing on Stable Diffusion. It includes practical examples using Jupyter Notebooks and various libraries to generate images and understand the theory behind diffusion models. ### Key Features: - Tutorials on diffusion models and their software libraries. - A step-by-step guide on implementing a diffusion model from scratch. - Scripts and notebooks for generating images with Stable Diffusion. - A basic guide to prompt engineering. - Resources for further learning on diffusion models. ### Tech Stack: Jupyter Notebook ### Why it Matters: This repository serves as a comprehensive resource for developers interested in image generation using AI, providing practical examples and foundational knowledge to build and improve models.

### Overview: GUNDAM is a data management system that leverages language models to prioritize and manage textual data efficiently. Built on PyTorch, GUNDAM includes a comprehensive data manager with a GPT-2 generator and a demonstration retriever, making it highly flexible and efficient. ### Key features: - **Comprehensive Data Manager**: Includes a one-to-one miner, GPT-2 generator, and demonstration retriever. - **Flexible Support**: Supports GPT-2 models of various sizes. - **Efficient Data Quality Check**: Provides an efficient one-to-one miner for data quality assessment. ### Tech stack: Python ### Why it matters: GUNDAM is a groundbreaking tool that enhances the efficiency and effectiveness of data management by leveraging language models, making it indispensable for applications requiring data prioritization and demonstration management.

### Overview: SubgraphRAG is a simple yet effective retrieval-and-reasoning pipeline designed for knowledge-graph-based retrieval-augmented generation, showcasing the roles of graphs and large language models in this context. ### Key Features: - Simple yet effective retrieval and reasoning pipeline. - Supports knowledge-graph-based retrieval-augmented generation. - Integrates the strengths of graphs and large language models for improved performance. ### Tech Stack: Python ### Why It Matters: This project demonstrates the practical application of combining knowledge graphs with large language models to enhance retrieval-augmented generation, offering a simple yet effective solution for knowledge-based generation tasks.

Overview: The AI Horde Worker reGen is a software client designed for volunteers to generate images for the crowdsourced AI-Horde platform. By running a worker on your local machine, you can earn kudos, which are used to prioritize your own requests. Key features: - Generate images from text prompts - Earn kudos for completing jobs - Support for Windows, Linux, AMD GPUs, and DirectML Tech stack: Python Why it matters: The AI Horde Worker reGen allows individuals to contribute their computing power to the AI-Horde project, generating images and helping to advance the technology. By running the worker, you can earn kudos that can be used to expedite your own requests on the platform. Repository: Haidra-Org/horde-worker-reGen Primary language: Python GitHub stars: 187

### Overview: PII Masker is an open-source tool designed to protect sensitive data by automatically detecting and masking Personally Identifiable Information (PII) using advanced AI models. Powered by DeBERTa-v3, it offers high-precision detection, scalable performance, and a simple Python API, ensuring privacy compliance in various industries. ### Key Features: - **Comprehensive Protection**: Detects and masks multiple PII types including names, addresses, phone numbers, and more. - **High Performance**: Utilizes DeBERTa-v3 with 1024 token support for processing longer documents. - **Precision Focused**: Fine-tuned AI model for PII detection. - **Structured Output**: Provides both masked text and a structured PII dictionary. - **Easy Integration**: Simple Python API for seamless integration into workflows. ### Tech Stack: - **Python**: 3.8+ - **Hugging Face Models**: Models fine-tuned for PII detection. - **Milvus**: Vector database for structured data storage. ### Why It Matters: - **High Precision**: Ensures accurate detection and masking of PII. - **Compliance Friendly**: Designed to meet privacy regulations. - **Flexible Integration**: Easy integration with existing systems through a Python API. ### Tech Stack: - **Python**: 3.8+ - **Hugging Face Models**: Models fine-tuned for PII detection. - **Milvus**: Vector database for structured data storage. ### Why Choose PII Masker? - **Comprehensive Protection**: Identifies and masks various PII types.

### Overview: Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement (DyPRAG) is a novel framework that leverages a lightweight parameter translator model to efficiently map documents into parameterized knowledge. This approach reduces inference, training, and storage costs while enhancing language models' knowledge at test-time, specifically addressing the issue of RAG hallucination. ### Key Features: - **Parameterized Knowledge Mapping**: DyPRAG models the relationship between documents and parameters, enabling efficient knowledge representation. - **Test-time Knowledge Enhancement**: It dynamically integrates parameterized knowledge to resolve conflicts between contextual and parametric knowledge, providing a practical solution to mitigate RAG hallucination. - **Parameter Translator Model**: A lightweight model is used to translate documents into parameters, reducing the computational overhead. - **Extensive Experiments**: DyPRAG has been extensively tested on multiple datasets, demonstrating its effectiveness and generalization in test-time knowledge enhancement. ### Tech Stack: - Python ### Why It Matters: DyPRAG offers a practical and effective solution to mitigate RAG hallucination in real-world applications, making it a valuable addition to the toolkit of language model developers and researchers.

### Overview: Hyper-Diffusion-Planner is an official implementation of "Unleashing the Potential of Diffusion Models for End-to-End Autonomous Driving." This project showcases how diffusion models can effectively serve as planners for complex, real-world autonomous driving tasks. ### Key Features: - **Real Vehicle Demo**: Demonstrates the performance of the model on real-world urban scenarios. - **Nuplan Implementation**: Provides a benchmark implementation based on the Nuplan platform. - **NAVSIM Implementation**: Offers a benchmark implementation based on the NAVSIM devkit, featuring supervised diffusion VLA and reward-based fine-tuning. ### Tech Stack: - Python ### Why It Matters: This project highlights the potential of diffusion models in autonomous driving, showcasing their effectiveness and scalability for end-to-end autonomous driving tasks.

Overview: MUXI is an open-source AI application server designed for deploying AI agents in production environments. Key features: - Native AI agents as primitives - Self-hosted server stack with orchestration - Auditable memory management - Group-based RBAC - Twelve SDKs already integrated - Comprehensive formation capabilities Tech stack: MUXI Why it matters: MUXI simplifies the process of deploying AI agents by providing a complete, self-hosted server stack, eliminating the need to reinvent infrastructure. It offers production capabilities through native AI agents, ensuring reliability and scalability.

Overview: The AdEMAMix Optimizer is a state-of-the-art optimization algorithm designed to be both faster and more effective than existing methods. Key features: - **Improved Convergence Speed**: Significantly reduces the time required to converge to optimal solutions. - **Enhanced Performance**: Offers better performance across various optimization problems. - **Older but Better**: Built on the foundation of older algorithms but with modern improvements. Tech stack: Python Why it matters: The AdEMAMix Optimizer is a significant advancement in optimization algorithms, offering faster and more efficient solutions to complex problems, making it invaluable for researchers and developers alike. Repository: nanowell/AdEMAMix-Optimizer-Pytorch Primary language: Python GitHub stars: 187

Overview: Use image models to texture a game level from a gray blockout, including planning cameras, painting, and baking textures with real shadows. Key features: - Plan cameras - Texture level - Bake textures with real shadows Tech stack: TypeScript Why it matters: This project demonstrates how to texture a game level using image models, providing a realistic look with real shadows, making the game more immersive and visually appealing.

### Overview: Quixotic AI is a complete, open-stack solution for local AI inference on the JVM, designed from first principles. It provides a unified API for various AI tasks and runs end-to-end on the JVM, eliminating the need for sidecar servers, ONNX, or external services. ### Key features: - **AI inference engine**: A complete inference engine for chat, vision, audio, embeddings, reranking, and text-to-speech. - **Token-perfect tokenizers**: Fast tokenizers for large language models, designed for perfect tokenization. - **Native matrix multiplication**: Optimized matrix multiplication routines for CPUs and GPUs, achieving performance comparable to `llama.cpp`. - **GraalVM's Native Image**: First-class support for GraalVM Native Image, resulting in self-contained binaries with a small footprint and fast startup times. ### Tech stack: Quixotic AI is built on Java, leveraging the GraalVM ecosystem for efficient matrix multiplication and native image compilation. ### Why it matters: Quixotic AI offers a pure Java-based inference solution that runs on the JVM, eliminating the need for Python, ONNX, or external services. This makes it ideal for applications that require high performance and security, such as mission-critical systems and enterprise environments.

Overview: This project embeds multiple modalities into large language models, including images, audio, and documents, to enhance their capabilities. Key features: - Embeds images, audio, and documents into large language models. - Supports various modalities and models. - Uses flash-attn for efficient computation. Tech stack: Python Why it matters: This library expands the functionality of large language models by integrating diverse data types, potentially opening new possibilities in AI applications. Repository: sshh12/multi_token Primary language: Python GitHub stars: 188 GitHub description: Embed arbitrary modalities (images, audio, documents, etc) into large language models.