Projects

### Overview: Prima.cpp is a distributed implementation of LLMs that allows you to run 70B-level models on your everyday devices, such as laptops, desktops, phones, and tablets. With its fast inference speed, it can generate 11 tokens per second for QwQ-32B and 1.5 tokens per second for Llama 3-70B, making it suitable for tasks like running a Home Siri or powering a LLM agent in your home. ### Key features: - **Distributed Implementation**: Prima.cpp is a distributed version of LLMs, allowing you to run large models on multiple devices. - **High Memory Efficiency**: It keeps memory pressure below 10%, ensuring you can run very large models without worrying about OOM. - **Scalability**: You can easily scale up by adding more devices, making it highly flexible and scalable. - **Inference Speed**: Prima.cpp is 15x faster than llama.cpp, generating 11 tokens per second for QwQ-32B and 1.5 tokens per second for Llama 3-70B, making it suitable for tasks like running a Home Siri. ### Tech stack: C++ Prima.cpp is built on top of llama.cpp, leveraging its performance and features. ### Why it matters: Prima.cpp is crucial for anyone looking to run large language models on their local devices, providing a practical solution for tasks that require high inference speed and memory efficiency. Its ability to scale up and run on various devices

Overview: This repository provides a collection of models and tools for using large language models (LLMs) in Colab, including AutoGPT, ChatGenTitle, GroundedSAM, MiniGPT-4, LangChain, segment-anything, Chinese-LLaMA-Alpaca, CPM-Bee, QWen-VL, and more. Key features: - AutoGPT - ChatGenTitle - GroundedSAM - MiniGPT-4 - LangChain - segment-anything - Chinese-LLaMA-Alpaca - CPM-Bee - QWen-VL Tech stack: Python, Jupyter Notebook Why it matters: This repository serves as a comprehensive guide for developers looking to integrate large language models into their Colab projects, offering a range of models and tools to enhance their applications.

Overview: InverseBench is a framework designed to benchmark plug-and-play diffusion priors for inverse problems in physical sciences, offering a comprehensive evaluation of 14 algorithms across five distinct scientific inverse problems. Key features: - 14 Inverse Problem Algorithms - Plug-and-Play Diffusion Priors - Five Distinct Scientific Inverse Problems Tech stack: Jupyter Notebook Why it matters: InverseBench provides valuable insights into the strengths and weaknesses of existing algorithms, offering a benchmark for solving inverse problems in physical sciences. Repository: devzhk/InverseBench Primary language: Jupyter Notebook GitHub stars: 128 GitHub description: InverseBench (ICLR 2025 spotlight) README excerpt:  [**InverseBench: Benchmarking Plug-and-Play Diffusion Priors for Inverse Problems in Physical Sciences**](https://openreview.net/pdf?id=U3PBITXNG6) (ICLR 2025 spotlight) Hongkai Zheng*, Wenda Chu*, Bingliang Zhang*, Zihui Wu*, Austin Wang, Berthy T. Feng, Caifeng Zou, Yu Sun, Nikola Kovachki, Zachary E. Ross, Katherine L. Bouman, Yisong Yue (*: Equal contribution) Abstract: *Plug-and-play diffusion priors (PnPDP) have emerged as a promising research direction for solving inverse problems. However, current studies primarily focus on natural image restoration, leaving the performance of these algorithms in

### Overview: Z-Image Studio is a comprehensive tool designed for generating high-quality images using the Z-Image-Turbo text-to-image generation model. It offers a hybrid interface, including a CLI, web UI, and MCP server, optimized for various hardware configurations. ### Key Features: - **CLI Interface**: Fast and direct image generation from the terminal. - **Web UI**: Modern web interface for interactive image generation. - **MCP Server**: Capable of being called by AI agents. - **Hybrid Interfaces**: Supports multiple languages, hardware-aware model recommendations, and customizable output directories. - **Automatic Memory Management**: Handles attention slicing for systems with lower RAM/VRAM to prevent out-of-memory errors and optimize performance. - **Multiple LoRA Support**: Uploads and manages LoRAs, applying up to 4 with per-LoRA strengths in a single generation. - **Seed Control**: Reproducible image generation via CLI or web UI. - **Customizable Output Directory**: Allows customization via config file and environment variable. - **Responsive Layout**: Mobile compatible and supports light, dark, and auto themes. ### Tech Stack: - Python 3.11 - FastAPI - PyTorch - Hugging Face - Apple Silicon (MPS) - NVIDIA CUDA - AMD ROCm - Intel XPU - Docker ### Why It Matters: Z-Image Studio is designed to be efficient and compatible with a wide range of hardware configurations, ensuring seamless performance across different operating systems and GPU setups. Its features, including automatic memory

Overview: Canvas is a macOS application that allows users to generate and edit images using DALL·E 3 and DALL·E 2, offering features such as image generation, editing, variations, and sharing capabilities. Key features: - Image Generation with DALL·E 3 & DALL·E 2 - Image Editing (for DALL·E 2) - Image Variations (for DALL·E 2) - Copy, save, and share generated images - Simple and easy to use - Free and open source - Native Tech stack: Swift Why it matters: Canvas provides a user-friendly interface for generating and editing images using advanced AI technology, making it a valuable tool for artists, designers, and anyone looking to explore the capabilities of DALL·E.

Overview: IMAGGarment is a project that addresses the challenges of multi-conditional controllability in personalized fashion design and digital apparel applications. It uses a two-stage training strategy to model global appearance and local details, enabling precise and customizable garment synthesis with high realism and controllability. Key features: - Fine-grained garment generation - Support for precise and customizable garment synthesis - Enables high realism and controllability for digital fashion design Tech stack: Python Why it matters: IMAGGarment is significant because it addresses the challenges of multi-conditional controllability in personalized fashion design and digital apparel applications, providing a solution for precise and customizable garment synthesis with high realism and controllability. Repository: muzishen/IMAGGarment Primary language: Python GitHub stars: 129 GitHub description: IMAGGarment is a project that addresses the challenges of multi-conditional controllability in personalized fashion design and digital apparel applications. It uses a two-stage training strategy to model global appearance and local details, enabling precise and customizable garment synthesis with high realism and controllability. The project is supported by a technical report, train and inference code, and a dataset for garment design. ## 🗓️ Release - [2026/3/4] 🔥 IMAGGarment is accepted by TVCG 2026. - [2025/4/18] 🔥 We released the [technical report](https://arxiv.org/pdf/2504.1317

Overview: This project is a web UI for Ollama, a model server, built in Java using Spring Boot and Vaadin. It aims to provide a fully functional web interface for Java and Spring developers. Key features: - Web UI for interacting with Ollama models - Integration with Ollama4j for enhanced capabilities - Support for models deployed on Ollama servers Tech stack: Java, Spring Boot, Vaadin, Ollama4j Why it matters: This project simplifies the interaction with Ollama models for developers familiar with Java and Spring, making it easier to integrate Ollama into their existing projects.

Overview: AI-CHATKIT is a full-stack AI agent chat project built with LangGraph, FastAPI, NextJS, and Chroma. It supports custom knowledge base Q&A, tool invocation, and chat streaming output. Key features: - Custom knowledge base Q&A capabilities for agents using ChromaDB. - Support for custom tool invocation for agents. - Python backend interface API with FastAPI. - Support for custom frontend applications using NextJS. - Chat history saved in local browser cache. Tech stack: Python, LangGraph, FastAPI, NextJS, ChromaDB, OpenAI, DashScope, DeepSeek, bge-m3. Why it matters: AI-CHATKIT provides a template for quickly building AI agent chat applications using LangGraph, enhancing Q&A capabilities with RAG, and supporting multi-agent collaboration, making it suitable for various chat application development needs.

Overview: PersonaForge is a platform for designing and deploying hyper-realistic AI conversational agents that inhabit a fictional social media ecosystem. It allows users to create detailed persona profiles and release them into a simulated news feed where they can engage in multi-party conversations. Key features: - **Persona Blueprint Engine** for defining each bot's age, profession, speech patterns, cognitive biases, and emotional baseline. - **Feed Simulation** for a scrolling social timeline where AI agents generate content based on their persona and recent events. - **Dynamic Memory Graph** for recalling past interactions, holding grudges, or changing topics based on contextual relevance. - **Multi-Model Backend Support** to swap underlying language models without rewriting persona logic. - **Responsive Web Interface** with mobile-first design, adapting seamlessly to any screen size. - **Multilingual Persona Templates** available in English, Spanish, French, and German with locale-aware humor and idioms. - **24/7 Autonomous Operation** with responsive agents that continue posting and replying on schedule. Tech stack: HTML Why it matters: PersonaForge is a unique tool for studying how AI personalities diverge, converge, and adapt under social pressure, generating rich, procedurally written dialogue for various applications, and providing a framework for prototyping user-facing AI characters without needing a full backend infrastructure.

### Overview: This repository is a curated list of research papers, datasets, and tools focused on applying machine learning and deep learning techniques to compilers and program optimization. It serves as a comprehensive resource for researchers and developers interested in advancing compiler optimization through AI-driven methods. ### Key features: - Papers: A collection of research papers from various years, covering topics such as compiler optimization, machine learning for compiler optimization, and performance tuning. - Books: No specific books mentioned. - Talks and Tutorials: No specific talks or tutorials mentioned. - Compiler Projects: No specific projects mentioned. - Benchmarks and Datasets: No specific benchmarks or datasets mentioned. - Conferences: No specific conferences mentioned. - Journals: No specific journals mentioned. - How to Contribute: No specific contribution guidelines mentioned. ### Tech stack: Unknown. ### Why it matters: This repository provides a valuable resource for researchers and developers looking to leverage machine learning and deep learning techniques to improve compiler optimization. It serves as a curated list of research papers, datasets, and tools that can be used to stay updated with the latest advancements in the field.

Overview: Avenkin is a private AI assistant designed for personal use, offering a range of features including visual assistance, voice translation, memory recall, and integration with various services. Key features: - Visual assistance with scene descriptions, text recognition, and smart capture. - Voice translation and spoken reading support. - Memory recall and context sharing. - Integration with Home Assistant, Siri Shortcuts, and MCP tool servers. - Offline AI capability for use on non-Wi-Fi devices. Tech stack: Swift. Why it matters: Avenkin simplifies communication and tasks by integrating seamlessly with your existing devices and services, making it a versatile and accessible AI assistant. Repository: straff2002/OpenGlasses GitHub stars: 126 GitHub description: Making Meta Ray-ban / Oakley glasses smarter. Non-Meta Model selection, agentic features, smart guidance.

### Overview: HUF is an open-source, self-hosted multi-agent AI infrastructure designed for teams and apps. It supports cloud and local models, integrates with various tools, and automates workflows across multiple business systems including Slack, ERPNext, Discord, and Gmail. ### Key Features: - **Open Source and Self-Hosted**: Your agents, data, and infrastructure are under your control. - **Local Model Support**: Connects to Ollama, LM Studio, or any OpenAI-compatible local endpoint. - **Pluggable Execution Environments**: Runs agents locally, in isolated virtual environments, over SSH, or inside Docker containers. - **No-Code Visual Control with Code-Level Depth**: Visual flow builders, agent configuration, and multi-channel gateways alongside hardened Python code execution sandboxes. - **Built-in Data Governance**: Role-based permissions, execution profiles, full audit trails, and cost/cache analytics. ### Tech Stack: Python ### Why It Matters: Centralize intelligence, execution, and communication into a single, self-hosted engine. AI can be trusted to operate inside real business systems without compromising data ownership or security. HUF enables teams to create intelligent multi-agent solutions, automate workflows, and manage complex interactions across various business systems.

### Overview: 7-docs is a powerful toolset for building a knowledge base using OpenAI APIs, allowing users to both ingest content and query it like ChatGPT. ### Key features: - Use `@7-docs/cli` to ingest content from the command-line. - Use `@7-docs/edge` for deploying functions to query the content (aka "chat completions"). ### Tech stack: TypeScript ### Why it matters: 7-docs provides a comprehensive solution for creating a knowledge base, enabling users to easily ingest and query their content using OpenAI APIs, similar to interacting with ChatGPT.

### Overview: This Python project is a versatile voice assistant that can perform a wide range of tasks without internet access. It includes features such as weather forecast reporting, Google search, YouTube video search, Wikipedia definition search, and translation between languages. ### Key Features: - **Weather Forecast Reporting**: Provides the weather forecast for any location in the world. - **Google Search**: Performs searches in Google. - **YouTube Video Search**: Searches for videos on YouTube. - **Wikipedia Definition Search**: Searches for definitions and reads the first two sentences. - **Translation**: Translates from one language to the user's native language. - **Miscellaneous Functions**: Includes additional features like flipping a coin, providing random greetings and farewells, and changing language settings. ### Tech Stack: The project uses Python 3, PyAudio, and Google. It leverages the Google API for weather forecasts and the Vosk model for offline speech recognition. ### Why it Matters: This project is designed to be a reliable and accessible voice assistant that can perform a variety of tasks without the need for an internet connection. It is useful for users who may not always have access to the internet, making it a valuable tool for those on the go or in remote locations. ### Repository: The repository is for the Voice Assistant Python App project, which includes all the necessary dependencies and features. It is hosted on GitHub and has received 127 stars, indicating its popularity among developers. ### Installation: To quickly set up all required dependencies, use the command: ```bash pip install requirements.txt

Overview: This repository contains PyTorch implementations of the Graves 2013 experiments for End-to-End Automatic Speech Recognition (ASR). It includes models, training scripts, and decoding utilities for both CTC and RNNT joint models. Key features: - Model implementation for Graves 2013 experiments - Training scripts for CTC and RNNT models - Evaluation script for decoding results - Feature extraction and transformation scripts - Requirements for running the experiments Tech stack: Python Why it matters: This repository provides a practical and detailed guide for developers interested in implementing and experimenting with end-to-end ASR models using PyTorch, specifically focusing on the Graves 2013 experiments. Repository: HawkAaron/E2E-ASR Primary language: Python GitHub stars: 127 GitHub description: PyTorch Implementations for End-to-End Automatic Speech Recognition

### Overview: `sieves` is a plug-and-play document AI framework that enables developers to quickly integrate structured document analysis without the need for extensive training or model setup. It provides a unified interface for various language model frameworks, allowing for easy switching and optimization of models. ### Key Features: - **Zero Training Required:** Immediate inference using zero-/few-shot models. - **Unified Generation Interface:** Seamlessly use multiple libraries such as `dspy`, `gliner2`, `langchain`, `outlines`, and `transformers`. - **Observable Pipelines:** Easy debugging and monitoring with conditional task execution. - **Integrated Tools:** Optional document parsing with `docling` and text chunking with `chonkie`. ### Tech Stack: Python ### Why It Matters: Recognizes that different language model frameworks excel at different aspects of language model development, making `sieves` a versatile solution for various document AI tasks.

Overview: YuE2-Turbo is an inference acceleration and serving layer for the YuE2 model, designed to improve performance by up to 3.31 times per GPU. Key features: - Faster inference speed per song - Increased number of songs per GPU - Uses vLLM for autoregressive stages - Resides weights on the GPU - Batched acoustic synthesis across requests - Provides an HTTP job API and browser Studio Tech stack: Python Why it matters: YuE2-Turbo leverages advanced inference techniques to enhance the performance of the YuE2 model, making it more efficient and capable of handling larger volumes of audio generation tasks.

### Overview: BabyGPT: Build Your Own GPT Large Language Model from Scratch BabyGPT is an educational project designed to guide developers through the process of building a GPT (Generative Pre-trained Transformer) model from scratch. This project starts with a simple Bigram Model and gradually incorporates advanced concepts from the Transformer architecture to create a more sophisticated model. ### Key Features: - **Simple Bigram Model**: The project begins with a Bigram Model to introduce the fundamental concepts of language modeling. - **Transformer Architecture**: Gradually incorporates advanced Transformer architecture to build a larger and more powerful model. - **Pre-Training**: Uses a pre-training approach to fine-tune the model on a large dataset, improving its ability to generate coherent text. ### Tech Stack: - **Python**: The primary programming language used for building the model. - **PyTorch**: Used for implementing the model architecture and training process. ### Why It Matters: - **Educational Tool**: Ideal for developers looking to understand the inner workings of GPT models and how they are built. - **Practical Application**: Provides a practical way to build and fine-tune a large language model, which can be applied to various applications like text generation, chatbots, and more. ### Repository: - **GitHub Repository**: [TatevKaren/BabyGPT-Build_GPT_From_Scratch](https://github.com/TatevKaren/BabyGPT-Build_GPT_From_Scratch) - **Stars**: 127 - **Description**: "BabyGPT: Build Your Own

### Overview: Assessing the reliability, accuracy, and safety of large language models, RAG pipelines, and AI agents is crucial for ensuring their responsible deployment. This repository serves as a comprehensive collection of evaluation tools, frameworks, and benchmarks designed to help developers and researchers evaluate the performance of these AI systems effectively. ### Key Features: - **Frameworks**: A variety of evaluation frameworks, including Aleph Alpha Eval Framework, Bloom, ChainForge, and more. - **Retrieval**: Tools for evaluating LLMs based on retrieval-based evaluation methods. - **Adversarial**: Frameworks and tools for evaluating AI agents and models under adversarial conditions. - **Platforms**: Self-hosted, managed, and cloud-based platforms for evaluating AI systems. - **Benchmarks**: General, long-context, specialized, agents, multimodal, and safety benchmarks for LLMs and AI agents. - **Leaderboards**: A platform for sharing and comparing evaluation results. - **Resources**: Papers, guides, and lists of evaluations and benchmarks. ### Tech Stack: Python and JavaScript ### Why It Matters: This repository provides a robust set of tools and resources for evaluating the reliability, accuracy, and safety of AI systems, ensuring that these technologies are developed and deployed responsibly and effectively.

Overview: This repository contains the implementation of the paper "When Drafts Evolve: Speculative Decoding Meets Online Learning" at ICML'26, focusing on a unified framework called ***OnlineSPEC*** that integrates speculative decoding with online learning to improve performance in large language model inference and adaptation. Key features: - Ens-EAGLE/Ens-EAGLE-3: Online ensemble methods for EAGLE and EAGLE-3. - Opt-Hydra: Online learning with optimism for Hydra. - Online-LR: Online DPO for Lookahead Reasoning. Tech stack: Python. Why it matters: This framework offers a novel approach to enhance speculative decoding performance by leveraging online learning techniques, making it particularly useful in large language model inference and adaptation scenarios. Repository: ZinYY/OnlineSPEC Primary language: Python GitHub stars: 127 GitHub description: [ICML 2026] When Drafts Evolve: Speculative Decoding Meets Online Learning