Projects

### Overview: Rust-powered CPU-only LLM inference engine, designed for efficient inference on consumer-grade CPUs. RAI is a 4-bit quantized language model with hand-written AVX2 kernels, offering a lightweight and fast alternative to GPU-based systems. ### Key features: - **4-bit Quantization**: Uses 4-bit quantization for lower memory footprint and faster inference. - **Hand-Written AVX2 Kernels**: Optimized kernels for maximum performance. - **Speculative Decoding**: Enhances throughput by allowing partial decoding. - **Local HTTP/MCP Server**: Provides a local HTTP server and message queue (MCP) for inter-process communication. ### Tech stack: Rust ### Why it matters: RAI is designed to be a fast, lightweight solution for CPU-based language models, offering a significant performance improvement over GPU-based systems. Its 4-bit quantization and hand-written AVX2 kernels make it suitable for applications requiring low memory and low latency. The lack of GPU, Python runtime, and reliance on Rust makes it an ideal choice for developers looking to create high-performance, low-resource systems.

Overview: This repository provides a comprehensive guide to full-stack AI engineering, covering model foundations, post-training, AI infrastructure, RAG, agents, memory, MCP, evaluation, and production. Key features: - Model foundations - Post-training techniques - AI infrastructure - RAG (Retrieval-Augmented Generation) - Agents - Memory management - Model Context Protocol (MCP) - Agent evaluation - Production applications Tech stack: Unknown Why it matters: This playbook serves as a practical guide for building production AI systems and agentic applications, providing a systems-level view of AI engineering from compute and data infrastructure to production applications and evaluation.

Overview: FinalDream is a cross-platform AI image generation tool that allows users to create high-quality images using off-line AI models on modern computers with Vulkan support. Key features: - Supports multiple platforms (MacOS, Windows, Linux) - Works with z-image and zimage-ncnn-vulkan - Compatible with Gemini 3 Pro, Claude Sonnet 4.5 (Thinking), Antigravity, and Cursor - Downloads and installs easily with a single command Tech stack: z-image, zimage-ncnn-vulkan, Gemini 3 Pro, Claude Sonnet 4.5 (Thinking), Antigravity, Cursor Why it matters: FinalDream simplifies the process of generating high-quality images, making it accessible to users without requiring internet connectivity or complicated setup.

Overview: OmniVLA is an innovative multimodal vision-language-action model designed for robotic manipulation tasks, integrating novel sensor inputs to enhance perception and manipulation capabilities. Key features: - Sensor-masked image representation - Unified representation that overlays physically meaningful, spatially grounded masks onto RGB images - Integration of infrared camera, mmWave radar, and microphone array for diverse sensor inputs - Lightweight per-sensor projectors for data-efficient learning Tech stack: Python Why it matters: OmniVLA achieves high success rates in challenging robotic manipulation tasks by leveraging novel sensor inputs, providing a more robust and efficient approach compared to traditional RGB-only methods. Repository: GuoHeyu/OmniVLA Primary language: Python GitHub stars: 16 GitHub description: [ICRA26] OmniVLA: Physically-Grounded Multimodal VLA with Unified Multi-Sensor Perception for Robotic Manipulation

Overview: Context0 is the world's first decentralized memory-sharing protocol for AI agents, ensuring memories live forever on the blockchain and are owned by users. It enables seamless sharing of memories across AI agents, providing perfect, permanent memory, global knowledge compounding, lightning-fast search, and built-in privacy. Key features: - Decentralized memory-sharing protocol - Blockchain-based memory storage - Permanent and immutable storage - Global knowledge compounding - Lightning-fast search - Built-in privacy Tech stack: TypeScript Why it matters: Context0 addresses the fundamental flaws of current AI memory systems, providing a decentralized, permanent, and privacy-preserving solution that ensures AI agents have perfect, permanent memory that survives system crashes, company shutdowns, and censorship.

### Overview: This repository is a curated collection of noteworthy resources in the field of Machine Learning Systems (MLSys) from various experts and enthusiasts. It aims to provide a valuable resource for those interested in the latest developments and insights in the field. ### Key Features: - 方佳瑞 (Feifei Bear) - 手抓饼熊 (TongSan Pang) - InterestingLSY (作者) - Yinmin Zhong (作者) - 赵轩磊 (Chui-Zou-Liao-Ka-Pei) - 谁来救救我 (Ming-Yin-2-93) - Fazzie (作者) - Ying Sheng (作者) - Lianmin Zheng (作者) - Chayenne Zhao (作者) - Swords (作者) - Byron Hsu (作者) - Pentium PRO (作者) - ZZYSonny (作者) -刀刀宁 (作者) - DefTruth (作者) - 陈乐群 (作者) - Zihao Ye (作者) - Crowowrk (作者) - BBuf (作者) - Yineng Zhang (作者) - ZOMI酱 (作者) - 初七123334 (作者) - ZOMI酱 (作者) - Kan Zhu (作者) - 老乔惜 (作者) - 爱情酱 (作者) - KuangjuX (作者) - SiriusNEO (作者) - SuSun (作者) - 鸦哥 (作者) - 进击

### Overview: GPT-Image-2 Studio 是一个轻量级的工具，专为调用 GPT-Image-2 提供支持。它允许用户批量调用 GPT-Image-2，支持 AI 拆分提示词、队列、重试和图生图功能。无需后端服务，只需提供 API key 和 Base URL 即可使用。 ### Key features: - 支持批量调用 GPT-Image-2 - AI 拆分提示词 - 队列功能 - 重试机制 - 图生图功能 ### Tech stack: HTML ### Why it matters: GPT-Image-2 Studio 是一个强大的工具，特别适合需要大量图像生成任务的场景，如产品海报设计、艺术创作等。它简化了调用 GPT-Image-2 的过程，提高了工作效率，同时提供了灵活的配置选项。

Overview: AI Draw 是一个基于 TypeScript 的 AI 画图 web 应用，支持多种模型，通过智能改写和抽卡功能提供创意灵感。 Key features: - 切换多个模型 - 自动改写中文 prompt 到英文 - 抽卡生成 prompt - 智能改写详细 prompt - API 调用次数限制 Tech stack: TypeScript Why it matters: AI Draw 是一个便捷的 AI 画图工具，提供创意灵感和抽卡功能，适合开发者和设计师使用。 Repository: Mereithhh/ai-draw Primary language: TypeScript GitHub stars: 16 GitHub description: AI 画图 web 应用，智能改写 prompt，可以抽卡。docker 一键部署 GitHub link: [https://github.com/Mereithhh/ai-draw](https://github.com/Mereithhh/ai-draw)

### Overview: The project aimed to fine-tune the Stable Diffusion model to generate LEGO-style images based on prompts. ### Key Features: - Fine-tuned Stable Diffusion model - LEGO dataset with mixed figures and constructions images - Image captioning using BLIP model - Gradio library for app development - Available models and dataset on Kaggle - Results include images of rubber duck aliens visiting Earth, a Disney-style cabin, a solarpunk utopia in the Amazon rainforest, and an insect robot preparing a meal. ### Tech Stack: Python ### Why It Matters: The project demonstrates the integration of AI models with LEGO imagery, showcasing how AI can enhance creativity and storytelling in visual media.

Overview: This project is a local implementation of the Claude Context MCP server, focusing on privacy and data sovereignty. It replaces cloud-based dependencies with PostgreSQL and Ollama, ensuring no external calls and full data control. Key features: - 100% Local Processing: All operations occur on the local machine. - PostgreSQL + pgvector Integration: Replaces Milvus/Zilliz Cloud with local PostgreSQL. - Ollama Embeddings: Uses free, open-source models instead of OpenAI. - Privacy-First: Your code never leaves your machine. - Zero API Costs: No subscription fees or usage charges. - Multimodal Support: Nomic embedding model supports text and images. Tech stack: TypeScript Why it matters: This project offers a local alternative to the cloud-based Claude Context, ensuring complete privacy and control over data without the need for external APIs or cloud resources. Repository: MikeO-AI/claude-context-local GitHub stars: 16

### Overview: This project is a personal knowledge management server built with TypeScript, designed to store and retrieve knowledge using semantic search powered by local embeddings. It is compatible with the Opencode ecosystem and requires no external API keys. ### Key features: - **🔍 Semantic Search** — Find knowledge using vector embeddings (BGE-small-en-v1.5). - **📝 Text Search** — Keyword-based search fallback. - **🏷️ Tag Organization** — Categorize entries with tags. - **🔌 Plug-and-Play** — No external services required (embeddings run 100% locally). - **💾 Persistent Storage** — Data stored in `~/.local/share/opencode-personal-knowledge/`. - **🔄 Automatic Indexing** — Entries are vectorized on creation. ### Tech stack: TypeScript ### Why it matters: This project allows users to store and retrieve knowledge efficiently, making it a valuable tool for personal knowledge management within the Opencode ecosystem.

### Overview: Doppelganger is an AI toolkit that fine-tunes large language models on your own chat history to create a digital twin that mimics your writing style. Built on LLaMA-Factory, it transforms raw chat exports into a ShareGPT-formatted dataset for supervised fine-tuning, then trains a LoRA adapter. ### Key Features: - **Source-agnostic ingestion**: One adapter per platform parses an export into a normalized message stream; the rest of the pipeline is shared. - **Conversation reconstruction**: Sessionizes messages by silence gaps and reply links, merges consecutive turns. - **Fine-tuning on your chats**: Captures writing tone, vocabulary, phrasing, response lengths, and conversational habits. ### Tech Stack: Python ### Why It Matters: Fine-tuning an LLM on your chat history can help create a digital twin that closely mirrors your writing style, potentially improving productivity and personalization in chat-based interactions. However, it's important to handle sensitive data responsibly, ensuring that the model's output is not taken as genuine from anyone.

### Overview: pyWikiMM is a Python library designed to collect and organize multimodal datasets of Wikipedia articles and their corresponding images. This dataset is optimized for efficient data collection and supports a wide range of languages, making it valuable for multimedia research and applications. ### Key Features: - Downloads Wikipedia articles and their images along with detailed metadata. - Supports caching for efficient updates. - Offers property selection to optimize resource usage. - Works with any language Wikipedia. ### Tech Stack: Python ### Why It Matters: This library can be used to extend existing cross-modal learning datasets or explore novel multimedia research directions, thereby improving the scope and accuracy of multimedia models and applications.

### Overview: Yet Another StableDiffusion Implementation is a project that provides a stable diffusion script based on Hugging Face's Diffusers, offering enhanced configurability and additional features. The script is compatible with various models and can be easily integrated into other scripts. ### Key Features: - Enhanced script with extra configurability - Single script for accessing all functionalities - Example code for a Discord bot demonstrating integration ### Tech Stack: - Python 3.9+ - Diffusers, Transformers, Hugging Face Hub, Accelerate, PyTorch, XFormers ### Why It Matters: This project simplifies the process of integrating stable diffusion models into other scripts, making it easier to use and extend the functionality of the script. The compatibility with various models and the ability to integrate it into other scripts make it a valuable tool for developers working with stable diffusion.

### Overview: LexicaAPI is a Python wrapper for the LexicaAPI, a comprehensive API for queries and support. It simplifies interaction with the API, making it easier for developers to integrate and utilize the services. ### Key features: - **Image Generation and LLMs:** Supports image generation and language models. - **Image Upscaling:** Enhances image quality by upscaling images. - **News App:** Provides access to news articles. - **Social Media Downloader:** Downloads videos and images from social media platforms. - **Telegram Bots:** Integrates with Telegram bots for various functionalities. ### Tech stack: Python ### Why it matters: LexicaAPI simplifies the integration and usage of the LexicaAPI, making it easier for developers to access and utilize its services. Its features cater to a wide range of applications, from image processing to news retrieval, making it a valuable tool for developers looking to enhance their projects.

### Overview: This repository provides a detailed runbook and installer for the fastest documented inference stack running on plain PCIe Tesla V100s, achieving impressive performance without NVLink. ### Key Features: - **PCIe Inference Stack**: The fastest documented inference stack for Volta, running on 2x PCIe Tesla V100-32GB. - **No NVLink Required**: Achieves 61-74 tok/s decode, with MTP speculative decoding enabled. - **1Cat-vLLM 1.2.2 Engine**: Utilizes the 1Cat-vLLM 1.2.2 engine for optimal performance. - **Native Precision Checkpoint**: Uses the native precision checkpoint from RadixArk/Qwen3.8-27B-NVFP4. - **GPU Specifications**: 2x Tesla V100-PCIE-32GB, NVIDIA driver 580.173.02, Ubuntu 24.04, CUDA 12.8. - **Performance Metrics**: Measured results show 61 avg / 74 peak tok/s decode, 476 tok/s prefill, 5.0 greedy / 2.4–2.8 sampling, and 131K max context. - **Boot Time**: Served OpenAI-compatible with MTP speculative decoding, achieving ~2.5 min boot time. ### Tech Stack: Shell ### Why It Matters: This repository provides the fastest documented inference stack for Volta running on plain

### Overview: Diffusers Tuner is a lightweight, configuration-driven, and flexible fine-tuning framework designed for the Diffusers ecosystem. It simplifies the process of fine-tuning diffusion models, eliminating the need to rewrite training pipelines and providing a streamlined, "plug-and-play" experience. ### Key features: - **Zero-Friction Adaptation**: Tunes existing diffusers pipelines without refactoring the core logic. - **Config-First Workflow**: Utilizes Hydra for flexible, hierarchical configuration management. - **Dataset Schema**: Adapts to various dataset formats by defining the dataset schema. - **Bucket Batched Dataset Training**: Supports bucket dataset training for larger batch sizes with different resolutions. - **Lightweight LoRA**: Simplified LoRA implementation focusing on what matters. - **Modern Model Support**: Specialized demos for Qwen-Image and FLUX.1. ### Tech stack: Python ### Why it matters: This framework streamlines the fine-tuning process, making it easier and more efficient for developers working with the Diffusers ecosystem. It reduces complexity and overhead, allowing for faster experimentation and deployment.

### Overview: Comic Studio AI is a multi-agent comic generator powered by Google Gemini API and FastAPI on Google Cloud Run. It ensures consistent character identity across panels and supports 7 languages for global localization. ### Key Features: - **Character Consistency**: Locks canonical character descriptors across every frame. - **7-Language Localization**: Supports 7 languages for educators, creators, and students worldwide. - **FastAPI Microservice**: Stateless orchestration runs as a containerized FastAPI service on Google Cloud Run with automatic scaling. ### Tech Stack: HTML ### Why It Matters: - Solves character consistency across multi-panel comics. - Native 7-language visual storytelling. - Stateless FastAPI microservice on Google Cloud Run for efficient and scalable deployment. Repository: RobinaMirbahar/Comic-Studio-Ai GitHub Stars: 16

Overview: SketchSSM aims to reduce state access traffic in hybrid-attention models by approximating reads through a compact sketch, while keeping full-state updates. This approach reduces traffic by about 10x while maintaining average accuracy. Key features: - Flush step: Reads the full state once, applies buffered updates, computes a compact sketch, and writes back the state and sketch. - Non-flush steps: Combines the sketch and coefficient map with the query to reconstruct the output without reading the full state. Tech stack: Python with CUDA kernels for decoding. Why it matters: By reducing state access traffic, SketchSSM enables larger decode batches, making it suitable for applications requiring high throughput and low latency. Repository: SNU-ARC/SketchSSM GitHub stars: 15 GitHub description: SketchSSM: Write to the Full State, Read from a Compact Sketch

### Overview: 本项目是一个全面的RAG系统教程，涵盖了从基础概念到高级技术的详细内容。通过Jupyter Notebook的形式，逐步讲解如何构建和优化一个高效的检索增强生成系统。所有使用的模型都是本地化部署的，可以在3090上运行。 ### Key features: - 文档加载和解析 - 文本分割技术 - 向量嵌入模型的使用 - 向量数据库（Chroma）的配置和使用 - 构建基本的问答系统流程 - 查询优化技术 - 查询转换和重写 - 上下文压缩检索 - 自查询检索器 - 多数据源路由 - 查询构建技术 - 结构化查询处理 - 条件过滤和元数据检索 - 智能分块技术 - 父文档检索器 - 压缩检索器 - 时间加权检索器 - 交叉编码器重排序 - 倒数排序融合（RRF） - 多查询检索集成 - 混合检索技术 - 微调BCE嵌入模型以适应特定领域 - 评估微调前后模型性能差异 ### Tech stack: Jupyter Notebook ### Why it matters: RAG（检索增强生成）是一种结合了信息检索和语言生成的技术，它允许大型语言模型在生成响应时参考外部知识源。这种方法解决了传统语言模型的一些关键限制，如知识截止日期、事实准确性等问题。