Projects

Overview: This project is a Node.js library designed for inference with large language models, specifically focused on models like LLaMA, Alpaca, GPT4All, and others. It leverages llama-rs, llama.cpp, and rwkv.cpp for backend support and uses napi-rs for communication between Node.js and the llama thread. Key features: - Supports multiple models including LLaMA, Alpaca, GPT4All, and others. - Compatible with both LLama-rs and llama.cpp backends. - Compatible with various platforms including macOS, Linux, and Windows. - Provides a simple npm package for easy installation. - Includes a Discord community for support and updates. Tech stack: Node.js, llama-rs, llama.cpp, rwkv.cpp, napi-rs. Why it matters: This library simplifies the process of using large language models in Node.js environments, making it accessible for developers without deep knowledge of AI model backend implementation.

Overview: This repository is a collection of code samples and notebooks designed to demonstrate the use of Generative AI on Google Cloud Vertex AI. It includes various applications such as prompt-based recipes, evaluation of generative AI models, and research operationalization. Key features: - Generative AI models and tools in Vertex AI - Prompt-based recipes for Gemini - Evaluation of generative AI models - Retrieval augmented generation - Vertex AI Search - Vertex AI evaluation services - Research operationalization of models and frameworks Tech stack: Jupyter Notebook Why it matters: This repository serves as a practical guide for developers and researchers interested in integrating Generative AI into their projects on Google Cloud, providing hands-on examples and best practices.

### Overview This repository is a curated list of papers related to neural network quantization, focusing on recent AI conferences and journals. It categorizes papers by model structures and application scenarios, providing a comprehensive overview of post-training quantization methods such as PTQ, non-uniform quantization, mixed-precision quantization (MP), and binary/ternary quantization (Extreme). Contributions to expand this list are highly encouraged. ### Key Features - **PTQ (Post-Training Quantization)**: Methods for quantizing models post-training. - **Non-uniform Quantization**: Techniques that vary the quantization granularity across different parts of the model. - **MP (Mixed-Precision Quantization)**: Quantization of models using different precision levels. - **Extreme (Binary/ternary Quantization)**: Techniques that use binary or ternary quantization to reduce the model size. ### Tech Stack - **Python**: Used for data processing and analysis. - **Markdown**: For documentation and formatting. ### Why It Matters This repository serves as a valuable resource for researchers, practitioners, and developers interested in efficient neural network inference. It provides a clear and organized overview of quantization methods, making it easier to find relevant papers and stay updated with the latest research in the field. ### Keywords - **PTQ**: post-training quantization - **Non-uniform**: non-uniform quantization - **MP**: mixed-precision quantization - **Extreme**: binary or ternary quantization --- ## Survey - "A Survey of Quantization Methods for Efficient Neural Network

Overview: This project demystifies advanced RAG pipelines by examining the mechanics, limitations, and costs that often remain opaque. It focuses on shedding light on the inner workings of these pipelines through a simplified example using the Sub-question Query Engine. Key features: - Vector retrieval - Data warehouse - Response generation Tech stack: Python Why it matters: This project provides a practical, user-friendly approach to understanding and building RAG pipelines, making it easier for developers to integrate advanced AI capabilities into their projects without the complexity of opaque frameworks. Repository: pchunduri6/rag-demystified Primary language: Python GitHub stars: 855 GitHub description: An LLM-powered advanced RAG pipeline built from scratch

Overview: Pipeless is an open-source computer vision framework designed to simplify the development and deployment of applications in real-time, enabling developers to create and run computer vision projects in minutes instead of weeks. Key features: - Multi-stream support - Dynamic stream configuration - Multi-language support - Built-in restart policies - Industry-standard models and custom models - Deployability to edge and IoT devices or the cloud Tech stack: Rust Why it matters: Pipeless streamlines the development and deployment process for computer vision applications, making it easier to create and run applications in real-time, reducing the time required and improving efficiency.

### Overview: 本项目是基于 DeepSeek 的 RAG 系统的代码仓库，旨在实现一个完整的检索增强生成系统。项目涵盖了 RAG 系统的多个关键组件，包括基础 RAG 系统、数据加载与预处理、文本切块、向量嵌入、向量存储、检索前处理、索引优化、检索后处理、响应生成和系统评估等。通过模块化设计，每个模块负责 RAG 系统的不同方面，确保了系统的高效和灵活。 ### Key features: - 基础 RAG 系统实现 - 数据加载与预处理 - 文档分块策略 - 文本向量化 - 向量数据库操作 - 检索优化 - 索引构建和优化 - 检索结果优化 - 答案生成 - 系统性能评估 ### Tech stack: - LangChain/LlamaIndex - pandas, PyPDF2 - HuggingFace, BGE - Milvus, Chroma - Query Expansion - 层次索引, 关键词索引 - LLM 集成 - RAGAS, TruLens ### Why it matters: RAG 的落地往往是面向业务做 RAG，而不是反过来面向 RAG 做业务。项目通过针对不同场景和问题的调整、优化和定制化，确保了 RAG 系统在实际应用中的有效性。通过深入探究细节，项目旨在提供

Overview: This repository contains various examples demonstrating how to use or integrate DeepSpeech, a speech recognition library, into applications. It serves as a practical guide for quick experimentation and inspiration for integrating DeepSpeech into different projects. Key features: - Microphone Voice Activity Detection (VAD) streaming - VAD transcriber - AutoSub Tech stack: Python Why it matters: These examples provide practical insights into using DeepSpeech for real-world applications, such as VAD detection and transcribing audio, making it easier to integrate DeepSpeech into projects without delving into detailed technical documentation. GitHub stars: 854 Repository: mozilla/DeepSpeech-examples

Overview: LiveTranslate is a real-time audio translation tool designed for Windows users. It captures system audio and optional microphone input, runs ASR (Speech-to-Text) using whisper, SenseVoice, or FunASR Nano, and translates the text via an LLM (Large Language Model) API. The results are displayed in a transparent overlay, making it perfect for VTubers, livestreamers, and watching foreign content. Key features: - Real-time pipeline: System audio → VAD → ASR → LLM translation → overlay - Multiple ASR engines: whisper, SenseVoice, FunASR Nano, Anime-Whisper - Remote ASR: offloads speech recognition to a GPU machine over HTTP - Any OpenAI-compatible API: DeepSeek, Grok, Qwen, GPT, Ollama, vLLM, etc. - Streaming translation display: Real-time character-by-character translation output - Per-model settings: Streaming, structured output (JSON), context history, disable thinking - Microphone mix-in: Optionally mix microphone input with system audio for ASR - Low-latency VAD: 32ms chunks + Silero VAD with adaptive silence detection - Transparent overlay: Always-on-top, click-through, draggable, 14 color themes - CUDA acceleration: GPU-accelerated ASR inference - Auto model management: Setup wizard, ModelScope / HuggingFace dual sources - Built-in benchmark: Compare translation model speed and quality Tech stack: Python 3.10+ (Portable build

Overview: Instruct-NeRF2NeRF is an implementation of the Instruct-NeRF2NeRF method presented at ICCV 2023, enabling the editing of 3D scenes using textual instructions. Key features: - Training and editing of NeRF scenes with text prompts - Support for custom datasets and downscaling of images for faster training - Option to run InstructPix2Pix on a separate GPU for improved performance Tech stack: Python Why it matters: This method allows for interactive and precise manipulation of 3D scenes through natural language instructions, enhancing the accessibility and flexibility of 3D rendering tools.

Overview: MAMMA is an open-source project that provides an accurate and markerless approach for multi-person motion acquisition. It includes multi-view capture, segmentation, 2D landmark detection, and 3D optimization, with a focus on real-time performance. Key features: - Multi-view capture - Per-person segmentation using SAM and YOLO - 2D landmark detection with MammaNet - Multi-view SMPL-X optimization - Real-time interactive scene overlays Tech stack: Python Why it matters: MAMMA offers a robust and efficient method for capturing and analyzing multi-person motion, making it valuable for applications in robotics, human-computer interaction, and more. Repository: `https://github.com/cuevhv/mamma` Primary language: Python GitHub stars: 846 GitHub description: Official code for MAMMA: Markerless Accurate Multi-person Motion Acquisition.

Overview: Byaldi is a pre-release library designed to simplify the use of late-interaction multi-modal models, such as ColPali, by providing a simple API that integrates with the ColPali repository. Key features: - Easy integration with ColPali models - Support for RAGatouille's pre-release version - Ability to handle PDFs, images, and directories of documents - No need for complex setup or installation of external libraries Tech stack: Python Why it matters: Byaldi simplifies the process of using complex multi-modal models like ColPali, making it accessible and easy to integrate into existing projects without requiring extensive setup or knowledge of deep learning frameworks.

Overview: YoloDotNet is a C# .NET 8.0 project designed for real-time computer vision tasks such as classification, object detection, OBB detection, segmentation, and pose estimation. It leverages ONNX Runtime and SkiaSharp, providing a lightweight solution without the need for Python or hidden preprocessing. Key features: - Real-time inference for YOLOv5u, YOLOv8-v26, YOLO-RT-DETR, YOLO-World, and YOLO-E models. - Explicit control over execution, memory, and preprocessing. - Built on .NET 8, ONNX Runtime, and SkiaSharp. - Ideal for desktop apps, backend services, and real-time vision pipelines requiring deterministic behavior and full control. Tech stack: C#, .NET 8, ONNX Runtime, SkiaSharp. Why it matters: YoloDotNet offers a pure .NET solution with real performance, explicit configuration, and production readiness, making it suitable for developers seeking fast, predictable YOLO inference in .NET environments.

### Overview: Magick is a cutting-edge toolkit designed for no-code data pipelines and multimodal agents, offering a visual AI development environment that connects to popular services and includes pre-built nodes and templates for intelligent agents, chatbots, reasoning systems, and realistic characters. ### Key Features: - **Realtime Agents**: Perform actions autonomously, interact with users and other agents in various modalities with a unified memory and self-awareness. - **Social Connectors**: Easily integrate with Discord, Twitter, and Twilio, with more connectors coming soon. - **Search Functions**: Access Google, Wikipedia, and Semantic Web data. - **Powerful Graph-Based IDE**: Ideal for complex data pipelines with embedded graphs and shared subgraphs for rapid community development. - **Pre-built Nodes and Templates**: Ready-to-use nodes and connectors for intelligent agents, chatbots, reasoning systems, and realistic characters. ### Tech Stack: TypeScript ### Why it Matters: Magick simplifies the process of creating AI applications without requiring deep coding knowledge, making it accessible to a broader audience. ### Repository: Oneirocom/Magick ### Primary Language: TypeScript ### GitHub Stars: 851 ### GitHub Description: Magick is a groundbreaking visual AIDE for no-code data pipelines and multimodal agents. Connect to other services, explore pre-built nodes and templates, and unleash the power of AI.

### Overview: AgentGym is a framework designed to help the community evaluate and develop generally-capable Large Language Model-based agents. It features diverse interactive environments and tasks with a unified format, ReAct. AgentGym supports real-time feedback and concurrency, and is easily scalable. It includes 14 environments across web navigating, text games, house-holding tasks, digital games, embodied tasks, tool-using and programming. ### Key features: - **AgentGym-RL**: The RL version of AgentGym, enabling agents to learn directly from interactive environments. - **AgentTraj-L**: A high-quality trajectory set for evaluating agent performance. - **AgentEval**: A benchmark suite for evaluating agent performance. - **AgentEvol-7B**: A model for investigating the potential of agent self-evolution. ### Tech stack: Python ### Why it matters: AgentGym provides a platform for evaluating and developing LLM-based agents, enabling real-time feedback and concurrent exploration. It supports diverse environments and tasks, making it a valuable tool for the AI community.

### Overview: Skylos is an open-source local-first pull-request scanner for Python projects. It identifies dead code, security bugs, secrets, quality regressions, and AI-code mistakes before a merge. Teams merging pull requests from AI coding agents can use `skylos done` to ensure their branch is ready for review. ### Key Features: - **Dead Code Detection**: Finds unused functions, classes, imports, and empty files. - **Security Checks**: Checks for security flaws and hard-coded secrets. - **Quality Fixes**: Ensures code quality by analyzing dependencies and dependencies. - **AI-Generated Code Mistakes**: Identifies common mistakes in AI-generated code, such as missing guards, fake helpers, and impossible dependency versions. ### Tech Stack: - Python ### Why It Matters: Skylos is designed to help teams and maintainers ensure their Python projects are clean, secure, and maintainable before merging changes. It provides a comprehensive check list for common issues, making it an essential tool for developers and maintainers of Python projects.

### Overview: The Engineering Handbook is a comprehensive set of open-source textbooks designed for software engineers. Each book covers system design, data structures, and AI engineering, providing in-depth explanations, comparisons, and practical examples. ### Key features: - **System Design, DSA, and AI Engineering**: Each book is a complete course, teaching from first principles to real-world production mistakes. - **Opinionated and Sourced**: Chapters recommend approaches and explain why, backed by primary sources and current research. - **Taught in Full**: Every topic is explained, with diagrams, worked examples, and trade-offs. - **Current and Open**: Covers modern technologies like LLMs, RAG, agents, and post-quantum cryptography, and welcomes contributions for updates. ### Tech stack: Markdown and GitHub ### Why it matters: The Engineering Handbook offers a unique, comprehensive, and practical approach to software engineering, making it an invaluable resource for engineers at all levels. ### Repository: handbook-academy/engineering-handbook ### GitHub stars: 848 ### Primary language: JavaScript ### License: CC BY-SA 4.0 ### Contributing: Anyone can suggest fixes and review changes, ensuring the books are always up-to-date and free to read and adapt.

### Overview: This repository is a curated list of awesome resources across various machine learning and deep learning topics, aiming to help beginners understand the branches and latest developments in machine learning, researchers follow new research directions, and engineers find suitable tutorials and libraries to solve practical problems. ### Key Features: - 📖 Machine Learning Resources - 🚀 Research Guides - 📝 Tutorials and Books - 🌐 Data Sets and Datasets - 🚀 Frameworks and Libraries - 🤝 Community and Forums ### Tech Stack: Python, JavaScript, Markdown, Git, and various machine learning frameworks and libraries. ### Why it Matters: This repository serves as an essential resource for anyone interested in machine learning, providing a comprehensive collection of resources to help beginners understand the field, researchers stay updated on the latest developments, and engineers find the tools they need to tackle real-world problems.

### Overview: 中文字幕语音识别项目，采用Python和TensorFlow技术实现。项目旨在提供高质量的中文语音识别服务。 ### Key features: - 使用TensorFlow 1.5.0进行语音识别模型训练和推理。 - 支持多种语言，包括中文。 - 提供详细的配置文件和运行脚本，方便用户进行本地开发和测试。 ### Tech stack: Python 3.5 ### Why it matters: 本项目是中文语音识别领域的关键技术实现，适用于多种应用场景，如智能客服、语音助手等。通过使用TensorFlow，该项目展示了深度学习在语音识别领域的强大应用潜力。

Overview: RQ-VAE Recommender is a PyTorch implementation of semantic-ID tokenization and generative retrieval for recommender-system research. It provides a complete experimental pipeline for researchers studying semantic tokenization, vector quantization, cold-item representations, and generative retrieval. Key features: - Residual quantization with K-means initialization and Euclidean or cosine assignment. - Gumbel-Softmax, straight-through, and rotation-trick gradient estimators. - Semantic-ID collision disambiguation and corpus-prefix-constrained generation. - A T5 encoder-decoder retrieval model with hit-rate and NDCG evaluation. - Automatic preprocessing for Amazon Reviews and MovieLens datasets. - Optional multi-device/mixed-precision training through Accelerate and optional Weights & Biases logging. - A published Amazon Beauty RQ-VAE checkpoint on Hugging Face. Tech stack: Python Why it matters: RQ-VAE Recommender simplifies the process of researching generative recommendation by providing a compact, inspectable PyTorch workflow that covers the necessary components for semantic tokenization, vector quantization, cold-item representations, and generative retrieval. It is designed for researchers and practitioners in the field of recommender systems. Repository: EdoardoBotta/RQ-VAE-Recommender Primary language: Python GitHub stars: 850 GitHub description: [Pytorch] Generative retrieval model using semantic IDs from "Recommender Systems with Generative Retrieval" README excerpt: [![DOI](https://zenodo.org/badge/

Overview: This project introduces a character-level transformer model for GPT training and inference in pure, dependency-free C, capable of training on 32k names in just a couple of seconds and generating new names at a rate of 10168430 tokens per second. Key features: - Pure dependency-free C implementation - Trains on 32k names in 20000 steps - Generates new names at 10168430 tok/sec - Supports macOS, Linux, and Windows (ARM64 with NEON and x86-64 with AVX2) Tech stack: C Why it matters: This project showcases the potential of C for high-performance natural language processing tasks, demonstrating that dependency-free C can be used to train and generate text with impressive efficiency and accuracy.