Projects

### Overview: This book, "Machine Learning Q and AI Beyond the Basics," is a comprehensive guide designed to help developers and learners stay current with the latest advancements in machine learning and artificial intelligence. It addresses 30 essential questions, each accompanied by diagrams and references for further reading, ensuring a fun and engaging learning experience. ### Key Features: - **Central Questions Addressed:** The book tackles 30 essential questions in machine learning and AI, providing clear answers and explanations. - **Rapid-Fire Chapters:** Each chapter focuses on a single question, making it easy to digest and apply the knowledge. - **Updated Version:** This is a fully edited and revised version of the original "Machine Learning Q and AI" book, available on Leanpub. - **Supplementary Materials:** Includes supplementary code for chapters 11, 12, and 13, covering topics like calculating parameters, equivalence of fully connected and convolutional layers, and large training sets for vision transformers. ### Tech Stack: - **Primary Language:** Jupyter Notebook ### Why It Matters: This book is essential for developers and learners looking to stay current with the latest technologies in machine learning and artificial intelligence. By addressing 30 essential questions, it provides a fun and engaging way to learn and stay updated. ### Repository: - **Repository:** rasbt/MachineLearning-QandAI-book - **GitHub Stars:** 973 - **GitHub Description:** Machine Learning Q and AI book - **GitHub URL:** https://github.com/rasbt/MachineLearning-QandAI-book

Overview: sandboxd is an open-source, self-hosted AI app builder that allows developers to build and deploy real apps on their own servers. It provides isolated sandboxes, live previews, and a simple command to self-host. Key features: - Build apps from prompts in isolated sandboxes - Run existing open-source apps with one click - Drive sandboxd from the API or CLI - Live demo accessible via a console Tech stack: Go Why it matters: sandboxd offers developers a self-hosted solution for building and deploying apps, providing complete control over the infrastructure and data, while offering a simple and efficient way to start building applications.

### Overview: This project provides a guide on how to run quantized versions of open-source LLMs on local CPU inference for document Q&A. It includes a step-by-step tutorial, tools used, and a quickstart guide for beginners. ### Key Features: - **Quantized LLMs**: Running Llama 2 and other open-source LLMs on CPU inference. - **LangChain**: Framework for developing applications powered by language models. - **C Transformers**: Python bindings for the Transformer models implemented in C/C++ using GGML library. - **FAISS**: Open-source library for efficient similarity search and clustering of dense vectors. - **Sentence-Transformers**: Pre-trained transformer model for embedding text to a 384-dimensional dense vector space. - **Llama-2-7B-Chat**: Fine-tuned Llama 2 model designed for chat dialogue. - **Poetry**: Dependency management and packaging tool for Python projects. ### Tech Stack: Python ### Why it Matters: This project offers a practical solution for self-hosting open-source LLMs on CPUs, reducing costs and providing flexibility for data privacy and residency rules.

### Overview: This repository contains comprehensive examples and tutorials on transformer models, fine-tuning, training, and prompt engineering, covering models from BERT to GPT-4, environments from Hugging Face to OpenAI, and detailed API examples for ChatGPT, GPT-3.5-turbo, GPT-4, and DALL-E. The repository also includes advanced prompt engineering and bonus sections on various AI tools and techniques. ### Key features: - Fine-tuning and training examples - Detailed API documentation for ChatGPT, GPT-3.5-turbo, GPT-4, and DALL-E - Comprehensive tutorials on transformer models and environments - Bonus sections on ChatGPT, GPT-3.5-turbo, GPT-4, and DALL-E - Advanced prompt engineering techniques with ChatGPT and GPT-4 APIs ### Tech stack: Jupyter Notebook ### Why it matters: This repository provides a valuable resource for developers looking to enhance their understanding of transformer models and their applications in natural language processing. The detailed examples and comprehensive API documentation make it an essential tool for anyone interested in AI and machine learning.

Overview: Data-Prep-Kit is an open-source project designed to accelerate the preparation of unstructured data for applications using Large Language Models (LLMs). This toolkit streamlines the process of preparing data for pre-training, fine-tuning, and instruct-tuning of LLMs, as well as building Retrieval Augmented Generation (RAG) applications. Key features: - A growing set of modules targeting laptop-scale to datacenter-scale processing. - Support for Natural Language, Code, and Image data modalities. - Built on common Python and Ray frameworks for scalable data processing. - Framework for developing custom transforms for Parquet, ZIP, NDJSON, and JSONL file formats. - Examples of deploying a single transform as a Python or Ray job on Kubernetes clusters and a pipeline using Tekton. Tech stack: Python, Ray, PyPI, Apache 2.0 license, pre-commit hooks. Why it matters: Data-Prep-Kit simplifies the data preparation process for LLM applications, enabling developers to efficiently train, fine-tune, and instruct-tune LLMs, and build RAG applications, all while ensuring the data is ready for use in modern AI and data science projects.

Overview: TinyChatEngine is a lightweight library designed for on-device inference of large language models (LLMs) and visual language models (VLMs). It leverages advanced compression techniques like SmoothQuant and AWQ to run these models efficiently on edge devices, providing better privacy and performance. Key features: - Universal support for x86, ARM (Apple M1/M2, Raspberry Pi), and CUDA (Nvidia GPU) environments. - No library dependency, enabling a self-contained implementation. - High performance, capable of real-time inference on Macbook and GeForce laptops. - Easy to use, requiring only a download and compile process. Tech stack: C++ Why it matters: TinyChatEngine simplifies the deployment of large language models and vision language models on edge devices, offering improved privacy and performance compared to running them on centralized servers. Repository: mit-han-lab/TinyChatEngine GitHub stars: 969 GitHub description: TinyChatEngine: On-Device LLM/VLM Inference Library

### Overview: AI Audio Datasets (AI-ADS) 🎵 is a comprehensive collection of speech, music, and sound effect datasets that can be used for training Generative AI models, AIGC, AI model training, intelligent audio tool development, and audio applications. It includes a variety of datasets such as AISHELL-1, AISHELL-3, Arabic speech corpus, Audio-FLAN, AudioMNIST, AVSpeech, ATIS, Carnatic Varnam Dataset, Casual Conversations, and CN-Celeb, providing a rich resource for researchers and developers in the field of AI and audio technology. ### Key features: - **Speech Datasets**: Includes AISHELL-1, AISHELL-3, and Audio-FLAN. - **Music Datasets**: Contains the Arabic Speech Corpus and AudioMNIST. - **Sound Effect Datasets**: Includes AVSpeech and ATIS. - **Cultural Datasets**: Features datasets like Carnatic Varnam Dataset and Casual Conversations. - **Instruction-Tuning Dataset**: Audio-FLAN, a large-scale instruction-tuning dataset covering 80 diverse tasks across speech, music, and sound domains. ### Tech stack: Python, TensorFlow, PyTorch, and other relevant technologies. ### Why it matters: AI Audio Datasets provide essential training data for developing advanced AI models, improving speech recognition, music synthesis, and audio applications, making it a valuable resource for researchers and developers in the field of AI and audio technology.

### Overview: This project is a TensorFlow implementation of pre-training of Deep Bidirectional Transformers for Language Understanding (BERT) and the Attention is All You Need (Transformer) approach. It aims to improve the performance of pre-training tasks and fine-tuning strategies, particularly for language understanding tasks. ### Key Features: - **Pre-training with Masked Language Model**: Enhances model performance with pre-training tasks like masked language model. - **Fine-tuning with Pre-trained Models**: Improves model performance by fine-tuning pre-trained models. - **Middle-Sized Data Set**: Utilizes a middle-sized dataset (450k) for pre-training, demonstrating significant performance gains. ### Tech Stack: Python ### Why It Matters: The project showcases how pre-training and fine-tuning strategies can significantly enhance the performance of language understanding tasks, particularly with a moderate-sized dataset. It provides a practical approach to improve model performance without requiring extensive training time and resources.

### Overview: AssetFurnace is a comprehensive open-source pipeline for converting images into textured, rigged, and animated game characters. It supports various 3D models and offers a wide range of features including auto-rigging, 21 move presets, and high-quality finishing options. The project is designed for indie game developers and 3D artists, providing free, locally generated assets with no credits or licensing fees. ### Key Features: - **Rigged, Moving Characters:** Generate a textured model with a skeleton and moves from 21 presets. - **Studio Library:** Your library, 3D viewer, and a steps panel. - **Low Poly, High Quality:** Pixel Match recovers real pixels, ensuring text, logos, and faces remain sharp. - **Nine Props from One Image:** Prop sheets turn a single image into separate props with LODs. - **Local Models:** Supports Pixal3D, TRELLIS.2, Hunyuan3D, and auto-rig. - **Game-Size Files:** Compress heavy models into light, easily manageable formats. - **Agent-Driven Assets:** Supports Claude Code, Codex, and other MCP clients for tweaking and checking. - **Know What You Can Ship:** Every file includes a record of the licenses behind it, ensuring non-negotiable provenance. ### Tech Stack: Python ### Why It Matters: AssetFurnace aims to provide a more inclusive and accessible approach to 3D asset creation, particularly for indie developers and artists who may not have access to high-end hardware or

Overview: RepoPrompt CE is a free, open-source native macOS app and agent orchestrator designed for context engineering, specifically tailored for AI coding agents. It helps coding agents understand your codebase before they act by assembling focused, reviewable context from various sources. Key features: - Context engineering for AI coding agents - Native macOS app and agent orchestrator - Bundled MCP server for agent harness - Supports MCP-compatible clients and CLI agents Tech stack: Swift Why it matters: RepoPrompt CE provides a user-friendly interface for context engineering, making it easier for developers to understand and modify their codebases before they are acted upon by AI tools or CLI agents. This feature is crucial for ensuring that AI-driven development is both effective and contextually informed.

Overview: Mycroft Precise is a lightweight, open-source RNN wake word listener designed to recognize specific phrases, such as "Hey, Mycroft," and activate Mycroft's software for command execution. Key features: - Recognizes specific phrases - Fully open-source - Trains to recognize any phrase, including names and coughs - Runs on Linux, including Debian, Ubuntu, and Raspbian Tech stack: Python Why it matters: Mycroft Precise is a practical solution for developers looking for a lightweight wake word listener, offering a balance between simplicity and functionality.

Overview: Vocalinux is a free, open-source desktop application for Linux that allows users to dictate text by speaking into their microphone. It supports multiple languages and engines, including Whisper, VOSK, and Parakeet, and works seamlessly with X11 and Wayland environments. Key features: - **On-device after model download**: Local engines; speech-to-text stays on your machine. - **X11 and Wayland**: Text injection via xdotool, IBus, wtype, ydotool, or clipboard fallback. - **Several engines**: Whisper, VOSK, and Parakeet. Tech stack: Python. Why it matters: Vocalinux is a valuable tool for users who prefer to dictate text without the need for a Voca account, as it provides a free, offline solution for dictation. The app's compatibility with multiple Linux distributions and its support for X11 and Wayland environments make it a versatile choice for users on various Linux systems.

### Overview: Athena is an open-source implementation of an end-to-end speech processing engine, designed to facilitate both industrial application and academic research on speech processing tasks. It includes models for Automatic Speech Recognition (ASR), Text-to-Speech (TTS), Voice Activity Detection (VAD), Key Word Spotting (KWS), and more. ### Key Features: - **Hybrid Attention/CTC based end-to-end and streaming methods (ASR)** - **Text-to-Speech (Fastspeech/FastSpeech2/Transformer)** - **Voice activity detection (VAD)** - **Key Word Spotting (KWS) with end-to-end and streaming methods** - **ASR Unsupervised pre-training (MPC)** - **Multi-GPU training on one machine or across multiple machines with Horovod** - **WFST creation and WFST-based decoding with C++** - **Deployment with Tensorflow C++ (Local server)** ### Tech Stack: - Implemented in Tensorflow >= 2.0.1 ### Why It Matters: Athena aims to democratize speech processing by providing accessible tools and models for various tasks, from ASR to KWS, making it easier for researchers and developers to apply advanced speech processing techniques.

Overview: IM.codes is a developer-focused project that provides a shared memory layer and managed tools for AI agents, enabling cross-provider supervision and cross-agent audit across AI providers. Key features: - Shared memory layer for completed work across multiple providers. - Managed MCP tool surface across supported AI providers. - Cross-agent audit and supervision across different AI providers. - Cross-provider Team audit for visibility and connection between human and agent work. Tech stack: TypeScript Why it matters: IM.codes simplifies the process of managing and auditing AI agent work across multiple providers, ensuring that both human and AI agents can collaborate effectively and transparently. Repository: im4codes/imcodes Primary language: TypeScript GitHub stars: 965 GitHub description: The IM for agents. Shared Agent Context & Memory, supervised execution, and cross-agent audit across AI providers. README excerpt:

### Overview: OmniQuant is a simple and powerful quantization technique designed specifically for Large Language Models (LLMs). This project aims to provide accurate quantization methods for weight-only and weight-activation quantization, making it easier to deploy models with lower precision while maintaining performance. ### Key Features: - **Weight-Only Quantization (`W2A16`, `W2A16g128`, `W2A16g64`)**: Accurately quantizes model weights. - **Weight-Activation Quantization (`W3A16`)**: Combines weight and activation quantization for improved accuracy. - **Pre-trained Model Zoo**: Offers pre-trained models for LLaMA-1&2, LLaMA-2-Chat, OPT, Falcon, and Mixtral-7Bx8. - **MLC-LLM Support**: Enables efficient inference with quantized models on GPUs and mobile phones using MLC-LLM. ### Tech Stack: - Python ### Why It Matters: OmniQuant simplifies the quantization process for LLMs, making it easier to deploy models with lower precision while maintaining performance. This is particularly useful for applications where memory and computational resources are limited.

Overview: DeepCache is a novel training-free and almost lossless paradigm that accelerates diffusion models from the perspective of model architecture. It utilizes the property of the U-Net to reuse high-level features while updating low-level features cheaply. Key features: - Accelerates Stable Diffusion v1.5 by 2.3x with only a 0.05 decline in CLIP Score. - Accelerates LDM-4-G(ImageNet) by 4.1x with a 0.22 decrease in FID. - Supports Stable Diffusion, Stable Diffusion XL, Stable Video Diffusion, Stable Diffusion Pipeline / XL Pipeline for Inpainting, Stable Diffusion Img2Img Pipeline, DDPM. Tech stack: Python Why it matters: DeepCache is a significant improvement over traditional training-free methods, offering nearly lossless acceleration with minimal performance degradation, making it suitable for applications requiring high-speed model inference without the need for expensive training. Repository: horseee/DeepCache Primary language: Python GitHub stars: 965 GitHub description: [CVPR 2024] DeepCache: Accelerating Diffusion Models for Free

### Overview ### Key features: - Streamlit-based local search tool for efficient paper analysis and statistics. - Comprehensive link collection for major conferences, ensuring coherence and consistency. - JSON-based data format for easy access and statistical analysis. ### Tech stack: Python ### Why it matters: This repository serves as a valuable resource for researchers and data analysts, providing a centralized and easily accessible platform for conference papers and statistics. It simplifies the process of finding relevant information and conducting statistical analysis, making it an essential tool for academic and professional use.

### Overview: SimPO is a preference optimization algorithm designed for reinforcement learning tasks, offering a simpler and more effective solution compared to DPO without the need for a reference model. It has been evaluated on various benchmarks and demonstrated superior performance across AlpacaEval 2, MT-Bench, and Arena-Hard, achieving high win rates. ### Key features: - SimPO is a preference optimization algorithm that outperforms DPO and its variants. - It is designed without the use of a reference model, making it more straightforward and potentially more effective. - SimPO has been evaluated on AlpacaEval 2, MT-Bench, and Arena-Hard benchmarks, consistently outperforming other methods. ### Tech stack: Python ### Why it matters: SimPO provides a simpler and more effective solution for preference optimization, making it a valuable tool for reinforcement learning tasks. Its performance across various benchmarks indicates its potential to enhance the efficiency and effectiveness of preference optimization algorithms. ### Repository: GitHub Repository: [princeton-nlp/SimPO](https://github.com/princeton-nlp/SimPO) ### GitHub stars: GitHub Stars: 964 ### Changelog: - **2024.10.12**: Training curves for Llama3-Instruct and Gemma2-IT are released. - **2024.07.17**: New SimPO model [gemma-2-9b-it-SimPO] achieves high win rates on AlpacaEval 2 and Arena-Hard. -

### Overview: The audio-ai-hub repository serves as a comprehensive hub for audio AI research, featuring curated papers, open models, benchmarks, and datasets across various audio LLMs, speech recognition, speech synthesis, and music generation. ### Key features: - **Whisper**: OpenAI's open-source speech recognition model trained on 680K hours of multilingual and multitask supervised data. - **VoxCPM2**: A fully open-source 2B-parameter multilingual, controllable speech generation foundation model extending VoxCPM's hierarchical architecture. - **MMS**: Massively Multilingual Speech, extending speech foundation models (wav2vec 2.0) to 1,107 languages for ASR and adds TTS and language independent functionalities. - **MiniCPM-o**: OpenBMB's compact (8B-class) full-duplex omni-modal LLM supporting real-time vision, speech, and text interaction. - **IndexTTS**: Bilibili's industrial-grade zero-shot TTS system optimized for production scenarios, featuring controllable prosody and low-latency inference. - **CosyVoice 3**: Scalable CosyVoice TTS stack with larger pre-training data and dedicated post-training stage, targeting industrial-grade performance. - **CosyVoice 2**: Alibaba's streaming TTS LLM, combining a unified speech tokenizer with a streaming-friendly LLM backbone for real-time interactions. - **MusicGen**: Meta's single-stage autoregressive transformer for controllable text-conditioned music generation, operating over discrete

### Overview: OpenRath is an open-source, PyTorch-like runtime designed for dynamic multi-agent and multi-session workflows. It turns agent runtime state into explicit, composable Python objects, making it easy to build and manage complex workflows with ease. ### Key Features: - **Session**: Carries conversation state and inter-agent collaboration lineage. - **Sandbox**: Decides where tools actually run. - **Memory**: Persists agent memory state across runs. - **Tool**: The operator-like callable surface exposed to the model. - **Agent**: A reusable, composable session transformation layer. - **Workflow**: Composes multiple agents and workflows into larger systems. - **Selector**: Routes between self-describing workflows at runtime, enabling dynamic `if` / `while` control flow. ### Tech Stack: - Python ### Why It Matters: OpenRath is designed for applications that need multiple agents, multiple branches, durable memory, sandboxed execution, and traceable lineage at the same time. It simplifies the process of managing complex workflows and makes it easy to build and deploy applications that require dynamic control flow and persistent state.