Projects

Overview: Quartz is a custom GPU-accelerated inference engine developed by Saient, designed to run large language models (LLMs) and image generation tasks entirely on-device, eliminating the need for cloud calls. This engine is part of Saient's mobile solution that generates full text-to-video clips, including the Wan2.1 T2V model with 1.3B parameters, all without the use of cloud services or API keys. Key features: - LLM chat inference: GGUF model loading, tokenization, forward pass, and an OpenAI-compatible HTTP server for streaming completions. - Image generation (SD1.5/SDXL): A CLIP text encoder, UNet, VAE, and scheduler implementation driven by the same binary through `--generate-sd15` or `--generate-sdxl`. Tech stack: Rust Why it matters: Quartz is crucial for Saient's mobile solution, enabling the generation of full text-to-video clips, including the Wan2.1 T2V model, and image and video generation tasks without the need for cloud calls, ensuring privacy and security. Repository: SaientAI/saient-quartz GitHub stars: 8 GitHub description: Quartz — custom CUDA/Vulkan GGUF inference engine + SD1.5/SDXL image and video generation (Rust, no llama.cpp, no ggml, no Candle, no PyTorch/libtorch) README excerpt: # Quartz Quartz is the on-device inference engine behind [Saient](https://saient.co.uk) —

### Overview: This project provides a version of NVIDIA's Nemotron TwoTower diffusion language model that can run on any CPU, including `ppc64le`, `arm64`, and `x86_64`, without the need for an NVIDIA GPU. It includes a pure-PyTorch kernel shim and a handful of one-line fixes to ensure compatibility with CPU-based setups. ### Key Features: - **Pure-PyTorch Kernel Shim**: A drop-in replacement for NVIDIA's `mamba_ssm` and `causal_conv1d` packages, ensuring the model runs on any CPU. - **One-Line Fixes**: Small changes to NVIDIA's `inference.py` script to make it compatible with CPU-based setups. - **Verification**: The project includes a script to verify the correctness of the kernel paths and ensure the patches are applied correctly. ### Tech Stack: - **Python**: The primary language used for development and execution. ### Why It Matters: This project is crucial for developers and researchers who need to run the Nemotron TwoTower model on CPUs without the need for NVIDIA GPUs, making it accessible for a wider range of hardware and environments.

Overview: RustyLLM is a GGUF inference runner written in Rust, designed to help developers understand how local language-models work. It's a small, educational project that reads and interprets GGUF files, converts text to token IDs, and predicts the next token, all in a way that's easy to follow. Key features: - GGUF file parsing - Tokenization and text-to-token conversion - Quantized inference paths for various tensor types - SIMD kernels for NEON and AVX2/FMA - Tokenizer support for SentencePiece and GPT-2 - Native GGUF loading with memory mapping Tech stack: Rust Why it matters: RustyLLM provides a practical, dependency-light way to learn about local language-model inference systems without needing a large production runtime. It's designed to be educational and easy to follow, making it a great resource for developers looking to understand how these systems work.

### Overview: HomeoRAG is a question-answering assistant designed for homeopathic remedies, built to streamline the process of finding appropriate treatments based on patient descriptions and symptoms. ### Key features: - **Natural Language Processing (NLP)**: HomeoRAG uses advanced NLP techniques to understand and interpret patient descriptions. - **Structured Knowledge Base**: The application relies on a structured knowledge base derived from the Materia Medica by William Boericke, providing precise and reliable answers. - **Retrieval-Augmented System (RAG)**: RAG technology enhances the system's ability to retrieve relevant information and answer questions accurately. ### Tech stack: Python ### Why it matters: HomeoRAG demonstrates how a full-stack RAG application can be developed, evaluated, and improved systematically, focusing on a meaningful and practical domain (homeopathic remedies). ### Repository: SrijanShovit/HomeoRAG ### Primary language: Python ### GitHub stars: 8 ### GitHub description: A RAG application to search documents for homeopathic remedies based on patient's description of symptoms and modalities.

Overview: SemanticQAGen is a Python library designed for generating high-quality question-answer pairs from various text documents like PDFs, DOCXs, MDs, and TXTs. It leverages advanced semantic understanding to create diverse questions across multiple cognitive levels. Key features: - Enhanced semantic chunking - Dynamic question generation - Validation of questions and answers - Flexible LLM routing capabilities Tech stack: Python Why it matters: SemanticQAGen simplifies the process of generating intelligent question-answer pairs, making it easier for developers to create engaging and informative content, especially for educational or knowledge-based applications. Repository: StephenGenusa/semantic-qa-gen Primary language: Python GitHub stars: 8 GitHub description: A Python library for generating high-quality question-answer pairs from PDF, DOCX, MD, and TXT files README excerpt: <div align="center"> <h1>SemanticQAGen</h1> <p><em>Intelligent Question-Answer Generation with Advanced Semantic Understanding</em></p> </div> <p align="center"> <a href="#overview">Overview</a> • <a href="#installation">Installation</a> • <a href="#quickstart">Quickstart</a> • <a href="#feature-overview">Feature Overview</a> • <a href="#core-capabilities">Core Capabilities</a> • <a href="#advanced-features">Advanced Features</a> • <a href="#project-file-

### Overview: Chat with Lex! A RAG app, using HyDE with milvus DB for vector store, VLLM for LLM inference, and FastEmbed for embeddings. ### Key Features: - **Vector Database (milvus)**: Milvus is Dockerized and supports Kubernetes for scaling. - **LLM Inference (VLLM)**: VLLM is chosen for its speed and scalability, with options for multi-node inference using Ray. - **Embedding Model (FastEmbed)**: FastEmbed is used for embedding model serving, with local serving for simplicity. - **Serving LLM (VLLM)**: VLLM is the fastest LLM serving engine, scalable with GPUs within a node and k8s load-based scaling. - **Onnxruntime GPU**: Optional, speeds up embeddings. - **Llamaindex**: Provides the RAG framework. - **Streamlit**: UI for the application. - **OpenAI API**: Optional, useful for isolating bugs from self-hosting models. ### Tech Stack: Python ### Why it Matters: This chat application leverages open-source, self-hostable models for a secure and scalable solution. It uses advanced techniques like RAG, HyDE, milvus, VLLM, and FastEmbed to provide a robust and efficient chat experience.

Overview: InsightForge-NLP is an advanced Natural Language Processing (NLP) system that combines multilingual sentiment analysis and question-answering capabilities with retrieval augmentation. Built with PyTorch, Transformers, FAISS, and FastAPI, it demonstrates state-of-the-art NLP techniques and a production-ready architecture. Key features: - Multilingual sentiment analysis with detailed aspect-based sentiment detection and context-aware processing. - Question-answering system with semantic search, knowledge base management, source attribution, and context integration. Tech stack: PyTorch, Transformers, FAISS, FastAPI. Why it matters: This system is designed to be scalable, maintainable, and extensible, making it suitable for large-scale applications requiring advanced NLP techniques and efficient document retrieval. Repository: TaimoorKhan10/InsightForge-NLP Primary language: Python GitHub stars: 8 GitHub description: A state-of-the-art NLP system that combines multilingual sentiment analysis and question-answering capabilities with retrieval augmentation. Built with industry-standard ML/AI engineering practices, this system demonstrates advanced NLP techniques and production-ready architecture.

Overview: Semantic Chunking is a Python library designed to segment text into meaningful chunks using embeddings from Sentence Transformers, aiding in the analysis and understanding of text data. Key features: - Splits text into semantic chunks using cosine similarity - Configurable chunk size and similarity thresholds - Based on `sentence-transformers` Tech stack: Python Why it matters: This library simplifies the process of text segmentation, making it easier to analyze and understand complex text data by breaking it down into meaningful chunks.

### Overview: Intelligent Document Processing and Analysis API leverages locally run Large Language Models (LLMs) to provide advanced document processing and analysis capabilities. It includes features such as summarization, analysis, sentiment detection, and document comparison, making it compatible with OpenAI API-standard LLM servers. ### Key Features: - **General LLM Completion**: Utilize the power of Large Language Models for various text generation tasks. - **Document Summarization**: Generate concise summaries of uploaded documents. - **Document Analysis**: Perform in-depth analysis of document content. - **Sentiment Analysis**: Determine the overall sentiment of a document. - **Dangerous Content Identification**: Identify potentially dangerous or illegal content within documents. - **Document Comparison**: Compare two documents and highlight similarities and differences. ### Tech Stack: - **Backend Framework**: ASP.NET Core 8.0 - Leverages the latest features of .NET for high-performance web API development. - Uses Entity Framework Core for database operations. - Implements dependency injection for loose coupling and better testability. - Utilizes the repository pattern for improved data access abstraction. - **Database**: SQLite - Lightweight, serverless database for storing LLM interactions. - Easily replaceable with other database systems supported by Entity Framework Core. - **LLM Integration**: - Compatible with LM Studio and any service following OpenAI's API standard. - Implemented using HttpClient for efficient API communication. - Flexible design allows easy switching between different LLM providers. - **Logging**: Serilog - Structured logging for

Overview: This tool is designed for developers who want to customize their own AI models without dealing with complicated code. Everything is done through a user-friendly web interface, ensuring safety and ease of use. Key features: - Safe and easy-to-use setup - Ability to upload or manually enter prompts for fine-tuning - Quick deployment of fine-tuned models Tech stack: TypeScript Why it matters: This tool simplifies the process of creating and deploying custom AI models, making it accessible to a broader audience without requiring extensive coding knowledge.

Overview: `ui-rl` is a framework designed for fine-tuning Computer Use (CUA) agent models. It simplifies the process of achieving high reliability on specific tasks by focusing on task-specific fine-tuning rather than boilerplate code. Key features: - Perform CUA rollouts at scale in custom task environments using state-of-the-art open models. - Rollout serialization/deserialization out of the box. - CUA specific Data Augmentation techniques for Supervised Fine-Tuning (SFT). - Training framework agnostic — pick your favorite torch-compatible trainer. - Pre-built torch `Dataset` implementations for rollouts — start training in a few lines. Tech stack: Python Why it matters: `ui-rl` simplifies the complex process of fine-tuning models for high reliability on specific tasks, allowing developers to focus on the core functionality of their applications without the hassle of boilerplate code. Repository: TobiasNorlund/ui-rl Primary language: Python GitHub stars: 8 GitHub description: Reinforcement Fine-tuning for Computer Use models README excerpt: <div align="center"> <img alt="ui-rl: Reinforcement Fine-tuning for Computer Use models" src="https://github.com/TobiasNorlund/ui-rl/raw/refs/heads/main/assets/uirl.svg"> </div> --- [](https://github.com/TobiasNorlund/ui-rl/actions/workflows/ci.yml

### Overview: VEX is a cross-standard vector database migration tool designed for AI agents, providing sovereign backup and memory portability across various vector stores. It supports multiple vector stores such as VEKTOR Slipstream, Pinecone, Qdrant, ChromaDB, Weaviate, PostgreSQL, Redis, Milvus/Zilliz, Neo4j, and more. ### Key features: - **Sovereign Hybrid Backup:** Encrypts and synchronizes memory to GitHub, Codeberg, or self-hosted Gitea. - **Vector Store Migrator:** Migrates directly between any two vector stores. - **Memory Export:** Export memory to the `.vmig.jsonl` portable format. - **Memory Import:** Import `.vmig.jsonl` into any store. - **LLM Format Conversion:** Convert to OpenAI, Anthropic, Groq, or plain text format. - **Export Signature Verification:** Sign export with BLAKE3 + Ed25519 and verify with `vex verify`. - **Memory Inspection:** Show stats, namespaces, dimensions. - **Memory Validation:** Lint all records in a `.vmig.jsonl` file. ### Tech stack: JavaScript ### Why it matters: VEX simplifies the migration and backup of AI agent memories across different vector stores, ensuring data sovereignty and portability. It supports a wide range of vector stores and provides a seamless experience for developers working with AI agents.

Overview: This project provides an official PyTorch implementation for "SwiftBrush v2," a method that enhances one-step diffusion models by improving their teacher's performance. It is designed to be used as a reference for developers looking to improve their own models. Key features: - PyTorch inference code - Enhances one-step diffusion models - Improves upon the teacher's performance Tech stack: Python Why it matters: This project is essential for developers who are interested in improving their one-step diffusion models, as it provides a clear and detailed implementation that can be easily integrated into their projects.

Overview: Explore practical fine-tuning of Large Language Models (LLMs) using LoRa (Low Rank Adaptation). Key features: - Trainable A & B (only) - Efficient model adaptation across diverse tasks Tech stack: Python, PyTorch, Hugging Face Transformers Why it matters: This repository provides foundational examples for fine-tuning LLMs using LoRa, showcasing efficient model adaptation across various tasks.

Overview: This repository contains 9 n8n automation workflows showcasing advanced AI-powered automation, vector database integration, and multi-modal processing. It includes WhatsApp agents, RAG chatbots, sentiment analysis, email automation systems, and more. Key features: - WhatsApp Agent - Vector Database Integration - Sentiment Analysis - Respond to Sponsorship - RAG Chatbot - Google Sheets Integration - Form Submission to Airtable - Customer Service Assistant - AI-Powered Email Automation System Tech stack: n8n, OpenAI GPT-4, Pinecone, Google Sheets, Airtable, WhatsApp, Gmail, Google Drive, Pinecone, Vector Databases, n8n Features Why it matters: This repository demonstrates the power of AI and n8n in automating complex business processes and integrating with multiple platforms, making it a valuable resource for developers looking to enhance their automation workflows.

### Overview: This repository is a curated collection of the latest AI Engineer Development Tools, including databases, vector databases, data ingestion, data processing, data warehousing, model training, model zoo, model tuning, model evaluation, model deployment, model monitoring, and an AI agent framework. ### Key Features: - **Database**: MongoDB, MySQL, PostgreSQL, and Chroma, LanceDB, Milvus, Pinecone. - **Vector Database**: Chroma, LanceDB, Milvus, Pinecone. - **Data Ingest**: No mention. - **Data Processing**: No mention. - **Data Warehouse**: No mention. - **Model Training**: No mention. - **Model Zoo**: No mention. - **Model Tuning**: No mention. - **Model Evaluation**: No mention. - **Model Deployment**: No mention. - **Model Monitor**: No mention. - **AI Agent Framework**: No mention. ### Tech Stack: GitHub stars: 8 ### Why it Matters: This repository serves as a comprehensive resource for developers looking to enhance their AI engineering toolkit. It provides a centralized location for accessing the latest tools and technologies in the field, ensuring that developers can stay up-to-date with the most advanced AI development practices.

Overview: Context Engineering is the art of providing all the necessary context for a Large Language Model (LLM) to solve tasks effectively. It involves understanding and incorporating various elements like instructions, user prompts, long-term memory, retrieved information, available tools, and structured outputs into the conversation. Key features: - Instructions / System Prompt - User Prompt - State / History (short-term Memory) - Long-Term Memory - Retrieved Information (RAG) - Available Tools - Structured Output Tech stack: Python and various LLM integrations. Why it matters: Context Engineering ensures that the LLM has all the necessary information to understand and solve tasks efficiently, leading to more effective and accurate responses.

Overview: This repository contains the code and evaluations for the paper "Where a New Concept Must Enter: Entry Point Gates Cross-Task Usability in Unified Multimodal Models". It focuses on separating tasks in a unified multimodal model to understand how different pathways influence cross-task usability. Key features: - Novel entity and rendered Objaverse asset paired with a screened pseudo-word. - Separation of tasks by construction: understanding and generation are bound through one task direction, and the untrained direction is measured. - Closed-form activation edit (`src/train/hidden_ti.py`) makes a concept drawable at layer 7 of 28 with every weight frozen, and is indistinguishable from the base model from layer 14 on. - Weight-carried version of the same objective (`src/train/anchor.py`) peaks at layers 10–14 across four models. - The window appears only where the understanding pathway is a semantic vision encoder. Tech stack: Python Why it matters: This repository provides a detailed analysis of how different pathways in a unified multimodal model affect cross-task usability, offering insights into the design of such models and their potential for improving cross-task usability in multimodal tasks. Repository: Zane-ZYQiu/entry-point-umm Primary language: Python GitHub stars: 8 GitHub description: Code and evaluations for "Where a New Concept Must Enter: Entry Point Gates Cross-Task Usability in Unified Multimodal Models" ([arXiv:2608.17564](https://arxiv.org/abs/

Overview: Academic RAG Assistant is an AI-powered tutoring system that transforms your course textbooks into an interactive AI conversation partner, providing contextual answers across multiple subjects including Linear Algebra, Discrete Structures, and Calculus & Analytical Geometry. Key features: - Multi-Subject Expertise - Agentic RAG System - Real-time Streaming - Modern UI - Persistent Chat History - Support for Multiple Google Gemini Models - Error Handling - Progress Tracking - Smart Retrieval - Memory System Tech stack: Streamlit, Python, asyncio, OpenAI Agents SDK, Google Gemini API, Pinecone, SQLite, LangChain, Sentence Transformers, HuggingFace. Why it matters: This system enhances the learning experience by providing personalized, contextually relevant answers, making complex academic topics more accessible and engaging.

Overview: This repository contains the code used to generate the plots in a paper titled "Optimal Scheduling Algorithms for LLM Inference: Theory and Practice." It includes scripts to reproduce key figures from the paper and evaluate different scheduling policies. Key features: - Scripts to generate and plot median TTFT and batch execution time vs. token budget. - Scripts to generate GPU memory utilization vs. number of active requests. - Scripts to generate batch execution time vs. number of decode tokens. Tech stack: Python Why it matters: This repository provides a practical tool for evaluating and comparing scheduling algorithms for large language model inference, based on the insights from the paper's findings.