Projects

### Overview: This project is a comprehensive guide for developers interested in AI and infrastructure, focusing on networking, GPU and accelerator performance, storage, training, inference, MLOps, and systems performance engineering. ### Key Features: - Network: RDMA, InfiniBand, RoCE, Clos fabric - GPU & Accelerators: Architecture, CUDA, profiling, kernel analysis - Storage: AI workload storage, checkpoint/data pipeline - Training: Distributed training, LLM/MoE/LoRA workloads - Inference: KV cache, batching, GPU profiling - MLOps: Lifecycle, lineage, CI/CD/CT, deployment, monitoring - Systems Performance: GPU/CUDA/PyTorch performance engineering ### Tech Stack: Python ### Why it Matters: This guide offers a deep dive into the technical aspects of AI infrastructure, providing essential knowledge for developers working in the field.

Overview: Kikiola is a high-performance vector database built in Go, designed for efficient storage, indexing, and similarity search. Key features: - Tensor Compression - Support multilingual embedding - Support for high-dimensional vectors - Handles concurrency and multiple writes - Simple and intuitive API for easy integration - Indexing techniques for fast similarity search - Embedding Reranking with a relevance score - Fast and efficient vector storage and retrieval - Text embedding support for text-based queries - Scalable architecture for handling large datasets - Distributed Storage: multiple nodes or shards for scalability Tech stack: Go Why it matters: Kikiola is optimized for performance and ease of integration, making it suitable for applications requiring high-speed similarity search, such as recommendation systems, AI, and machine learning.

### Overview: Random AI Prompt is an open-source tool designed to generate rich, detailed AI prompts. It automatically builds prompts from a large library of scenes, subjects, and styles, enhancing the user's ability to create compelling visuals and text descriptions. ### Key Features: - **Comprehensive Prompt Library**: Access a vast collection of prompts to choose from. - **Model Integration**: Utilizes 40+ models including Midjourney, DALL-E, Gemini, FLUX, and Stable Diffusion. - **Automatic Generation**: Generates prompts quickly and efficiently, saving time and effort. ### Tech Stack: - **JavaScript**: The primary language used for development. ### Why It Matters: Random AI Prompt simplifies the process of generating AI prompts, making it easier for developers and artists to create high-quality visuals and descriptions. By automating the prompt generation, it streamlines workflows and enhances productivity, especially for those who frequently need to create detailed AI-generated content. ### Repository: - **GitHub Repository**: 1fairyfox/random-ai-prompt - **Stars**: 11 - **Description**: An open-source generator for AI image and text prompts that automatically builds richer, more detailed prompts than most people write by hand. Compose from a large library of scenes, subjects, and styles, then run them through 40+ models including Midjourney, DALL-E, Gemini, FLUX, and Stable Diffusion. ### Open Issues: - **Open Issues**: 11 - **Closed Issues**: 0 ### Netlify Deployment: - **Deployment Status

Overview: NativeLab is a fully local, privacy-first LLM desktop application powered by llama.cpp, offering a polished PyQt6 GUI and a Claude-Code-style terminal CLI, with support for multi-model pipelines, document references, long-document summarization, and a Labs experimentation layer. Key features: - **Polished PyQt6 GUI** for a user-friendly interface. - **Claude-Code-style terminal CLI** for direct command-line interaction. - **Support for multi-model pipelines** for seamless integration of different models. - **Document references** for easy navigation and referencing. - **Long-document summarization** for efficient reading. - **Labs experimentation layer** for advanced development and research. Tech stack: Python, llama.cpp, PyQt6, Hugging Face Transformers, and other libraries. Why it matters: NativeLab provides a privacy-first, local-first approach to LLM usage, ensuring that all computations are performed on your machine, enhancing security and privacy. It offers a comprehensive toolkit for developers and researchers to experiment with large language models and create custom applications. Repository: 7ZoneSystems/NativeLab Primary language: Python GitHub stars: 11 GitHub description: A fully local, privacy-first LLM desktop application powered by llama.cpp.

Overview: This project implements an end-to-end differentiable framework for fine-tuning 3D texture generation, addressing the limitations of existing methods in capturing human preferences and integrating 3D structure. Key features: - End-to-end differentiable framework for 3D texture generation - Incorporates human feedback through differentiable reward functions - Generates textures respecting 3D geometry and aligning with desired criteria - Three novel geometry-aware reward functions for controllable and interpretable 3D content creation Tech stack: Python Why it matters: This approach offers a more flexible and interpretable method for generating high-quality 3D textures, improving the alignment of generated content with human preferences and the 3D structure of input objects. Repository: AHHHZ975/Differentiable-Texture-Learning Primary language: Python GitHub stars: 11 GitHub description: The official implementation of the paper titled "End-to-End Fine-Tuning of 3D Texture Generation using Differentiable Rewards" accepted to IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2026.

Overview: This n8n node allows integration of Fusionbrain.ai into workflows, offering high-resolution image generation capabilities. Key features: - Text2Image: Generates images based on prompts with customizable styles and dimensions. - listModels: Retrieves a list of available Fusionbrain.ai models for image generation. - listStyles: Retrieves a list of available Fusionbrain.ai styles for image customization. Tech stack: TypeScript Why it matters: Fusionbrain.ai is a reliable API for image generation, but the API has become unstable, making this node obsolete. It's recommended to use faster and more reliable alternatives. Repository: AKAME007/n8n-nodes-fusionbrain GitHub stars: 11 GitHub description: Use fusionbrain.ai in your n8n workflows.

### Overview: This project is a one-command recipe for running GLM-5.3-Flash on four 96 GB RTX PRO 6000 Blackwell GPUs, utilizing TensorFold for text and vision tasks. The recipe includes optimizations from Aevonix Research and Mia's AI Lab, making it suitable for a Linux host with 40 concurrent requests and 1,048,576 tokens per request. ### Key Features: - **Checkpoint:** Mia-AiLab/GLM-5.3-Flash-EXL3-4bpw-TensorFold - **Drafter:** incoai/GLM-5.3-Flash-DFlash2 - **API Model ID:** glm-5.3-flash - **Context Tokens:** 1,048,576 tokens per request - **Concurrency:** 40 requests by default - **One Command:** `./start.sh`; restart with `./start.sh restart`, stop with `./stop.sh` ### Tech Stack: Python ### Why It Matters: This recipe provides a streamlined, efficient way to run GLM-5.3-Flash on multiple GPUs, leveraging the strengths of TensorFold and Mia's AI Lab's expertise. The combination of these components results in a powerful and versatile model that can handle tasks like text and vision processing with high performance and accuracy.

### Overview: Qwen3.8-Flash-Next is a high-performance language model optimized for two NVIDIA CMP 170HX cards, achieving impressive throughput and efficiency. This model excels in decoding tasks, with a throughput of 234 tokens per second for one user and 827 tokens per second across eight users. The model supports image, video, and text input, as well as HumanEval and GSM8K benchmarks, demonstrating its versatility and effectiveness. ### Key Features: - **Pipeline Parallelism**: The model achieves high throughput through parallel processing, with 234 tokens/s for one user and 827 tokens/s across eight users. - **Efficient Prefill**: The model can prefill up to 6,200–7,400 tokens from 4k to 64k prompts, making it highly efficient for large input sizes. - **Scalability**: The model can handle up to eight users simultaneously, with a throughput of 877 tokens/s. - **Benchmark Performance**: Measured on two NVIDIA CMP 170HX cards, the model consistently performs well in benchmarks, achieving median times of 79 ms for decoding, 106 ms for decoding with JSON, 143 ms for prose decoding, and 197 ms for structured decoding. ### Tech Stack: The model is built using Python, leveraging the vLLM plugin for vLLM pipeline parallelism. ### Why It Matters: Qwen3.8-Flash-Next is a significant

Overview: RagDocs is an AI-powered search engine designed to interact with documentation using RAG and local LLMs. It offers a seamless, private, and efficient way to search for documentation, leveraging vector search and Next.js for a modern user interface. Key features: - 🤖 Intelligent Chat Interface: Conversational interface tailored for technical documentation. - 🔍 Cutting-Edge Retrieval Augmented Generation: Accurate and context-aware responses. - 💾 Local LLM Support: Integrated with Ollama for privacy and efficiency. - 📂 Vector Search Powered by Milvus: Rapid and scalable document querying. - 🚀 Modern Next.js Frontend: Sleek, real-time user experience. - 📚 Multi-Document Support: Seamless handling of diverse tech stacks. - 🔛 Fast Document Processing: Efficient ingestion and analysis of updates. - 🔄 Incremental Updates: Keep your documentation in sync effortlessly. Tech stack: Python, Node.js, Milvus, Ollama, FastAPI, Next.js Why it matters: With complete data privacy and no usage fees, developers can easily deploy and access their documentation, making the process of finding and understanding their tech stacks more efficient and private.

Overview: This project is an AI-powered video search engine that enables users to search video content using natural language queries. It utilizes Google SigLIP, ONNX Runtime, and Qdrant for semantic embedding and efficient video ingestion, making it suitable for low-resource environments. Key features: - Natural language-based video search - Efficient semantic embedding and retrieval using SigLIP - Low resource consumption with zero GPU dependency - High performance with low latency and fast execution Tech stack: Python Why it matters: This project addresses the need for efficient video search solutions that can handle large video datasets without requiring specialized hardware, making it highly valuable in various applications where computational resources are limited.

Overview: The `selective-peft-toolkit` is an official implementation of the paper "Step-by-Step Unmasking for Parameter-Efficient Fine-tuning of Large Language Models". This toolkit provides a flexible framework for selectively fine-tuning large language models using different selective Parameter-Efficient Fine-Tuning (PEFT) methods. Key features: - **Selective Optimizers**: Wrappers around standard optimizers that selectively update a budgeted number of parameters. - **Heuristic-Based Selection**: The selective optimizers update parameters according to various heuristics and selection strategies. - **Integration with Transformers**: Compatible with `transformers.Trainer` for easy integration into existing pipelines. - **Efficient Storage**: Stores modified weights in a summary object that occupies only O(B) space, where B is the budget. - **Model-Agnostic**: Can be used in conjunction with reparameterization-based PEFT techniques that modify the model, since the selective PEFT techniques implemented here operate directly on the optimization process. - **Multi-GPU Support**: Seamlessly handles models sharded across multiple devices. Tech stack: Python Why it matters: This toolkit provides a practical solution for efficiently fine-tuning large language models, making it easier to experiment with selective PEFT methods and integrate them into existing pipelines without the need for extensive modifications to the model architecture. Repository: Aradhye2002/selective-peft-toolkit GitHub stars: 11 GitHub description: Official implementation of the paper "Step-by-Step Unmasking for Parameter-Efficient Fine

### Overview: This project introduces GSQHalo.cpp, a fork of halo-box/strix-llama.cpp optimized for the GSQ-quantized Qwen3.8-Flash-Next model running on an AMD Strix Halo (gfx1151) GPU. The fork addresses three critical issues: faster ROCm prefill, MTP, and a persistent KV cache on SSD, enabling four 256K chats with Q8_0 KV and one KV stream per slot, significantly improving performance and memory efficiency. ### Key Features: - **Faster ROCm Prefill**: Fixed MoE tile selection for low-bit experts, slice decoders, and optimized hyper-connections for faster prefill. - **MTP Memory Management**: Reduced memory usage by 520 MiB and improved performance by 1386 t/s. - **Persistent KV Cache on SSD**: Enables four 256K chats with Q8_0 KV and one KV stream per slot, significantly improving chat efficiency. ### Tech Stack: - **Backend**: ROCm/HIP - **GPU**: gfx1151 ### Why It Matters: - **Performance Improvement**: GSQHalo.cpp achieves a 1401 t/s prefill speed, a 1386 t/s decode performance, and a 4.9 GiB server RSS, compared to the base’s 1003 t/s and 10.5 GiB RSS. - **Memory Efficiency**: The fork reduces memory usage by 520 Mi

Overview: This project fine-tunes Google's Gemma-3-4B-IT language model to solve 4x4 Mini Sudoku puzzles using Generative Reinforcement Policy Optimization (GRPO). It demonstrates how to train a model to follow a specific output format, apply logical reasoning, and generate valid Sudoku solutions with high accuracy. Key features: - Fine-tuning of Google's Gemma-3-4B-IT language model - Training with Generative Reinforcement Policy Optimization (GRPO) - Output format: reasoning and solution tags, step-by-step solution, valid Sudoku solution with high accuracy Tech stack: Python Why it matters: This project showcases how to leverage large language models for solving complex puzzles, demonstrating the potential of AI in educational and puzzle-solving applications.

Overview: This project is an open-source Optimism AI chatbot app built with Next.js, OpenAI Assistants, and FlowiseAI, utilizing the latest technological stack and fine-tuned with data from the whole optimism Blockchain and its ecosystem. Key features: - An open-source Optimism AI chatbot app - Built with Next.js, OpenAI Assistants, and FlowiseAI - Fine-tuned with data from the optimism Blockchain and its ecosystem Tech stack: JavaScript Why it matters: This project offers a practical application of cutting-edge AI and blockchain technologies, providing a tool for developers to build chatbots using the latest tools and data from the optimism ecosystem. Repository: AsharibAli/optimism-chatbot-ai Primary language: JavaScript GitHub stars: 11

### Overview: Woodcock is a speculative decoding technique designed for Apple Silicon, offering a 2x speedup on Qwen3-1.7B and 2.23x on Qwen3-4B, while maintaining 0 quality loss. This lossless method optimizes performance on a Mac by adding a tiny drafter layer that guesses the next tokens, reducing the need for multiple forward passes. ### Key Features: - **Lossless by Construction**: The greedy output is identical to the model's greedy output, ensuring no quality loss. - **Built for Unified Memory**: Synchronizes one host per round, limiting the draft head to tokens used by the model, and choosing chain length based on measured latencies. - **Trainable on Mac**: Users can train the drafter on their Mac or a rented GPU using publicly available data. - **MIT-Licensed**: The project is open-source, with every design element traced to papers or marked as our own. ### Tech Stack: - Python 3.12 ### Why It Matters: - **2x Speedup on Qwen3-1.7B**: Achieves a 2x speedup on a Mac with a trained hidden-state drafter. - **2.23x Speedup on Qwen3-4B**: Provides a significant 2.23x speedup on a Qwen3-4B model. - **Lossless by Construction**: Ensures that the output remains exactly the same as the original model's greedy output. - **Built for Unified Memory**: Optim

Overview: This Terraform module is designed to deploy QDrant, a vector database, on an EC2 instance in the AWS region "ap-south-1". It is intended for development purposes only. Key features: - QDrant vector database deployment - AWS EC2 instance with QDrant installed - SSH access via generated key pair Tech stack: Terraform, AWS Why it matters: This module provides a quick and easy way to set up a QDrant instance for development, making it accessible via SSH for further testing and development. GitHub stars: 11 Repository: CTOFriendly/terraform-aws-qdrant

### Overview: Fast local language inference on AMD Radeon RX 7900 XTX / XT with RDNA3, providing a tuned build of Qwen3.8-Flash-Next on one or two consumer GPUs. Up to 4 parallel agents with 262k context each, or one 1M-token context, OpenAI/Anthropic API, NVFP4 and EXL3 checkpoints. ### Key features: - **Local Server**: Runs Qwen3.8-Flash-Next model on consumer GPUs for local inference. - **OpenAI/Anthropic API Compatibility**: Supports OpenAI and Anthropic API for chat and coding agents. - **Tuned Build**: Uses FreeToken MoE engine optimized for RDNA3. - **GPU and RAM Requirements**: Requires one or two RX 7900 XTX / XT GPUs, 128 GB RAM, and 85 to 135 GB for the model and image. - **Quick Start**: Easy Docker-based setup, including `rdna3/serve.sh --check` and `rdna3/download.sh` for model and image download. - **Client Integration**: Supports integration with chat apps and coding agents like OpenCode and Claude Code. - **Long Conversations**: Handles up to 262k tokens per conversation on two cards, 131k on one, with 6.9 GiB RAM usage. - **Parallel Agents**: Supports up to 4 parallel agents, allowing sub-agents to work in parallel. - **Context Switch

### Overview: A self-hosted image generation engine that turns text prompts into images using OpenAI-compatible APIs, with intelligent caching and optional cloud hosting. ### Key features: - **LLM Prompt Optimization** (Optional): Enhances user prompts for better image quality. - **De-NSFW Content Recovery** (Optional): Rewrites prompts for safety when blocked. - **Multi-Backend Support** (Weighted load balancing): Supports multiple OpenAI-compatible endpoints. - **Smart Retry Logic** (Configurable): Retry attempts for blocked content across different backends. - **Multi-Stage Fallback** (Seamless retries): Replaces blocked content with safety-rewritten prompts before falling back to Pollinations.ai. - **Base64 & URL Support** (Automatic handling): Handles both Base64 image data and URL responses. - **Model Mapping** (Per backend): Maps model names to ensure compatibility. - **Dual Cache Strategy** (Local Filesystem or Deno KV): Uses the preferred cache method. - **Content Moderation** (Caches blocked content): Instantly triggers recovery/fallback on subsequent requests. - **Pluggable Image Hosting** (Built-in support for smms, picgo, and cloudflare_imgbed). - **Reproducible Results** (Control over seed, model, width, height): Full control over configuration. - **Secure by Default** (Requires `key` for access, `X-Admin-Token` for admin operations): Secure access. - **Health Endpoint** (Simple JSON check): `/status` provides health and

Overview: CrossPool is a serving system designed to efficiently manage and co-locate multiple SGLang models, optimizing resource usage and improving performance by separating KV Cache and FFN weight management. Key features: - Logical KV/prefix-cache ownership is kept separate from FFN weight shards, allowing physical backing to be shared across Instances. - FFN weight shards are retained by FfnAgents, while SGLang models are kept on a shared FFN execution tier. - CrossPool shares execution infrastructure and physical capacity, but not cache contents, to maintain logical isolation. - Seamless integration with SGLang, ensuring architecture-specific adapters are installed without altering the request or KV Cache runtime. - Independent resource placement for attention/KV and FFN/weight sides, allowing co-located models to be configured and sized separately. - SLO-aware elastic KV Cache backing, allowing physical capacity to be reallocated between Instances while maintaining logical allocation and prefix-cache ownership. - Layer-wise FFN execution, with FfnAgents retaining model-specific weight shards and executing model-defined FFN layers through GraphTemplates. Tech stack: Python Why it matters: CrossPool simplifies the management of multiple SGLang models by separating KV Cache and FFN weight management, improving performance and resource efficiency through independent placement and elastic backing. Repository: Coekjan/CrossPool GitHub stars: 11 GitHub description: Efficient GPU Memory Pooling for Multi-LLM Serving via KV Cache and Weight Disaggregation

### Overview: This project is a deep learning model based on YOLOv12 designed for real-time detection and classification of teeth as normal or cavity using a web application. The model is integrated into a Flask-based dashboard for easy access and real-time dental screening. The dashboard enhances early diagnosis accessibility and is intended to be expanded with improved dataset, annotation quality, and robustness to real-world conditions. ### Key features: - Real-time prediction of dental images - Bounding box visualization with color-coded bounding boxes (red for cavity, green for normal) - Interactive probability plot for visualizing confidence scores - In-memory processing of images, ensuring no temporary files are saved to disk - Clean and intuitive user interface built with Flask, HTML, CSS, and JavaScript ### Tech stack: - **Backend**: Python (Flask) - **YOLOv12**: Ultralytics library for deep learning object detection - **OpenCV (`cv2`)**: For image processing and drawing bounding boxes - **NumPy**: For numerical operations, especially image handling - **Base64**: For encoding/decoding image data for web transfer - **Frontend**: HTML, CSS (Bootstrap), JavaScript (Chart.js) - **Chart.js**: For interactive data visualization (probability plots) - **Model**: Pre-trained YOLOv12 model (`best.pt`) ### Why it matters: This project aims to enhance early diagnosis accessibility by providing a user-friendly interface for real-time dental screening. The model integration into a web application ensures a clean