Projects

### Overview: DeepSeek-V4-Flash-0731 is a high-performance language model optimized for 1,048,576-token context on two NVIDIA DGX Spark units with TP=2, achieving peak decode speeds of up to 653.2 tok/s (c=16) and 398.2 tok/s (c=16) for list and JSON, and 291.1 tok/s (c=32) for code and prose. The model features a 4-bit MLA KV cache and is designed for speculative decoding and deterministic KV-pin sizing, with a 3.23x concurrency at 1M tokens. ### Key features: - **Max Token Context**: 1,048,576 tokens - **GPU KV Cache Pool**: 3,386,611 tokens (3.23x @ 1M) - **Peak Decode Speed**: 653.2 tok/s (c=16) for list, 398.2 tok/s (c=16) for JSON, 291.1 tok/s (c=32) for code and prose - **Mixed-Kind Aggregate**: ~310 tok/s at c=12 - **Speculative Decoding Acceptance**: 0.63 / 4.16 tokens per step - **Long-Context Retrieval**: 994,397-token prompt verified on this config

### Overview: This project is an AI-powered book recommendation engine that uses semantic understanding and emotion intelligence to deliver highly relevant and personalized book suggestions. Leveraging LangChain, OpenAI embeddings, and emotion-aware NLP across 7,000 titles, it offers an interactive Gradio experience for intuitive exploration. ### Key Features: - Semantic vector search for content-based similarity. - Emotion and sentiment analysis from book descriptions. - Interactive Gradio dashboard for user-friendly book recommendations. - Integration of Python, Pandas, LangChain, OpenAI embeddings, transformers, Chroma vector database, and Gradio. ### Tech Stack: Python, Pandas, LangChain, OpenAI embeddings, transformers, Chroma vector database, Gradio. ### Why it Matters: By combining embedding-based retrieval, zero-shot text classification, sentiment and emotion analysis, and an interactive Gradio dashboard, this project demonstrates a complete end-to-end AI pipeline for an intelligent, emotion-aware semantic recommendation system.

Overview: Kurtis is a fine-tuning, inference, and evaluation tool designed for Small Language Models, specifically Huggingface's SmolLM2. It focuses on creating a space for genuine interaction and understanding, rather than complex tasks. Key features: - Fine-tuning for specific language models - Inference capabilities - Evaluation tools for various models Tech stack: Python Why it matters: Kurtis aims to provide a supportive and empathetic conversational AI, focusing on understanding and meaningful dialogue, rather than complex tasks or generating specific outputs like images or code.

### Overview: MiniChat is a minimalistic LLM chat UI inspired by Fooocus and Pi. It focuses on content and simplicity, using bold typography and a minimal design. ### Key features: - **Minimalistic Design**: Focuses on content with minimal design elements. - **OpenAI Compatible**: Supports OpenAI-compatible APIs, including Ollama, Portkey, and HuggingFace Text Generation Inference. - **Docker Support**: Runs with Docker, providing a lean image size of less than 2MB. ### Tech stack: TypeScript ### Why it matters: By keeping configuration minimal and focusing on content, MiniChat aims to provide a focused and efficient chat interface.

Overview: lollm is a small, readable inference engine designed specifically for studying how LLMs and related models run inference. It includes a thin loader, router, and generate loop, along with self-contained family packages optimized for clarity and verification against `transformers`. Key features: - Thin loader and router - Shared generate loop - Self-contained family packages (qwen2, qwen3, gemma2, gemma3, gemma4, qwen3_5) - GGUF and safetensors support for all families - Tokenizer optimized for clarity and verification against `AutoTokenizer` Tech stack: Python Why it matters: lollm provides a clear and verified way to study LLM inference, ensuring that the implementation is as close as possible to the original models while maintaining readability and verification against established benchmarks. Repository: giapnguyen74/lollm GitHub stars: 6 GitHub description: an AI inference for learning README excerpt: <img src="docs/lollm-logo.svg" alt="lollm — learn only LLM" height="72">

### Overview: This project explores the development and evaluation of Convolutional Neural Networks (CNNs) for skin lesion classification using the HAM10000 dataset. The dataset includes 7 skin cancer classes, including melanoma, which is the primary focus. Techniques such as data augmentation, generative adversarial networks, and transfer learning are utilized to improve model performance. ### Key features: - Data augmentation with image transformations and Generative Adversarial Networks. - Segmentation and transfer learning techniques. - Metrics extracted for accuracy, loss, sensitivity (Recall), specificity, F1-score, AUC, and precision. - Utilization of ConvNeXt, ResNetV2, ViT_B/32, and Xception architectures. ### Tech stack: Jupyter Notebook ### Why it matters: This project provides a comprehensive analysis of various CNN models and techniques for skin lesion classification, demonstrating the impact of configurations and transfer learning on model performance. The use of HAM10000 dataset and detailed metrics extraction ensures a thorough evaluation of the models' effectiveness.

### Overview: Fine-tuning ResNet50 for Food Classification means adapting a pre-trained ResNet50 model to recognize specific food categories by adjusting its final layers and training it on a specialized dataset of food images for improved accuracy. ### Key Features: - Fine-tuning ResNet50 model - Training on a specialized dataset of food images - Adjusting final layers for better accuracy - Predicting food classes and estimating calorie content ### Tech Stack: Jupyter Notebook ### Why It Matters: This project enables users to track their dietary intake and make informed food choices by accurately recognizing food items from images and estimating their calorie content.

Overview: This project demonstrates how to enhance Retrieval-Augmented Generation (RAG) systems by implementing Anthropic's Contextual Retrieval approach. It shows how to add crucial context before embedding documents into a vector database, improving the accuracy and relevance of retrieved information. Key features: - Enhancing RAG systems with Contextual Retrieval - Using Exxa batch inference API for efficient cost-effectiveness Tech stack: Python Why it matters: This method addresses the issue of losing context in traditional RAG systems by adding chunk-specific explanatory context before embedding, making the information more useful for retrieval.

Overview: This monorepo is designed for the EA platform, focusing on a developer-focused workflow engine built on Kubernetes. Key features: - Node-based AI agent/workflow builder for integrating with ChatGPT, Grok, Claude, Gemini, and Ollama, along with third-party APIs via a low-code/no-code JSON interface. - Kubernetes-native job engine for distributed agent workflows across nodes. - Designed with Kubernetes patterns like custom resources and operators. - Implements best practices for platform scaling and security, including JWT/JWKS auth, NetworkPolicy, Istio, and Kubernetes Gateway API. Tech stack: Go Why it matters: This monorepo streamlines the development and management of AI workflows and agent engines, making it easier for developers to build, test, and scale their applications using Kubernetes. Repository: im-knots/ea-monorepo GitHub stars: 6 GitHub description: A thing do-er workflow engine designed for kubernetes

Overview: This repository provides a supplementary training and evaluation surface for Breeze TTS 2, including LoRA and full-SFT fine-tuning, without including model weights, adapters, or checkpoints. It offers a detailed implementation of supervised labels, feasibility gates, LoRA and adapter releases, full-SFT sweeps, and evaluation metrics. Key features: - Supervised labels for reconstructing text and 16-codebook audio targets from an exact transcript and target WAV. - Feasibility gate for one BF16 forward and backward step, finite losses, and fresh-process reload. - LoRA for backbone, depth decoder, text projection, depth-input projection, and output projection targets. - LoRA lifecycle for trainable-parameter receipt, checkpoint, resume, adapter export, merge, and fresh-process merged reload. - Adapter release runtime for manifest-driven adapter loading with base revision, base-file, and adapter checksum validation. - Full-SFT for BF16 synthesis model with frozen codec and text encoder, FP32-master SGD or Adafactor, family learning-rate multipliers, and bounded checkpoints. - Full-SFT sweeps for JSON plans, multiple seeds, independent schedule horizons, validation-only trials, and lightweight finalist exports. - Evaluation for matched prompts and seeds, ASR, speaker similarity, acoustic diagnostics, confidence intervals, and an opaque blind-listening pack. Tech stack: Python Why it matters: This repository offers a comprehensive solution for fine-tuning Breeze TTS 2, providing a detailed implementation of various features and metrics, ensuring reproducibility and safety through strict adherence to

### Overview: `hf-providers` is a comprehensive tool designed to help developers and researchers compare different API providers, local GPUs, and cloud services for running any model. It provides real-time pricing, throughput, and status updates for models, as well as detailed information about the models themselves, including their parameter counts, weight sizes, and licensing details. ### Key Features: - **Model Comparison**: Shows the live status, pricing, throughput, and tool support for any model provider. - **Fastest and Cheapest Providers**: Routes API calls through the cheapest or fastest available provider. - **Local GPU Estimates**: Estimates the performance of a model on various GPUs. - **Model Metadata**: Provides detailed metadata about the model, including parameter counts and weight sizes. - **Variant Detection**: Automatically detects related models from the same family. - **Interactive Tree Browser**: Allows users to expand models into providers and providers into language options. ### Tech Stack: JavaScript ### Why It Matters: `hf-providers` is essential for developers who need to quickly and efficiently compare different options for running models, whether it’s through local hardware, cloud services, or GPU providers. It streamlines the process of finding the best solution for their needs, saving time and reducing the risk of errors. ### Repository: - **Repository**: `https://github.com/jadnohra/hf-providers` - **GitHub Stars**: 6 - **GitHub Description**: Compare API providers, local GPUs, and cloud for any model - **Primary Language**: JavaScript ### Installation: - **Brew Install**: `brew install

### Overview: Groq QA Generator is a Python library and CLI tool designed to automate the creation of question-answer pairs from text, specifically for fine-tuning large language models (LLMs). It leverages Groq and Hugging Face datasets to generate high-quality QA pairs, offering customizable train/test split ratios and prompt templates for flexible question generation. ### Key Features: - **CLI Tool**: Use the `groq-qa` command for quick and easy QA pair generation. - **Python Library**: Import directly into your project and extend your code. - **Advanced Models**: Supports large models like LLaMA 3.1 (70B parameters). - **Automated QA Generation**: Generate question-answer pairs from input text. - **Hugging Face Datasets**: Convert QA pairs to Hugging Face datasets and export/upload datasets to Hugging Face. - **Custom Split Ratios**: Define custom train/test split ratios when generating QA pairs. - **Prompt Templates**: Flexible question generation through customizable prompt templates. ### Tech Stack: - Python ### Why It Matters: Groq QA Generator streamlines the process of generating high-quality QA pairs, making it an essential tool for researchers, educators, and domain-specific applications. Its customizable features and integration with Groq and Hugging Face datasets ensure efficient and effective fine-tuning of large language models.

Overview: This project fine-tunes the wav2vec2 model for pathological speech processing, addressing the need for accurate and sensitive analysis of speech in medical contexts. Key features: - Fine-tuning of wav2vec2 model - Integration with pathological speech datasets - Enhanced accuracy and sensitivity for speech analysis Tech stack: Jupyter Notebook Why it matters: The project addresses a critical need in medical speech processing, providing a more accurate and sensitive analysis of speech in pathological conditions, which can significantly improve diagnostic accuracy and patient care.

Overview: Atlas RAG is a document retrieval and augmented generation application designed for PDF-based Q&A workflows, research assistance, and knowledge retrieval. Key features: - Upload and ingest PDF documents - Text chunking and embedding generation - Vector store creation with ChromaDB or FAISS - Semantic retrieval for relevant document chunks - Question answering on extracted knowledge - Streamlit-based web interface - Groq model integration for fast LLM responses Tech stack: Python, LangChain, Streamlit, ChromaDB, FAISS, PyMuPDF / PyPDF, Sentence Transformers, Groq Why it matters: This project streamlines the process of extracting and answering questions from PDF-based documents, making it ideal for researchers, support assistants, and anyone needing quick access to information stored in PDFs. Repository: kabirnagdev/atlas Primary language: Jupyter Notebook GitHub stars: 6 GitHub description: RAG Pipeline with Chroma DB & Chatgroq

Overview: This project compiles a stable LangGraph/YAML agent into a small model that self-orchestrates with no runtime orchestrator, offering a 100× cheaper inference cost compared to traditional methods. Key features: - Compiles LangGraph/YAML agent procedures into small open models - Removes runtime orchestrator, enabling private, offline inference - Reports near-frontier quality at 128-462× lower inference cost - Supports LangGraph and YAML procedures Tech stack: Python Why it matters: This project provides a solution for developers looking to reduce inference costs and streamline their workflows, offering a 100× cheaper inference cost compared to traditional methods. Repository: kamaalg/agent2model GitHub stars: 6 GitHub description: Compile a stable LangGraph/YAML agent into a small model that self-orchestrates with no runtime orchestrator — private + offline, ~100× cheaper inference (per the paper; reproduction in progress).

### Overview: Morphogenesis is a learning-focused LLM inference engine built from scratch in Rust, designed to explore tensors, memory, model loading, attention, KV cache, and serving. The engine is memory-bandwidth-bound, with every weight read exactly once per token and used for a single multiply-accumulate against a vector. The goal is to understand how an LLM actually runs by simulating a disk-based engine. ### Key Features: - **Disk-Based Inference**: Forces the engine to account for every single byte that moves, providing a clear understanding of how an LLM operates. - **Quantization and Sparsity**: Reduces memory usage by quantizing weights and using sparsity. - **Model Loading**: Supports various file formats including pickle, safetensors, GGUF, and ONNX, with exact byte layouts. - **Tokenizer Verification**: A reference implementation has been verified to ensure accuracy. ### Tech Stack: Rust ### Why It Matters: Understanding the intricacies of how an LLM runs, particularly in terms of memory and bandwidth usage, is crucial for optimizing and improving performance. By simulating a disk-based engine, one can gain insights into the bottlenecks and optimizations that are essential for efficient inference. ### Repository: Repository: kashifnawazz/morphogenesis GitHub Stars: 6 GitHub Description: Learning-focused LLM inference engine built from scratch in Rust, exploring tensors, memory, model loading, attention, KV cache, and serving. ### Status: **Early. No inference yet.** Currently: the model is downloaded, the

Overview: Gemma-3n-local-training is a lightweight, GPU-focused framework designed for small-scale deployments such as chatbots, assistants, or domain-specific Q&A systems. It leverages Google's Gemma 3n family models, including 1.1B and 2B, and supports GPU-only inference and LoRA fine-tuning with PEFT, as well as optional 4-bit quantization with bitsandbytes. Key features: - GPU-only inference and training for high performance. - LoRA fine-tuning with PEFT for memory-efficient adaptation. - Optional 4-bit quantization with bitsandbytes. - Simple Gradio Web App for an easy, no-code workflow. - Functionality to push trained adapters directly to the Hugging Face Hub. Tech stack: Python Why it matters: This framework simplifies the process of fine-tuning large language models, making it accessible for developers and researchers without requiring extensive knowledge of machine learning or deep learning frameworks. It is particularly useful for applications that require quick deployment and can benefit from fine-tuning on a smaller dataset.

Overview: SecGPT-Lite 是一个面向中文漏洞知识增强的领域大模型微调与评测项目，旨在通过在免费国产算力上完成数据合成、27B LoRA 微调和防泄漏评测，提升模型在网络安全领域的知识掌握能力。该项目在专家盲审中获得了 8.08/10 的高分，B 类题目全部来自未参与训练的 holdout 文档，实现了防泄漏评测的突破。 Key features: - 实现了 27B LoRA 微调 - 防泄漏评测设计，B 类题目出自未参与训练的 holdout 文档 - 专家盲审 10 分制，四轮度量修正后定稿 - 28 个工程坑全部有解且记录在案 - 服务可用性，推理服务扛得住 opencode 等 coding agent 的真实折腾 Tech stack: Python Why it matters: SecGPT-Lite 通过在免费国产算力上完成数据合成、27B LoRA 微调和防泄漏评测，实现了防泄漏评测的突破，提升了模型在网络安全领域的知识掌握能力。该项目在专家盲审中获得了高分，为网络安全领域的大模型应用提供了新的思路和方法。

Overview: 이 레포지토리에서 BERT를 huggingface PyTorch 라이브러리로 빠르고 효율적으로 모델을 fine-tuning하여 문장 분류에서 우수한 성능에 근접하는 방법을 보여줍니다. Key features: - BERT를 사용하여 텍스트 분류기를 학습합니다. - Pre-training된 BERT 모델의 가중치를 사용하여 훈련을 수행합니다. - 적은 데이터 세트에서 우수한 성능을 발휘하도록 모델을 학습합니다. - 다양한 NLP 작업에 대한 최소한의 조정으로 우수한 결과를 얻을 수 있습니다. Tech stack: PyTorch, Hugging Face Why it matters: 이 레포지토리에서 제공하는 방법은 NLP에서 transfer learning 모델을 사용하여 빠르고 효율적으로 우수한 성능을 발휘하는 방법을 보여줍니다. 이를 통해 연구자 및 실무자들이 NLP 작업을 쉽게 훈련시킬 수 있는 기술을 배우고 적용할 수 있습니다.

Overview: Chirrup is a lightweight, fast, and responsive inference frontend for RWKV models, designed to be a drop-in replacement for existing LLM workflows. It leverages the powerful inference engine from Albatross to achieve high performance and continuous batching, making it suitable for various applications requiring high throughput. Key features: - High Performance: Utilizes the blazing-fast inference engine from Albatross. - Continuous Batching: Maximizes GPU utilization by dynamically batching incoming requests. - State Cache: Reuses computation states for long-context inputs, improving throughput as context length increases. - OpenAI-Compatible API: Compatible with existing LLM workflows without requiring code changes. Tech stack: Python, PyTorch, CUDA, Albatross. Why it matters: Chirrup is designed to be highly performant and efficient, making it ideal for applications that require real-time processing and high throughput, such as chatbots, language models, and more. Repository: leonsama/chirrup GitHub stars: 6 GitHub description: 🐦RWKV-7 inference | Like a chirping bird — lightweight, fast, and always responsive.