Projects

### Overview: Fine-tuning ResNet50 for Food Classification means adapting a pre-trained ResNet50 model to recognize specific food categories by adjusting its final layers and training it on a specialized dataset of food images for improved accuracy. ### Key Features: - Fine-tuning ResNet50 model - Training on a specialized dataset of food images - Adjusting final layers for better accuracy - Predicting food classes and estimating calorie content ### Tech Stack: Jupyter Notebook ### Why It Matters: This project enables users to track their dietary intake and make informed food choices by accurately recognizing food items from images and estimating their calorie content.

Overview: This project demonstrates how to enhance Retrieval-Augmented Generation (RAG) systems by implementing Anthropic's Contextual Retrieval approach. It shows how to add crucial context before embedding documents into a vector database, improving the accuracy and relevance of retrieved information. Key features: - Enhancing RAG systems with Contextual Retrieval - Using Exxa batch inference API for efficient cost-effectiveness Tech stack: Python Why it matters: This method addresses the issue of losing context in traditional RAG systems by adding chunk-specific explanatory context before embedding, making the information more useful for retrieval.

Overview: This monorepo is designed for the EA platform, focusing on a developer-focused workflow engine built on Kubernetes. Key features: - Node-based AI agent/workflow builder for integrating with ChatGPT, Grok, Claude, Gemini, and Ollama, along with third-party APIs via a low-code/no-code JSON interface. - Kubernetes-native job engine for distributed agent workflows across nodes. - Designed with Kubernetes patterns like custom resources and operators. - Implements best practices for platform scaling and security, including JWT/JWKS auth, NetworkPolicy, Istio, and Kubernetes Gateway API. Tech stack: Go Why it matters: This monorepo streamlines the development and management of AI workflows and agent engines, making it easier for developers to build, test, and scale their applications using Kubernetes. Repository: im-knots/ea-monorepo GitHub stars: 6 GitHub description: A thing do-er workflow engine designed for kubernetes

### Overview: `hf-providers` is a comprehensive tool designed to help developers and researchers compare different API providers, local GPUs, and cloud services for running any model. It provides real-time pricing, throughput, and status updates for models, as well as detailed information about the models themselves, including their parameter counts, weight sizes, and licensing details. ### Key Features: - **Model Comparison**: Shows the live status, pricing, throughput, and tool support for any model provider. - **Fastest and Cheapest Providers**: Routes API calls through the cheapest or fastest available provider. - **Local GPU Estimates**: Estimates the performance of a model on various GPUs. - **Model Metadata**: Provides detailed metadata about the model, including parameter counts and weight sizes. - **Variant Detection**: Automatically detects related models from the same family. - **Interactive Tree Browser**: Allows users to expand models into providers and providers into language options. ### Tech Stack: JavaScript ### Why It Matters: `hf-providers` is essential for developers who need to quickly and efficiently compare different options for running models, whether it’s through local hardware, cloud services, or GPU providers. It streamlines the process of finding the best solution for their needs, saving time and reducing the risk of errors. ### Repository: - **Repository**: `https://github.com/jadnohra/hf-providers` - **GitHub Stars**: 6 - **GitHub Description**: Compare API providers, local GPUs, and cloud for any model - **Primary Language**: JavaScript ### Installation: - **Brew Install**: `brew install

### Overview: Groq QA Generator is a Python library and CLI tool designed to automate the creation of question-answer pairs from text, specifically for fine-tuning large language models (LLMs). It leverages Groq and Hugging Face datasets to generate high-quality QA pairs, offering customizable train/test split ratios and prompt templates for flexible question generation. ### Key Features: - **CLI Tool**: Use the `groq-qa` command for quick and easy QA pair generation. - **Python Library**: Import directly into your project and extend your code. - **Advanced Models**: Supports large models like LLaMA 3.1 (70B parameters). - **Automated QA Generation**: Generate question-answer pairs from input text. - **Hugging Face Datasets**: Convert QA pairs to Hugging Face datasets and export/upload datasets to Hugging Face. - **Custom Split Ratios**: Define custom train/test split ratios when generating QA pairs. - **Prompt Templates**: Flexible question generation through customizable prompt templates. ### Tech Stack: - Python ### Why It Matters: Groq QA Generator streamlines the process of generating high-quality QA pairs, making it an essential tool for researchers, educators, and domain-specific applications. Its customizable features and integration with Groq and Hugging Face datasets ensure efficient and effective fine-tuning of large language models.

Overview: This project fine-tunes the wav2vec2 model for pathological speech processing, addressing the need for accurate and sensitive analysis of speech in medical contexts. Key features: - Fine-tuning of wav2vec2 model - Integration with pathological speech datasets - Enhanced accuracy and sensitivity for speech analysis Tech stack: Jupyter Notebook Why it matters: The project addresses a critical need in medical speech processing, providing a more accurate and sensitive analysis of speech in pathological conditions, which can significantly improve diagnostic accuracy and patient care.

Overview: Atlas RAG is a document retrieval and augmented generation application designed for PDF-based Q&A workflows, research assistance, and knowledge retrieval. Key features: - Upload and ingest PDF documents - Text chunking and embedding generation - Vector store creation with ChromaDB or FAISS - Semantic retrieval for relevant document chunks - Question answering on extracted knowledge - Streamlit-based web interface - Groq model integration for fast LLM responses Tech stack: Python, LangChain, Streamlit, ChromaDB, FAISS, PyMuPDF / PyPDF, Sentence Transformers, Groq Why it matters: This project streamlines the process of extracting and answering questions from PDF-based documents, making it ideal for researchers, support assistants, and anyone needing quick access to information stored in PDFs. Repository: kabirnagdev/atlas Primary language: Jupyter Notebook GitHub stars: 6 GitHub description: RAG Pipeline with Chroma DB & Chatgroq

Overview: This project compiles a stable LangGraph/YAML agent into a small model that self-orchestrates with no runtime orchestrator, offering a 100× cheaper inference cost compared to traditional methods. Key features: - Compiles LangGraph/YAML agent procedures into small open models - Removes runtime orchestrator, enabling private, offline inference - Reports near-frontier quality at 128-462× lower inference cost - Supports LangGraph and YAML procedures Tech stack: Python Why it matters: This project provides a solution for developers looking to reduce inference costs and streamline their workflows, offering a 100× cheaper inference cost compared to traditional methods. Repository: kamaalg/agent2model GitHub stars: 6 GitHub description: Compile a stable LangGraph/YAML agent into a small model that self-orchestrates with no runtime orchestrator — private + offline, ~100× cheaper inference (per the paper; reproduction in progress).

### Overview: Morphogenesis is a learning-focused LLM inference engine built from scratch in Rust, designed to explore tensors, memory, model loading, attention, KV cache, and serving. The engine is memory-bandwidth-bound, with every weight read exactly once per token and used for a single multiply-accumulate against a vector. The goal is to understand how an LLM actually runs by simulating a disk-based engine. ### Key Features: - **Disk-Based Inference**: Forces the engine to account for every single byte that moves, providing a clear understanding of how an LLM operates. - **Quantization and Sparsity**: Reduces memory usage by quantizing weights and using sparsity. - **Model Loading**: Supports various file formats including pickle, safetensors, GGUF, and ONNX, with exact byte layouts. - **Tokenizer Verification**: A reference implementation has been verified to ensure accuracy. ### Tech Stack: Rust ### Why It Matters: Understanding the intricacies of how an LLM runs, particularly in terms of memory and bandwidth usage, is crucial for optimizing and improving performance. By simulating a disk-based engine, one can gain insights into the bottlenecks and optimizations that are essential for efficient inference. ### Repository: Repository: kashifnawazz/morphogenesis GitHub Stars: 6 GitHub Description: Learning-focused LLM inference engine built from scratch in Rust, exploring tensors, memory, model loading, attention, KV cache, and serving. ### Status: **Early. No inference yet.** Currently: the model is downloaded, the

Overview: Gemma-3n-local-training is a lightweight, GPU-focused framework designed for small-scale deployments such as chatbots, assistants, or domain-specific Q&A systems. It leverages Google's Gemma 3n family models, including 1.1B and 2B, and supports GPU-only inference and LoRA fine-tuning with PEFT, as well as optional 4-bit quantization with bitsandbytes. Key features: - GPU-only inference and training for high performance. - LoRA fine-tuning with PEFT for memory-efficient adaptation. - Optional 4-bit quantization with bitsandbytes. - Simple Gradio Web App for an easy, no-code workflow. - Functionality to push trained adapters directly to the Hugging Face Hub. Tech stack: Python Why it matters: This framework simplifies the process of fine-tuning large language models, making it accessible for developers and researchers without requiring extensive knowledge of machine learning or deep learning frameworks. It is particularly useful for applications that require quick deployment and can benefit from fine-tuning on a smaller dataset.

Overview: SecGPT-Lite 是一个面向中文漏洞知识增强的领域大模型微调与评测项目，旨在通过在免费国产算力上完成数据合成、27B LoRA 微调和防泄漏评测，提升模型在网络安全领域的知识掌握能力。该项目在专家盲审中获得了 8.08/10 的高分，B 类题目全部来自未参与训练的 holdout 文档，实现了防泄漏评测的突破。 Key features: - 实现了 27B LoRA 微调 - 防泄漏评测设计，B 类题目出自未参与训练的 holdout 文档 - 专家盲审 10 分制，四轮度量修正后定稿 - 28 个工程坑全部有解且记录在案 - 服务可用性，推理服务扛得住 opencode 等 coding agent 的真实折腾 Tech stack: Python Why it matters: SecGPT-Lite 通过在免费国产算力上完成数据合成、27B LoRA 微调和防泄漏评测，实现了防泄漏评测的突破，提升了模型在网络安全领域的知识掌握能力。该项目在专家盲审中获得了高分，为网络安全领域的大模型应用提供了新的思路和方法。

Overview: 이 레포지토리에서 BERT를 huggingface PyTorch 라이브러리로 빠르고 효율적으로 모델을 fine-tuning하여 문장 분류에서 우수한 성능에 근접하는 방법을 보여줍니다. Key features: - BERT를 사용하여 텍스트 분류기를 학습합니다. - Pre-training된 BERT 모델의 가중치를 사용하여 훈련을 수행합니다. - 적은 데이터 세트에서 우수한 성능을 발휘하도록 모델을 학습합니다. - 다양한 NLP 작업에 대한 최소한의 조정으로 우수한 결과를 얻을 수 있습니다. Tech stack: PyTorch, Hugging Face Why it matters: 이 레포지토리에서 제공하는 방법은 NLP에서 transfer learning 모델을 사용하여 빠르고 효율적으로 우수한 성능을 발휘하는 방법을 보여줍니다. 이를 통해 연구자 및 실무자들이 NLP 작업을 쉽게 훈련시킬 수 있는 기술을 배우고 적용할 수 있습니다.

Overview: Chirrup is a lightweight, fast, and responsive inference frontend for RWKV models, designed to be a drop-in replacement for existing LLM workflows. It leverages the powerful inference engine from Albatross to achieve high performance and continuous batching, making it suitable for various applications requiring high throughput. Key features: - High Performance: Utilizes the blazing-fast inference engine from Albatross. - Continuous Batching: Maximizes GPU utilization by dynamically batching incoming requests. - State Cache: Reuses computation states for long-context inputs, improving throughput as context length increases. - OpenAI-Compatible API: Compatible with existing LLM workflows without requiring code changes. Tech stack: Python, PyTorch, CUDA, Albatross. Why it matters: Chirrup is designed to be highly performant and efficient, making it ideal for applications that require real-time processing and high throughput, such as chatbots, language models, and more. Repository: leonsama/chirrup GitHub stars: 6 GitHub description: 🐦RWKV-7 inference | Like a chirping bird — lightweight, fast, and always responsive.

### Overview: This project implements an experimental speculative model serving system, aiming to optimize performance and execution time for machine learning applications. ### Key features: - **Guaranteed Execution Time**: Ensures consistent execution times regardless of the model's performance. - **Consensus-Based Model Serving**: Achieves consensus among multiple models to provide a reliable prediction. - **Quality-Based Model Serving**: Selects the model serving result with the highest quality based on evaluation criteria. ### Tech stack: Scala ### Why it matters: This approach leverages speculative execution to enhance the efficiency and reliability of model serving, ensuring consistent performance and decision-making based on quality metrics.

### Overview: `lil` is a Go-based launcher designed for local and Spark/RDMA vLLM deployments in the Local Inference Lab. It reads launch manifests from the catalog repository `local-inference-lab/lil-catalog`, combines them with a discovered machine topology, and produces a typed vLLM invocation. The executable embeds the model families in `configs/models/_bases.yaml`, and the catalog entries and checkpoint facts come from Hugging Face at runtime. ### Key features: - **Local and Spark/RDMA Deployment**: `lil` supports both local and Spark/RDMA vLLM deployments. - **Discovered Topology**: It reads the topology from the catalog, including hardware facts and host tuning. - **Model Execution**: The launcher combines the topology with the model checkpoint metadata to produce a typed vLLM invocation. - **Environment and Command**: It provides a way to run models in a pretty-printed environment without downloading model weights or executing them. ### Tech stack: Go ### Why it matters: `lil` simplifies the deployment and execution of vLLM models in a local environment, making it easier to manage and test models without the need for complex setup or external dependencies.

Overview: dserve is a Deep Learning models Serving framework built with Python, utilizing gunicorn, uvicorn, and FastAPI for efficient model deployment and parallel processing. Key features: - Fast and efficient model serving with multiple workers - Automatic loading of gunicorn configuration from src/gunicorn_conf.py - Startup script located at src/gunicorn.sh for easy integration - Memory sharing between processes for faster model loading - Built-in support for parallel model inference using parallel and curl Tech stack: Python, gunicorn, uvicorn, FastAPI Why it matters: dserve simplifies the deployment and parallel processing of deep learning models, making it an ideal solution for developers looking to efficiently serve and train models in production environments.

Overview: This project automates the process of collecting, processing, and transforming Trustpilot reviews into high-quality QA datasets suitable for fine-tuning LLMs, specifically Gemma 3, using Bright Data, OpenAI, and Hugging Face tools. Key features: - Collect customer reviews from Trustpilot using Bright Data API. - Convert reviews to Markdown for easy inspection or sharing. - Chunk and refine reviews with OpenAI LLMs for improved coherence and quality. - Generate insightful QA pairs for fine-tuning customer service or product feedback models. - Upload the dataset directly to Hugging Face Hub with a train/test split. Tech stack: Python, requests, python-dotenv, openai, tenacity, langchain-text-splitters, datasets, huggingface-hub. Why it matters: This workflow streamlines the process of collecting, processing, and fine-tuning customer feedback for LLMs, making it easier and more efficient for developers working on customer service or product improvement projects.

Overview: Photon is a model serving service built on Kubernetes, designed to run various machine learning models efficiently. Key features: - Serving different model kinds (e.g., PMML, TensorFlow) - Static features store - A/B testing - Monitoring Tech stack: Go Why it matters: Photon simplifies the deployment and management of machine learning models, making it easier to scale and maintain complex models in a Kubernetes environment.

### Overview: Vapor is a macOS floating app that enhances AI productivity by transforming voice, screenshots, and browser research into context-rich prompts, sending them directly into any AI chat without leaving your workflow. ### Key Features: - **Voice Dictation**: Dictate thoughts directly into the editor as you speak, without the need for cloud services or privacy concerns. - **Screenshot Shelf**: Auto-detects and captures screenshots, allowing you to quickly add context to your prompts with a single keypress. - **Context Tray**: Stores and searches for captured pages, articles, and research in a searchable sidebar, making it easy to inject context into your prompts. - **Agent Memory API**: Exposes local agent sessions through an authenticated API, allowing agents to access and search previous conversation history and tool context. - **Research Interrogation**: Automatically discovers structured data from live browser tabs, making it easier to inject context into your prompts. ### Tech Stack: Swift ### Why It Matters: Vapor streamlines your workflow by integrating AI tools seamlessly into your daily tasks, enhancing productivity and efficiency without the need for manual intervention.

### Overview: This repository contains custom recipes for DGX Spark-class systems, designed to optimize performance and functionality for AI applications. The recipes are reviewed and trusted, ensuring compatibility and reliability. ### Key Features: - **Qwen3.8-27B**: A TensorFold launcher for single Spark, optimized for MiaAI-Lab's Qwen3.8 model, with a specific image pinned by digest for compatibility. - **Fish Audio Drama 3**: A native TTS recipe for single Spark, leveraging Fish Audio and Drama 3 snapshots for efficient audio processing. ### Tech Stack: Python ### Why it Matters: These recipes are tailored to enhance the performance and functionality of AI applications on DGX Spark-class systems, ensuring optimal performance and reliability.