Projects

### Overview: This project implements an experimental speculative model serving system, aiming to optimize performance and execution time for machine learning applications. ### Key features: - **Guaranteed Execution Time**: Ensures consistent execution times regardless of the model's performance. - **Consensus-Based Model Serving**: Achieves consensus among multiple models to provide a reliable prediction. - **Quality-Based Model Serving**: Selects the model serving result with the highest quality based on evaluation criteria. ### Tech stack: Scala ### Why it matters: This approach leverages speculative execution to enhance the efficiency and reliability of model serving, ensuring consistent performance and decision-making based on quality metrics.

### Overview: `lil` is a Go-based launcher designed for local and Spark/RDMA vLLM deployments in the Local Inference Lab. It reads launch manifests from the catalog repository `local-inference-lab/lil-catalog`, combines them with a discovered machine topology, and produces a typed vLLM invocation. The executable embeds the model families in `configs/models/_bases.yaml`, and the catalog entries and checkpoint facts come from Hugging Face at runtime. ### Key features: - **Local and Spark/RDMA Deployment**: `lil` supports both local and Spark/RDMA vLLM deployments. - **Discovered Topology**: It reads the topology from the catalog, including hardware facts and host tuning. - **Model Execution**: The launcher combines the topology with the model checkpoint metadata to produce a typed vLLM invocation. - **Environment and Command**: It provides a way to run models in a pretty-printed environment without downloading model weights or executing them. ### Tech stack: Go ### Why it matters: `lil` simplifies the deployment and execution of vLLM models in a local environment, making it easier to manage and test models without the need for complex setup or external dependencies.

Overview: dserve is a Deep Learning models Serving framework built with Python, utilizing gunicorn, uvicorn, and FastAPI for efficient model deployment and parallel processing. Key features: - Fast and efficient model serving with multiple workers - Automatic loading of gunicorn configuration from src/gunicorn_conf.py - Startup script located at src/gunicorn.sh for easy integration - Memory sharing between processes for faster model loading - Built-in support for parallel model inference using parallel and curl Tech stack: Python, gunicorn, uvicorn, FastAPI Why it matters: dserve simplifies the deployment and parallel processing of deep learning models, making it an ideal solution for developers looking to efficiently serve and train models in production environments.

Overview: This project automates the process of collecting, processing, and transforming Trustpilot reviews into high-quality QA datasets suitable for fine-tuning LLMs, specifically Gemma 3, using Bright Data, OpenAI, and Hugging Face tools. Key features: - Collect customer reviews from Trustpilot using Bright Data API. - Convert reviews to Markdown for easy inspection or sharing. - Chunk and refine reviews with OpenAI LLMs for improved coherence and quality. - Generate insightful QA pairs for fine-tuning customer service or product feedback models. - Upload the dataset directly to Hugging Face Hub with a train/test split. Tech stack: Python, requests, python-dotenv, openai, tenacity, langchain-text-splitters, datasets, huggingface-hub. Why it matters: This workflow streamlines the process of collecting, processing, and fine-tuning customer feedback for LLMs, making it easier and more efficient for developers working on customer service or product improvement projects.

Overview: Photon is a model serving service built on Kubernetes, designed to run various machine learning models efficiently. Key features: - Serving different model kinds (e.g., PMML, TensorFlow) - Static features store - A/B testing - Monitoring Tech stack: Go Why it matters: Photon simplifies the deployment and management of machine learning models, making it easier to scale and maintain complex models in a Kubernetes environment.

### Overview: Vapor is a macOS floating app that enhances AI productivity by transforming voice, screenshots, and browser research into context-rich prompts, sending them directly into any AI chat without leaving your workflow. ### Key Features: - **Voice Dictation**: Dictate thoughts directly into the editor as you speak, without the need for cloud services or privacy concerns. - **Screenshot Shelf**: Auto-detects and captures screenshots, allowing you to quickly add context to your prompts with a single keypress. - **Context Tray**: Stores and searches for captured pages, articles, and research in a searchable sidebar, making it easy to inject context into your prompts. - **Agent Memory API**: Exposes local agent sessions through an authenticated API, allowing agents to access and search previous conversation history and tool context. - **Research Interrogation**: Automatically discovers structured data from live browser tabs, making it easier to inject context into your prompts. ### Tech Stack: Swift ### Why It Matters: Vapor streamlines your workflow by integrating AI tools seamlessly into your daily tasks, enhancing productivity and efficiency without the need for manual intervention.

### Overview: This repository contains custom recipes for DGX Spark-class systems, designed to optimize performance and functionality for AI applications. The recipes are reviewed and trusted, ensuring compatibility and reliability. ### Key Features: - **Qwen3.8-27B**: A TensorFold launcher for single Spark, optimized for MiaAI-Lab's Qwen3.8 model, with a specific image pinned by digest for compatibility. - **Fish Audio Drama 3**: A native TTS recipe for single Spark, leveraging Fish Audio and Drama 3 snapshots for efficient audio processing. ### Tech Stack: Python ### Why it Matters: These recipes are tailored to enhance the performance and functionality of AI applications on DGX Spark-class systems, ensuring optimal performance and reliability.

### Overview: This repository provides interactive visualizations of the GPT-2 Medium model, allowing developers to understand its architecture, attention mechanisms, token activations, and overall structure. ### Key Features: - **Model Summary**: Provides a concise overview of the GPT-2 model's architecture and specifications. - **Token Activations**: Shows the activations of tokens across layers, highlighting the importance of each token. - **Attention Visualization**: Visualizes how the model focuses on different parts of a given text. - **3D Model Architecture**: Offers a spatial representation of the model's layers and neurons. - **Token Embeddings**: Projects token embeddings into 2D or 3D space to understand relationships between tokens. - **Gradients and Saliency**: Visualizes the influence of each token on the model's output using gradients. ### Tech Stack: Jupyter Notebook ### Why It Matters: These visualizations provide valuable insights into the GPT-2 model's operations, helping developers understand its architecture, attention mechanisms, and token processing. They are particularly useful for debugging, interpreting, and appreciating the model's behavior on specific tasks.

Overview: MegaDetector-Classifier is a training toolkit for fine-tuning ResNet-based species classifiers on custom camera-trap image datasets. The output weights integrate directly with the PyTorch-Wildlife framework, making it straightforward to deploy a classifier trained on your own data. Key features: - ResNet-18 and ResNet-50 classifier training using PyTorch Lightning - Three data-splitting strategies designed for camera-trap realities: random, location-based, and sequence-based - YAML-based configuration — no code changes required for most use cases - Demo data included for immediate testing without your own dataset Tech stack: Python Why it matters: MegaDetector-Classifier is part of the Microsoft AI for Good Lab's ecosystem, providing a comprehensive solution for wildlife identification and conservation, making it easier and more accessible for researchers and practitioners to adapt existing classifiers to new geographic regions or add new species to the PyTorch-Wildlife model zoo.

### Overview: This is a fine-tuned version of the Mistral-7B model on the Kto-12 dataset, achieving impressive results with a loss of 0.2786, rewards of 5.3903, and logps of -59.0879. The model is designed for LLM continual learning and long-term memory. ### Key Features: - Fine-tuned Mistral-7B model - Achieves high performance on evaluation set - Implements continual learning and long-term memory capabilities - Optimized for LLMs' continuous training and memory retention ### Tech Stack: - Jupyter Notebook ### Why It Matters: This model is designed for LLMs' continual learning and long-term memory, making it ideal for applications requiring continuous adaptation and memory retention. Its performance on the Kto-12 dataset is impressive, indicating its effectiveness in real-world scenarios. ### GitHub Stars: 6 ### Repository: - Repository: neoheartbeats/neoheartbeats-kernel - Primary Language: Jupyter Notebook - GitHub Stars: 6

Overview: MooseNet is an official repository for a trainable metric for synthesized speech, specifically designed with a PLDA module, aiming to improve speech synthesis performance. Key features: - MooseNet: A trainable metric for synthesized speech. - PLDA Module: Enhances the performance of speech synthesis models. Tech stack: Python Why it matters: MooseNet is significant for speech synthesis research, offering a new approach to improving the quality and realism of synthesized speech through the use of a trainable metric and a PLDA module, which can lead to advancements in natural language processing and speech synthesis technologies. Repository: oplatek/moosenet-plda Primary language: Python GitHub stars: 6 GitHub description: Official repo for “MooseNet: A Trainable Metric for Synthesized Speech with a PLDA Module” README excerpt:

Overview: PaglaMLX is a macOS menu-bar application designed to orchestrate local and cloud-based machine learning models, providing a smart reverse proxy for efficient model access and management. Key features: - Local text generation models - Cloud fallback routing - Native protocol translation on Apple Silicon - Smart Gateway (`:2525/v1`) supporting OpenAI and Anthropic - Model orchestration with one-click load/unload from a directory - Live route table updates for model availability - One-click integrations with popular development tools Tech stack: Swift, Python 3.12+, macOS 14.0+, Apple Silicon (M1/M2/M3/M4) Why it matters: PaglaMLX simplifies the process of managing and accessing machine learning models on macOS, providing a unified API that integrates seamlessly with existing workflows, making it easier to test and deploy local models without the need for manual server setup and port management. Repository: paglaai/PaglaMLX GitHub stars: 6 GitHub description: Local LLM orchestration for Apple Silicon — native macOS menu-bar app serving MLX models through a smart reverse proxy with Auto-Router, BYOK, and 14+ integration targets.

### Overview: RelayOps is a production-shaped control plane designed for AI support agents in telecom and subscription billing. It includes scoped permissions, route safety, decision traces, audit export, and human handoff features to ensure safe, auditable, and handoff-ready outcomes in regulated domains. ### Key Features: - **Scoped Permissions**: Ensures that agents can only perform actions within predefined scopes. - **Route Safety**: Ensures that routes are safe and efficient. - **Decision Traces**: Records every state and decision made by the broker. - **Audit Export**: Provides an audit trail of every action taken. - **Human Handoff**: Allows for the final reply to be generated from the broker decision and recorded in the audit trail. ### Tech Stack: - Python: The primary programming language used for development. - Streamlit: Used for the live demo and interactive features. ### Why It Matters: RelayOps treats AI support as a control system, ensuring safe, auditable, and handoff-ready outcomes in regulated domains. It mirrors production controls, including access gates, scoped tools, guardrails, audit trails, and human handoff, making it a robust solution for AI support agents in telecom and subscription billing. ### GitHub Stars: 6 ### Repository: - **Repository**: patibandlavenkatamanideep/relayops - **Primary Language**: Python - **GitHub Stars**: 6 ### Live Demo: - **Demo**: [relayops-production.up.railway.app](https://relayops-production.up.railway.app) ### Additional Resources: - **

### Overview: FinChain is an AI-powered chat-driven expense tracker designed to help you manage your finances in any language without the need for explicit category selection. By simply typing your expenses in any language, FinChain automatically categorizes and analyzes them using an LLM-based NLP backend powered by Hugging Face Transformers. This innovative approach eliminates the need for clunky forms or manual categorization, making expense tracking more intuitive and efficient. ### Key features: - **Chat Your Expenses, Any Language:** Type "Spent 670 on burgers today", "বার্গার খাইসি ১০০০ টাকা দি

### Overview: This project implements Reinforcement Learning from Human Feedback (RLHF) on custom data, replicating the pipeline described in the paper "Learning to Summarize from Human Feedback" by OpenAI. The RLHF process involves three key steps: Supervised Fine-Tuning (SFT), Reward Model Training, and Policy Optimization, enabling the model to generate summaries that align with human preferences. ### Key Features: - Supervised Fine-Tuning (SFT) - Reward Model Training - Policy Optimization ### Tech Stack: Jupyter Notebook ### Why It Matters: Traditional supervised learning for summarization relies on human-labeled datasets, but RLHF addresses the limitations of this approach by using a reward model to capture human preferences, fine-tuning the model with reinforcement learning, and dynamically adapting to evolving human feedback. This method ensures the model generates responses that are not only accurate but also highly aligned with what users value, making it suitable for custom domains and handling ambiguous or subjective tasks. ### Repository: ranzeet013/RLHF-CustomData ### GitHub Stars: 6 ### GitHub Description: Building an LLM with RLHF involves fine-tuning using human-labeled preferences. Based on Learning to Summarize from Human Feedback, it uses supervised learning, reward modeling, and PPO to improve response quality and alignment.

Overview: This project is a sample implementation of FastAPI serving a PyTorch model using micro batching. It demonstrates the use of Service Streamer for efficient model serving. Key features: - Micro batching for model serving - FastAPI integration - PyTorch model with micro batch processing - Service Streamer for deployment Tech stack: Python, FastAPI, PyTorch, Service Streamer Why it matters: This project showcases how to efficiently deploy and serve PyTorch models using FastAPI and micro batching, highlighting the benefits of using micro batching for better performance and resource utilization.

### Overview: tunelab is a Claude Code plugin designed to fine-tune, distill, and evaluate Large Language Models (LLMs) locally. It helps developers decide whether fine-tuning is necessary, distills LLM logs into small local models using MLX/LoRA, and verifies the models' performance on held-out data before deployment. ### Key Features: - **Fine-Tuning Locally**: Tunelab fine-tunes LLMs on local models using MLX/LoRA, ensuring the models are more accurate and efficient. - **Distillation of Logs**: It distills LLM logs into small local models, making it easier to fine-tune. - **Accuracy Verification**: Before deploying, it verifies the models' performance on held-out data, ensuring the best model is chosen. - **Versatile Use Cases**: Suitable for various tasks such as classification, extraction, and more. ### Tech Stack: Python ### Why It Matters: - **Accuracy and Efficiency**: Tunelab ensures that the models are more accurate and efficient, reducing the cost of fine-tuning. - **Versatility**: It can be used with Claude Code, Gemini CLI, or any other tool that reads skills, making it highly versatile. - **Cost-Effective**: It provides a cost-effective solution by fine-tuning models locally, reducing the need for expensive fine-tuning on the cloud. ### Repository: rchaz/tunelab ### GitHub Stars: 6 ### GitHub Description: Claude Code plugin for LLM fine-tuning, distillation, and evaluation —

Overview: CTF_Agents is a multi-agent framework designed for authorized Capture The Flag work, supporting various security challenges across different domains. Key features: - Cryptography - Quantum protocols - Adversarial AI/ML - Industrial control and operational technology - Reverse engineering - Web challenges - Pwn challenges - Hardware challenges - Forensics - Blockchain challenges - Secure coding challenges - Networking challenges - OSINT challenges - Log-analysis challenges - General coding challenges Tech stack: Python Why it matters: CTF_Agents provides a comprehensive framework for authorized CTF work, supporting a wide range of security challenges across different domains. The project's ability to route challenges to the appropriate specialist and execute bounded tools and playbooks makes it highly versatile and useful for both training and real-world scenarios. Repository: rmjohnson12/CTF_Agents Primary language: Python GitHub stars: 6 GitHub description: Agentic CTF security tooling project with natural language support README excerpt: # CTF_Agents CTF_Agents is a multi-agent framework for authorized Capture The Flag work. It classifies a challenge, selects a specialist, executes bounded tools and playbooks, records evidence, and reports either a verified result or a concrete failure reason. The project supports cryptography, quantum protocols, adversarial AI/ML, industrial control and operational technology, reverse engineering, web, pwn, hardware, forensics, blockchain, secure-coding, networking, OSINT, log-analysis, and

## Overview: This project fine-tunes OpenAI’s primary models to produce JSON-formatted output, enhancing cost-efficiency and data consistency for applications that utilize the OpenAI API. ## Key features: - Fine-tuning of OpenAI models - JSON output format - Node.js implementation ## Tech stack: JavaScript (Node.js) ## Why it matters: Improves efficiency and consistency in applications that use the OpenAI API, reducing costs and ensuring data quality.

Overview: This repository contains a comprehensive collection of ML-related active learning datasets, including algorithms, Jupyter Notebook (.ipynb) pipelines, Python (.py) scripts, and curated, ethically aligned synthetic data. The repository is structured into three main modules for easy understanding, reproducibility, and experimental training. Key features: - Synthetic Datasets: Ethically aligned synthetic datasets for active learning experiments. - Inference Generation Scripts: Python scripts for generating synthetic datasets and executing active learning runs. - Raw Reasoning (Jupyter Notebooks): Detailed Jupyter Notebook files for experimental active learning setups, providing context and full visibility into the reasoning and decisions made. Tech stack: Jupyter Notebook Why it matters: This repository serves as a valuable resource for researchers and practitioners in the field of active learning, offering a structured and transparent way to access and experiment with ML datasets and active learning algorithms.