Projects

### Overview: Morphogenesis is a learning-focused LLM inference engine built from scratch in Rust, designed to explore tensors, memory, model loading, attention, KV cache, and serving. The engine is memory-bandwidth-bound, with every weight read exactly once per token and used for a single multiply-accumulate against a vector. The goal is to understand how an LLM actually runs by simulating a disk-based engine. ### Key Features: - **Disk-Based Inference**: Forces the engine to account for every single byte that moves, providing a clear understanding of how an LLM operates. - **Quantization and Sparsity**: Reduces memory usage by quantizing weights and using sparsity. - **Model Loading**: Supports various file formats including pickle, safetensors, GGUF, and ONNX, with exact byte layouts. - **Tokenizer Verification**: A reference implementation has been verified to ensure accuracy. ### Tech Stack: Rust ### Why It Matters: Understanding the intricacies of how an LLM runs, particularly in terms of memory and bandwidth usage, is crucial for optimizing and improving performance. By simulating a disk-based engine, one can gain insights into the bottlenecks and optimizations that are essential for efficient inference. ### Repository: Repository: kashifnawazz/morphogenesis GitHub Stars: 6 GitHub Description: Learning-focused LLM inference engine built from scratch in Rust, exploring tensors, memory, model loading, attention, KV cache, and serving. ### Status: **Early. No inference yet.** Currently: the model is downloaded, the

Overview: Gemma-3n-local-training is a lightweight, GPU-focused framework designed for small-scale deployments such as chatbots, assistants, or domain-specific Q&A systems. It leverages Google's Gemma 3n family models, including 1.1B and 2B, and supports GPU-only inference and LoRA fine-tuning with PEFT, as well as optional 4-bit quantization with bitsandbytes. Key features: - GPU-only inference and training for high performance. - LoRA fine-tuning with PEFT for memory-efficient adaptation. - Optional 4-bit quantization with bitsandbytes. - Simple Gradio Web App for an easy, no-code workflow. - Functionality to push trained adapters directly to the Hugging Face Hub. Tech stack: Python Why it matters: This framework simplifies the process of fine-tuning large language models, making it accessible for developers and researchers without requiring extensive knowledge of machine learning or deep learning frameworks. It is particularly useful for applications that require quick deployment and can benefit from fine-tuning on a smaller dataset.

Overview: SecGPT-Lite 是一个面向中文漏洞知识增强的领域大模型微调与评测项目，旨在通过在免费国产算力上完成数据合成、27B LoRA 微调和防泄漏评测，提升模型在网络安全领域的知识掌握能力。该项目在专家盲审中获得了 8.08/10 的高分，B 类题目全部来自未参与训练的 holdout 文档，实现了防泄漏评测的突破。 Key features: - 实现了 27B LoRA 微调 - 防泄漏评测设计，B 类题目出自未参与训练的 holdout 文档 - 专家盲审 10 分制，四轮度量修正后定稿 - 28 个工程坑全部有解且记录在案 - 服务可用性，推理服务扛得住 opencode 等 coding agent 的真实折腾 Tech stack: Python Why it matters: SecGPT-Lite 通过在免费国产算力上完成数据合成、27B LoRA 微调和防泄漏评测，实现了防泄漏评测的突破，提升了模型在网络安全领域的知识掌握能力。该项目在专家盲审中获得了高分，为网络安全领域的大模型应用提供了新的思路和方法。

Overview: 이 레포지토리에서 BERT를 huggingface PyTorch 라이브러리로 빠르고 효율적으로 모델을 fine-tuning하여 문장 분류에서 우수한 성능에 근접하는 방법을 보여줍니다. Key features: - BERT를 사용하여 텍스트 분류기를 학습합니다. - Pre-training된 BERT 모델의 가중치를 사용하여 훈련을 수행합니다. - 적은 데이터 세트에서 우수한 성능을 발휘하도록 모델을 학습합니다. - 다양한 NLP 작업에 대한 최소한의 조정으로 우수한 결과를 얻을 수 있습니다. Tech stack: PyTorch, Hugging Face Why it matters: 이 레포지토리에서 제공하는 방법은 NLP에서 transfer learning 모델을 사용하여 빠르고 효율적으로 우수한 성능을 발휘하는 방법을 보여줍니다. 이를 통해 연구자 및 실무자들이 NLP 작업을 쉽게 훈련시킬 수 있는 기술을 배우고 적용할 수 있습니다.

Overview: Chirrup is a lightweight, fast, and responsive inference frontend for RWKV models, designed to be a drop-in replacement for existing LLM workflows. It leverages the powerful inference engine from Albatross to achieve high performance and continuous batching, making it suitable for various applications requiring high throughput. Key features: - High Performance: Utilizes the blazing-fast inference engine from Albatross. - Continuous Batching: Maximizes GPU utilization by dynamically batching incoming requests. - State Cache: Reuses computation states for long-context inputs, improving throughput as context length increases. - OpenAI-Compatible API: Compatible with existing LLM workflows without requiring code changes. Tech stack: Python, PyTorch, CUDA, Albatross. Why it matters: Chirrup is designed to be highly performant and efficient, making it ideal for applications that require real-time processing and high throughput, such as chatbots, language models, and more. Repository: leonsama/chirrup GitHub stars: 6 GitHub description: 🐦RWKV-7 inference | Like a chirping bird — lightweight, fast, and always responsive.

### Overview: This project implements an experimental speculative model serving system, aiming to optimize performance and execution time for machine learning applications. ### Key features: - **Guaranteed Execution Time**: Ensures consistent execution times regardless of the model's performance. - **Consensus-Based Model Serving**: Achieves consensus among multiple models to provide a reliable prediction. - **Quality-Based Model Serving**: Selects the model serving result with the highest quality based on evaluation criteria. ### Tech stack: Scala ### Why it matters: This approach leverages speculative execution to enhance the efficiency and reliability of model serving, ensuring consistent performance and decision-making based on quality metrics.

### Overview: `lil` is a Go-based launcher designed for local and Spark/RDMA vLLM deployments in the Local Inference Lab. It reads launch manifests from the catalog repository `local-inference-lab/lil-catalog`, combines them with a discovered machine topology, and produces a typed vLLM invocation. The executable embeds the model families in `configs/models/_bases.yaml`, and the catalog entries and checkpoint facts come from Hugging Face at runtime. ### Key features: - **Local and Spark/RDMA Deployment**: `lil` supports both local and Spark/RDMA vLLM deployments. - **Discovered Topology**: It reads the topology from the catalog, including hardware facts and host tuning. - **Model Execution**: The launcher combines the topology with the model checkpoint metadata to produce a typed vLLM invocation. - **Environment and Command**: It provides a way to run models in a pretty-printed environment without downloading model weights or executing them. ### Tech stack: Go ### Why it matters: `lil` simplifies the deployment and execution of vLLM models in a local environment, making it easier to manage and test models without the need for complex setup or external dependencies.

Overview: dserve is a Deep Learning models Serving framework built with Python, utilizing gunicorn, uvicorn, and FastAPI for efficient model deployment and parallel processing. Key features: - Fast and efficient model serving with multiple workers - Automatic loading of gunicorn configuration from src/gunicorn_conf.py - Startup script located at src/gunicorn.sh for easy integration - Memory sharing between processes for faster model loading - Built-in support for parallel model inference using parallel and curl Tech stack: Python, gunicorn, uvicorn, FastAPI Why it matters: dserve simplifies the deployment and parallel processing of deep learning models, making it an ideal solution for developers looking to efficiently serve and train models in production environments.

Overview: This project automates the process of collecting, processing, and transforming Trustpilot reviews into high-quality QA datasets suitable for fine-tuning LLMs, specifically Gemma 3, using Bright Data, OpenAI, and Hugging Face tools. Key features: - Collect customer reviews from Trustpilot using Bright Data API. - Convert reviews to Markdown for easy inspection or sharing. - Chunk and refine reviews with OpenAI LLMs for improved coherence and quality. - Generate insightful QA pairs for fine-tuning customer service or product feedback models. - Upload the dataset directly to Hugging Face Hub with a train/test split. Tech stack: Python, requests, python-dotenv, openai, tenacity, langchain-text-splitters, datasets, huggingface-hub. Why it matters: This workflow streamlines the process of collecting, processing, and fine-tuning customer feedback for LLMs, making it easier and more efficient for developers working on customer service or product improvement projects.

Overview: Photon is a model serving service built on Kubernetes, designed to run various machine learning models efficiently. Key features: - Serving different model kinds (e.g., PMML, TensorFlow) - Static features store - A/B testing - Monitoring Tech stack: Go Why it matters: Photon simplifies the deployment and management of machine learning models, making it easier to scale and maintain complex models in a Kubernetes environment.

### Overview: Vapor is a macOS floating app that enhances AI productivity by transforming voice, screenshots, and browser research into context-rich prompts, sending them directly into any AI chat without leaving your workflow. ### Key Features: - **Voice Dictation**: Dictate thoughts directly into the editor as you speak, without the need for cloud services or privacy concerns. - **Screenshot Shelf**: Auto-detects and captures screenshots, allowing you to quickly add context to your prompts with a single keypress. - **Context Tray**: Stores and searches for captured pages, articles, and research in a searchable sidebar, making it easy to inject context into your prompts. - **Agent Memory API**: Exposes local agent sessions through an authenticated API, allowing agents to access and search previous conversation history and tool context. - **Research Interrogation**: Automatically discovers structured data from live browser tabs, making it easier to inject context into your prompts. ### Tech Stack: Swift ### Why It Matters: Vapor streamlines your workflow by integrating AI tools seamlessly into your daily tasks, enhancing productivity and efficiency without the need for manual intervention.

### Overview: This repository contains custom recipes for DGX Spark-class systems, designed to optimize performance and functionality for AI applications. The recipes are reviewed and trusted, ensuring compatibility and reliability. ### Key Features: - **Qwen3.8-27B**: A TensorFold launcher for single Spark, optimized for MiaAI-Lab's Qwen3.8 model, with a specific image pinned by digest for compatibility. - **Fish Audio Drama 3**: A native TTS recipe for single Spark, leveraging Fish Audio and Drama 3 snapshots for efficient audio processing. ### Tech Stack: Python ### Why it Matters: These recipes are tailored to enhance the performance and functionality of AI applications on DGX Spark-class systems, ensuring optimal performance and reliability.

### Overview: This repository provides interactive visualizations of the GPT-2 Medium model, allowing developers to understand its architecture, attention mechanisms, token activations, and overall structure. ### Key Features: - **Model Summary**: Provides a concise overview of the GPT-2 model's architecture and specifications. - **Token Activations**: Shows the activations of tokens across layers, highlighting the importance of each token. - **Attention Visualization**: Visualizes how the model focuses on different parts of a given text. - **3D Model Architecture**: Offers a spatial representation of the model's layers and neurons. - **Token Embeddings**: Projects token embeddings into 2D or 3D space to understand relationships between tokens. - **Gradients and Saliency**: Visualizes the influence of each token on the model's output using gradients. ### Tech Stack: Jupyter Notebook ### Why It Matters: These visualizations provide valuable insights into the GPT-2 model's operations, helping developers understand its architecture, attention mechanisms, and token processing. They are particularly useful for debugging, interpreting, and appreciating the model's behavior on specific tasks.

Overview: MegaDetector-Classifier is a training toolkit for fine-tuning ResNet-based species classifiers on custom camera-trap image datasets. The output weights integrate directly with the PyTorch-Wildlife framework, making it straightforward to deploy a classifier trained on your own data. Key features: - ResNet-18 and ResNet-50 classifier training using PyTorch Lightning - Three data-splitting strategies designed for camera-trap realities: random, location-based, and sequence-based - YAML-based configuration — no code changes required for most use cases - Demo data included for immediate testing without your own dataset Tech stack: Python Why it matters: MegaDetector-Classifier is part of the Microsoft AI for Good Lab's ecosystem, providing a comprehensive solution for wildlife identification and conservation, making it easier and more accessible for researchers and practitioners to adapt existing classifiers to new geographic regions or add new species to the PyTorch-Wildlife model zoo.

Overview: Ladder is a small open model fine-tuned for competitive programming, designed to run on free GPUs for Codeforces-style problems. It uses QLoRA training and verifiable evaluation, with a focus on running on Kaggle/Colab GPUs. Key features: - Fine-tuned models for competitive programming - Runs on free GPUs for Codeforces-style problems - Verifiable evaluation without relying on another model or BLEU scores - Data prepared using reasoning traces from Codeforces problems - Training on 4-bit QLoRA with loss computed on assistant turns only - Evaluated by generating a program and running it against test cases Tech stack: Python Why it matters: Ladder is designed to provide a reliable and efficient way to fine-tune models for competitive programming, ensuring that the models can be run on free GPUs, making it accessible for a wide range of users. Repository: natedemoss/ladder Primary language: Python GitHub stars: 6 GitHub description: Ladder: small open models fine-tuned for competitive programming. QLoRA training + verifiable eval that runs on Kaggle/Colab GPUs.

### Overview: This is a fine-tuned version of the Mistral-7B model on the Kto-12 dataset, achieving impressive results with a loss of 0.2786, rewards of 5.3903, and logps of -59.0879. The model is designed for LLM continual learning and long-term memory. ### Key Features: - Fine-tuned Mistral-7B model - Achieves high performance on evaluation set - Implements continual learning and long-term memory capabilities - Optimized for LLMs' continuous training and memory retention ### Tech Stack: - Jupyter Notebook ### Why It Matters: This model is designed for LLMs' continual learning and long-term memory, making it ideal for applications requiring continuous adaptation and memory retention. Its performance on the Kto-12 dataset is impressive, indicating its effectiveness in real-world scenarios. ### GitHub Stars: 6 ### Repository: - Repository: neoheartbeats/neoheartbeats-kernel - Primary Language: Jupyter Notebook - GitHub Stars: 6

Overview: MooseNet is an official repository for a trainable metric for synthesized speech, specifically designed with a PLDA module, aiming to improve speech synthesis performance. Key features: - MooseNet: A trainable metric for synthesized speech. - PLDA Module: Enhances the performance of speech synthesis models. Tech stack: Python Why it matters: MooseNet is significant for speech synthesis research, offering a new approach to improving the quality and realism of synthesized speech through the use of a trainable metric and a PLDA module, which can lead to advancements in natural language processing and speech synthesis technologies. Repository: oplatek/moosenet-plda Primary language: Python GitHub stars: 6 GitHub description: Official repo for “MooseNet: A Trainable Metric for Synthesized Speech with a PLDA Module” README excerpt:

Overview: PaglaMLX is a macOS menu-bar application designed to orchestrate local and cloud-based machine learning models, providing a smart reverse proxy for efficient model access and management. Key features: - Local text generation models - Cloud fallback routing - Native protocol translation on Apple Silicon - Smart Gateway (`:2525/v1`) supporting OpenAI and Anthropic - Model orchestration with one-click load/unload from a directory - Live route table updates for model availability - One-click integrations with popular development tools Tech stack: Swift, Python 3.12+, macOS 14.0+, Apple Silicon (M1/M2/M3/M4) Why it matters: PaglaMLX simplifies the process of managing and accessing machine learning models on macOS, providing a unified API that integrates seamlessly with existing workflows, making it easier to test and deploy local models without the need for manual server setup and port management. Repository: paglaai/PaglaMLX GitHub stars: 6 GitHub description: Local LLM orchestration for Apple Silicon — native macOS menu-bar app serving MLX models through a smart reverse proxy with Auto-Router, BYOK, and 14+ integration targets.

### Overview: RelayOps is a production-shaped control plane designed for AI support agents in telecom and subscription billing. It includes scoped permissions, route safety, decision traces, audit export, and human handoff features to ensure safe, auditable, and handoff-ready outcomes in regulated domains. ### Key Features: - **Scoped Permissions**: Ensures that agents can only perform actions within predefined scopes. - **Route Safety**: Ensures that routes are safe and efficient. - **Decision Traces**: Records every state and decision made by the broker. - **Audit Export**: Provides an audit trail of every action taken. - **Human Handoff**: Allows for the final reply to be generated from the broker decision and recorded in the audit trail. ### Tech Stack: - Python: The primary programming language used for development. - Streamlit: Used for the live demo and interactive features. ### Why It Matters: RelayOps treats AI support as a control system, ensuring safe, auditable, and handoff-ready outcomes in regulated domains. It mirrors production controls, including access gates, scoped tools, guardrails, audit trails, and human handoff, making it a robust solution for AI support agents in telecom and subscription billing. ### GitHub Stars: 6 ### Repository: - **Repository**: patibandlavenkatamanideep/relayops - **Primary Language**: Python - **GitHub Stars**: 6 ### Live Demo: - **Demo**: [relayops-production.up.railway.app](https://relayops-production.up.railway.app) ### Additional Resources: - **

### Overview: FinChain is an AI-powered chat-driven expense tracker designed to help you manage your finances in any language without the need for explicit category selection. By simply typing your expenses in any language, FinChain automatically categorizes and analyzes them using an LLM-based NLP backend powered by Hugging Face Transformers. This innovative approach eliminates the need for clunky forms or manual categorization, making expense tracking more intuitive and efficient. ### Key features: - **Chat Your Expenses, Any Language:** Type "Spent 670 on burgers today", "বার্গার খাইসি ১০০০ টাকা দি