Projects

### Overview: This project implements Reinforcement Learning from Human Feedback (RLHF) on custom data, replicating the pipeline described in the paper "Learning to Summarize from Human Feedback" by OpenAI. The RLHF process involves three key steps: Supervised Fine-Tuning (SFT), Reward Model Training, and Policy Optimization, enabling the model to generate summaries that align with human preferences. ### Key Features: - Supervised Fine-Tuning (SFT) - Reward Model Training - Policy Optimization ### Tech Stack: Jupyter Notebook ### Why It Matters: Traditional supervised learning for summarization relies on human-labeled datasets, but RLHF addresses the limitations of this approach by using a reward model to capture human preferences, fine-tuning the model with reinforcement learning, and dynamically adapting to evolving human feedback. This method ensures the model generates responses that are not only accurate but also highly aligned with what users value, making it suitable for custom domains and handling ambiguous or subjective tasks. ### Repository: ranzeet013/RLHF-CustomData ### GitHub Stars: 6 ### GitHub Description: Building an LLM with RLHF involves fine-tuning using human-labeled preferences. Based on Learning to Summarize from Human Feedback, it uses supervised learning, reward modeling, and PPO to improve response quality and alignment.

Overview: This project is a sample implementation of FastAPI serving a PyTorch model using micro batching. It demonstrates the use of Service Streamer for efficient model serving. Key features: - Micro batching for model serving - FastAPI integration - PyTorch model with micro batch processing - Service Streamer for deployment Tech stack: Python, FastAPI, PyTorch, Service Streamer Why it matters: This project showcases how to efficiently deploy and serve PyTorch models using FastAPI and micro batching, highlighting the benefits of using micro batching for better performance and resource utilization.

### Overview: tunelab is a Claude Code plugin designed to fine-tune, distill, and evaluate Large Language Models (LLMs) locally. It helps developers decide whether fine-tuning is necessary, distills LLM logs into small local models using MLX/LoRA, and verifies the models' performance on held-out data before deployment. ### Key Features: - **Fine-Tuning Locally**: Tunelab fine-tunes LLMs on local models using MLX/LoRA, ensuring the models are more accurate and efficient. - **Distillation of Logs**: It distills LLM logs into small local models, making it easier to fine-tune. - **Accuracy Verification**: Before deploying, it verifies the models' performance on held-out data, ensuring the best model is chosen. - **Versatile Use Cases**: Suitable for various tasks such as classification, extraction, and more. ### Tech Stack: Python ### Why It Matters: - **Accuracy and Efficiency**: Tunelab ensures that the models are more accurate and efficient, reducing the cost of fine-tuning. - **Versatility**: It can be used with Claude Code, Gemini CLI, or any other tool that reads skills, making it highly versatile. - **Cost-Effective**: It provides a cost-effective solution by fine-tuning models locally, reducing the need for expensive fine-tuning on the cloud. ### Repository: rchaz/tunelab ### GitHub Stars: 6 ### GitHub Description: Claude Code plugin for LLM fine-tuning, distillation, and evaluation —

Overview: CTF_Agents is a multi-agent framework designed for authorized Capture The Flag work, supporting various security challenges across different domains. Key features: - Cryptography - Quantum protocols - Adversarial AI/ML - Industrial control and operational technology - Reverse engineering - Web challenges - Pwn challenges - Hardware challenges - Forensics - Blockchain challenges - Secure coding challenges - Networking challenges - OSINT challenges - Log-analysis challenges - General coding challenges Tech stack: Python Why it matters: CTF_Agents provides a comprehensive framework for authorized CTF work, supporting a wide range of security challenges across different domains. The project's ability to route challenges to the appropriate specialist and execute bounded tools and playbooks makes it highly versatile and useful for both training and real-world scenarios. Repository: rmjohnson12/CTF_Agents Primary language: Python GitHub stars: 6 GitHub description: Agentic CTF security tooling project with natural language support README excerpt: # CTF_Agents CTF_Agents is a multi-agent framework for authorized Capture The Flag work. It classifies a challenge, selects a specialist, executes bounded tools and playbooks, records evidence, and reports either a verified result or a concrete failure reason. The project supports cryptography, quantum protocols, adversarial AI/ML, industrial control and operational technology, reverse engineering, web, pwn, hardware, forensics, blockchain, secure-coding, networking, OSINT, log-analysis, and

## Overview: This project fine-tunes OpenAI’s primary models to produce JSON-formatted output, enhancing cost-efficiency and data consistency for applications that utilize the OpenAI API. ## Key features: - Fine-tuning of OpenAI models - JSON output format - Node.js implementation ## Tech stack: JavaScript (Node.js) ## Why it matters: Improves efficiency and consistency in applications that use the OpenAI API, reducing costs and ensuring data quality.

Overview: This repository contains a comprehensive collection of ML-related active learning datasets, including algorithms, Jupyter Notebook (.ipynb) pipelines, Python (.py) scripts, and curated, ethically aligned synthetic data. The repository is structured into three main modules for easy understanding, reproducibility, and experimental training. Key features: - Synthetic Datasets: Ethically aligned synthetic datasets for active learning experiments. - Inference Generation Scripts: Python scripts for generating synthetic datasets and executing active learning runs. - Raw Reasoning (Jupyter Notebooks): Detailed Jupyter Notebook files for experimental active learning setups, providing context and full visibility into the reasoning and decisions made. Tech stack: Jupyter Notebook Why it matters: This repository serves as a valuable resource for researchers and practitioners in the field of active learning, offering a structured and transparent way to access and experiment with ML datasets and active learning algorithms.

Overview: Formless is a proof-of-concept application designed to simplify client onboarding by utilizing AI prompts and a Rails application. It eliminates the need for extensive form filling, making the process more efficient and user-friendly. Key features: - AI-driven client onboarding process - User prompts for structured responses - Secure and privacy-focused data handling - Rails-based application for client object creation and user request management Tech stack: Ruby on Rails, OpenAI API, Rails Caching, Sidekiq, Octokit, Devise, Kaminari, RSpec, Factorybot-rails, Ruby-openai Why it matters: Formless addresses the inefficiencies of traditional client onboarding processes by automating the initial data entry, reducing the burden on users and streamlining the onboarding experience.

Overview: This repository contains two independent crates, `bonsai-pot` and `bonsai2-pot`, designed to run from-scratch, dependency-light inference engines for the Bonsai model family. These crates utilize wgpu compute shaders to achieve fast performance on consumer GPUs, with realistic throughput numbers. The repository is structured with a shared workspace manifest and includes benchmarks, architecture notes, and usage details. Key features: - No dequantization of weights, directly consuming packed weights for matvec/matmul kernels. - Fast performance on consumer GPUs with realistic throughput. - Small, embeddable library with tokenization kept separate from Rust crates. - Tokenization is handled by `scripts/bpe.py` in each crate and can be piped into CLI. Tech stack: Rust + wgpu. Why it matters: This repository provides a lightweight, fast inference engine for the Bonsai model family, suitable for consumer GPUs, and includes detailed benchmarks and architecture notes for optimal performance. Repository: ruihe774/bonsai-pot GitHub stars: 6 GitHub description: A from-scratch, dependency-light, dequant-free Bonsai (Qwen3-architecture) Q1_0 inference engine running on wgpu compute shaders

Overview: This script provides a framework for miners to fine-tune models using the dataset generated by validators in the Bittensor ecosystem. Key features: - Dataset generated by validators - Models used by miners - Miners train models on their own compute (GPU) - Model can be used as the base for Bittensor miners Tech stack: Python Why it matters: This script offers a practical solution for miners to fine-tune models using the validated dataset, potentially improving the performance of their Bittensor miners. Repository: salahawk/bittensor-model-finetune Primary language: Python GitHub stars: 6 GitHub description: Model Fine-tuning for Bittensor miners using the dataset generated by validators

Overview: This repository contains an implementation of YOLOv8 trained to detect fire, using the YOLO algorithm for real-time object detection. Key features: - Fire detection using YOLOv8 - Real-time object detection - Video stream processing Tech stack: Python Why it matters: This project showcases the application of YOLOv8 for real-time fire detection, which is crucial in ensuring safety and security in various environments.

Overview: This project aims to develop a streamlit application that utilizes deep learning neural networks to classify diseases in chicken fecal images. The application will help detect diseases such as Salmonella, Newcastle, and Coccidiosis, which are common in poultry farming systems, and contribute to early detection and control of diseases, thereby improving farm productivity and reducing mortality rates. Key features: - Streamlit application for real-time disease classification - Convolutional Neural Network (CNN) model for image classification - Transfer Learning and Fine-tuning techniques for improved accuracy - Image augmentation for balanced data distribution Tech stack: Python (Jupyter Notebook), TensorFlow, Keras, Streamlit Why it matters: This project is crucial for enhancing the efficiency and productivity of small-scale poultry farming in Thailand, particularly in regions like Arusha and Kilimanjaro in Tanzania. By providing an early detection system, it helps prevent the spread of diseases and ensures better quality control, ultimately benefiting the Thai poultry industry and the economy as a whole.

### Overview: This project demonstrates how to fine-tune the PaliGemma model for image captioning, a model developed by Google Research designed to handle images and generate corresponding captions. ### Key features: - Fine-tuning of PaliGemma model - Customizable dataset for training and validation - Streamlit interface for uploading images and generating captions ### Tech stack: Jupyter Notebook ### Why it matters: This project provides a practical guide for developers interested in image captioning and fine-tuning machine learning models, offering a step-by-step approach to training and using the PaliGemma model effectively.

Overview: RiskDelta is a multi-tenant AI runtime control plane designed for production LLM systems, AI agents, copilots, and enterprise AI applications. It scores risky model behavior, enforces policies, blocks unsafe tool calls, requires human approval, preserves evidence, and creates traceability across users, models, agents, and tools. Key features: - Risk scoring - Policy enforcement - Runtime guardrails - Human approvals - Audit trails - TraceVault evidence preservation Tech stack: TypeScript Why it matters: RiskDelta provides a comprehensive solution for the execution layer of AI, ensuring secure, auditable, and controllable AI systems.

### Overview: Loom is a distributed expert cache system designed for large models like Qwen, enabling efficient inference on ordinary machines. It uses a static Zig binary with Metal/Vulkan backends, serving an OpenAI-compatible API. ### Key features: - **Distributed Expert Cache**: Stores only the necessary weights, reducing the need for large model checkpoints. - **Local Compute**: All computation stays local to each node, reducing network traffic. - **Gossip Discovery**: Nodes discover each other using a gossip protocol. - **Weight Repair**: Repairs weights in case of network churn. - **Merkle Manifest**: Verifies weights against a Merkle manifest for security. ### Tech stack: Zig 0.16.0 with Metal/Vulkan backends. ### Why it matters: Loom simplifies the management of large models, making them accessible and efficient on standard hardware. It reduces the need for expensive GPU resources and minimizes network traffic, making it ideal for distributed computing environments. ### Repository: svayatta/loom ### GitHub stars: 6 ### GitHub description: Distributed expert cache for large MoE inference: swarm-shared GGUF weights, node-local compute, gossip discovery, OpenAI-compatible API. One static Zig binary with Metal/Vulkan backends.

### Overview: This intensive three-day training course combines theory with extensive hands-on practice to teach participants how to build production-ready generative AI applications using state-of-the-art models and techniques. ### Key features: - **Hands-on Labs:** 32 hands-on notebooks covering all aspects of AI foundations, deep learning, natural language processing, large language models, prompt engineering, fine-tuning, retrieval augmented generation, and optimization. - **Modern Topics:** Includes modern topics such as diffusion models, multimodal AI, retrieval augmented generation (RAG), AI agents, and production optimization. - **Practical Applications:** Designed for developers, data engineers/analysts, and tech product owners, focusing on practical skills for building AI applications. ### Tech stack: Jupyter Notebook ### Why it matters: This course equips participants with the skills and knowledge necessary to build and deploy production-ready generative AI applications, making them valuable assets in the tech industry.

### Overview: Sentient Core is a terminal-based interface designed for interacting with AI characters using LLMs, aiming to provide a locally processed AI experience. ### Key Features: - GGUF Llama models or any model supported by Llama.cpp. - GPU layer offloading for accelerated local text generation. - Optionally use koboldcpp as a backend for text generation. - Streaming-mode for text inference. ### Tech Stack: Rust ### Why it Matters: Sentient Core offers a unique and user-friendly interface for interacting with AI characters, leveraging advanced LLMs for a locally processed experience.

Overview: This is a comprehensive AI Engineer roadmap that covers 113 beginner-friendly topics on RAG, LLMs, AI agents, frameworks, and production AI. It includes runnable Python code, diagrams, and interview questions. Key features: - 113 topics - Python 3.10 or newer - Diagrams and code examples - 3 interview questions per topic - Structured learning path Tech stack: Python Why it matters: This roadmap provides a clear, structured approach for beginners and developers moving into AI engineering, ensuring they can understand and apply these concepts effectively.

Overview: LLMJson is a C# library designed to facilitate the sharing and interpretation of JSON models with Large Language Models (LLMs). It aims to address the limitations of LLMs in generating well-formed JSON, making it easier for developers to integrate LLMs into their applications. Key features: - Serializes and generates descriptions for JSON values. - Handles parsing of up to 2GB of JSON files. - Supports parsing of abstract classes and interfaces. - Provides robust handling of field values, including non-standard formats. - Offers special property wrappers for runtime manipulation of fields. Tech stack: C# Why it matters: LLMJson simplifies the process of integrating LLMs into applications by providing a way to serialize and interpret JSON models, making it easier for developers to leverage the capabilities of LLMs in their projects.

Overview: `tensorcore` is a software layer designed to bridge the gap between Metal GPUs and traditional CUDA hardware, enabling efficient tensor operations on Apple Silicon. It provides a CUDA-equivalent library that simplifies the development of AI models. Key features: - `tc_gemm` for matrix operations - `tc_attention_*` for attention mechanisms - `tc_conv2d_*` for convolutional layers - `tc_rmsnorm / RoPE` for normalization - `tc_swiglu / softmax / AdamW` for activation functions - `tc_gemv_quantized` for quantized inference - `tc_gguf_*` for GGUF reader - `tc_allreduce / broadcast / allgather` for communication primitives Tech stack: Python Why it matters: By providing a seamless integration between Metal GPUs and traditional CUDA hardware, `tensorcore` enables teams to leverage the full potential of Apple Silicon for training large AI models, reducing the need for expensive hardware upgrades and improving overall efficiency. Repository: tsotchke/tensorcore GitHub stars: 6 GitHub description: CUDA-equivalent tensor-core acceleration for Apple Silicon. C-ABI kernel library wrapping simdgroup_matrix (M1+) and mpp::tensor_ops (M5+): GEMM, FlashAttention, Conv2D, Q4_0/Q8_0 quantized inference, GGUF reader, full transformer training kernels. One binary, M1 → M5.

Overview: This project fine-tunes the Tesseract Japanese vertical (jpn_vert) model with 16 types of Japanese fonts to achieve a BCER (Best Character Error Rate) of 1% while improving text segmentation performance. Key features: - Fine-tuned Tesseract jpn_vert model with 16 Japanese fonts - Achieved BCER of 1% after fine-tuning - Improved text segmentation performance Tech stack: Jupyter Notebook Why it matters: This fine-tuning process enhances the accuracy and segmentation capabilities of the Tesseract Japanese vertical model, making it more suitable for Japanese vertical text recognition tasks.