Projects

Overview: This project fine-tunes OpenAI's Whisper model on the LibriSpeech dataset, enabling improved speech recognition capabilities. Key features: - Fine-tuning of Whisper model - LibriSpeech dataset integration - Improved speech recognition accuracy Tech stack: Python Why it matters: This project enhances the performance of speech recognition systems, particularly in noisy environments, by leveraging the expertise of OpenAI and the comprehensive dataset of LibriSpeech.

Overview: This project explores the technique of lightweight fine-tuning, which is crucial for modifying foundation models without requiring significant computational resources. Key features: - Fine-tuning models - Parameter-efficient fine-tuning - Inference using a fine-tuned model Tech stack: Jupyter Notebook Why it matters: Lightweight fine-tuning is essential for adapting foundation models to specific needs without demanding substantial computational resources, making it a valuable technique for developers and researchers.

Overview: DomainPostTrain is a sample pipeline for domain-neutral LLM post-training, featuring CPT, Fact-SFT, DPO, GRPO, adapter merge, quality evaluation, GGUF/ONNX export, and OpenAI-compatible inference. Key features: - CPT document discovery and safety pre-check - Static mock CPT/SFT/DPO/GRPO/eval data - PEFT LoRA/QLoRA domain continuation training - Fact-SFT assistant-only loss - DPO preference training default enabled but optional - GRPO reward optimization default enabled but optional - Adapter merge, post-training quality evaluation, single inference, OpenAI-compatible Flask inference service - GGUF/ONNX export from merged model Tech stack: Python Why it matters: This pipeline is useful for developers looking to streamline the post-training process for LLMs, ensuring consistency and quality in the final model. Repository: EternalmBlue/DomainPostTrain GitHub stars: 5 GitHub description: Domain-neutral LLM post-training pipeline for CPT, Fact-SFT, optional DPO, adapter merge, quality evaluation, GGUF/ONNX export, and OpenAI-compatible inference.

### Overview: RAGognizer is an official repository for the paper "Hallucination-Aware Fine-Tuning via Detection Head Integration". It provides a clean, minimal setup for token-level hallucination detection and hallucination-aware fine-tuning. The project is designed to be accessible to researchers, builders, and curious users. ### Key Features: - Token-level hallucination detection - Hallucination-aware fine-tuning integration - A dataset of naturally occurring closed-domain hallucinations with token-level annotations - Models for fine-tuning and inference - Dataset pipeline for generating data - Fine-tuning capabilities - Generation and inference tools ### Tech Stack: Python ### Why it Matters: RAGognizer offers a practical solution for researchers and practitioners looking to enhance the quality and reliability of language models by addressing hallucination issues through integrated detection and fine-tuning. The project's emphasis on token-level detection and its integration with LLMs makes it a valuable resource for improving the output of large language models. ### Repository: F4biian/RAGognizer ### GitHub Stars: 5 ### GitHub Description: Official Repo of the Paper "Hallucination-Aware Fine-Tuning via Detection Head Integration" ### Primary Language: Python ### GitHub Description: Official Repo of the Paper "Hallucination-Aware Fine-Tuning via Detection Head Integration" ### Quick Links: - 🍃 Browser Demo: [Space: F4biian/RAGognizer](https://huggingface.co/spaces/F4biian/RAGognizer) - 🚀 Try it

Overview: FrameTrain is a desktop application designed for local AI model training and fine-tuning. It allows users to train and fine-tune large language models without needing a cloud service, using either Hugging Face models or local ones. The app features a visual neural network builder, dataset management, live training monitoring, and automatic model versioning. Key features: - Local AI model training and fine-tuning - LoRA and QLoRA fine-tuning of large language models - PyTorch-based training with GPU acceleration (NVIDIA CUDA and Apple Silicon / Metal MPS) - Visual neural-network builder - Dataset import and management - Live training monitoring and AI training analysis - Automatic model versioning and comparison Tech stack: Tauri 2 (Rust) for the desktop app, React 18, TypeScript, Vite, Tailwind CSS for the frontend, React Flow for the visual builder, PyTorch, Hugging Face, PEFT/LoRA, QLoRA, BitsAndBytes for the training core, Vitest, Testing Library for testing. Why it matters: FrameTrain is designed to be GDPR-compliant, ensuring that user data never leaves their local device. It offers a seamless, local training experience, making it ideal for users who prefer to train models without the need for cloud services. The app's features, including the visual neural network builder and live monitoring, make it a powerful tool for AI developers and enthusiasts. Repository: FrameSphere/FrameTrain-App GitHub stars: 5 GitHub description: App repo for Frame

Overview: This project fine-tunes a PaliGemma model for improved language generation capabilities. Key features: - Fine-tuning of PaliGemma model - Enhanced language generation quality Tech stack: Jupyter Notebook Why it matters: This project enhances the performance of PaliGemma, making it more effective for various language tasks.

### Overview: This project fine-tunes five small Language Models (LMs) to generate syntactically valid and schema-conformant JSON. It employs Group Relative Policy Optimization (GRPO) to fine-tune the models, using rule-based reward components instead of a neural reward model. The models were evaluated across four distinct training strategies, resulting in a tight 87-99% Pass@1 band after 2,500 training steps. ### Key Features: - **Rule-Based Reward Components**: Score each completion, providing a dense, deterministic additive signal. - **Four Training Strategies**: No-Think Standard, No-Think Curriculum, Think Standard, and Think Curriculum. - **Models**: SmolLM2-135M-Instruct, SmolLM2-360M-Instruct, Qwen2.5-0.5B-Instruct, TinyLlama-1.1B-Chat-v1.0, Gemma-2-2B-it. ### Tech Stack: - **Jupyter Notebook**: Primary language used for prototyping and experimentation. ### Why It Matters: - **Fine-Tuning of Small Models**: The project demonstrates that rule-based optimization can effectively fine-tune small LLMs for specific tasks, such as generating valid JSON. - **Rule-Based Reward System**: The framework uses a purely rule-based approach to reward, avoiding the overhead of a neural reward model, making it suitable for small models. - **Comprehensive Evaluation**: The project evaluates four distinct training strategies across 2,500 training steps,

### Overview: dgx-spark-ai is a project designed to run GPT-OSS 120B on a local NVIDIA DGX Spark system. It provides a simple way to fine-tune models and use Cline CLI for local AI tasks, offering a local control and offline-friendly environment. ### Key Features: - Run large language models on your own hardware. - Fine-tune models with your own data. - Use Cline CLI with a local setup. - GPU-aware setup for optimal performance. - Offline-friendly use without constant internet connection. ### Tech Stack: Shell-based project. ### Why It Matters: This project is ideal for individuals and teams looking for local control over their AI work, ensuring that their data and models remain on their machine without the need to send data to a cloud service.

Overview: WFT is a Python library designed to simplify the fine-tuning process of OpenAI's Whisper models, making it easy to fine-tune Whisper models on custom datasets without requiring code changes. Key features: - 🤗 Hugging Face Integration: Syncs with the Hugging Face HUB for organization settings. - 📄 Easy Dataset Preparation and Preprocessing: Quick dataset setup for fine-tuning. - 🛠️ Fine-Tuning Using LoRA: Efficiently fine-tunes Whisper models with low-rank adaptation techniques. - 🚀 Flexible Configuration Options: Customize various aspects of the fine-tuning process. - 📊 Evaluation Metrics: Supports Character Error Rate (CER) or Word Error Rate (WER) for model evaluation. - 📊 TensorBoard Logging: Tracks training progress in real-time. - 🤖 Automatic Model Merging and Saving: Automatically merges and saves the final model. - 🔄 Resume Training: Seamlessly resume training from interrupted runs. Tech stack: Python Why it matters: WFT simplifies the fine-tuning process of Whisper models, making it accessible to developers and researchers without requiring extensive coding knowledge. This library streamlines the workflow, allowing for faster development cycles and improved model performance.

Overview: This challenge is designed to fine-tune a pre-trained HuggingFace transformer model, specifically the Orca Mini 3B, on the evolved codealpaca dataset to create a Code Generation language model. Additionally, it involves building a retrieval-augmented generation (RAG) application using LangChain to answer questions on the Battle Line game rules. Key features: - Fine-tuning of Orca Mini 3B on the evolved codealpaca dataset. - Building a Code Generation language model. - Creating a RAG application using LangChain for answering Battle Line game rules questions. Tech stack: Jupyter Notebook Why it matters: This challenge provides a practical application of fine-tuning pre-trained models and integrating with LangChain for advanced text generation tasks, demonstrating the capabilities of these tools in real-world scenarios.

### Overview: LLM Router Server is a high-performance routing service designed for multi-model deployment scenarios, used to uniformly manage and orchestrate multiple local Large Language Model (LLM) services, Embedding models, Re-ranking models, and other inference services. ### Key Features: - **Unified Routing Management**: Integrates multiple independent vLLM services, Embedding services, and Reranker services. - **OpenAI-Compatible API**: Provides fully compatible OpenAI API interfaces (`/v1/chat/completions`, `/v1/completions`, `/v1/embeddings`). - **Configuration-Based Deployment**: Manages startup parameters, ports, GPU allocation, etc. through YAML configuration files. - **Multi-Model Parallelism**: Supports multiple model instances running simultaneously, each using independent processes and GPU resources. - **Intelligent Load Balancing**: Automatically selects the least-loaded instance based on real-time metrics (running requests, waiting requests, KV cache usage). - **High-Performance Forwarding**: Uses FastAPI + Gunicorn + Uvloop for high-performance asynchronous architecture. - **Streaming Response Optimization**: Optimizes streaming requests to ensure low latency and stable token output. ### Tech Stack: Python ### Why it Matters: LLM Router Server simplifies the deployment and management of multiple LLM and related models, ensuring efficient and reliable service delivery.

**Overview:** DeltaLoop is a continuous fine-tuning layer that automatically converts AI agent logs into LoRA adapters, eliminating the need for manual prompt rewriting and reducing prompt bloat by 80%. **Key features:** - Framework agnostic - Fully automated workflow - Lightweight LoRA adapters (~17MB) - Open source and cost-effective - Continuous improvement over time **Tech stack:** Python **Why it matters:** DeltaLoop simplifies the optimization process, reduces manual labor, and ensures that models learn from real-world interactions, leading to more effective and efficient AI agents.

### Overview: Hops Run is a low-poly racer game that utilizes decision models deployed on Hopsworks for real-time gameplay. The game's moves are determined by models deployed on Hopsworks, which provide quick predictions in under 30 milliseconds. ### Key Features: - **Hops Run Game**: A low-poly racing game available at [game.hopsworks.ai](https://game.hopsworks.ai). - **Decision Models**: Deployed on Hopsworks, these models provide real-time predictions for the game's moves. - **Track Tracks**: Two tracks available: classic and a live edition published daily by the designer. - **Leaderboards**: Players, model pilots, and bots have leaderboards based on their performance. - **Models Overview**: - **qwen**: Reads the situation in plain English and scores each move from the model's logits. - **kumo**: Learns from 78 labelled situations without training step. - **clef**: Answers as a SystemOne choice question with its joint schema head. ### Tech Stack: - **JavaScript**: The primary language used for the game. ### Why It Matters: Hops Run showcases the integration of AI models with a game, demonstrating how real-time predictions can enhance gameplay. The models deployed on Hopsworks provide immediate feedback, making the game more engaging and dynamic.

### Overview: This repository contains the Korean implementation of Large MultiModal Fine-Tuning (LMM) for the Ovis2-FFT model, focusing on Korean language processing tasks. The code is designed to fine-tune the model on private datasets and evaluate its performance using various benchmarks. ### Key Features: - **Fine-Tuning:** The model is fine-tuned on the MarkrAI WizardLM training dataset. - **Benchmarking:** The repository includes detailed benchmark results for both the Korean and English versions of the model. - **Code Structure:** The code is structured to be easily reusable and adaptable for different datasets and models. ### Tech Stack: - Python ### Why It Matters: This implementation is crucial for developers and researchers looking to improve Korean language processing capabilities using large language models. It provides a practical solution for fine-tuning models on private datasets and evaluating their performance against established benchmarks. ### Repository: - **Repository:** Marker-Inc-Korea/Ovis2-FFT-Korean - **Stars:** 5 - **Description:** Korean Large MultiModal Fine-Tuning Code - **Language:** Python ### Blog: - **Link:** [Gukbap-LMM blog🔥](https://kyujinpy.tistory.com/169) ### Result: - The model performs well across various benchmarks, including English VLM Evaluation and Korean VLM Evaluation. - The repository includes detailed benchmark results and a comparison with other models. ### Citation: - **VLMEvalKit:** [https://github.com/open-compass/VLMEvalKit](https://github

### Overview: This repository contains all the code and data referenced in the paper titled "Large Language Models for Scientific Question Answering: an Extensive Analysis of the SciQA Benchmark" authored by Jens Lehmann, Antonello Meloni, Enrico Motta, Francesco Osborne, Diego Reforgiato Recupero, Angelo Antonio Salatino, and Sahar Vahdati. ### Key features: - Large Language Models for Scientific Question Answering: an Extensive Analysis of the SciQA Benchmark - Models employed: T5-base, GPT-2-large, Dolly-v2-3b, GPT-3.5 Turbo - Structure of the repository: Contains all the code and data referenced in the paper - How to cite: The paper delves into the evaluation of large language models on the SciQA benchmark, focusing on scientific question answering over knowledge graphs. It discusses the dataset composition, including manually crafted and automatically generated questions, and explores the transformer architecture's role in semantic parsing and question answering tasks. A comparison between human-generated and automatically generated questions is presented, emphasizing the distribution across various question templates. ### Tech stack: Jupyter Notebook ### Why it matters: The paper provides an analysis of the performance of language models on the SciQA benchmark, which aims to represent a challenging task for next-generation question-answering systems. The study discusses the dataset composition, including manually crafted and automatically generated questions, and explores the transformer architecture's role in semantic parsing and question answering tasks. A comparison between human-generated and automatically generated questions is presented, emphasizing the

### Overview: nybbloris is a project that fine-tunes and serves NVFP4 models on a single DGX Spark, leveraging runtime LoRA and a generic fallback mechanism to ensure the adapter remains functional and validated across various target models and architectures. ### Key Features: - Fine-tune a 100B+ MoE on the 4-bit weights NVIDIA ships. - Train LoRA directly on the served 4-bit weights without the need for a BF16 base. - Serve runtime-LoRA on NVFP4, FP8, and bf16 models. - Proven end-to-end on Command-A, achieving a -28.8 -> -17.4, +11.4 nats improvement in base vs adapter. - Fine-tune the vision tower of a VLM and train the LLM backbone and tower jointly. - Use a generic fallback mechanism to validate and serve models from unregistered checkpoints. ### Tech Stack: Python ### Why it Matters: nybbloris provides a reliable and validated solution for fine-tuning and serving NVFP4 models on a single DGX Spark, ensuring the adapter remains functional and the model's performance is maintained across various configurations and architectures.

Overview: This repository is an OpenMOSS fork of SGLang, specifically designed to support the MOSS-TTS Family and MOSS-Audio, including models such as MOSS-Audio, MOSS-TTS (Delay), MOSS-SoundEffect, MOSS-TTSD v1.0, and MOSS-TTSD v0.7. Key features: - MOSS-Audio - MOSS-TTS (Delay) / MOSS-SoundEffect - MOSS-TTSD v1.0 - MOSS-TTSD v0.7 Tech stack: Python Why it matters: This repository provides comprehensive support for the MOSS-TTS Family and MOSS-Audio models, enhancing the functionality of SGLang for audio understanding and generation tasks. Repository: OpenMOSS/sglang GitHub stars: 5 GitHub description: An OpenMOSS fork of SGLang with serving support for the MOSS-TTS Family and MOSS-Audio.

### Overview: This project fine-tunes the Gemini model with a medical named entity recognition dataset to accurately identify and classify medical entities in text. ### Key Features: - Fine-tuning of the Gemini model for medical named entity recognition. - Integration of a medical dataset for improved accuracy. - Extraction of medical entities such as diseases, symptoms, drugs, and body parts. - Output format: Entities with their corresponding labels. ### Tech Stack: Jupyter Notebook ### Why it Matters: This project is crucial for healthcare professionals and researchers who need to extract structured information from unstructured medical text, enhancing their ability to analyze and understand patient records effectively.

Overview: This project involves finetuning the Pixtral-12B-2409 model using the unsloth package for visual Question Answering (NLP Task). Key features: Finetuning on the unsloth/llava-instruct-mix-vsft-mini dataset with the Pixtral-12B-2409 model. Tech stack: Jupyter Notebook Why it matters: This project aims to enhance the performance of visual Question Answering by fine-tuning a pre-trained model on a specific dataset, potentially improving accuracy and relevance in NLP tasks.

### Overview: SafetyPulse is a tool designed to detect safety drift during the fine-tuning of large language models (LLMs). It uses a Circuit Integrity Score (CIS) to monitor the model's refusal mechanism, which is cheaper and faster than a full red-team audit. ### Key Features: - **Circuit Integrity Score (CIS):** A score computed from 50 prompt pairs to gauge the model's refusal mechanism integrity. - **SafeTune Repair:** A repair mechanism that can be used to correct unsafe checkpoints identified by SafetyPulse. - **AuditKIT Audit:** An audit tool to verify the model's safety status. ### Tech Stack: Python 3.10+ with Jupyter Notebook ### Why It Matters: Early detection of safety drift is crucial for maintaining the integrity and safety of LLMs, preventing harmful outputs before they become measurable. SafetyPulse provides a cost-effective solution to ensure model safety during fine-tuning.