Projects

Overview: This lightweight voice companion is designed for macOS, powered by local LLM inference, speech-to-text, offline translations, and high-quality multilingual TTS. Key features: - Local LLM inference via [mlx-lm](https://github.com/ml-explore/mlx-lm) - Speech-to-text via [MLX Whisper](https://github.com/ml-explore/mlx-examples/tree/main/whisper) - Offline translations via [Unbabel/TowerInstruct-7B-v0.2](https://huggingface.co/Unbabel/TowerInstruct-7B-v0.2) - High-quality multilingual TTS (currently using XTTS v2) Tech stack: Python Why it matters: This project prioritizes simplicity, speed, and on-device privacy, making it ideal for Apple Silicon Macs. Repository: ethicalabs-ai/Kurtis-E1-MLX-Voice-Agent GitHub stars: 10 GitHub description: A lightweight voice companion, optimized for macOS. Primary language: Python GitHub description: A lightweight voice companion, optimized for macOS.

Overview: Fiorino.AI is an open-source solution designed to help SaaS owners track and optimize their AI costs. Key features: - Open Source - Flexible Integration - Detailed Tracking - Usage Limits - Custom Insights - AI Cost Analyst Tech stack: Python Why it matters: Fiorino.AI helps businesses monitor and optimize their AI spending, ensuring profitability and compliance with various LLM providers. Repository: fiorino-ai/fiorino-ai GitHub stars: 10

Overview: This paper introduces β-CFG, a dynamic guidance method for text-to-image diffusion models that adapts guidance strength using a β-distribution. It improves image quality, maintains prompt alignment, and achieves better FID scores compared to standard CFG. Key features: - β-CFG dynamically adjusts guidance strength over time. - It uses a β-distribution to balance prompt matching and image quality. - It stabilizes the effects of guiding during the diffusion process. Tech stack: Python Why it matters: β-CFG provides a flexible and principled way to enhance text-to-image generation in diffusion models, improving image quality and maintaining prompt alignment while maintaining similar CLIP scores to standard CFG.

Overview: ModelLoom is a distributed local LLM inference solution that leverages llama.cpp, GGUF, CUDA, and Vulkan for GPU-based inference. It provides an intuitive GUI for easy deployment and usage. Key features: - Distributed inference across multiple GPUs and PCs - Supports GGUF models - Easy setup with user and developer guides Tech stack: TypeScript Why it matters: ModelLoom simplifies the deployment and inference of large language models, making it accessible for developers and researchers alike. Repository: haowern98/ModelLoom Primary language: TypeScript GitHub stars: 8 GitHub description: Distributed local LLM inference across multiple GPUs and PCs using llama.cpp, GGUF, CUDA, and Vulkan, with an intuitive GUI. README excerpt:

Overview: This repository provides a hands-on local fine-tuning course for large language models (LLMs). It covers ten lessons, from datasets and tokenizers to SFT, LoRA, PEFT, DPO, and QLoRA, with a focus on practical application and browser training studio visualization. Key features: - Datasets pipeline - AutoTokenizer - Batch / Collator - Trainer闭环 - Handwriting LoRA - PEFT LoRA - SFT baseline - DPO - Reward / RLHF concepts - QLoRA engineering boundary Tech stack: Python, Jupyter Notebook, Flask, HTML, JavaScript, CSS, Markdown, and Jekyll. Why it matters: This course is designed for developers who want to understand and apply LLM fine-tuning in a practical way, focusing on the local training process and browser-based visualization of loss, tensor, and adapter. It is not intended for large-scale fine-tuning of 7B or 13B models and is suitable for those looking to build a foundation for subsequent projects like Agents, RAG, and model applications. GitHub stars: 9 Repository: harrisliangsu/llm-finetune-study Primary language: HTML Last commit: 2023-XX-XX License: MIT Read online: https://harrisliangsu.github.io/llm-finetune-study/ Support: https://afdian.com/a/sharkchiligaga ## In the repository: - [Online Reading](#online-

Overview: GTO is a framework designed to address draft policy misalignment in speculative decoding, providing faster and more efficient decoding compared to vanilla decoding and EAGLE-3. Key features: - Faster decoding speed - 7% faster than EAGLE-3 Tech stack: Python Why it matters: GTO offers significant improvements in performance for speculative decoding, making it a valuable tool for researchers and developers working on efficient language models.

Overview: This project is a research-grade implementation that fuses structured knowledge graphs with large language models to improve QA systems, focusing on reducing hallucination, improving faithfulness, and delivering auditable answers in medical domains. Key features: - KG-Augmented Tree-of-Thoughts (KG+ToT): Unifies symbolic KG paths with neural ToT exploration to boost interpretability and faithfulness. - Multi-Agent QA: Role-specialized agents (Specialist → Reviewer/Critic → Coordinator) enforce review-before-synthesis and safer decisions. - Evidence-grounded prompting (MindMap): Structures entities, relations, multi-hop KG paths, and supporting passages into first-class evidence in prompts. - Comprehensive evaluation: LLM ranking, BERTScore, and Self-Check/NLI for semantic alignment, contradiction, and factual support. Tech stack: Python Why it matters: This project provides a reproducible and engineering-ready solution for integrating structured knowledge graphs with large language models, making it suitable for both academic research and practitioner deployment in knowledge-intensive domains. Repository: ictup/Enhancing-QA-Systems-through-Integrated-Reasoning-over-Knowledge-Bases-and-Large-Language-Models GitHub stars: 10 GitHub description: The repository packages the methods of my thesis, *Enhancing Question Answering Systems through Integrated Reasoning over Knowledge Bases and Large Language Models*, into an engineering-ready implementation: modular code, scripts, notebooks, and guidance to adapt to your own domain KG. It is written to satisfy both academic reproduction and practitioner deployment.

### Overview: RAG Engine is a comprehensive RAG (Retrieval-Augmented Generation) template designed for developers. It includes FastAPI, LangChain, Chroma, MySQL, MinIO, and Next.js 14, providing a fully configured and integrated solution for knowledge retrieval and generation. ### Key Features: - **FastAPI + LangChain LCEL + Next.js 14**: The backend is built with FastAPI for RESTful APIs, LangChain for RAG pipelines, and Next.js for the frontend. - **MySQL + Chroma + MinIO**: The backend uses MySQL for business data, Chroma for vector storage, and MinIO for raw file storage. - **Model Configuration**: Configuration for LLMs and embeddings is stored in the database, allowing easy switching between different API providers. - **Multi-Source Retrieval**: Supports hybrid search with dense and BM25/RRF, FlashRank reranking, and SSE streaming. - **RAGAS Evaluation**: Provides RAGAS evaluation for quality assessment. - **JWT Authentication**: Secure JWT authentication for user management. ### Tech Stack: - **API Framework**: FastAPI + Pydantic v2 - **RAG Framework**: LangChain 1.x (LCEL) - **Vector Storage**: Chroma - **Relational Database**: MySQL + SQLAlchemy 2 + Alembic - **Object Storage**: MinIO - **Model Configuration**: Endpoint and keys stored in the database - **Frontend**: Next.js 14 + React 18 + Tailwind ###

Overview: This project demonstrates the fine-tuning of a pre-trained Llama 3.2 model for text generation using the Unsloth AI library. The notebook covers all necessary steps from setting up the environment to fine-tuning and generating text. Key features: - Fine-tuning of Llama 3.2 model - Text generation using the Unsloth AI library - Detailed instructions and explanations for each step Tech stack: Python, Hugging Face transformers, trl, datasets libraries Why it matters: This project provides a practical guide for developers to fine-tune models for text generation, enhancing their ability to generate coherent and contextually relevant text. Repository: jkanalakis/finetuning-llama-model-for-text-generation-using-unsloth Primary language: Jupyter Notebook GitHub stars: 10 GitHub description: Fine-tuning Llama 3.2 3B Instruct model for text generation using Unsloth AI

### Overview: DevIntel AI is an AI-powered developer productivity platform that connects GitHub repositories, indexes codebases, and integrates with an AI assistant using Retrieval Augmented Generation (RAG). It provides instant code insights, refactoring suggestions, and intelligent explanations. ### Key Features: - **Code Insights:** Instant code insights and refactoring suggestions. - **RAG Integration:** Retrieval Augmented Generation (RAG) for codebase questions. - **Code Health Metrics:** Calculates multi-dimensional code health metrics. - **Architecture Diagrams:** Generates AST-based architecture diagrams. - **Pull Request Reviews:** Automates pull request reviews. ### Tech Stack: Python 3.11, FastAPI 0.109, SQLAlchemy 2.0 (asyncpg), pgvector, Google GenAI SDK, OpenAI SDK. ### Why it Matters: DevIntel AI streamlines the development process by enhancing developer productivity and providing valuable insights into codebases, making it easier for developers to collaborate and improve their projects efficiently.

### Overview: This project aims to fine-tune a large language model (LLM) for the detection of code vulnerabilities. It uses the QLoRA method, which quantizes the model to 4-bit floats and incorporates LoRA (Low-Rank Adaptation) for fine-tuning. The repository contains the code and documentation necessary to run experiments and reproduce the results described in the original research paper. ### Key features: - Fine-tuning of a large language model for vulnerability detection. - Utilizes QLoRA method for quantization and LoRA fine-tuning. - Includes adaptation of upstream code for reproducibility. - Contains detailed documentation and experiment results. ### Tech stack: Python ### Why it matters: This project provides a practical application of fine-tuning large language models for vulnerability detection, demonstrating the effectiveness of the QLoRA method in reducing model size and improving performance.

### Overview: EduMIND is an enterprise-grade, highly modular bilingual lecture assistant and active learning pipeline designed for academic environments with mixed language content, such as Code-Mixed Vietnamese-English lectures. ### Key Features: - **Bilingual Audio Lecture**: Transcribes bilingual speech. - **VietMix Machine Translation**: Implements token-level language identification and computes the Code-Mixing Index (CMI). - **Anti-Forget RAG Engine**: Handles layout-aware PDF chunking and retrieves relevant information from Qdrant Vector Database. ### Tech Stack: Python ### Why it Matters: EduMIND is designed to handle the complexities of academic environments with mixed language content, ensuring accurate transcription, translation, and retrieval of relevant information. It leverages cutting-edge technologies like Whisper ASR, VietMix Translation, Anti-Forget RAG Engine, and Label Studio for active learning, making it an essential tool for modern educational environments.

### Overview: NeuralSymbolic_OmniIPS_Knowledge_Representation_Playground is a production-ready Neuro-Symbolic AI and GraphRAG-powered Multi-domain Expert System designed for automated Euclidean plane geometry theorem proving. It leverages FastAPI, Streamlit, Neo4j, Qdrant, Docker, and Astral UV for efficient inference and visualization. ### Key Features: - Pure First-Order Logic (FOL) forward-chaining deduction kernel ensuring all intermediate proof steps are mathematically sound. - Integrated computer algebra engine executing real-time angle chasing, segment proportion reduction, and metric height quadratic systems. - Automatically generates essential auxiliary points, perpendiculars, circumcircles, and midpoints using a LLM-powered auxiliary construction agent. - Semantic knowledge base consisting of 1,128 ontology nodes and 1,532 relationships over Neo4j Property Graph and Qdrant Cloud Vector Database. - Streamlit interface for natural language geometry input, live SSE proof streaming, and interactive step-by-step LaTeX proof tree exploration. - Solves problems across 4 difficulty tiers ranging from high-school entrance exams to Olympiad-tier theorems, achieving a 15/15 IMO Benchmark Passing Rate. ### Tech Stack: Python, FastAPI, Streamlit, Neo4j, Qdrant, Docker, Astral UV. ### Why it Matters: This system is designed to be enterprise-grade, state-of-the-art, and capable of solving complex geometry problems with high accuracy and efficiency. It excels in proving Euclidean plane geometry theore

Overview: This repository offers a comprehensive course on enhancing your skills in prompt engineering for vision models, focusing on techniques for models like SAM, OWL-ViT, and Stable Diffusion 2.0. Key features: - Image Generation - Image Segmentation - Object Detection - In-painting - Fine-tuning with DreamBooth - Experiment Tracking with Comet Tech stack: Jupyter Notebook Why it matters: This course is essential for developers looking to improve their ability to generate, segment, and detect objects in images using vision models, providing practical techniques and tools for effective prompt engineering.

### Overview Directional Textual Inversion (DTI) addresses the issue of embedding norm inflation in pre-norm Transformers, which is a common problem in Textual Inversion (TI). DTI decouples magnitude and direction, fixes embedding magnitude to an in-distribution scale, and optimizes only the direction on the unit hypersphere using Riemannian SGD. This approach leads to superior text fidelity and subject preservation on models like Stable Diffusion XL (SDXL), SANA, and Wan2.1-T2V-1.3B. ### Key Features 1. **Decoupling Magnitude and Direction**: DTI separates the magnitude and direction of the learned tokens, ensuring that the magnitude remains within the expected range. 2. **Spherical Optimization**: It optimizes only the direction on the unit hypersphere, avoiding the drift to out-of-distribution magnitudes. 3. **vMF Prior**: A von Mises-Fisher prior is used to maintain semantic coherence and improve the quality of the generated images. ### Tech Stack - **GitHub Repository**: `kunheek/dti` - **Primary Language**: Jupyter Notebook - **GitHub Stars**: 10 ### Why It Matters DTI improves the fidelity and subject preservation of text-to-image generation models, making them more effective and reliable for applications like personalized text-to-image conversion. By addressing the limitations of standard TI, DTI offers a significant enhancement in the quality of the generated images and text. ### Repository - **Repository**: `kunheek/dti` - **

### Overview: Domicile is a private AI stack that provides a vector database, RAG, and local language model (LLM) solution entirely running in the browser. It ensures zero egress, zero third-party processors, and zero cloud costs, making it ideal for on-prem privacy and data custody. ### Key Features: - **Vector Database**: Persistent client-side storage with a pure-TypeScript HNSW graph index, holding up to 100K documents with cosine, euclidean, or dot metrics. - **Local Embeddings**: Transformers.js with WebGPU acceleration and a WASM fallback, caching model weights on-device for fast repeat queries. - **Dual LLM Runtime**: WebLLM for GPU-fast inference and Wllama for CPU portability, automatic fallback for generation. - **RAG with Citations**: Answers grounded in source documents, with citations linking every claim back to the document that grounded it, ensuring auditable and non-black-box results. - **MCP Integration**: Exposes the custody layer as Model Context Protocol tools, allowing integration into Claude Desktop and team-built agent stacks. - **Resident by Design**: Data domiciled on the device, ensuring no cloud egress, no third-party processors, and no residency drift. ### Tech Stack: TypeScript ### Why It Matters: - **Private**: Data never leaves the device, ensuring zero egress and no third-party processors. - **Fast**: WebGPU acceleration with WASM SIMD fallback for fast queries. - **Free**: Zero cloud costs, no API keys, no per-seat fees

### Overview: This repository contains samples for fine-tuning HuggingFace models with Azure Machine Learning (AzureML). The samples are designed to be easily adapted for various NLP tasks and can be fine-tuned for other applications as well. ### Key features: - AzureML v2 CLI examples for fine-tuning HuggingFace models - Quickstart ARM Templates for fine-tuning HuggingFace models - Fine-tuned HuggingFace models and results available at [huggingface.co/linydub](https://huggingface.co/linydub) ### Tech stack: - AzureML v2 CLI - ARM Templates - HuggingFace models ### Why it matters: These samples provide a practical way to fine-tune HuggingFace models using AzureML, making it easier to adapt the models for different NLP tasks or product scenarios. The samples also include integrations with MLflow and CodeCarbon, which help in benchmarking and carbon accounting, respectively. ### Repository: The repository is hosted on GitHub and has been starred by 10 developers. The project is currently in the preview stage and aims to add more fine-tuned models and benchmark results in the future.

Overview: This is the official repository of "HoliSDiP: Image Super-Resolution via Holistic Semantics and Diffusion Prior". The project aims to address the issues of pre-trained text-to-image diffusion models in Real-ISR, providing improved image quality through semantic guidance. Key features: - Semantic segmentation as text prompts - Dense semantic guidance through segmentation masks - Segmentation-CLIP Map for enhanced spatial control Tech stack: Python Why it matters: By leveraging semantic labels for text prompts and incorporating segmentation masks, HoliSDiP significantly enhances the quality of image super-resolution results, especially in terms of reduced prompt noise and improved spatial control.

**Overview:** This repository contains the ATC '25 Artifact, which is a resource multiplexing approach for tuning and serving large language models. The project aims to optimize the performance of LLMStation, a customized version of PyTorch and vLLM, by leveraging multiple GPUs. **Key features:** - Resource multiplexing approach for LLMStation - Customized PyTorch 2.4.0 and vLLM 0.6.3 - Baselines using vanilla PyTorch 2.4.0 and vLLM 0.6.3 - Installation scripts for Ubuntu 22.04, Conda, CUDA 12.6, cmake 3.26, gcc-13.3.0, and g++-13.3.0 - Hardware requirements: Running these scripts requires 2 GPUs, each with 24 GB HBM **Tech stack:** Python **Why it matters:** This project provides a comprehensive solution for optimizing the performance of large language models, specifically LLMStation, by leveraging resource multiplexing techniques. The detailed installation scripts and hardware requirements ensure that users can effectively run and benchmark the project's performance on their systems. **Repository:** llm-db/llmstation **Primary language:** Python **GitHub stars:** 10

Overview: This repository contains hands-on implementations, research notebooks, and production recipes for fine-tuning Large Language Models (LLMs) and Vision-Language Models (VLMs). It includes examples for supervised fine-tuning, reinforcement learning, preference alignment, dense retrieval, and autonomous multi-agent systems. Key features: - Domain QA factual injection via Wikipedia scraping & QLoRA - Financial sentiment classification with Chain-of-Thought reasoning - Cardiology SFT with Google's Tunix library & JAX/XLA GPU acceleration - Clinical dialogue adaptation with token-level perplexity validation - Multimodal VLM adaptation for handwritten formula LaTeX OCR - Direct Preference Optimization (DPO) pairwise alignment - Mathematical reasoning RL via Group Relative Policy Optimization - Structured tool use and ChatML JSON function calling on Gemma 3 270M - Dense embedding model adaptation with synthetic queries & MNRL loss - Autonomous multi-agent coding system with dynamic LoRA swapping (Unsloth) and (TRL & PEFT) Tech stack: Jupyter Notebook Why it matters: This repository provides a comprehensive set of examples and recipes for fine-tuning LLMs and VLMs, making it valuable for developers looking to enhance their skills in this area.