Projects

### Overview: helmstudio is a local launcher and platform for open-weight generative models. It allows users to install, build, and run video, image, and speech models from their own repositories. The system keeps everything in one gallery, ensuring consistency across different instances. ### Key features: - **Studios Stay Independent Repositories:** Helmstudio does not vendor, fork, or patch any of the studio code. A studio remains fully usable by anyone who clones it and ignores helmstudio. - **Daemon is Never a Model Runtime:** The daemon fetches, builds, downloads, supervises, stores, and serves everything needed for the model, without understanding the model format itself. This ensures that the model runs independently of the daemon. - **Studio Gallery and Timeline:** Everything created by a studio is stored in a single gallery, making it easy to keep track of the work. ### Tech stack: Go ### Why it matters: helmstudio simplifies the process of using open-weight generative models by providing a local launcher and platform. It ensures that the models stay independent repositories and that the daemon is never a model runtime, making it easier to manage and use these models.

### Overview: Physics-Constrained Fine-Tuning (PCFT) is a method for fine-tuning flow-matching generative models to enforce physical constraints and solve inverse problems in scientific systems. This approach leverages a pretrained flow-matching model to enforce physical consistency and recover unknown parameters in a data-driven yet physics-aware manner. ### Key Features: - **Post-training enforcement of physical constraints:** Fine-tunes a pretrained flow-matching model by tilting the generative distribution toward PDE-consistent samples using weak-form PDE residuals, improving physical validity while preserving diversity. - **Adjoint-Matching fine-tuning with theoretical grounding:** Builds on Adjoint Matching to cast reward-based fine-tuning as a stochastic optimal control problem, enabling stable, low-variance fine-tuning for physics-based rewards. - **Joint generation for inverse problems:** Introduces a joint evolution of states and latent physical parameters via an inverse predictor and an auxiliary parameter flow, enabling generation of solution–parameter pairs without requiring paired training data. ### Tech Stack: Python ### Why It Matters: By bridging generative modeling and scientific inference, PCFT opens new avenues for simulation-augmented discovery and data-efficient modeling of physical systems. The method effectively addresses ill-posed inverse problems in a datadriven yet physics-aware manner, demonstrating improved satisfaction of physical constraints and accurate recovery of latent coefficients.

Overview: A vector search SQLite extension developed by VSAG, designed to enhance database performance and search capabilities. Key features: - Vector search indexing - Efficient query execution - Integration with SQLite database Tech stack: Rust Why it matters: This extension leverages Rust's performance and SQLite's database capabilities to provide a powerful and efficient vector search solution, making it ideal for applications requiring fast and accurate search functionalities.

Overview: zLLM is a lightweight, high-performance LLM inference engine specifically designed for macOS Apple Silicon, utilizing Zig 0.16.0 and Metal for optimal performance. Key features: - Native Metal Acceleration: Custom Metal Shading Language (MSL) kernels for optimized tensor operations. - Zig 0.16 "Juicy Main": Leveraging the latest Zig standard library features. - Zero-Copy Architecture: Maps GGUF model files directly into Metal buffers for minimal CPU-GPU data transfer overhead. - GGUF Support: Pure Zig parser for the GGUF file format. - Quantization: Built-in support for GGUF quantization formats like Q4_K and Q8_0. - Zero Dependencies: Built entirely with Zig and macOS system frameworks (Foundation, Metal, QuartzCore). Tech stack: Zig 0.16.0 and Metal. Why it matters: zLLM offers a fast, efficient, and lightweight solution for LLM inference, leveraging Zig's advanced features and Metal's performance to deliver high-quality results without the need for heavy external libraries. Repository: jiacai2050/zllm GitHub stars: 7 GitHub description: A lightweight, high-performance LLM inference engine for macOS Apple Silicon, built with Zig 0.16.0 and Metal.

### Overview: This repository contains the slides and code for a PyCon Colombia 2024 talk titled "Embeddings are all you need: Artificial Intelligence for text, audio, and images." It explores the use of embeddings in developing applications using AI techniques such as RAG, LLMs, ViT, and PANNs. ### Key Features: - **Embeddings for Text, Audio, and Images**: Represent words, phrases, and documents as vectors for semantic meaning. - **Chatbot Implementation**: Uses RAG architecture with LLMs, embeddings, vector databases, and prompt engineering to build a chatbot for PyCon Colombia 2024. - **Reverse Image Search**: Utilizes ViT models, embeddings, and vector databases to find similar products in a fashion store. - **Music Recommender**: Implements a music recommender using PANNs models, embeddings, and vector databases to find similar songs to favorite tracks. ### Tech Stack: - Python ### Why it Matters: This talk and repository demonstrate practical applications of embeddings in AI, showcasing how they can be used to develop applications in text, audio, and image processing, making it valuable for developers looking to incorporate AI into their projects.

### Overview: vLLM-Fit is a CLI tool designed to simplify the recommendation and profiling of vLLM engine arguments for any HuggingFace model on a user's current hardware. It provides static estimation, dynamic profiling, multi-GPU support, and free-memory awareness. ### Key features: - **Static Estimation**: Provides instant recommendations based on the model's configuration and checkpoint size. - **Dynamic Profiling**: Launches vLLM with candidate settings to confirm they start, shrinking only on memory failures. - **Multi-GPU Support**: Sizes for NVIDIA GPUs, Apple GPUs, and CPU backend when no accelerator is present. - **Free-Memory Aware**: Capped at GPU memory utilization to ensure vLLM's startup check passes even when other processes hold GPU memory. ### Tech stack: Python ### Why it matters: vLLM-Fit simplifies the process of finding optimal vLLM engine arguments, ensuring efficient resource utilization and minimal risk of failures.

### Overview: Passive Aggressive Clippy is a small project that aims to help you fine-tune your corporate email messages with gentle feedback. ### Key features: - Clippy-like assistance for proofreading emails - Friendly, yet slightly stern, feedback - Vue 3 + TypeScript implementation ### Tech stack: Vue 3 + TypeScript ### Why it matters: This project serves as a tribute to the thoughtful and sometimes slightly harsh guidance provided by Clippy, making it easier for users to maintain a friendly tone in their emails.

Overview: SecForge is a Python toolkit designed to fine-tune a security-focused coding language model using QLoRA, enabling users to understand the entire pipeline from data preparation to serving locally without cloud lock-in. Key features: - Fine-tuning of a security-focused coding language model using QLoRA - Clean, documented pipeline from data preparation to serving - Supports interactive chat and evaluation - Runs on a 4 GB VRAM laptop and free Colab T4 Tech stack: Python Why it matters: SecForge simplifies the process of fine-tuning and deploying a security-focused language model, providing a clear and understandable pipeline for users who want to understand the entire process rather than just relying on a cloud-based API.

### Overview Cookidoo Agent is a Python application that integrates OpenAI, Milvus vector database, and the Cookidoo API to fetch, process, and search for recipes based on natural language queries. This project is designed with modern Python best practices, including asynchronous programming, modular architecture, and type hinting, to deliver an impressive and efficient solution. ### Key Features - **Asynchronous Operations:** Utilizes `asyncio` and `aiohttp` for efficient network operations. - **Advanced API Integrations:** Leverages OpenAI for both text embeddings and chat-based completions. - **Vector Search:** Uses Milvus for fast, scalable similarity searches on recipe embeddings. - **Modular Codebase:** Clear separation of concerns with well-structured packages and modules. - **Comprehensive Logging & Error Handling:** Centralized configuration ensures clear diagnostics. ### Tech Stack - Python 3.8+ - OpenAI API - Milvus vector database - Cookidoo API ### Why It Matters This application is ideal for demonstrating advanced Python programming, asynchronous I/O, and integration of multiple external APIs and services.

### Overview: This repository documents the results of a project aimed at understanding which model architectures are optimized for ROCmFP4 4-bit quantization on AMD Strix Halo hardware, including measured benchmarks and a collection of 154 ready-to-run GGUF models. ### Key Features: - **Model Porting and Benchmarks**: The repository details the porting of various model architectures to ROCmFP4 format on AMD Strix Halo hardware, including their performance metrics. - **Benchmarks and Models**: It includes measured benchmarks and 154 ready-to-run GGUF models, providing a comprehensive resource for developers. - **Build Recipes**: Detailed build recipes and serving instructions are provided to ensure smooth integration into existing systems. ### Tech Stack: Python ### Why It Matters: This project is crucial for developers looking to optimize their models for specific hardware, ensuring better performance and resource efficiency. The detailed documentation and benchmarks provide a solid foundation for both theoretical and practical applications.

Overview: Nano Banana API (`gemini-2.5-flash-image-preview`) is a fast, OpenAI-compatible endpoint that allows developers to generate images and edit reference images, all at a cost of $0.0125 per image. It supports multiple models and resolutions, and the API key can be obtained through the provided link. Key features: - Supports multiple models and resolutions - Charges $0.0125 per image - Provides a free tier with a minimum of $1 - Asynchronous by design, no need for retries or special integrations Tech stack: Python Why it matters: Nano Banana API offers a convenient and cost-effective way for developers to integrate image generation and editing into their applications, leveraging the power of OpenAI-compatible models without the need for a monthly subscription or prepaid plans.

Overview: Seedance 2.0 Prompts is a comprehensive resource for developers, offering tested image prompt recipes, including 8 examples with detailed renders, aspect ratios, and the model ID (`seedance-2.0`). The cost per image is provided, making it accessible and efficient. Key features: - 8 tested recipes - Detailed renders and example prompts - Model ID: `seedance-2.0` - Effective price: $0.066 per second - Estimated cost for the set: $0.264 Tech stack: Python Why it matters: Seedance 2.0 Prompts provides a reliable and cost-effective way to generate high-quality images, making it ideal for developers looking to streamline their workflows. Repository: kqo0u3/seedance-2-0-prompts-es Primary language: Python GitHub stars: 7 GitHub description: Dreamina — Seedance 2.0 prompts: tested image prompt recipes with example renders, aspect ratios, the model id (`seedance-2.0`) and the cost per image.

Overview: Edgewise is a TypeScript-based package that enables developers to run language, vision, speech, judging, embedding, image, and forecasting models on-device, seamlessly integrating with browsers, Bun, and Node via WebGPU and ONNX Runtime. It ensures that the same code runs across browsers, Bun, and Node, and provides detailed information on the models' capabilities and runtimes. Key features: - Runs language, vision, speech, judging, embedding, image, and forecasting models on-device. - Supports WebGPU and WebAssembly for browser integration. - Compatible with Bun and Node via ONNX Runtime. - Pinpoints and verifies models to ensure real-world performance. - Simplifies model integration with a single package and entry points. Tech stack: TypeScript Why it matters: Edgewise simplifies the integration of AI models into web applications by providing a unified package and entry points, ensuring real-world performance through pinned and verified models, and making it easy to run models on multiple runtimes. Repository: kucukkanat/edgewise GitHub stars: 7 GitHub description: Six verbs for on-device AI in TypeScript. Edgewise runs language, vision, speech, judging, embedding, image and forecasting models where your code runs: in the browser through WebGPU or WebAssembly, and in Bun or Node through ONNX Runtime, with vgpu for WebGPU on the server.

### Overview: This project is a comprehensive set of reproducible Qwen3.8 deployments on a single RTX PRO 6000 workstation, including flash-next W4 PLE, SGLang patches, and detailed benchmarks and field reports. The repository provides single-GPU inference reports, reproducibility artifacts, and experimental inspection tools. ### Key Features: - Single-GPU inference reports - Reproducibility artifacts - Benchmarks and field reports ### Tech Stack: C++ ### Why It Matters: This project offers detailed insights into Qwen3.8 deployments on a high-performance GPU, ensuring reproducibility and providing essential tools for developers to test and verify their models.

Overview: This repository provides a PyTorch implementation for classifying the Oxford IIIT Pet Dataset using K-Nearest Neighbors (KNN) and ResNet. The goal is to evaluate the results obtained using these two approaches. Key features: - K-Nearest Neighbors (KNN) classification - ResNet-34 architecture for fine-tuning on the Oxford IIIT Pet Dataset Tech stack: PyTorch Why it matters: This project aims to differentiate the outcomes of using KNN and ResNet for classifying pet images, providing insights into the performance of these models on the Oxford IIIT Pet Dataset.

Overview: Lexo is a comprehensive collection of Jupyter notebooks designed for learning and applying Large Language Models (LLMs) in real-world scenarios. It covers RAG systems, fine-tuning, AI agents, multimodal processing, and ML benchmarks. Key features: - 🌐 Website Summarizer - 🧾 Brochure Generator - 💡 Tech Assistant - 🤖 AI Chat - 🌤️ Weather AI - 📝 Meeting Minutes Assistant - 🧪 Synthetic Data Generator - 🧠 Internal RAG Assistant - 💲 Price Predictor - 🏷️ Snapr Tech stack: Python 3.11.x, uv package manager, Make, Jupyter Notebook Why it matters: Lexo provides hands-on projects and projects for learning and applying LLMs in various real-world scenarios, including RAG systems, fine-tuning, AI agents, and more. It is inspired by the LLM engineering certification course by Ed Donner and includes projects that cover website summarization, brochure generation, tech assistant, AI chat, weather AI, meeting minutes assistant, synthetic data generator, internal RAG assistant, price predictor, and more. Repository: lisekarimi/lexo GitHub stars: 7 GitHub description: 🗯️LLM toolkit for RAG, tuning, agents, and more GitHub link: [https://github.com/lisekarimi/lexo](https://github.com/lisekarimi/lexo)

Overview: This repository contains a fine-tuned M3GNet-IAP model specifically designed for halide perovskites (HaPs). The model is trained on a large-scale in-house DFT dataset, enabling accurate predictions and geometry optimization across diverse structural motifs. Key features: - Fine-tuning M3GNet-IAP on a Mannodi group HT-DFT dataset - Generalizable across bulk alloys, defects, impurities, and surfaces - Enables efficient exploration of the perovskite potential energy surface for energy prediction, bulk alloy screening, defect screening, and surface reconstructions Tech stack: Jupyter Notebook Why it matters: This model provides a powerful tool for researchers and developers working with perovskite materials, enabling accurate predictions and efficient exploration of the perovskite potential energy surface, which is crucial for various applications such as energy prediction, bulk alloy screening, defect screening, and surface reconstruction.

### Overview: This repository contains experimental MLX custom Metal kernels designed for Apple Silicon, focusing on fast attention, decode, KV-cache, and inference primitives. It includes attention kernels, decode attention, KV-cache operations, and quantized matvec kernels, aiming to explore the limits of Apple Silicon GPUs through correctness-first development. ### Key features: - MLX custom Metal kernels for Apple Silicon - Attention kernels - Decode attention and KV-cache operations - Quantized matvec kernels - Paged KV-cache and q4/q8 quantization - Experimental fused q4/q8 MLP kernels - RMSNorm, RoPE, SwiGLU, and residual helpers - Fused and quantized decode blocks - Quantized MLP blocks - Toy transformer-layer benchmarks - Backend autotuning infrastructure - Llama-like model integration scaffolding ### Tech stack: Python ### Why it matters: This project is an experimental research repo focused on building, validating, benchmarking, and iterating on Mac-native transformer inference primitives. It aims to explore the limits of Apple Silicon GPUs through correctness-first development and correctness-first kernel lab.

### Overview: LM-DiskANN.jl is a Julia library designed for approximate nearest-neighbor (ANN) indexing on disk. It efficiently stores vectors and their adjacency lists in memory-mapped files, reducing the in-memory footprint to a minimum. The library supports LevelDB databases for user-key ↔ numeric ID lookups, making it easy to associate each embedding with a custom string ID. ### Key Features: - **Disk-Resident**: Vectors and adjacency lists are stored in memory-mapped files, reducing RAM usage. - **Graph-Based Search**: Uses BFS expansion for approximate neighbor lookups. - **Insert & Delete**: Supports dynamic insertion and deletion without needing to rebuild the index. - **User Keys**: Links a string key (e.g., `"image123"`) to an internal node ID; retrieve or delete by either integer ID or key. ### Tech Stack: Julia ### Why It Matters: LM-DiskANN.jl is a lightweight and efficient solution for approximate nearest-neighbor search on disk, making it suitable for applications where memory usage is a critical concern. Its disk-resident storage and user-friendly key association make it particularly useful for real-time applications and systems with limited RAM. ### GitHub Stars: 7 ### Repository: - Repository: mantzaris/LMDiskANN.jl - Primary language: Julia - GitHub stars: 7 ### Documentation: - Stable: https://mantzaris.github.io/LMDiskANN.jl/stable - Development: https://mantzaris.github.io/LMDiskANN.jl/dev ###

Overview: 本项目通过使用约120亿参数的Krea 2模型，对消费者级别的GPU（NVIDIA GeForce RTX 3060 12GB）上进行了LoRA（低资源加权平均）微调实验，生成了消费者可用的高质量图像。项目包括详细的论文、数据集、推论工作流程和详细的实验结果。 Key features: - 使用约120亿参数的Krea 2模型进行LoRA微调 - 在NVIDIA GeForce RTX 3060 12GB上完成，无需有偿云服务 - 生成消费者可用的高质量图像 - 通过FP8量化和块交换技术实现12GB模型在RTX 3060上的学习 - 使用13张原生LINEスタンプ作为训练数据 - 通过蒸馏前的RAW模型和8步蒸馏版Turbo推论实现高质量生成 - 通过LoRA强度和文本条件优化推论性能 - 提供详细的论文、数据集、推论工作流程和provenance记录 Tech stack: PyTorch 2.6.0 + CUDA 12.4 / Python 3.11.15，Krea 2模型，LoRA，Qwen3-VL-4B，Qwen-Image VAE，FP8量化，块交换，PyTorch `expandable_segments`，NVIDIA GeForce RTX 3060 12GB Why