Projects

Overview: `llm-gpu-lab` is a hands-on, end-to-end LLM toolkit that allows developers to run a full local LLM workflow on a single NVIDIA GPU. It includes pretraining, SFT, evaluation, and benchmarking without the need for cloud training or hosted inference. Key features: - Tiny GPT pretraining from scratch - Generation from the trained checkpoint - LoRA / QLoRA fine-tuning of a small open model - Lightweight evaluation - GGUF export and llama.cpp serving - GPU benchmarking and a reproducible HTML report Tech stack: Python Why it matters: This toolkit provides a practical, hands-on approach to LLM development, ensuring that results are based on real-world performance metrics and not hypothetical or fabricated numbers. It simplifies the process of creating reproducible and verifiable benchmarks, making it ideal for developers looking to understand and improve their models locally.

### Overview: Reproduce Lium's LocalMaxxing leaderboard runs (7 boards, 20 Verified) on a rented Lium GPU in one script — vLLM/llama.cpp recipes, measured cost per million tokens. ### Key features: - Reproduce Lium's LocalMaxxing leaderboard runs - Use vLLM/llama.cpp recipes - Measure cost per million tokens - Rent a GPU for an hour on Lium - Serve Qwen3.6-35B-A3B at 14,499 tok/s - Stream Nemotron 3.5 Lightning to a single user at 681.5 tok/s - Every number below is a live, third-party-hosted run ### Tech stack: Shell ### Why it matters: - One command, any class of card, by the hour - You keep what you measured - Cheap enough to explore - The same node is a serving node ### Repository: Datura-ai/lium-localmaxxing ### GitHub stars: 36 ### GitHub description: Reproduce Lium's LocalMaxxing leaderboard runs (7 boards, 20 Verified) on a rented Lium GPU in one script — vLLM/llama.cpp recipes, measured cost per million tokens

Overview: DreamSampler is an innovative framework that seamlessly integrates reverse sampling and score-distillation techniques for image manipulation using latent diffusion models, as presented in the ECCV24 paper. Key features: - Unifies reverse sampling and score-distillation for image manipulation. - Seamlessly integrates two distinct approaches through regularized latent optimization. - Supports both text-guided image editing and inpainting tasks. Tech stack: Python, CLIP. Why it matters: DreamSampler offers a comprehensive solution for image manipulation tasks, combining the strengths of reverse sampling and score-distillation to achieve superior results, making it a valuable tool for researchers and practitioners in the field of image processing and AI.

### Overview: Socratic LLM is a training pipeline designed to fine-tune Phi-3-mini-instruct to follow the Socratic method. This project aims to enhance the ability of AI models to engage in interactive and reflective dialogue, making them more effective in educational and research contexts. ### Key features: - Fine-tuning Phi-3-mini-instruct to follow the Socratic method. - Enhances interactive and reflective dialogue capabilities. - Suitable for educational and research applications. ### Tech stack: Python ### Why it matters: Socratic LLM is crucial for applications requiring interactive and reflective dialogue, such as educational tools, research assistants, and more. It leverages the Socratic method to foster deeper understanding and engagement, making it an invaluable tool in various fields.

### Overview: AI NourishBot is an AI-powered nutrition assistant that uses advanced vision models and natural language processing to detect ingredients from food images, filter ingredients based on dietary restrictions, estimate calories, provide detailed nutrient analysis, and generate recipe suggestions. ### Key features: - **Ingredient Detection**: Detects ingredients from user-uploaded images using a vision AI model. - **Dietary Filtering**: Filters detected ingredients based on user-defined dietary restrictions. - **Calorie Estimation**: Estimates total calories from the detected ingredients. - **Nutrient Analysis**: Provides a detailed breakdown of key nutrients such as protein, carbohydrates, fats, vitamins, and minerals. - **Health Evaluation**: Summarizes the overall healthiness of the meal and provides a health evaluation summary. - **Recipe Suggestion**: Generates recipe ideas based on the filtered ingredients and dietary restrictions. ### Tech stack: Python ### Why it matters: AI NourishBot demonstrates the integration of advanced AI tools to deliver insightful and personalized nutritional feedback, making it a valuable tool for individuals looking to improve their dietary choices and overall health.

Overview: The Brain is a multimodal RAG dashboard and 3D Knowledge Graph visualizer that processes and queries documents locally or via cloud APIs using LightRAG, RAG-Anything, and Neo4j. Key features: - Integrates with Ollama for local execution or cloud-based execution via OpenAI-compatible APIs. - Supports offline and cloud-based document processing. - Provides an interactive interface for querying and exploring knowledge. - Built on top of LightRAG, RAG-Anything, and Neo4j for advanced querying and visualization. Tech stack: JavaScript Why it matters: The Brain simplifies the process of querying and understanding complex information from multimodal documents, making it accessible and efficient for developers and researchers alike. Repository: Hastur-HP/The-Brain GitHub stars: 35 GitHub description: A multimodal RAG dashboard and 3D Knowledge Graph visualizer. Process documents locally with Ollama or via Cloud APIs, powered by LightRAG, RAG-Anything and Neo4j.

### Overview This repository is a curated list of resources focusing on SLAM-related tasks employing the Diffusion Model, including view/feature correspondences, depth estimation, 3D reconstruction, pose estimation, and more. It aims to provide a comprehensive overview of the latest advancements in the field, ensuring that users can stay updated with the most relevant research. ### Key Features - **Matching**: Papers that address data association or correspondence, such as MATCHA, Sd4match, Emergent Correspondence, SD+DINOv2, SD+DINO, and SD+DINO. - **Depth Estimation**: Papers that focus on 3D reconstruction, including NOVA3R, SpatialCrafter, and Geo4D. - **Pose Estimation**: Papers that deal with pose estimation, such as SD+DINOv2, SD+DINO, and SD+DINO. ### Tech Stack The repository uses the following technologies: - **GitHub**: Stars: 36 ### Why It Matters This repository serves as a valuable resource for developers and researchers interested in the latest advancements in SLAM using the Diffusion Model. It provides a curated list of papers, repositories, and resources, ensuring that users can easily find and access the most relevant research in the field. ### Contributions - **Star**: A simple star is the best affirmation for finding this repository useful. - **Pull Requests**: Feel free to create pull requests or open issues to contribute to making the list more comprehensive. - **Issues**: Feel free to share this list with others!

Overview: This is an official implementation of LatentDB, a method for fast unpaired image-to-image translation using latent diffusion models. The project aims to reduce the computational cost of traditional diffusion models by approximating them with pre-trained Stable Diffusion. Key features: - LatentSB: A method that approximates Schrodinger Bridges ODEs using pre-trained Stable Diffusion. - Competitive performance in unsupervised I2I translation. - Results demonstrated on FID scores with reduced NFEs compared to baseline methods. Tech stack: Python Why it matters: The project offers a significant improvement in computational efficiency for image-to-image translation tasks, making it a valuable resource for developers working on such projects. Repository: LatentSB/LatentSB GitHub stars: 36 GitHub description: Official repository for "Prompting Latent Diffusion for Fast Unpaired Image-to-Image Translation"

### Overview: FlashVSR-Pro is an enhanced, production-ready re-implementation of the FlashVSR algorithm, featuring Docker support, NVENC hardware acceleration, low-VRAM tiling, and unified inference for real-time video super-resolution. ### Key Features: - **Unified Inference Script**: Simplifies the user interface with a single, parameterized `infer.py` script. - **Audio Track Preservation**: Automatically preserves the audio track from the input video. - **Tiled Inference for Reduced VRAM**: Implements a tiling mechanism for DiT model, lowering GPU memory requirements. - **Optimized Docker Container**: Fully-configured Dockerfile for seamless setup. - **Automated Block-Sparse Attention Installation**: Optimizes and automates the installation of the Block-Sparse-Attention backend. - **Enterprise-Grade Performance Optimizations**: Maximizes throughput with Zero-Copy Pipeline, Hardware Acceleration, Asynchronous Execution, and NVENC Video Encoding. - **Strict Precision Alignment**: Ensures output frame count and resolution match input, preventing data loss. ### Tech Stack: HTML ### Why it Matters: FlashVSR-Pro offers improved usability, additional features, and better compatibility for real-world deployment, making it suitable for high-end GPUs and production-grade video processing.

### Overview: YazSes is a free, open-source, fully-offline-by-default voice dictation tool designed for Linux, macOS, and Windows. It allows users to hold a key, speak, and release, instantly typing their voice commands into any app without leaving their computer. YazSes runs speech recognition locally, ensuring your voice never leaves your device, making it perfect for those who prefer privacy and security. ### Key Features: - **Voice Typing**: Type with your voice by holding a key, speaking, and releasing. - **Voice Commands**: Say commands like "scratch that" or "go to line 42" to perform actions without typing. - **Transcribe Recordings**: Transcribe audio or video files into text. - **Meeting Notes**: Record and transcribe meetings with speaker labels. - **Private by Default**: Everything runs on your CPU, ensuring no audio or text leaves your device. ### Tech Stack: Python ### Why It Matters: YazSes provides a secure and private solution for dictating and transcribing text, eliminating the need for cloud-based services. It's ideal for those who want to keep their data and communications offline and secure.

Overview: This repository utilizes a Stable Diffusion model with DreamBooth to generate distinctive and personalized avatars based on input training images. Key features: - Fine-tuning of a Stable Diffusion model - Personalized avatar generation from training images - Stabilized training process with regularization images Tech stack: Jupyter Notebook Why it matters: This project offers a practical solution for creating unique avatars tailored to individual preferences, leveraging advanced AI techniques for image generation.

Overview: This repository contains the implementation of the paper "Unite and Conquer: Plug & Play Multi-Modal Synthesis using Diffusion Models" presented at CVPR 2023. The project proposes a diffusion-based solution for image generation under the presence of multimodal priors, addressing the need for paired data for multimodal synthesis. Key features: - Diffusion-based solution for image generation under multimodal priors - Tackles the problem of paired data requirement for multimodal synthesis - Easily scalable and can be incorporated with off-the-shelf models - Supports semantic face generation and text-guided generation Tech stack: Python Why it matters: This project addresses a significant challenge in multimodal image synthesis by proposing a flexible and scalable solution that can be easily integrated with existing models, making it accessible for researchers and practitioners in the field of computer vision and artificial intelligence. Repository: Nithin-GK/UniteandConquer Primary language: Python GitHub stars: 36 GitHub description: [CVPR '23] Unite and Conquer: Plug & Play Multi-Modal Synthesis using Diffusion Models README excerpt: This repository contains the implementation of the paper: "Unite and Conquer: Plug & Play Multi-Modal Synthesis using Diffusion Models" presented at CVPR 2023. The project proposes a diffusion-based solution for image generation under the presence of multimodal priors, addressing the need for paired data for multimodal synthesis.

### Overview: This project automates the entire process of VITS (Voice Conversion Training) including Data Preprocessing, Whisper ASR, Text Preprocessing, and Training, with a single command. It simplifies the complex VITS training process into a straightforward workflow, making it accessible for developers and researchers. ### Key Features: - **Data Preprocessing**: Automatically cleans and processes audio files. - **Whisper ASR**: Transcribes audio to text using the Whisper model. - **Text Preprocessing**: Enhances the text for better model performance. - **Modification config.json**: Configures the model for specific modifications. - **Training**: Trains the VITS model using the modified configuration. - **Inference**: Generates predictions using the trained model. ### Tech Stack: Python 3.8 with Anaconda, PyTorch, CUDA 11.7, and Zlib DLL. ### Why it Matters: This tool streamlines the VITS training process, reducing the time and effort required for developers and researchers to achieve high-quality voice conversion models. It simplifies the complex pipeline into a single, user-friendly command, making it easier to experiment with different configurations and models.

Overview: Octopus is a speech-to-index engine developed by Picovoice, designed to index speech directly without relying on text representation. This acoustic-only approach enhances accuracy by overcoming out-of-vocabulary limitations and resolving the issue of competing hypotheses. Key features: - Direct acoustic indexing - Supports multiple languages: English, French, German, Italian, Japanese, Korean, Portuguese, Spanish, and additional languages for commercial use - Python demo: Install `pvoctopusdemo` package, run `octopus_demo --access_key {AccessKey} --audio_paths ${AUDIO_PATHS}` - C demo: Build and run `octopus_index_demo` and `octopus_search_demo` scripts - Android demo: Open `OctopusDemo` as an Android project, replace AccessKey in `MainActivity.java` - Web demo: Available for commercial customers on a case-by-case basis Tech stack: Python Why it matters: Octopus leverages deep learning to index speech, improving accuracy and efficiency in speech recognition applications. Repository: Picovoice/octopus GitHub stars: 36 GitHub description: On-device Speech-to-Index engine powered by deep learning

Overview: QuickButlerPace65/AnythingLLM-Desktop is an enhanced all-in-one AI desktop app designed to streamline productivity and enhance user experience. Key features: - Integration with AI-powered productivity tools - Customizable interface for tailored workflows - Seamless integration with existing software and devices Tech stack: Python, Flask, TensorFlow, and various AI libraries Why it matters: This project offers a comprehensive solution for developers and tech enthusiasts looking to leverage AI for enhanced productivity without the need for specialized knowledge.

Overview: SoundSwallower is a small speech recognizer designed for embedding in web applications, focusing on finite-state grammar recognition and word-level force alignment. Key features: - Finite-state grammar recognition - Word-level force alignment - Supports JSGF grammars and word-level force alignment - Python module and command-line interface for easy integration Tech stack: C Why it matters: SoundSwallower is a compact speech recognition tool that is ideal for applications requiring minimal footprint, making it suitable for embedding in web applications or other small-scale projects.

Overview: Emost-Bot is a Discord music bot that utilizes voice commands for interaction, specifically designed for voice recognition and command execution. Key features: - Voice commands for music management - Discord integration - Voice recognition using Porcupine for hotword detection Tech stack: JavaScript Why it matters: Emost-Bot simplifies music management in Discord by allowing users to control music through voice commands, making it easier and more intuitive for voice-activated interactions. Repository: SteTR/Emost-Bot GitHub stars: 36 GitHub description: Discord Music Bot using Voice Recognition to receive commands.

### Overview: LLM Fine-Tuner simplifies the process of fine-tuning large language models (LLMs) for specific applications. It provides a user-friendly interface for no-code users, allowing them to fine-tune models without needing programming skills. The tool supports various types of models, including SFT, DPO, ORPO, KTO, GRPO, and reward models, and can be trained on data from any source. It also supports distillation, vision, and embedding models, as well as exporting models in GGUF and FP8 formats. ### Key features: - 📝 Fine-tuning of SFT, DPO, ORPO, KTO, GRPO, and reward models - 📱 Gradio UI and CLI interface - 💡 Support for distillation, vision, and embedding models - 📂 Export models in GGUF and FP8 formats - 🚀 Fast training times, up to 2-5x faster than traditional methods - 🚀 Works on regular hardware, even a gaming laptop with 8 GB GPU memory - 🚀 Your data stays yours, and the model can be run anywhere - 🚀 Export models for use in Ollama, LM Studio, or sharing online ### Tech stack: Python ### Why it matters: LLM Fine-Tuner simplifies the process of fine-tuning large language models for specific applications, making it accessible to no-code users. It supports various types of models and provides a fast, user-friendly interface for training and exporting models.

Overview: This repository tracks the latest advancements in AI multimodal models, including Foundation Models, Large Language Models, Agents, Audio, Images, Videos, Music, and 3D content. Key features: - Multimodal Model - LLM - Agent - Audio - Image - Video - Music - 3D Tech stack: Unknown Why it matters: This repository serves as a comprehensive resource for staying updated with the latest developments in AI multimodal models, providing insights into how these models are evolving and integrating different modalities to achieve more comprehensive understanding and generation of information.

### Overview: This repository is a comprehensive paper list for the field of multimodal dialogue, covering various aspects including methods, datasets, and metrics. It categorizes the tasks into Visual-Grounded Dialogue (VGD), Visual Question Generation, Multimodal Conversation (MMC), and Visual Navigation (VN). ### Key Features: - **Datasets**: Comprehensive collection of datasets for visual QA (VQA), visual dialogue, visual reference resolution, CLEVR, IGC, and more. - **Metrics**: Various metrics related to multimodal dialogue tasks. - **Tasks**: Visual-Grounded Dialogue (VGD), Visual Question Generation, Multimodal Conversation (MMC), and Visual Navigation (VN). ### Tech Stack: - **Language**: Unknown - **Stars**: 36 ### Why It Matters: This repository serves as an invaluable resource for researchers and practitioners in the field of multimodal dialogue, providing a wealth of information and tools to advance the state of the art in dialogue systems.