Projects

Overview: This project provides an efficient audio tokenization tool, enabling the conversion of audio into acoustic and semantic tokens. Key features: - Fast encoding and decoding of audio files - Support for acoustic and semantic tokenizers - Installation via pip - Usage examples in the provided notebook Tech stack: Python Why it matters: This tool is crucial for applications requiring real-time acoustic and semantic analysis of audio data, such as speech recognition, natural language processing, and audio classification tasks. Repository: indri-voice/audiotoken Primary language: Python GitHub stars: 56 GitHub description: Audio tokenization, in the fastest way possible!

Overview: Sibila is a Python library designed to extract structured data from local or remote Large Language Models (LLMs). It ensures predictable behavior by providing a consistent output format. Key features: - Extracts structured data into Pydantic objects, dataclasses, or simple types. - Supports access to models from OpenAI, Anthropic, and Mistral AI, as well as local models like Llama-3 and Phi-3. - Integrates with vision models to extract structured data from images. - Provides a general-purpose model access library for generating both plain text and free JSON results. Tech stack: Python Why it matters: Extracting structured data from LLMs can help ensure more predictable and consistent behavior from models, which is crucial for serious use of LLMs. Repository: jndiogo/sibila GitHub stars: 56

Overview: This project is designed to reduce token usage for Chinese, Japanese, and Korean (CJK) languages when using Claude Code, aiming to lower costs and improve context windows. Key features: - Reduces input token count by up to 2x effective context window - Preserves code blocks, file paths, and URLs - Auto-detects English technical terms - Uses Apple NaturalLanguage framework for named entity recognition - Caches translations locally to eliminate redundant API calls - Sends only prompt text for translation; code artifacts stay local - Adds 100-300ms latency per translation Tech stack: Rust Why it matters: By reducing token usage, this project helps lower costs and improve context windows for RAG/agent workflows, making Claude Code more efficient and effective for CJK languages. Repository: jserv/cjk-token-reducer GitHub stars: 56 GitHub description: Reduce Claude Code token usage by 35-50% when using CJK languages.

Overview: Pinecone.NET is a C# library designed for interacting with the Pinecone vector database, providing comprehensive support for indexing, querying, and managing vectors. Key features: - Standard operations on pod-based and serverless indexes - gRPC and REST transports for vector operations - Support for sparse-dense vectors - Metadata support - Client-side gRPC load balancing - Efficient vector serialization - NativeAOT compatibility (e.g., for AWS Lambda) - Automatic batching and parallelization for upsert, fetch, and delete operations - Exceptions to save partial results and recover from failures on batched and streamed operations Tech stack: C# Why it matters: Pinecone.NET simplifies the integration of Pinecone's vector database into C# applications, offering efficient vector management and easy-to-use features for developers.

Overview: FluxForge is a configuration-driven fine-tuning framework designed for efficient and customizable fine-tuning of Stable Diffusion models. It offers a declarative pipeline description, making the process more accessible and transparent. Key features: - **Configuration as a Living Document:** The entire tuning run is described in a single, versionable YAML file. - **Layer-Freezing:** Allows freezing, partially training, or fully fine-tuning individual components of the model. - **Adaptive Data Carousels:** Dynamically samples data based on per-sample loss, preventing catastrophic forgetting. - **Memory-Elastic Gradient Accumulation:** Automatically adjusts batch sizes based on GPU memory availability. Tech stack: Python, PyTorch, Stable Diffusion, YAML. Why it matters: FluxForge provides a more flexible and understandable fine-tuning process, allowing for better experimentation and reproducibility compared to monolithic tuning frameworks.

Overview: This project automates the process of generating YouTube Shorts videos by automatically fetching videos from Pexels, generating voice-over and subtitles using AI, and compiling them into a single video using `moviepy`. Key features: - Automatically fetches videos from Pexels - Generates voice-over and subtitle using AI - Compiles videos into a single video using `moviepy` Tech stack: Python Why it matters: This project streamlines the video creation process, making it accessible to users without video editing skills, and automates the video compilation, saving time and effort.

Overview: Mold is a local AI image and video generation tool that supports multiple platforms including CUDA, Metal on macOS, desktop, web, iOS, macOS, and more. It enables users to create high-quality images and videos locally using their own GPU, making it accessible and powerful. Key features: - Supports multiple GPU backends including CUDA, Metal, and desktop. - Offers a CLI, native desktop app, web studio, mobile companions, Discord bot, and REST/SSE API. - Provides a seamless experience across different platforms and devices. - Supports various models and prompt editing functionalities. Tech stack: Rust, Nix, and Docker. Why it matters: Mold simplifies the process of generating high-quality images and videos locally, making it a valuable tool for developers and content creators who need quick and efficient AI-driven content creation without the need for expensive cloud services or complex setups. Repository: utensils/mold GitHub stars: 54 GitHub description: CLI-native local AI image and video generation for people, scripts, and agents — CUDA on Linux, Metal on macOS, plus desktop, web, iPhone, REST/SSE, and MCP.

### Overview: MechVQA is a comprehensive benchmarking and enhancement project for multimodal Large Language Models (LLMs) designed for the understanding of mechanical engineering drawings. It consists of a benchmark dataset with 3.3K high-density drawings and 21K question-answers, along with a strong domain-specialized model, MechVL-4B-RL, that achieves an impressive Total score of 84.85 on the MechVQA dataset. ### Key Features: - **MechVQA Benchmark**: The first comprehensive dataset for mechanical-drawing understanding, featuring 3.3K high-density drawings and 21K question-answers across three capability levels (Recognition, Reasoning, Judging). - **MechVL Model**: A strong domain-specialized baseline built via multi-stage training (SFT → two-stage self-play RL), achieving a Total score of 84.85, surpassing the strongest closed-source baseline by 7.57 points and the strongest open-source baseline by 5.94 points. ### Tech Stack: - **Python** for development and experimentation. ### Why It Matters: MechVQA addresses the challenge of brittle general Multimodal LLMs on mechanical engineering drawings, providing a comprehensive dataset and a strong model that outperform existing benchmarks.

Overview: Difformer is an open-source implementation of a text generation model that leverages diffusion models on the embedding space, as presented in the NAACL 2024 paper "Empowering Diffusion Models on the Embedding Space for Text Generation". Key features: - Supports multiple language pairs (e.g., English-Deutsch, German-English) - Uses fairseq for training and evaluation - Implements knowledge distillation using Transformer models - Provides preprocessed datasets and scripts for easy data preparation and model training Tech stack: Python 3.8, PyTorch 1.11, Fairseq 0.10.2 Why it matters: Difformer offers a novel approach to text generation by leveraging diffusion models, potentially improving the quality and diversity of generated text compared to traditional methods.

Overview: This repository contains a performance recipe for Qwen/Qwen3.8-27B-FP8 running on a single DGX Spark (GB10 / SM121). It includes a benchmark harness to measure token throughput and serves as a reference for single-stream performance. Key features: - Single-stream performance of 7.88 to 58.5 tok/s - Speculative decoding and prefix caching for improved performance - Quantization advantage on concurrency, with FP8 being the only build that can serve DFlash 2 under vLLM at all - Performance comparison between different configurations and drafter options Tech stack: Python Why it matters: This repository provides a practical guide for developers to measure and optimize the performance of Qwen/Qwen3.8-27B-FP8 on a single DGX Spark, focusing on single-stream throughput and benchmarking different configurations and drafter options. Repository: 0xBakeer/Qwen3.8-27B-FP8-on-a-single-DGX-Spark GitHub stars: 55 GitHub description: Serving Qwen3.8-27B-FP8 on a single DGX Spark (GB10 / SM121): 7.88 to 58.5 tok/s single-stream from decode strategy alone, weights untouched. Speculative decoding and prefix caching benchmarked, plus DFlash 2 — the only Qwen3.8-27B build that can serve it under vLLM.

### Overview: Lightmetal is a Java-based GPU LLM inference engine optimized for Apple Silicon, featuring zero dependencies. It leverages Metal to bind a Metal-enabled `libllama.dylib` and supports models like Mistral, Gemma, Nemotron, and Mellum. ### Key Features: - **Zero Dependencies**: Runs without any external libraries or dependencies. - **Metal Integration**: Binds a Metal-enabled `libllama.dylib` through Foreign Function & Memory API. - **GPU Acceleration**: Utilizes Apple Silicon for efficient GPU acceleration. ### Tech Stack: Java 25 ### Why it Matters: Lightmetal simplifies GPU LLM inference by eliminating the need for external libraries, making it highly efficient and lightweight. Its native integration with Metal ensures optimal performance on Apple Silicon, making it ideal for applications requiring high GPU acceleration. Repository: AdamBien/lightmetal

Overview: PerLDiff is a novel approach for generating controllable street view synthesis using perspective-layout diffusion models. This project introduces a method that leverages diffusion models to control the perspective and layout of street views, making it highly versatile and effective for various applications. Key features: - Perspective-layout diffusion models for street view synthesis - Controllable perspective and layout adjustments - Adapted for the KITTI dataset - Preprint and code availability Tech stack: Jupyter Notebook Why it matters: This project showcases the potential of diffusion models in generating controllable street views, offering significant improvements in the quality and flexibility of street view synthesis compared to traditional methods. The ability to control perspective and layout makes PerLDiff highly valuable for applications requiring precise and customizable street view generation. Repository: CVL-UESTC/PerLDiff Primary language: Jupyter Notebook GitHub stars: 55 GitHub description: ICCV 2025-PerLDiff: Controllable Street View Synthesis Using Perspective-Layout Diffusion Models README excerpt: <div align="center"> <h2> PerLDiff：Controllable Street View Synthesis Using Perspective-Layout Diffusion Models (ICCV 2025) </h2> [Jinhua Zhang](https://scholar.google.com/citations?user=tyYxiXoAAAAJ), [Hualian Sheng](https://scholar.google.com/citations?user=73JaDUQAAAAJ&hl=zh-CN&oi=sra), [Sijia Cai](https://

### Overview This project is a comprehensive deepfake creation using a custom diffusion model, focusing on satellite imagery of Earth. The goal is to create a realistic yet deceptive image of satellite data, showcasing advancements in computer vision and deep learning. ### Key Features - **Custom Diffusion Model**: A model created from scratch using the principles of diffusion models. - **Satellite Imagery**: The project involves gathering and processing satellite imagery data. - **End-Of-Year Project**: The project was completed within a tight 4-week timeline, constrained by vacation periods. - **No Public Datasets**: All data was gathered manually. - **No School-Learned Frameworks**: The project did not use any frameworks taught in school. - **No Third-Party Software**: The project avoided using third-party software for data collection. ### Tech Stack - Python ### Why It Matters This project demonstrates the capabilities of custom diffusion models in creating realistic yet deceptive images, showcasing advancements in computer vision and deep learning. It also highlights the importance of self-imposed constraints and the necessity of learning from scratch. ### Repository The repository is named `Camaltra/this-is-not-real-aerial-imagery`. ### GitHub Stars The repository has been starred by 55 developers. ### Primary Language The primary language used in this project is Python.

### Overview: The Guide-and-Rescale project is an official implementation for a self-guidance mechanism designed to enhance the effectiveness of tuning-free real image editing. This approach aims to preserve the overall structure and local appearance of the input image while avoiding the need for time-consuming hyperparameter tuning or fine-tuning of diffusion models. ### Key features: - **Self-Guidance Mechanism:** The method introduces a self-guidance technique to maintain the integrity of the input image's structure and appearance. - **Layout Preserving Energy Functions:** Explicitly introduces layout-preserving energy functions to ensure local and global structures of the source image are preserved. - **Noise Rescaling Mechanism:** A noise rescaling mechanism balances the norms of classifier-free guidance and proposed guiders to achieve more consistent and better editing results. ### Tech stack: Jupyter Notebook ### Why it matters: The proposed method provides a fast and high-quality editing mechanism that can produce desired edits while maintaining the original image's quality and appearance, significantly improving the user experience compared to existing approaches.

Overview: IISAN is an efficient multimodal foundation model adaptation framework designed for sequential recommendation tasks, aiming to balance parameter efficiency, GPU memory usage, and training speed. Key features: - IISAN (Intra- and Inter-modal Side Adapted Network for Multimodal Representation) - Decoupled PEFT (Parameter-efficient Fine-tuning) - GPU memory usage reduction from 47GB to 3GB - Training speed improvement from 443s to 22s per epoch Tech stack: Python Why it matters: IISAN provides a practical solution for efficiently adapting multimodal foundation models in recommendation tasks, addressing the trade-offs between parameter efficiency, GPU memory usage, and training speed.

### Overview: This repository provides comprehensive examples on how to use the Hotpot.ai API, including Stable Diffusion, background removal, image upscaling, and more. ### Key Features: - **Stable Diffusion API**: Offers three flavors: budget, standard, and premium. - **Art Maker**: Demonstrates how to monitor and render URLs using the Stable Diffusion API. - **Background Remover**: Provides examples in multiple languages, including Curl, Node.js, Python, PHP, and C# with Flurl.Http. ### Tech Stack: Node.js, Python, PHP, C#, and C# with Flurl.Http. ### Why it Matters: This repository serves as a practical guide for developers looking to integrate the Hotpot.ai API into their projects, showcasing various functionalities and languages to enhance their applications.

### Overview: Continuous Locomotive Crowd Behavior Generation is an official code for "Continuous Locomotive Crowd Behavior Generation" presented at CVPR 2025. This project aims to generate realistic, continuous crowd behaviors with learned dynamics, focusing on the continuous simulation of crowd trajectories. ### Key Features: - **Crowd Emitter Diffusion Model**: Iteratively "emits" new agents by sampling when and where they appear on spatial layouts. - **State-Switching Crowd Simulator**: Generates continuous trajectories with agents dynamically switching behavior modes. - **Real2Sim Evaluation**: Assesses performance on both scene-level realism and agent-level accuracy. - **Two-Tier Evaluation**: Evaluates on scene-level metrics (density, frequency, coverage, population metrics) and agent-level metrics (kinematics, DTW, diversity, collision rate). ### Tech Stack: Python ### Why It Matters: This project provides a comprehensive framework for generating lifelike crowd behaviors, which is crucial for various applications such as urban planning, event simulation, and virtual reality environments. The ability to simulate continuous crowd dynamics and evaluate performance across different metrics makes it highly valuable in the field of crowd simulation and behavior generation.

Overview: AgnesStudio 是一个基于 Rust 和 Dioxus 构建的桌面客户端，为 Agnes AI 的图像与视频生成模型提供精美的图形化操作界面。 Key features: - 图像生成：支持 Agnes Image 2.5 / 2.1 / 2.0 Flash，文生图与图生图 - 视频生成：支持 Agnes Video 2.5 Flash / 2.5、V2.0，文生视频、首尾帧、图片/音频/视频参考 - 液态玻璃 UI：苹果式磨砂玻璃质感，中性配色；生成按钮常驻侧栏底部，永远可见 - 主题切换：浅色 / 深色 / 跟随系统，顶栏一键切换 - 本地配置：API Key、保存目录等配置保存在本机，重启不丢失 - 丰富尺寸：2.1 / 2.5 Flash 档位式尺寸（1K–4K × 8 种宽高比），2.0 Flash 精确尺寸预设与自定义 - 跨平台内核：基于 Dioxus desktop，主要面向 Windows Tech stack: Rust + Dioxus Why it matters: AgnesStudio 提供了一个简单、高效且易于使用的界面，帮助用户快速生成高质量的图像和视频，无需复杂的命令行操作。其轻量级设计和本地配置功能使其适用于各种环境，特别适合需要频繁生成

Overview: FinetuneForge is a comprehensive repository designed to guide developers through the intricate process of fine-tuning Hugging Face Transformers, from the basics to advanced techniques, ensuring a smooth and efficient learning experience. Key features: - 📚 Layered Abstraction Walkthroughs: Each notebook is paired with a companion markdown file explaining every decision. - 🧩 Modular Config System: All hyperparameters are externalized into YAML files, allowing easy switching between different models. - 🌐 Multilingual Sandbox: Scripts for fine-tuning on non-English corpora. Tech stack: Python, Jupyter Notebooks, YAML, and Hugging Face Transformers. Why it matters: FinetuneForge provides a practical, step-by-step guide that bridges the gap between theoretical knowledge and real-world application, ensuring developers can fine-tune their models effectively and efficiently.

Overview: RiddleWeaver is a comprehensive framework designed to fine-tune a language model on math riddles, enabling it to generate riddles that are not only novel but also internally consistent, making them suitable for educational tools, gamified learning platforms, and AI-driven tutoring systems. Key features: - Dual-Decoder Architecture (DPCE) to separate narrative from mathematical skeleton. - Coherence Penalty Loss to ensure semantic alignment between question and answer. - Multilingual Riddle Scaffolding (MRS) to support multiple languages in the solution. - Style-Guided Solution Expansion (SGSE) to provide three distinct solution paths. Tech stack: HTML Why it matters: By treating riddles as thought experiments and math problems as logical puzzles, RiddleWeaver ensures that the generated riddles are not only novel but also internally consistent, making them ideal for educational applications where accuracy and coherence are crucial. GitHub stars: 55