Projects

Overview: Enhance BoxDiff to leverage Stable Diffusion XL for improved layout-to-image generation. Key features: - Extend BoxDiff to work with Stable Diffusion XL - Experiment with different parameters like --max_iter_to_alter, weight_loss, and various seeds and token indices Tech stack: Python Why it matters: This project aims to enhance the existing BoxDiff tool by integrating it with the advanced capabilities of Stable Diffusion XL, potentially leading to more versatile and high-quality image generation.

### Overview: A zero-dependency Python tool designed to resolve relative Markdown links and inline images as Base64 strings, ideal for web scraping, RAG pipelines, and LLM data preparation. ### Key features: - Resolves relative Markdown links and inline images to absolute URLs. - Perfect for web scraping, RAG pipelines, and LLM data preparation. - Supports automatic inline image resolution as Base64 strings for Multimodal LLMs. - Includes advanced features for handling authentication and graceful fallbacks. ### Tech stack: Python ### Why it matters: This tool simplifies the process of integrating Markdown content into more complex applications, ensuring that links and images are correctly resolved and displayed, regardless of the source or authentication requirements.

### Overview: Identity Forge is a Python-based ComfyUI character generator that allows you to create believable, coherent characters with just a few dropdowns. It automatically generates detailed descriptions and prompts for your image model, making it easy to create cosplayers or creatures. ### Key Features: - **Character Creation:** Identity Forge turns dropdowns into clean, natural-language character descriptions that feed directly into your image model. - **Cosplay Nodes:** Provides detailed costume descriptions for characters, ensuring they can be rendered accurately. - **Franchise Support:** Supports characters from Marvel, DC, Star Wars, anime, video games, fantasy, sci-fi, and horror. - **Crossplay:** Allows setting gender independently and adapting costumes. - **Named Beasts:** Characters render as the animal itself, not as someone dressed as it. - **Turnaround:** Generates multiple angles of the same character in a single queue. ### Tech Stack: Pure Python, no LLM or dependencies. ### Why It Matters: Identity Forge simplifies the process of creating believable characters, making it easier and more efficient to generate cosplayers or creatures in ComfyUI. The detailed descriptions and automatic costume creation ensure that the characters can be rendered accurately, and the crossplay feature allows for gender-neutral character creation.

Overview: MulT is a Pytorch implementation of the paper "Multimodal Transformer for Unaligned Multimodal Language Sequences", focusing on the unaligned version of MOSEI dataset. Key features: - Pytorch implementation - Supports unaligned MOSEI dataset - Includes pre-trained models and datasets - Runs on Python 3.6 - Requires CUDA 10.0 or above Tech stack: Python 3.6, Pytorch (>=1.0.0), torchvision, CUDA 10.0 Why it matters: This implementation provides a practical tool for researchers and developers interested in unaligned multimodal sequence processing, offering a valuable resource for further research and application development.

### Overview: This project aims to enhance the performance of Intel Arc Pro B70 GPUs running on Ubuntu Server, specifically for LLM inference tasks. The kit includes 11 cherry-picked commits that fix major issues, such as MoE slot-init SEGV, Q8_0 reorder crashes, and other performance bottlenecks. It also includes Mesa 26.0.5, SYCL/Vulkan backend selection rules, and optimized runtime scripts for various models. ### Key Features: - **Performance Boost:** Up to 2-7x speedup on 4x B70 GPUs. - **LLM Inference Kit:** Built for Ubuntu Server environments, ensuring smooth operation and production-level performance. - **LLM Start Scripts:** Tuned flags and environment variables for optimal performance. - **Systemd Units:** Ensures tiers survive reboots. - **Syntactic and Semantic Workarounds:** Fixes for MoE model crashes and SYCL slot-init SEGVs. - **Benchmark Verification:** Benchmarks verified to show significant performance improvements. ### Tech Stack: Python ### Why It Matters: This kit is essential for developers and enthusiasts looking to maximize the performance of Intel Arc Pro B70 GPUs on Ubuntu Server for LLM inference tasks. It addresses common issues that can significantly impact the speed and reliability of inference workloads, making the GPUs more competitive in server environments.

Overview: Vehicle-Speed-Detection-System is an intelligent computer vision project designed to detect vehicles and estimate their speed in real time using advanced object detection and tracking algorithms. Key features: - Vehicle Detection - Speed Estimation - Object Tracking - Real-Time Video Processing Tech stack: Python, OpenCV, NumPy, PyTorch / TensorFlow, YOLO Why it matters: This project demonstrates the integration of deep learning, motion tracking, and computer vision for real-world transportation analytics, making it useful for smart traffic management and road safety systems.

### Overview: Real-Gemini is a real-time video understanding and interaction framework utilizing a large multi-modal model. It enables users to ask and answer questions and engage in conversations through text, audio, image, and video. ### Key features: - Real-time video understanding and interaction - Multi-modal model integration - Text, audio, image, and video-based communication ### Tech stack: Python ### Why it matters: Real-Gemini provides a powerful framework for real-time video understanding and interaction, making it an essential tool for developers looking to enhance their applications with advanced multimedia capabilities.

Overview: PRISM is a context-aware cloud-edge inference framework designed for LLM deployments, focusing on balancing privacy and inference quality. Key features: - Edge-side entity-level sensitivity profiling - Entropy-regularised soft gating to select an execution path - Adaptive two-layer local differential privacy for collaborative paths - Cloud-edge semantic sketch collaboration for final response generation Tech stack: Python Why it matters: PRISM addresses the privacy concerns of LLM deployments by dynamically routing prompts based on their privacy risk, ensuring both privacy and inference quality are maintained. Repository: Junfei-Z/PRISM GitHub stars: 28 GitHub description: [AAAI2026] Official repo for paper "PRISM: Privacy-Aware Routing for Adaptive Cloud–Edge LLM Inference via Semantic Sketch Collaboration" Project Page: [junfei-z.github.io/prism/] License: MIT Repository Structure: - Code directory contains scripts and models for execution - Dataset directory includes evaluation test set and labelled routing dataset - prism.pdf and Appendix_PRISM.pdf provide the paper and supplementary material - requirements_prism.txt lists Python dependencies needed for installation Installation: Clone the repository and install Python dependencies using the provided requirements_prism.txt file.

### Overview: A deep learning model designed to identify and detect lexical stress in spoken English words. ### Key features: - **Phoneme Alignment**: Aligns audio files into phonemes. - **Feature Extraction**: Extracts spectral and non-spectral features for each phoneme. - **Model Training**: Uses a combination of CNN and DNN to classify lexical stress. ### Tech stack: Python ### Why it matters: This model helps in improving the accuracy of speech recognition and natural language processing tasks, making it valuable for applications like language translation, voice recognition, and speech-to-text conversion.

### Overview: Zelda Rose is a Python-based tool designed for training transformer-based models. It simplifies the process of setting up and running language modeling tasks, making it accessible for developers and researchers. ### Key Features: - **Training Models**: Supports both tokenizer and transformer models. - **Tokenizer Creation**: Automatically generates a tokenizer from raw text files. - **Model Training**: Runs language modeling tasks using pre-trained models from Hugging Face’s transformers library. - **Distributed Training**: Offers options for distributed training using DDP (Data Parallelism) and DDP Spawn strategies. - **Data Management**: Utilizes Hugging Face’s datasets for efficient data management and cache management. ### Tech Stack: Python ### Why It Matters: Zelda Rose streamlines the process of training transformer-based models, making it easier for developers to focus on the model training rather than the setup. It simplifies the configuration and management of distributed training, ensuring efficient use of resources and reducing the likelihood of errors. ### Repository: Repository: LoicGrobol/zeldarose GitHub Stars: 28 GitHub Description: Train transformer-based models.

Overview: DelphiMistralAI is a library that provides unified access to Mistral's text, vision, and audio endpoints, along with agentic workflows and state-of-the-art models. It includes features like batch processing, fine-tuning, and live request monitoring, making it a powerful tool for developers working with Delphi. Key features: - Unified access to text, vision, and audio endpoints - Agentic workflows via the `v1/conversations` endpoint - Built-in tools for web search, code interpreter, image generation, and document library - Support for Mistral's reasoning-centric models, including voxtral, devstral, mistral-ocr, and magistral - Ready-made `Sync`, `Async`, and `Await` code snippets - Batch processing, function calling, file management, and content moderation - Mock-friendly design with dependency injection for testing Tech stack: The library is built using Pascal and is MIT-licensed. Why it matters: DelphiMistralAI simplifies the integration of Mistral's advanced AI capabilities into Delphi applications, making it easier for developers to leverage these powerful tools without needing to dive into complex APIs or libraries.

### Overview: Lynn Engine 是一个由 Lynn 自主开发的高性能语言模型推理引擎，专为 NVIDIA Blackwell 系列 GPU 设计。它通过 MoE（多模态扩展）和自定义 CUDA/Triton 内核实现了快速的推理性能，同时保持了与原生 LLM 的良好兼容性。 ### Key Features: 1. **高性能单流推理**：通过自研内核实现了单流 decode（单 token 生成）的性能，实测端到端编排时间缩短至 26.6s，相较于原版 DeepSeek-V4-Pro 提升了 2.3 倍。 2. **并发服务系统**：在 NVFP4 的基础上，实现了高效的并发吞吐能力，每秒输出 token 数量达到 2434，比成熟引擎提升了 1.14-1.34 倍。 3. **MoE（多模态扩展）**：通过 MoE 实现了模型的扩展，提高了模型的处理能力和推理速度。 ### Tech Stack: Python ### Why it Matters: Lynn Engine 是一个高性能的推理引擎，通过 MoE 和自定义内核实现了在 NVIDIA Blackwell 系列 GPU 上的快速推理性能。它不仅提高了推理效率，还保持了与原生 LLM 的良好兼容性，使得 Lynn 在推理性能上达到了业界领先水平。

### Overview: JibarOS is an Android 16 fork where AI is treated as a platform primitive, not an app-specific feature. It exposes twelve AI capabilities through a single binder AIDL, ensuring models and runtime are shared across every app on the device. This approach reduces duplication and improves scalability, making it the cheapest signal that on-device AI belongs at the platform tier. ### Key features: - **Shared Runtime and Models**: Models load once at the platform tier and are shared across every app that asks. - **Single Native Daemon**: One system service, one native daemon, four pluggable backends, pooled residency, per-UID rate limits, and priority-aware scheduling. - **OEM-Configurable Capabilities**: OEMs can configure per capability, ensuring the right model is used for each app. - **Android 16 Fork**: Forked from Android 16, it supports the latest Android features and APIs. ### Tech stack: - Shell-based development environment. ### Why it matters: - Reduces app duplication, saving on storage and computation costs. - Improves scalability by sharing AI resources across multiple apps. - Ensures models are loaded once and used efficiently, without the need for separate app-specific implementations. - Supports OEM-specific configurations, making it flexible and adaptable to different hardware and software environments.

Overview: MaxFusion is a plug and play framework designed for multimodal generation using text to image diffusion models. It addresses the issue of conflicting spatial conditioning in text to image models and proposes a novel variance-based feature merging strategy. Key features: - Multimodal generation - Saliency in variance maps - Plug and play framework - Combining information from different feature layers Tech stack: Jupyter Notebook Why it matters: MaxFusion simplifies the process of multimodal generation by allowing easy integration with existing diffusion models, making it accessible for various applications without the need for extensive training with paired data. Repository: Nithin-GK/MaxFusion GitHub stars: 28 GitHub description: [ECCV'24] MaxFusion: Plug & Play multimodal generation in text to image diffusion models

Overview: vlut.cpp is a lightweight extension of llama.cpp that implements Vec-LUT for ultra-low-bit LLM inference on edge devices. It targets parallel scenarios such as prefilling, serving, and test-time scaling. Key features: - Fast lookup table-based design. - Efficient vector LUT paradigm. - Optimized memory access patterns. - Compatible with llama.cpp kernel integration. Tech stack: C++ Why it matters: vlut.cpp offers a lightweight and efficient solution for ultra-low-bit LLM inference, making it suitable for edge devices with limited resources.

### Overview: PocketLLM Lite is a premium, privacy-first AI chat application for Android/iOS, designed to run directly on your device. It integrates seamlessly with Ollama via Termux, offering powerful AI models without the need for an account or hosted inference. The app features a beautiful, customizable interface and supports various document types for chat with them, including PDF, TXT, Markdown, and CSV. It also includes strict offline mode, no ads, no analytics, and secure storage with AES-256-GCM backups. ### Key Features: - **Local AI Models**: Cactus-format model bundles run on-device through the Cactus runtime, ensuring no need for an account or hosted inference. - **Strict Offline Mode**: Blocks all non-loopback traffic, ensuring strict offline operation. - **User-Defined Connections**: Connects to an OpenAI-compatible endpoint or your own machine for more flexibility. - **Customizable Interface**: Features a beautiful, customizable interface and supports various document types for chat. - **Privacy and Security**: Strict offline mode, network audit log, secure storage with AES-256-GCM backups, and password-derived AES-256-GCM `.pllm` backups with rollback on failed restore. ### Tech Stack: - Dart ### Why It Matters: - **Runs Directly on Your Device**: Models and documents run on-device, ensuring privacy and security. - **User-Defined Connections**: Allows connecting to OpenAI-compatible endpoints or your own machine for more flexibility. - **Strict Offline Mode**: Ensures strict offline operation, preventing network traffic

### Overview: Tritium is a foundational library for converting dense neural networks into compact additive-ternary models. It supports refining or training models with PyTorch, and executing packed weights across native and portable runtimes. Tritium treats the recipe, calibration evidence, ancestry, packed bytes, runtime memory, and measured model quality as one auditable artifact lifecycle. ### Key features: - **Additive PTQ with one to three ternary planes** - **Calibration receipts** - **Strict physical-byte accounting** - **Packed artifacts** - **Scale-only refinement** - **Hard-PV refinement** - **Differentiable PyTorch QAT with STE estimators** - **Tied-weight preservation** - **Strict checkpointing** - **Hugging Face integration** - **Packed inference kernels and runtime adapters** - **Native SALT formats and compact B3 packing** - **Sparse/additive layouts** - **Seek-backed payloads** - **Canonical manifests** - **Bounded readers** ### Tech stack: Rust ### Why it matters: Tritium is a foundational library for converting dense neural networks into compact additive-ternary models. It supports refining or training models with PyTorch, and executing packed weights across native and portable runtimes. Tritium treats the recipe, calibration evidence, ancestry, packed bytes, runtime memory, and measured model quality as one auditable artifact lifecycle. ### Repository: Quitetall/tritium ### GitHub stars: 28 ### GitHub description: Foundational ternary-model inference & training library — CUDA + CPU, Bit

Overview: This project implements the "Learning From Failure" method, which integrates negative examples during the fine-tuning process of large language models as agents. The method aims to improve data efficiency and performance. Key features: - Incorporates negative examples in fine-tuning large language models as agents. - Utilizes GPT-3.5 to collect trajectories for math and question-answering tasks. - Provides datasets for math and question-answering tasks. - Models trained with both positive and negative samples. - Models available in the repository: NAT-math-7b, NAT-math-13b, NAT-qa-7b, NAT-qa-13b, NAT-strategy-qa-7b, NAT-strategy-qa-13b. Tech stack: Python. Why it matters: This method enhances the performance of large language models by leveraging negative examples, improving data efficiency and achieving superior results compared to direct negative sample inclusion. Repository: Reason-Wang/NAT, stars: 28.

### Overview: SABER-labs/Drogon-torch-serve is a C++ project that serves PyTorch/Torch models using Drogon, an HTTP server framework for C++. This implementation includes an example for ResNet18 model for ImageNet, showcasing improved throughput and latency by up to 6-10x at peak load. ### Key Features: - Serve PyTorch/Torch models as REST APIs using Drogon. - Example for ResNet18 model for ImageNet. - Benchmarks show significant performance improvements over traditional approaches. ### Tech Stack: - C++ - Drogon ### Why It Matters: This project demonstrates the effectiveness of using Drogon for serving PyTorch models, highlighting the potential for improving performance and efficiency in server-side applications. ### GitHub Stars: 28 ### Build & Run Instructions: ```bash # Create optimized models for your machine. $ python3 optimize_model_for_inference.py # Build and Run Server $ docker compose run --service-ports blaze ``` ### Development Instructions: - Add Docker to CLion toolchain for setting up necessary dependencies. ### Client Instructions: ```bash curl "localhost:8088/classify" -F "image=@images/cat.jpg" ``` ### Benchmarking Instructions: ```bash # Drogon + libtorch for i in {0..8}; do curl "localhost:8088/classify" -F "image=@images/cat.jpg"; done # Run once to warmup. wrk -t8 -c10

Overview: BackdoorMBTI is an open-source project that extends unimodal backdoor learning to a multimodal context, aiming to facilitate the analysis and development of backdoor defense methods within a realistic and diverse scenario. Key features: - Multi-modal backdoor learning framework - Support for new and realistic scenarios and tasks - Comprehensive experimental results and documentation Tech stack: Python Why it matters: BackdoorMBTI provides a comprehensive tool kit for evaluating and developing backdoor defense methods in a multimodal context, making it valuable for researchers and practitioners in the field of cybersecurity and machine learning.