Projects

Overview: Zihuan Next is a high-performance, highly customizable Rust-based AI Agent platform that includes a Workspace Agent (for coding) and an IM Agent (for QQ bots, etc.). It supports various models and can be easily tweaked for different scenarios. Key features: - Workspace Agent (for coding) - IM Agent (for QQ bots, etc.) - Supports multiple models (Candle, llama.cpp, online APIs in various formats) Tech stack: Rust Why it matters: Zihuan Next is designed to be highly customizable and efficient, making it suitable for a wide range of AI agent use cases, including coding and instant messaging bots. Repository: FredYakumo/zihuan-next GitHub stars: 40 GitHub description: Zihuan is a high-performance, highly customizable Rust-based AI Agent platform. Zihuan Impls a interesting Workspace Agent (zihuan code for coding) and an IM Agent (QQ bots, etc.) as the basic framework. And Zihuan can easily tweak agent behavior and model inference details—whether using Candle, llama.cpp, or online APIs in various formats. Downloads: [Latest release](https://github.com/FredYakumo/zihuan-next/releases/latest) GitHub stars: 40 License: AGPL-3.0 Contributors: Thanks to everyone who has contributed to Zihuan Next!

### Overview: This project provides a high-quality image generation tool using Stable Diffusion, GFPGAN, Real-ESR, and SwinIR. It allows users to generate 4K and full HD images and artworks for free, without needing to generate token keys. The project is based on Jupyter Notebook and is compatible with Google Colab. ### Key features: - **Stable Diffusion**: Generates high-quality images from text prompts. - **GFPGAN**: Enhances human faces in images. - **Real-ESR**: Upscales images to 4K resolution. - **SwinIR**: Further enhances image quality. ### Tech stack: Jupyter Notebook ### Why it matters: This project is useful for anyone looking to generate high-quality images and artworks without the need for expensive token keys or specialized hardware. It leverages cutting-edge techniques in image generation and restoration, making it a valuable tool for artists, designers, and anyone interested in creating visually stunning content.

### Overview: ProteinReDiff is a state-of-the-art method for redesigning complex-based ligand-binding proteins using equivariant diffusion-based generative models. This project has been published in the Structural Dynamics journal, showcasing its effectiveness in generating high-quality protein-ligand complex structures. ### Key Features: - **Equivariant Diffusion Models**: Utilizes equivariant diffusion-based generative models to redesign complex-based ligand-binding proteins. - **Featurization of Ligand SMILES**: Featurizes the ligand using SMILES notation for efficient representation. - **Sequence Masking and Featurization**: Stochastically masks and featurizes protein sequences to enhance diversity. - **Single Representation Attention and Outer Product Mean**: Adaptations of Single Representation Attention and Outer Product Mean from AF2. - **Parameterization of β_T Diffusion**: Parametersize β_T diffusion instead of using variational lower bound in DPL. ### Tech Stack: - **Python**: The primary programming language used for implementing the model and generating protein-ligand complex structures. ### Why It Matters: ProteinReDiff offers a significant advancement in the field of protein-ligand complex structure generation, providing more efficient and diverse results compared to existing methods. Its ability to redesign complex-based ligand-binding proteins makes it highly valuable for drug discovery and drug design applications. ### Repository: - **Repository**: HySonLab/Protein_Redesign - **Stars**: 40 - **Description**: ProteinReDiff: Complex-based ligand-binding proteins redesign by equivariant diffusion-based generative models. -

### Overview: This repository contains the code for the paper "Asymmetry in Low-Rank Adapters of Foundation Models". It explores the asymmetry of the LoRA adapter matrices `B` and `A` in Foundation Models, providing insights into their initialization and update mechanisms. ### Key Features: - **Asymmetry Analysis**: Discover and analyze the asymmetry of LoRA adapter matrices `B` and `A`. - **Flexible Initialization**: Provide options to initialize matrices `A` and `B` with different settings. - `A` can be initialized as a random orthonormal matrix (`rand`) or frozen during training (`zero`). - `B` starts at zero and is updated during training. - **Wrapper for Transformers**: Offers a wrapper to compile with other models from Huggingface's transformers library, making it easy to integrate LoRASYM with various models. ### Tech Stack: - Python ### Why It Matters: The repository provides a detailed analysis of the initialization and update mechanisms of LoRA adapter matrices in Foundation Models, shedding light on their asymmetry and providing a flexible way to initialize and update them. This can be particularly useful for researchers and developers looking to improve the performance and stability of Foundation Models.

### Overview: BoundaryDiffusion is a learning-free method for semantic control with Diffusion Models, presented at NeurIPS 2023. It leverages pre-trained frozen diffusion models to achieve efficient and effective semantic control with minimal supervision. ### Key Features: - **Boundary Guided Learning-Free Semantic Control**: Automatically searches for and applies semantic boundaries in diffusion models. - **Efficient and Resource-Friendly**: Light-weight, efficient, and resource-friendly approach. - **State-of-the-Art Performance**: Achieves strong performance compared to learning-based methods. ### Tech Stack: Python ### Why It Matters: BoundaryDiffusion provides a novel and efficient method for semantic control in diffusion models, offering a learning-free approach that significantly enhances the control of diffusion models without the need for extensive training.

Overview: LitModels is a powerful tool designed to save, load, host, and share AI model checkpoints efficiently, ensuring minimal training slowdowns. It simplifies the process of deploying models on Lightning AI or your own enterprise-grade cloud storage. Key features: - Save model checkpoints without slowing down training - Load models from anywhere - Host models on Lightning AI or your own cloud with enterprise-grade access controls Tech stack: Python Why it matters: LitModels is essential for developers and researchers who need to quickly deploy and share AI models without compromising the performance of their training processes. It streamlines the deployment process, making it easier to share models across different environments and teams. Repository: GitHub - https://github.com/Lightning-AI/LitModels Primary language: Python GitHub stars: 40 GitHub description: Save, load, host, and share models without slowing down training. Host on Lightning AI or your own cloud with enterprise-grade access controls.

Overview: The rlx-models repository contains concrete model graph builders and weight loaders for the RLX multi-backend compiler, including support for various architectures such as language models, vision-language/omni models, and time-series forecasters. This repository serves as the "what actually runs" layer in the RLX monorepo. Key features: - 215 model families across 17 categories, including language models, vision-language/omni models, and time-series forecasters. - Comprehensive catalog of models, including embeddings, vision models, and audio codecs. - Standalone repo for easy access and integration with the RLX monorepo. Tech stack: Rust Why it matters: This repository provides the necessary infrastructure and models for the RLX monorepo, ensuring that the core components of the system are well-documented and easily accessible. Repository: MIT-RLX/rlx-models GitHub stars: 40 GitHub description: ML model ports (ASR · TTS · LLM · VLM · vision · audio codecs) on the rlx multi-backend compiler + runtime — CPU/Metal/MLX/CUDA/wgpu Primary language: Rust Build and test: Clone the repository and navigate to the `rlx-models` directory to run the tests. Install: Follow the instructions in the README to set up the environment and clone the repository. Quickstart — embeddings: Get started with embeddings by following the instructions in the README. High-level runner API: Utilize the high-level runner API for running models. Adding a new model

Overview: Nanotune is a simple, interactive CLI designed for fine-tuning small language models on Apple Silicon. It automates the MLX and llama.cpp tooling, making the process straightforward and user-friendly. Key features: - Auto-generated project setup - Interactive TUI for fine-tuning - No YAML configs or complex flags - Handles MLX and llama.cpp tooling automatically Tech stack: TypeScript Why it matters: Nanotune simplifies the process of fine-tuning language models, making it accessible to developers without requiring advanced knowledge of AI tooling or configuration.

Overview: This repository is a comprehensive resource for the self-supervised graph foundation models literature, organized by a knowledge-based taxonomy. It includes detailed pretexts and papers, categorized into microscopic, mesoscopic, and macroscopic pre-training, as well as multi-task pre-training and downstream tuning. Key features: - Microscopic pre-training: Node features, node properties, links - Mesoscopic pre-training: Context, long-range similarities, motifs, clusters - Macroscopic pre-training: Global structure, manifolds - Multi-task pre-training - Downstream tuning Tech stack: Python, Markdown, LaTeX Why it matters: This resource provides a structured overview of the latest research in self-supervised graph foundation models, making it an essential tool for researchers and practitioners in the field.

Overview: Empower your productivity with local AI assistants, designed to run offline and secure your data. Key features: - Offline AI capabilities - Seamless task automation - Secure data management Tech stack: TypeScript Why it matters: Opla is a cutting-edge productivity tool that offers a personalized approach to AI, ensuring your data privacy and security while enhancing your workflow efficiency.

Overview: Oqura's deepresearch-datagen-cli is a terminal tool designed to generate structured datasets from real-world data using deep research. It automatically suggests a schema, refines it with multi-step research, and outputs clean, usable data for quick experimentation, training tasks, or dataset needs without manual formatting. Key features: - Automatically suggests a schema based on the query - Refines the schema with follow-up adjustments - Breaks the dataset into focused sections and subtopics - Assigns a research agent to each section - Each agent runs web search, extracts key data, and summarizes it - Generates section-wise structured data - Merges all sections into a single final dataset - Saves the dataset in the `output_files` directory via the terminal Tech stack: Python Why it matters: This tool simplifies the process of generating structured datasets, making it easier and faster to experiment with and train large language models without the need for manual data collection and formatting. Repository: Oqura-ai/deepresearch-datagen-cli Primary language: Python GitHub stars: 40 GitHub description: Using deep research workflow to generate datasets for finetuning LLMs. GitHub description excerpt: Using deep research workflow to generate datasets for finetuning LLMs.

Overview: This project is an IBus Input Method that uses VOSK for speech-to-text functionality, written in Python. It supports multiple languages and formats text with automatic formatting and voice commands. Key features: - Speech-to-text functionality - Automatic text formatting - Voice commands for managing case, punctuation, and diacritics - Support for various languages - Built-in support for formatting numbers Tech stack: Python Why it matters: This project leverages VOSK for local voice recognition, improving accuracy and reducing reliance on external services. The automatic text formatting and voice commands make it versatile and user-friendly for various applications and languages. Repository: PhilippeRo/IBus-Speech-To-Text GitHub stars: 40 GitHub description: A speech to text IBus Input Method using VOSK, written in Python.

Overview: This repository introduces an enhanced version of the F5-TTS zero-shot voice cloning model, specifically designed for explicit emotion conditioning. The model, named F5-TTS-Emotion-CFG, builds upon the F5-TTS framework and fine-tunes it on the ESD dataset to support five emotions: Neutral, Happy, Sad, Angry, and Surprised. Key features: - Explicit emotion conditioning - Fine-tuned on the ESD dataset - Supports five emotions: Neutral, Happy, Sad, Angry, and Surprised Tech stack: Python Why it matters: This model offers a practical solution for voice cloning and text-to-speech applications, particularly in scenarios where the emotional tone of the generated voice is crucial. By incorporating explicit emotion conditioning, it enhances the realism and naturalness of the voice cloning, making it suitable for various applications such as voice acting, voiceovers, and more. Repository: RaduBolbo/F5-TTS-Emotional-CFG

Overview: LLaMA.MMEngine is an experimental repository that leverages the MMEngine training engine to train and fine-tune language models, with a focus on exploring compatibility and fine-tuning techniques. Key features: - Support for loading LLaMA models with parameter sizes ranging from 7B to 65B - Instruct tuning support - Low-rank adaptation (LoRA) fine-tuning support Tech stack: Python Why it matters: This repository aims to explore the compatibility of MMEngine with language models and learn fine-tuning techniques, fostering knowledge sharing and collaboration within the open-source community.

Overview: This project presents an end-to-end pipeline for automatic车牌 recognition on images and video streams. The system is built on top of modern neural network models and optimized for CPU performance. Key features: - **High Accuracy:** The system achieves >98% accuracy on test data. - **Two-Stage Pipeline:** Utilizes a custom model combination: 1. **Detector (YOLOv8n)** for precise localization of license plates. 2. **Recognizer (CRNN)** for reading characters. - **CPU Optimization:** The OCR model is quantized to INT8, resulting in ~2x acceleration on CPU while maintaining accuracy. - **Advanced Preprocessing:** Built-in perspective correction mechanism using OpenCV to straighten license plates captured at an angle. - **Video Processing:** Includes a tracking and stabilization algorithm for reliable车牌 recognition in video streams. - **Clean Code:** Project follows SOLID, DRY, KISS, and OOP principles for easy maintenance and extension. Tech stack: Jupyter Notebook Why it matters: This system is designed to provide high accuracy in车牌 recognition, making it suitable for various applications such as traffic management, license plate recognition systems, and more. Repository: Runoi/ANPR-System GitHub stars: 40 GitHub description: High-accuracy Russian ANPR system built with YOLOv8 for detection and an optimized, custom-trained PyTorch CRNN for OCR.

**Overview:** RPI (Resonant Permutation Inference) is a zero-multiply text generation engine that optimizes inference by replacing matrix multiplication with permutation table lookups and integer accumulation. This approach eliminates the need for billions of multiply-accumulate operations, making it highly efficient and scalable across various hardware architectures. **Key features:** - **Zero-multiply inference**: Replaces matrix multiplication with permutation table lookups and integer accumulation, eliminating the need for floating-point operations. - **Table-driven**: Uses a permutation table to route tokens, making it cache-resonant and hardware-timed. - **Standalone or speculative engine**: Runs on N64, POWER8, x86, and ARM, providing flexibility and performance across different hardware platforms. - **18,000 tok/s performance**: On POWER8, achieving 84+ tok/s with the `vsx` vec_perm instruction. - **0.3-3 MB model size**: Compact model size compared to traditional LLMs, reducing memory and power consumption. **Tech stack:** RPI is built using C, leveraging the cache-resonant architecture of POWER8, G4, and ARM processors. **Why it matters:** RPI's core insight is that inference is reordering, not arithmetic, making it highly efficient and scalable. By eliminating the need for billions of multiply-accumulate operations, RPI significantly reduces computational complexity and energy consumption, making it a game-changer for large language models.

Overview: x0-GPT is an advanced AI-powered tool that allows seamless interaction with websites and documents, including PDFs, using natural language. It supports chat with websites, PDFs, CSVs, and notes, making data extraction and automation easy. Key features: - User-friendly interface - Real-time updates - Dynamic routing - Supports almost all AI - Chat with websites - Chat with PDFs - Chat with CSVs - Chat with notes Tech stack: TypeScript Why it matters: x0-GPT is free and accessible, making it an ideal tool for anyone looking to interact with websites and documents using natural language.

Overview: This project presents a novel approach to frame interpolation using the Consecutive Brownian Bridge Diffusion model, aiming to improve the quality of interpolated frames in video sequences. Key features: - The autoencoder with flow estimation enhances the visual quality of frames decoded from the latent space. - The Consecutive Brownian Bridge Diffusion reduces cumulative variance during sampling, making it suitable for the frame interpolation task where there is a deterministic ground truth. - During inference, the decoder receives estimated latent features from the Consecutive Brownian Bridge Diffusion. Tech stack: Python Why it matters: This method achieves state-of-the-art performance in various quality metrics compared to recent state-of-the-art techniques, making it a significant advancement in the field of video frame interpolation.

Overview: This project provides a JAX implementation of a diffusion-based controller (DM-AC) for robust quadrotor dynamics learning. The controller is designed to handle disturbances such as payload variations and wind gusts, ensuring reliable trajectory tracking. The repository includes a clear, modular, and extensible codebase suitable for rapid prototyping of diffusion-based system identification for real-time deployment. Key features: - JAX implementation - Robust trajectory tracking under disturbances - Configurable training and inference - Integration with PX4 SITL via Gazebo and ROS 1/MAVROS for real-time control Tech stack: Python Why it matters: This project aims to provide a clear, modular, and extensible codebase for rapid prototyping of diffusion-based system identification for real-time deployment, making it suitable for various robotic platforms.

### Overview: This repository is a research fork of `antirez/ds4` designed to run the DeepSeek V4 Flash model on a Linux laptop with 7.7 GiB of RAM and no GPU, using demand paging backed by NVMe storage. The GGUF (Generalized Graphical Universal Format) size is 78.62 GiB, and the logical parameters are 284.33B. The model is CPU-only and uses Linux faults to page demand-loaded weights into memory. ### Key Features: - CPU-only execution of the DeepSeek V4 Flash GGUF - NVMe-backed demand paging - 78.62 GiB GGUF size - 7.7 GiB of physical RAM - No GPU support ### Tech Stack: C ### Why it Matters: This repository demonstrates the execution of a large model size (78.62 GiB) on a constrained system (7.7 GiB RAM) using demand paging backed by NVMe storage, showcasing the model's ability to handle extreme memory pressure and demonstrate the performance under a severe model-size-to-RAM mismatch scenario.