Projects

### Overview: x-unet is a comprehensive implementation of a U-Net model, featuring efficient attention mechanisms and the latest research findings. This project is designed to be efficient and scalable, making it suitable for various applications such as image segmentation, 3D processing, and more. ### Key features: - **Efficient Attention**: Utilizes cross-stage attention residuals to enhance model performance. - **Nested U-Net Depth**: Supports different depths of the nested U-Net structure, providing flexibility in model architecture. - **Cross-Stage Attention Residuals**: Enables the use of learned attention residuals for skip connections, improving model efficiency. - **3D Support**: Compatible with 3D data processing, including videos and medical imaging. - **Weight Standardization**: Implements weight standardization to improve model stability and performance. ### Tech stack: - **Programming Language**: Python ### Why it matters: x-unet offers advanced attention mechanisms and efficient attention residuals, making it a powerful tool for image and 3D data processing. Its flexibility and efficiency make it suitable for a wide range of applications, from medical imaging to video analysis. ### GitHub Stars: 294 ### Repository: - **Repository**: lucidrains/x-unet - **Primary Language**: Python - **GitHub Stars**: 294 ### Installation: ```bash $ pip install x-unet ``` ### Usage: ```python import torch from x_unet import XUnet # Basic usage unet = XUnet( dim = 64, channels

### Overview: VideoGPT+ is an enhanced model designed to improve video understanding by integrating image and video encoders. It processes videos in segments using adaptive pooling on features from both encoders, achieving excellent results across various benchmarks. ### Key Features: - **VideoGPT+ Model**: A dual-encoding scheme that leverages both image and video features for detailed spatial and temporal understanding. - **VCG+ 112K Dataset**: A novel dataset with 4,354 human-annotated QA pairs across 18 video categories, providing comprehensive evaluation for video-conversation tasks. - **VCGBench-Diverse Benchmark**: A diverse benchmark with 4,354 human-annotated QA pairs across 18 video categories, designed to evaluate the performance of video-conversation models. ### Tech Stack: - Python ### Why It Matters: VideoGPT+ addresses the limitations of existing benchmarks by providing a diverse and comprehensive dataset, ensuring that the model's performance is evaluated across a wide range of video categories and scenarios. This makes it a valuable tool for researchers and developers looking to improve video understanding and performance in various applications.

### Overview: Komputation is a neural network framework for the Java Virtual Machine written in Kotlin and CUDA C. It provides a comprehensive set of layers and functionalities for building and training neural networks, including feed-forward, convolutional, and recurrent networks, as well as activation functions and dropout layers. The framework is designed to be easily integrated into Java applications, making it accessible to developers familiar with Kotlin and Java. ### Key Features: - **Layers**: The framework includes standard feed-forward networks, convolutional neural networks (CNNs), and recurrent neural networks (RNNs), along with dropout layers and activation functions. - **Activation Functions**: Support for various activation functions such as Identity, Rectified Linear Units (ReLUs), Sigmoid, Softmax, and Tanh. - **CPU Demos**: Demonstrations of basic CPU operations like Boolean functions, total functions, running totals, and increment operations, providing a practical understanding of the framework's capabilities. ### Tech Stack: - **Kotlin**: The primary language for development. - **CUDA C**: Used for GPU acceleration, enabling efficient computation on modern GPUs. ### Why It Matters: Komputation simplifies the process of building neural networks by providing a comprehensive set of layers and functionalities. Its integration with Kotlin and CUDA C makes it accessible to developers familiar with these technologies, making it a valuable tool for developers looking to implement neural networks in their Java applications.

### Overview: Whisper Dictation is a versatile AI-powered voice assistant that integrates various functionalities such as hands-free recording, voice keyboard, AI chat, image generation, webcam control, and more. It runs on an old laptop from 2013 with at least 4 GiB of VRAM, making it a powerful tool for both personal and professional use. ### Key Features: - **Hands-Free Recording**: Use `record.py` to capture audio. - **Voice Keyboard**: Utilizes `whisper.cpp` for text input. - **AI Chat**: Interacts with LLM models like `ChatGPT` and `Google Gemini`. - **Webcam Control**: Voice commands to take pictures and perform searches. - **Voice-Controlled Apps**: Launches and controls apps using `pyautogui`. - **OpenAI Integration**: Optional integration with `mimic3` for voice responses. - **Translation**: Supports translation into English for enhanced functionality. - **Stable Branch**: The primary development branch, `main`, is available for stable features. ### Tech Stack: The project is built using Python, GStreamer, and various AI libraries like Whisper, Llama.cpp, and Stable Diffusion WebUI. ### Why It Matters: Whisper Dictation offers a unique combination of AI capabilities, voice control, and versatile features, making it a valuable tool for both personal and professional use. Its ability to perform tasks like dictation, image generation, and voice control makes it a versatile assistant that can be used in various settings. ### Repository: The project

### Overview: This repository is an awesome collection of papers and code related to diffusion model-based video editing, also known as video-to-video (V2V) translation. It serves as a comprehensive resource for researchers and developers interested in this field. ### Key features: - A collection of papers on diffusion model-based video editing. - A benchmark code for video editing. - Papers listed in reverse chronological order, with conference/journal year format. ### Tech stack: Python ### Why it matters: This repository provides a valuable resource for researchers and developers looking to explore and contribute to the field of video editing using diffusion models. It serves as a starting point for understanding the latest advancements and techniques in this area. ### GitHub stars: 294 ### Repository: wenhao728/awesome-diffusion-v2v ### Primary language: Python ### GitHub description: Awesome diffusion Video-to-Video (V2V). A collection of paper on diffusion model-based video editing, aka. video-to-video (V2V) translation. And a video editing benchmark code. ### Repository assets: -  ### Citation: If you find this repository helpful, please consider citing our paper: ```bibtex @article{sun20

### Overview In speech processing, keyword spotting involves identifying specific keywords within spoken utterances. This repository is a curated list of outstanding papers on Speech Keyword Spotting (Wake-Up Word Detection). ### Key Features - **Keyword Spotting Papers**: Includes a comprehensive collection of research articles on keyword spotting. - **Publications**: Covers a wide range of years from 2018 to 2022, showcasing advancements in the field. - **Efficient Algorithms**: Discusses various techniques for improving keyword spotting accuracy and efficiency. - **Real-World Applications**: Highlights projects that use keyword spotting in practical applications, such as wake-up word detection. ### Tech Stack - **Programming Languages**: Python, Java, C++, TensorFlow, PyTorch, etc. - **Frameworks**: Keras, PyTorch, TensorFlow, etc. - **Libraries**: Librosa, FastSpeech, etc. ### Why It Matters - **Innovation in Speech Processing**: Provides insights into cutting-edge research in speech processing. - **Practical Applications**: Demonstrates how keyword spotting is applied in real-world scenarios, enhancing user experience and accessibility. ### Repository: zycv/awesome-keyword-spotting - **Stars**: 294 - **Description**: A curated list of awesome Speech Keyword Spotting (Wake-Up Word Detection) papers. ### Primary Language: Unknown - **GitHub Stars**: 294

### Overview: This repository provides a production-ready blueprint for the Agentic Era, allowing developers to replace rigid, hard-coded workflows with a dynamic, transparent, observable, and sovereign Context Engine. By building universal, domain-agnostic Multi-Agent Systems through high-level semantic orchestration, you can save thousands of lines of code while maintaining 100% observability. ### Key Features: - **Transparent and Observable Context Engine**: A dynamic, transparent, observable, and sovereign Context Engine that integrates with Natural-Language-Programmed LLMs and domain-agnostic dual-RAG MAS environments. - **Universal and Domain-Agnostic Multi-Agent Systems (MAS)**: High-level semantic orchestration to build universal, domain-agnostic MAS. - **Production-Ready Blueprint**: Provides a production-ready blueprint for the Agentic Era, ensuring 100% observability. ### Tech Stack: - Jupyter Notebook ### Why It Matters: By using this repository, developers can save thousands of lines of code while maintaining 100% observability and transparency in their multi-agent systems. This approach allows for a more flexible and dynamic architecture compared to traditional hard-coded workflows, enabling the execution of natural-language-programmed LLMs and domain-agnostic dual-RAG MAS environments.

Overview: Revornix is an open-source, local-first AI information/markdown workspace designed to help users collect fragmented inputs, turn them into structured knowledge, generate reports with images and podcast audio, and deliver the output through automated notifications. Key features: - Collect and ingest various types of content - Convert and clean the collected content into Markdown format - Organize and store content in Milvus and Neo4j databases - Generate AI-generated summaries, illustrated reports, podcasts, and notifications - Support for multiple OpenAI-compatible providers Tech stack: TypeScript Why it matters: Revornix simplifies the process of collecting, organizing, and delivering information, making it easier and more efficient for users to manage their knowledge and share it with others. Repository: Qingyon-AI/Revornix GitHub stars: 292 GitHub description: Revornix is an open-source, local-first AI information/markdown workspace. It helps you collect fragmented inputs, turn them into structured knowledge, generate reports with images and podcast audio, and deliver the output through automated notifications.

Overview: ROScribe is a tool that allows developers to write ROS packages in minutes using a natural language interface. Key features: - Automatically generates ROS packages for robotic projects - Combines LLMs with prompt tuning techniques - Supports identification and visualization of ROS nodes and topics - Generates code or open-source repositories for each ROS node - Integrates code base over generated and open-source assets - Creates ROS launch file and package files - Provides user support throughout the project lifecycle Tech stack: Python Why it matters: ROScribe simplifies the process of writing ROS packages, making it accessible to both beginners and experienced engineers. It automates the creation of ROS packages, reducing the time and effort required, and streamlines the development process by integrating open-source assets.

Overview: RaBitQ-Library is an official lightweight library for the RaBitQ algorithm, providing vector search functionalities and applications in vector quantization and approximate nearest-neighbor search. Key features: - Modular C++ components - Stable core API v1 - Optional OpenMP for numerical components - Unified index counts - Includes indexes for IVF-RaBitQ, HNSW-RaBitQ, and SymphonyQG - Supports L2 and inner product searches - Normalizes vectors for cosine search - Provides different QPS–recall tradeoffs for storage and updates Tech stack: C++ Why it matters: RaBitQ-Library offers a comprehensive solution for vector quantization and approximate nearest-neighbor search, backed by research-backed algorithms. It simplifies the implementation of vector search functionalities, making it accessible to developers and researchers alike.

Overview: This repository contains the official implementation of the **Contrastive Audio-Visual Masked Autoencoder (CAV-MAE)**, proposed in the ICLR 2023 paper [Contrastive Audio-Visual Masked Autoencoder](https://openreview.net/forum?id=QPtMRyk5rb). The repository includes the code and pre-trained models for audio-visual event classification tasks. Key features: - **CAV-MAE Model**: The implementation of the CAV-MAE model in PyTorch. - **Data Preparation**: Steps for extracting audio tracks and image frames from videos, building label sets, and generating JSON files. - **Pretraining**: Instructions for adapting Vision-MAE checkpoints, setting up a virtual environment, and running pretraining. - **Audio-Visual Event Classification**: Detailed steps for audio-set and vgg-sound datasets. - **Retrieval**: Techniques for audio-visual retrieval. - **Inpainting**: Methods for image inpainting. - **Pretrained Models**: Comprehensive list of pre-trained models, including main, ablation study, pretrained+finetuned, and data lists for AudioSet and VGGSound. Tech stack: Python Why it matters: The CAV-MAE model achieves a new state-of-the-art accuracy of 65.9% on VGGSound and is comparable with the previous best supervised pretrained model on AudioSet, demonstrating its effectiveness in audio-visual event classification tasks. GitHub stars: 292 Repository: YuanGongND/cav-m

### Overview: PostBot 3000 is an open-source project that demonstrates how to build a powerful AI agent capable of generating responses and artifacts. This project simplifies the implementation of similar solutions for developers. ### Key Features: 1. **OpenSource:** All the code (`agent-ui` & `agent-service`) is open-source and available on GitHub. 2. **Artifacts:** The posts content generated by the agent is streamed in Artifacts, such as Claude. 3. **Agent Python Backend:** The agent is built using LangGraph for AI workflows and FastAPI for creating a robust API. ### Tech Stack: - LangGraph - Vercel AI SDK - gpt-4o-mini - FastAPI - Next.js - TailwindCSS - Clerk Auth - Upstash Redis ### Why it Matters: This project provides a comprehensive guide for anyone looking to implement similar AI agent solutions, making it easier to build powerful AI agents and stream responses and generate artifacts. ### Repository: ahmad2b/postbot3000 ### Primary Language: TypeScript ### GitHub Stars: 292 ### GitHub Description: PostBot 3000 is an open-source project that shows how to build a powerful AI agent and stream responses and generate artifacts. This project makes it easier for anyone looking to implement similar solutions.

Overview: Modular Diffusion is a Python library designed for creating and training custom Diffusion Models using PyTorch. It offers a modular design, pre-built modules, and integration with PyTorch, making it suitable for both enthusiasts and researchers. Key features: - Highly modular design for swapping components of the diffusion process. - A growing library of pre-built modules. - Easy creation of custom modules by inheriting from a base class. - Integration with PyTorch for developing custom modules. - Applications ranging from image generation to non-autoregressive text synthesis. Tech stack: Python, PyTorch. Why it matters: This framework provides a user-friendly interface for designing and training custom Diffusion Models, making it accessible to both beginners and advanced users in the field of machine learning and diffusion models.

### Overview: This project introduces an online speech recognition system utilizing RNN Transducer (RNN-T) trained on diverse voice datasets. The system achieves significant speed improvements over YouTube's captioning, running online speech recognition on YouTube Live videos with up to 4-10 seconds faster than YouTube's captioning on a 2.3GHz dual-core Intel Core i5 processor. ### Key Features: - **Online Speech Recognition**: The system supports real-time speech recognition. - **Model Training**: The model is trained on 2000+ hours of audio speech data and has been pre-trained on diverse voice datasets. - **Model Release**: The trained model is available for release, including pre-trained weights for offline use. - **Training Tips**: The project provides detailed training tips and insights, including balanced encoder and predictor, larger vocabulary size, and batch size optimization. ### Tech Stack: - **Programming Languages**: Python - **Model**: RNN Transducer (RNN-T) ### Why It Matters: This project demonstrates the effectiveness of RNN Transducer in real-time speech recognition, offering a significant performance improvement over YouTube's captioning. The system's ability to run online speech recognition on YouTube Live videos is particularly noteworthy, making it a valuable tool for live streaming and transcription needs.

### Overview: i1 is a 3B-parameter text-to-image diffusion model designed to explore the design space of text-to-image models and understand how different modeling and data choices affect their capabilities. This model performs competitively with leading models at 1024-resolution, as measured by GenEval, DPG-Bench, PRISM, CVTG-2K, and LongText-Bench. ### Key Features: - **3B Parameter Model**: i1 is a 3B-parameter model that performs well at 1024-resolution. - **Fully Open-Source**: The code, data, and recipes are fully open-source, allowing others to reproduce the model. - **General Image Generation**: Generates a wide variety of images, including general images and text-rendering. - **Dataset**: Includes a dataset of captions and image-caption pairs. ### Tech Stack: Python ### Why it Matters: i1 provides a valuable resource for researchers and developers looking to improve the performance and capabilities of text-to-image models. Its open-source nature allows for easy replication and further research, making it a significant contribution to the field of text-to-image generation.

Overview: RepoMind is an open-source AI-powered platform that uses Agentic CAG to analyze public GitHub repositories and developer profiles, providing deep code analysis, visual architecture maps, and security audits. Key features: - Context-aware analysis - Visual understanding - Security scanning built in - Live step feedback - Real-time repository search Tech stack: TypeScript Why it matters: RepoMind simplifies the process of understanding and interacting with public GitHub repositories and developer profiles, offering a fast, browser-first experience with AI-powered insights. Repository: 403errors/repomind GitHub stars: 291

### Overview: InsTag: A Tool for Data Analysis in LLM Supervised Fine-tuning InsTag is a tool designed to analyze and tag data in the context of large language models (LLMs) fine-tuned for supervised fine-tuning (SFT). It leverages semantic and intent-based tagging to enhance the understanding and analysis of SFT datasets, particularly focusing on query diversity and complexity. ### Key Features: - **InsTagger**: A LLaMa-2-based tagging model fine-tuned on InsTag's tagging results, optimized for local tagging deployments. - **TagLM-13B-v1.0** and **TagLM-13B-v2.0**: Fine-tuned models based on sub-sampled SFT data from open-source datasets, achieving superior performance on MT-Bench compared to open-source models. - **InsTag**: An open-set fine-grained tagger that generates 6.6K comprehensive tags to describe user queries, providing a basis for selecting diverse and complex samples for model fine-tuning. ### Tech Stack: LLaMa-2 InsTag is built on the 7B version of LLaMa-2, utilizing FastChat for training and inference. ### Why It Matters: InsTag's focus on query diversity and complexity is crucial for improving the performance of LLMs in supervised fine-tuning. By selecting and fine-tuning models on a more diverse and complex dataset, InsTag demonstrates the importance of query diversity and complexity in achieving better results.

### Overview: OpenASR is a local-first speech-to-text solution that runs entirely on your device. It offers a seamless transcription experience without requiring a terminal or network connectivity, making it perfect for private use and remote collaboration. ### Key Features: - **Transcribe Audio Files**: Convert audio files or entire folders into text transcripts. - **Live Captions**: Real-time transcription from your microphone, ideal for live events. - **System Audio Capture**: Caption meetings, lectures, and podcasts by recording what your computer plays. - **Speaker Separation**: Automatically label who said what. - **Translation**: Transcribe and translate to English in one step. - **Local API**: Compatible with OpenAI SDKs, providing an OpenAI-compatible `/v1/audio/transcriptions` endpoint. ### Tech Stack: Rust CLI, local OpenAI-compatible HTTP API, and ggml inference engine. ### Why It Matters: - **Private**: Audio stays on your machine, ensuring privacy and no telemetry. - **Broad**: Supports multiple models, including Whisper, Qwen3-ASR, Parakeet, SenseVoice, FireRed, Dolphin, Moonshine, and more. - **Open**: The engine is Apache-2.0 licensed, and all model packs come with their own upstream licenses. ### For Developers: - **CLI Quickstart**: ```bash # Option A: Homebrew brew install quintinshaw/tap/openasr # Option B: One-line Installer curl -fsSL https://dl.openasr.org/install.sh | sh

### Overview: This repository is a curated list of papers related to large graph models, focusing on the intersection of large language models and graph machine learning. It aims to provide researchers and practitioners with a comprehensive overview of the latest advancements in this field. ### Key Features: - **Perspective and Survey**: Includes seminal papers on the intersection of large language models and graph models, such as "Graph Meets LLMs: Towards Large Graph Models" and "Towards Graph Foundation Models: A Survey and Beyond". - **Model**: Covers various models that leverage large language models for graph reasoning, including "LLMs as Graph Models" and "ChatGraph". - **Technologies**: Utilizes GitHub stars and forks to indicate the popularity and engagement of the repository. ### Tech Stack: Python, Markdown, and GitHub. ### Why it Matters: This repository serves as a valuable resource for researchers and practitioners interested in the intersection of large language models and graph machine learning. It provides a curated list of papers that explore the latest advancements in this field, making it easier for individuals to stay updated and contribute to the ongoing research.

Overview: AgentDesk is an open-source AI customer support system designed for teams requiring online support, knowledge-based Q&A, human collaboration, and service tracking in one system. It combines AI-first support with human-ready operations. Key features: - AI-first support: AI Agents handle common questions and knowledge-based answers. - Knowledge-constrained replies: Use RAG and Answerability Gate to ensure accurate responses. - Natural human handoff: Move to human agents when knowledge is insufficient. - Conversation-to-ticket loop: All support interactions are tracked in one system. - Built for extension: Supports Go backend, Next.js frontend, and OpenAI-compatible model access. - Self-host friendly: SQLite/MySQL and Qdrant options for local trials, intranet, and enterprise self-hosting. - Core capabilities: AI Agent support, online conversation system, agent workspace, knowledge-base RAG, Answerability Gate, ticket system, support organization management, AI extensibility, and multiple entry points. Tech stack: Go, Next.js, Skills, MCP, OpenAI-compatible model access, SQLite/MySQL, Qdrant. Why it matters: AgentDesk is an AI-first support system that combines AI with human collaboration, making it suitable for teams needing knowledge-based Q&A and human collaboration in one system.