Project directory

Projects

Open-source AI projects tracked by the leaderboard, ordered by GitHub stars.

Python 91 stars

invoker-LL/WSI-finetuning

This is the official repository for our CVPR 2023 paper 'Task-Specific Fine-Tuning via Variational Information Bottleneck for Weakly-Supervised Pathology Whole Slide Image Classification'.

Overview: This repository contains the official source code for the CVPR 2023 paper "Task-Specific Fine-Tuning via Variational Information Bottleneck for Weakly-Supervised Pathology Whole Slide Image Classification". Key features: - Implementation of the proposed task-specific fine-tuning method using variational information bottleneck. - Baseline model training using CLAM framework. - Variational Information Bottleneck (VIB) training for improved performance. - End-to-end training with top-k patches for better accuracy. - Fine-tuning of patch backbone for improved classification. Tech stack: Python Why it matters: This repository provides a comprehensive solution for whole slide image classification using variational information bottleneck, showcasing the effectiveness of the proposed method in improving weakly-supervised classification tasks.

Python 91 stars

liyanlin06/UniM

[CVPR 2026] Official repo of UniM

### Overview: UniM is a unified any-to-any interleaved multimodal benchmark designed to evaluate both multimodal understanding and generation under a single paradigm. This project aims to provide a comprehensive and standardized evaluation platform for researchers in the field of multimodal AI. ### Key Features: - **Unified Benchmark**: A single framework for evaluating multimodal understanding and generation. - **Interleaved Architecture**: Combines different modalities in a sequential manner to simulate real-world interactions. - **Modality-Blended Batch Sampler**: Allows for the creation of varied batch sizes and compositions of input modalities. ### Tech Stack: Python ### Why It Matters: UniM offers a standardized benchmark that helps researchers across various domains (e.g., computer vision, natural language processing, robotics) to evaluate the performance of their models in a consistent and comparable manner. This makes it an essential tool for advancing the field of multimodal AI and improving the overall quality of research in this area. Repository: liyanlin06/UniM Primary Language: Python GitHub Stars: 50 GitHub Description: [CVPR 2026] Official repo of UniM

TypeScript 91 stars

nathannlu/aperture

🔬 Visualize attention layers from Stable Diffusion

Overview: This project visualizes attention layers from Stable Diffusion models, providing insights into how attention is distributed across words in the input prompt. Key features: - Visualizes attention layers from Stable Diffusion models - Supports models Flux Dev and Stable Diffusion 2.1 Tech stack: TypeScript Why it matters: This tool helps developers understand how attention mechanisms work in Stable Diffusion models, which can be crucial for optimizing and improving model performance.

Python 91 stars

neis-lab/mmcows

MmCows: A Multimodal Dataset for Dairy Cattle Monitoring

### Overview: MmCows is a multimodal dataset designed for the monitoring of dairy cattle. It comprises two main parts: sensor data from wearable sensors and visual data captured by multiple cameras. The dataset includes 14 days of sensor data, 15-second interval visual data, and internet-time synchronized frames. The visual data is further divided into UWB-synchronized images and 1-second interval videos. Additionally, high-resolution photos of individual cows are provided for reference, and cropped bounding boxes and pre-trained weights for behavior classification and identification are included. ### Key features: - **Sensor Data**: 14-day data from wearable sensors. - **Visual Data**: 15-second interval visual data on 7/25 with 20,000 annotated images. - **UWB-Synchronized Images**: 14-day UWB-synchronized images with a sampling rate of 15 seconds. - **1-Second Interval Videos**: Internet-time synced frames throughout 14 days with a sampling rate of 1 second. - **Cropped Bounding Boxes**: 13 GB of cropped bounding boxes for behavior classification, lying cow identification, and non-lying cow identification. - **Pre-trained Weights**: 0.4 GB of pre-trained weights for non-vision models. - **Benchmarking**: Provides benchmarks for UWB-related models, including performance testing and correlation analysis between cows' behavior changes and THI. ### Tech Stack: Python ### Why it Matters: MmCows offers a comprehensive dataset for monitoring dairy cattle, enabling researchers and

TypeScript 91 stars

oil-oil/dsh-vision

Near-native image understanding for DeepSeek Harness

Overview: dsh-vision is a plugin for DeepSeek Harness that enables native image understanding, providing a seamless integration with text-only models. Key features: - Native image understanding for DeepSeek Harness - Supports both image and text-only models - Automatically routes to the most appropriate model based on the input type - Integrates with Vision Recognition cards for additional capabilities Tech stack: TypeScript Why it matters: dsh-vision simplifies the integration of image understanding capabilities with text-only models in DeepSeek Harness, enhancing the overall user experience and flexibility of the system. Repository: oil-oil/dsh-vision GitHub stars: 91 GitHub description: Near-native image understanding for DeepSeek Harness README excerpt:

Python 91 stars

rustyneuron01/AI-Generated-Content-Detection

Multi-modal AI-generated content detection: image, video, and audio. Benchmarks, training code (DINOv2, DINOv3, ReStraV, BreathNet), and evaluation pipeline for real vs. synthetic classification with calibration-aware metrics.

### Overview: This repository contains benchmark tooling, model training code, and evaluation pipelines for binary classifiers that distinguish genuine media from synthetic or AI-generated content across image, video, and audio modalities. The project aims to evaluate models that answer the question: *Is this image, video, or audio real (human-captured) or AI-generated?* It includes a unified benchmark suite and focuses on generalization across diverse generators and datasets. ### Key features: - **Three modalities**: Separate models for image, video, and audio. - **Unified evaluation**: A single benchmark suite (GASBench) runs all models on the same datasets and metrics. - **Focus on generalization**: Training and evaluation are designed to perform well on both public datasets and unseen (holdout) data, avoiding dataset memorization. ### Tech stack: - **Models**: - **Image**: DINOv3 (ViT backbone) with LoRA fine-tuning for better generalization. - **Video**: DINOv2 + temporal head (GenD-style) or ReStraV-style with frozen DINOv2 and trajectory geometry. - **Audio**: BreathNet. - **Training methods**: - **LoRA**: Used for efficient fine-tuning of vision backbones. - **Reinforcement learning**: Applied in the training pipeline to optimize for the combined benchmark metric. ### Why it matters: The project is crucial for content authentication, media safety, and research on generalization across diverse generators and datasets.

Python 91 stars

sangyun884/fast-ode

Official PyTorch implementation for the paper Minimizing Trajectory Curvature of ODE-based Generative Models, ICML 2023

### Overview: Fast-ODE is an official PyTorch implementation of the paper "Minimizing Trajectory Curvature of ODE-based Generative Models" presented at ICML 2023. This repository offers a comprehensive set of tools for training and generating samples from ODE-based generative models, specifically focusing on minimizing the curvature of the generative trajectories. ### Key features: - **Training Code**: Provides scripts for training the forward process in ODE-based generative models. - **Image Generation**: Includes a script for generating images using the trained forward process. - **FID Calculation**: Offers a script to calculate the FID score for evaluating the quality of generated images. - **Training Examples**: Contains scripts for training on specific datasets like CIFAR-10, MNIST, and FFHQ. - **Distillation**: Provides a script for distilling the learned flow model from a pre-trained model to a new dataset. - **Sample Generation**: Includes a script for generating samples from the trained forward process, with options to save trajectories and visualize them. ### Tech stack: Python ### Why it matters: This implementation is crucial for researchers and practitioners interested in generating images and videos using ODE-based generative models. By minimizing the curvature of the generative trajectories, it aims to reduce the computational cost of sampling, making the models more efficient and practical for real-world applications.

Go 91 stars

vorojar/VoiceSnap

Open-source offline voice dictation — a free alternative to Typeless. 100% local, zero internet, ideal for air-gapped & classified environments. SenseVoice + DirectML GPU, ~34MB, hotkey-driven. 纯离线语音输入，Typeless 免费平替，数据永不出设备，适合党政机关等涉密场景。

### Overview: VoiceSnap 是一款纯离线语音输入工具，无需联网即可实现语音转文字功能。它支持多种输入模式，包括长按模式和自由说话模式，同时提供自定义快捷键、录音音效、剪贴板保护等功能，适用于党政机关等涉密场景。 ### Key features: - **完全离线**：SenseVoice模型本地推理，无数据上传。 - **DirectML GPU 加速**：自动检测显卡，CPU自动回退。 - **两种输入模式**：长按说话 + 短按自由说话。 - **自定义快捷键**：支持 Ctrl / Alt / Shift / 功能键 / 字母键。 - **麦克风选择**：下拉切换输入设备。 - **录音音效**：马林巴风格提示音（可关闭）。 - **录音计时**：实时显示录音时长（0:00 → 0:03 → ...）。 - **Escape 取消**：录音中按 Esc 立即取消。 - **剪贴板保护**：粘贴后自动恢复原剪贴板内容。 - **识别历史**：自动保存最近 50 条，可复制、删除、设置保留时长。 - **语气词过滤**：自动去除"嗯、啊、呃"及 um/uh/hmm 等填充词。 - **静音自动停止**：连续 3 秒无声自动结束并识别。 - **中英混合优化**：自动

Shell 91 stars

xlba2khlnrdsd8f/seedream5.0-seedream-5.0-api

seedream-5.0 API (seedream5.0 / seedream 5.0): 1K-layer $0.0146; 1K $0.0293; 2K-layer $0.0293. Model id, api settings, curl and Python examples over an OpenAI-compatible gateway, $

Overview: Seedream 5.0 API provides high-quality image generation at a cost-effective rate, suitable for various applications. Key features: - 1K-layer $0.0146 - 1K $0.0293 - 2K-layer $0.0293 Tech stack: Shell Why it matters: This API offers excellent image generation at competitive prices, making it ideal for businesses and developers seeking reliable and affordable solutions.

TypeScript 90 stars

EM-GeekLab/LLMOne

Enterprise-grade LLM automated deployment tool that makes AI servers truly "plug-and-play".

### Overview: LLMOne 是一款开源、轻量的企业级大语言模型部署与服务平台。它旨在解决传统大模型私有化部署中的周期长、配置复杂、性能难以保障、运维成本高等痛点。 ### Key Features: - **极速自动化部署：** 全栈自动化，小时级完成从系统到模型的部署，真正实现“开箱即用”。 - **专业级推理性能：** 集成 vLLM 等领先推理引擎，深度优化主流 AI 硬件，保障企业级性能与高可靠性。 - **广泛兼容适配：** 支持主流开源模型与硬件，如 NVIDIA DGX 级别服务器、Mac Studio/Mini 等。 ### Tech Stack: TypeScript ### Why it Matters: 通过高度自动化的部署流程和优化的运行环境，LLMOne 将复杂的部署过程简化为几次点击操作，确保企业级性能和稳定性，让企业能更专注于大模型应用的创新和价值实现。 Repository: EM-GeekLab/LLMOne

TypeScript 90 stars

FishWoWater/Open3DStudio

A self-hosted cross-platform 3DAIGC software. Working with 3DAIGC algorithms completely deployed locally. Supported 3D workflows include MeshGen, Mesh Retopology, UV Unwrapping, TexGen, MeshSeg, AutoRig, Mesh Editing etc.

Overview: Open3DStudio is a self-hosted cross-platform 3D AIGC application that integrates with the 3DAIGC-API to provide locally deployed and free 3D workflows. It supports text-to-3D, image-to-3D, mesh segmentation, texture generation, auto-rigging, mesh re-topologize, and mesh UV unwrapping. Key features: - Mesh Editing (text-based and image-based) - Mesh Retopology and Mesh UV Unwrapping - Support for running in user management mode - Support for reusing mesh from a previous task as input - Improved 3D preview, UV layout viewing, and two-sided rendering - Multiple rendering modes and task management with progress and history Tech stack: TypeScript Why it matters: Open3DStudio offers a locally deployed and free solution for advanced 3D AIGC workflows, making it accessible and scalable for developers and creators. GitHub stars: 90 Repository: FishWoWater/Open3DStudio

Python 90 stars

IS2AI/TurkicASR

A multilingual ASR model that can recognize ten Turkic languages—Azerbaijani, Bashkir, Chuvash, Kazakh, Kyrgyz, Sakha, Tatar, Turkish, Uyghur, and Uzbek.

Overview: TurkicASR is a multilingual Automatic Speech Recognition (ASR) model designed to recognize ten Turkic languages, including Azerbaijani, Bashkir, Chuvash, Kazakh, Kyrgyz, Sakha, Tatar, Turkish, Uyghur, and Uzbek. Key features: - Supports ten Turkic languages - Provides pre-trained models for efficient inference - Compatible with various datasets and training scripts Tech stack: Python Why it matters: This repository offers a comprehensive solution for multilingual speech recognition, making it valuable for researchers and developers working with Turkic languages.

Java 90 stars

Kaljurand/speechutils

Android library for speech-to-text and text-to-speech apps

Overview: Speechutils is an Android library designed to facilitate the development of apps that require speech-to-text and text-to-speech functionalities. Key features: - Provides methods for audio recording and encoding - Aggregates speech-to-text and text-to-speech services - Offers audio cues before and after speech-to-text - Supports pausing background audio during speech-to-text Tech stack: Java Why it matters: Speechutils simplifies the implementation of speech-to-text and text-to-speech features in Android apps, making it easier for developers to create applications that can interact with speech recognition and synthesis technologies.

Python 90 stars

Open-Speech-EkStep/vakyansh-wav2vec2-experimentation

Repository containing experimentation platform on how to train, infer on wav2vec2 models.

Overview: This repository contains an experimentation platform designed to train, infer on, and generate outputs from Wav2Vec2 models, focusing on various Indic languages. Key features: - Pretrained models for various Indic languages - Installation and setup instructions for Python environment - Directory structure for organization of models, data, logs, and scripts - Support for pretraining, finetuning, and inference processes - Single file inference capability for quick results Tech stack: Python Why it matters: This platform provides a comprehensive solution for developers and researchers working with Wav2Vec2 models, enabling efficient training, fine-tuning, and inference across multiple languages, making it valuable for language modeling and natural language processing applications.

Python 90 stars

Tako-Research/TakoVM

A secure file system for your agents to execute code

Overview: Tako-Research/TakoVM is a secure file system designed for agents to execute code, providing job queues, Docker isolation, execution history, retries, and more. Key features: - Job queue - Execution history - Docker isolation - Network isolation - Self-hosted Tech stack: Python Why it matters: Tako VM offers a complete solution for secure, isolated code execution, including job queues, execution history, and network isolation, making it suitable for enterprise-level deployments. Repository: Tako-Research/TakoVM GitHub stars: 90 GitHub description: A secure file system for your agents to execute code README excerpt: A secure file system for your agents to execute code. Every job runs in its own isolated Docker container, with an ephemeral workspace, optional gVisor sandboxing, job queues, retries, and execution history included. Tech stack: Python Why it matters: Tako VM offers a complete solution for secure, isolated code execution, including job queues, execution history, and network isolation, making it suitable for enterprise-level deployments. Repository: Tako-Research/TakoVM GitHub stars: 90 GitHub description: A secure file system for your agents to execute code README excerpt: A secure file system for your agents to execute code. Every job runs in its own isolated Docker container, with an ephemeral workspace, optional gVisor sandboxing, job queues, retries, and execution history included. Tech stack: Python Why it matters: Tako VM offers a complete solution for secure, isolated

Python 90 stars

Vicky0522/I2VEdit

[SIGGRAPH Asia 2024] I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion Models

### Overview: I2VEdit is an official PyTorch implementation of the SIGGRAPH Asia 2024 paper "[I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion Models](https://arxiv.org/abs/2405.16537)". It provides a first-frame-guided video editing framework that leverages image-to-video diffusion models to propagate edits from a single frame to the entire video. ### Key features: - **First-Frame-Guided Editing**: Propagates edits from a single frame to the entire video. - **Image-to-Video Diffusion Models**: Utilizes diffusion models to refine the appearance and motion consistency of edits. - **Resource-Efficient**: Optimizes time and resource costs for both training and inference. ### Tech stack: Python ### Why it matters: I2VEdit offers a novel approach to video editing by leveraging diffusion models to propagate edits from a single frame to the entire video, making it a significant advancement in the field of video editing and image-to-video diffusion models.

Python 90 stars

WindQAQ/listen-attend-and-spell

Tensorflow implementation of "Listen, Attend and Spell" authored by William Chan. This project utilizes input pipeline and estimator API of Tensorflow, which makes the training and evaluation truly end-to-end.

Overview: The project implements the Listen, Attend, and Spell (LAS) model using Tensorflow, providing an end-to-end training and evaluation process. Key features: - Input pipeline and estimator API integration for Tensorflow - Conversion of data into TFRecord format - Creation of a vocabulary table for labels - Training and evaluation of the LAS model using Tensorflow Estimator API - Visualization of training progress with Tensorboard Tech stack: Python Why it matters: This project provides a practical implementation of the LAS model using Tensorflow, making it easier for developers to understand and utilize the model.

Go 90 stars

acheong08/vectordb

A simple vector database: Text encoding, semantic search, document storage

Overview: A simple vector database designed for Go, offering text encoding, semantic search, and document storage. Key features: - Text encoding - Semantic search - Document storage Tech stack: Go Why it matters: This library provides efficient text similarity search capabilities, making it suitable for applications requiring fast and accurate semantic search and document management. Repository: acheong08/vectordb GitHub stars: 90

Python 90 stars

ai-bot-pro/achatbot

An open source chat bot architecture for voice/vision (and multimodal) assistants, local(CPU/GPU bound) and remote(I/O bound) to run.

Overview: This project is an open-source chat bot architecture designed for voice, vision, and multimodal assistants. It supports various models and protocols for local and remote execution, making it versatile for different environments. Key features: - Support for Model Context Protocol (MCP) and Agent2Agent (A2A) Protocol - Features listed in the README Tech stack: Python Why it matters: This project offers a robust framework for developing chat bots with a wide range of functionalities, making it suitable for various applications in the field of AI and voice/vision assistants. Repository: ai-bot-pro/achatbot GitHub stars: 90 GitHub description: An open source chat bot architecture for voice/vision (and multimodal) assistants, local (CPU/GPU bound) and remote (I/O bound) to run.

JavaScript 90 stars

asterisk/asterisk-external-media

### Overview: This package demonstrates how to use the ARI External Media feature to transcribe the audio from a bridge using the Google Speech APIs. ### Key features: - Transcribes audio from a bridge using the Google Speech APIs. - Supports various audio formats and codec settings. - Provides speaker diarization for better transcription accuracy. - Integrates with Asterisk's ARI client for seamless integration. ### Tech stack: Node.js and Google Cloud Speech API. ### Why it matters: This sample provides a practical example of how to use Asterisk's External Media feature with Google Speech APIs, making it useful for developers looking to integrate speech recognition into their Asterisk-based communication systems.