Project directory

Projects

Open-source AI projects tracked by the leaderboard, ordered by GitHub stars.

Python 39 stars

al35s98diini7d/skyreels-v4-std-api-es

SkyReels V4 Std API (`skyreels-v4-std`): model id, pricing, parameters and copy-paste examples for the OpenAI-compatible APIMart endpoint.

Overview: SkyReels V4 Std API is a comprehensive model id, pricing, and parameterized endpoint for the OpenAI-compatible APIMart, offering 1080p video generation with native audio and cinematic coherence for $0.088 per second. Key features: - Model id: skyreels-v4-std - Pricing: $0.088 per second - Parameters: model, prompt, duration, resolution, size, image_urls, seed - Billing: per generated second, by resolution Tech stack: Python Why it matters: This API provides a unified model for video and audio generation, offering high-quality output at a cost-effective rate, making it ideal for developers and content creators looking to streamline their workflows. GitHub stars: 39

Python 39 stars

dwain-barnes/sky-livekit-agent-perplexica

Sky LiveKit Agent Perplexica is a local, free solution integrating LiveKit with advanced internet search. It uses a local Perplexica instance with function calling to retrieve and summarise search results in natural language. Powered by Faster Whisper, Ollama (Qwen 2.5) and Kokoro-82M TTS.

Overview: Sky LiveKit Agent Perplexica is a local, free solution that integrates LiveKit with advanced internet search capabilities, using a local Perplexica instance with function calling to retrieve and summarise search results in natural, conversational language. Key features: - Local Search - Free of API Costs - Speech Recognition - Language Processing - Text-to-Speech Tech stack: Python Why it matters: This solution offers a local, free alternative to external APIs, making it ideal for environments where API costs are prohibitive or where privacy is a concern. Repository: dwain-barnes/sky-livekit-agent-perplexica Primary language: Python GitHub stars: 39

Shell 39 stars

foamliu/Listen-Attend-Spell-v2

PyTorch implementation of Listen Attend and Spell Automatic Speech Recognition (ASR).

### Overview: Listen Attend and Spell is a PyTorch implementation of the Listen Attend and Spell (LAS) model, designed for Automatic Speech Recognition (ASR) tasks. This model is based on the research paper "Listen, Attend and Spell" by Chan et al., published in ICASSP 2016. ### Key features: - **Model Architecture**: A neural network architecture specifically designed for ASR tasks. - **Training Data**: Uses the Aishell dataset, a Mandarin speech corpus with high-quality manual transcriptions. - **Automatic Speech Recognition**: Employs advanced techniques for recognizing speech in real-time applications. ### Tech stack: Python 3.6, PyTorch 1.0.0 ### Why it matters: Listen Attend and Spell is significant in the field of ASR because it demonstrates the potential of deep learning models in speech recognition, providing a more efficient and accurate solution compared to traditional methods. Its implementation in PyTorch showcases the versatility of this model across various speech recognition tasks.

Jupyter Notebook 39 stars

grib0ed0v/kaldi-for-russian

Overview: This project provides a comprehensive manual for building and training the Kaldi speech recognition system on Russian language data, including detailed instructions on cloning repositories, building tools, and installing dependencies. Key features: - Cloning Kaldi and Voxforge Ru recipes - Building tools and dependencies - Building Kaldi source code - Copying or creating symbolic links for utils and steps - Installing additional dependencies - Running the training script - Recording audio for script execution Tech stack: Jupyter Notebook Why it matters: This project serves as a step-by-step guide for developers to set up and train the Kaldi speech recognition system on Russian language data, ensuring a smooth and efficient development process.

Python 39 stars

halt95/merlin-engine

vLLM 0.32 fork for consumer GPUs: 3 x 262K sessions on 4x RTX 3090, Qwen 3.8 Flash-Next at 197 tok/s, up to 17 on bigger cards. Seven checkpoints incl. Qwen3.8-27B, Nemotron 3.5 Super VL (1M) and Ornith 1.5. One command: merlin serve.

Overview: Merlin is a vLLM fork optimized for consumer GPUs, capable of running seven models at their full 262K context, including Qwen 3.8 Flash-Next and others. Key features: - Seven open models tuned for consumer GPUs - Supports 4x RTX 3090s with 3x 262K sessions - Runs Qwen 3.8 Flash-Next at 197 tok/s on 4x RTX 3090s - Supports seven checkpoints including Qwen3.8-27B, Nemotron 3.5 Super VL, and Ornith 1.5 - One command: `merlin serve` for easy deployment Tech stack: Python Why it matters: Merlin provides a powerful, efficient way to run large language models on consumer GPUs, enabling faster and more resource-efficient model inference. Repository: halt95/merlin-engine Primary language: Python GitHub stars: 37 GitHub description: vLLM 0.32 fork for consumer GPUs: 3 x 262K sessions on 4x RTX 3090, Qwen 3.8 Flash-Next at 197 tok/s, up to 17 on bigger cards. Seven checkpoints incl. Qwen3.8-27B, Nemotron 3.5 Super VL (1M) and Ornith 1.5. One command: merlin serve.

Shell 39 stars

hwdsl2/scribecrate-live

ScribeCrate Live: self-hosted real-time transcription server powered by WhisperLive and faster-whisper. WebSocket transcription and speech-to-English translation, plus an OpenAI-compatible REST API for file transcription. All Whisper models, VAD, and offline mode with cached models. Docker on CPU (amd64/arm64) or NVIDIA GPU (CUDA, amd64).

Overview: ScribeCrate Live is a self-hosted real-time speech-to-text server powered by WhisperLive and faster-whisper, providing WebSocket transcription, speech-to-English translation, and an OpenAI-compatible REST API for file transcription. It supports CPU and NVIDIA GPU deployment. Key features: - Real-time WebSocket transcription - Speech-to-English translation - OpenAI-compatible API for file transcription - Private, local processing - Voice activity detection (VAD) - Support for all Whisper models - CPU and GPU support - Offline operation with pre-cached models - Automated builds through GitHub Actions Tech stack: Shell Why it matters: ScribeCrate Live offers a reliable, self-hosted solution for real-time transcription, making it ideal for developers and organizations looking to integrate AI into their workflows without relying on third-party services. Repository: hwdsl2/scribecrate-live GitHub stars: 39 GitHub description: ScribeCrate Live: self-hosted real-time transcription server powered by WhisperLive and faster-whisper. WebSocket transcription and speech-to-English translation, plus an OpenAI-compatible REST API for audio-file transcription. All Whisper models, VAD, and offline mode with cached models. Docker on CPU (amd64/arm64) or NVIDIA GPU (CUDA, amd64).

Python 39 stars

jdh-algo/JoyType

JoyType: A Robust Design for Multilingual Visual Text Creation

Overview: JoyType is a robust design for multilingual visual text creation, utilizing a comprehensive framework that includes data collection, training pipeline, and inference pipeline. Key features: - Robust multilingual visual text creation - Utilizes a Visual Language Model (e.g., CogVLM) to generate textual descriptions - Employing a canny algorithm to extract edges from text regions - Supports inference through both online and offline methods Tech stack: Python Why it matters: JoyType offers a powerful solution for creating multilingual visual text, enhancing the accessibility and understanding of information across different languages and visual elements. Repository: jdh-algo/JoyType Primary language: Python GitHub stars: 39

TypeScript 39 stars

kingcharleslzy-ai/dsh-qwen-studio

Qwen Image 2.1 for DeepSeek Harness: native image editing on Apple Silicon MLX/MFLUX or self-hosted CUDA, LoRA, ControlNet, inpainting, presets & live agent progress. 本机生图 · 原生参考编辑

Overview: Qwen Image 2.1 for DeepSeek Harness is a powerful image editing tool designed for seamless integration with Apple Silicon MLX/MFLUX or self-hosted CUDA environments. It offers native image editing capabilities, including reference-based editing, LoRA, ControlNet, inpainting, presets, and real-time agent progress tracking. The project is built on TypeScript and has been actively maintained with 37 GitHub stars, making it a valuable resource for developers looking to enhance their image editing workflow. Key features: - Native Qwen Image 2.1 generation and reference editing - Support for Apple Silicon MLX/MFLUX and self-hosted CUDA environments - LoRA, ControlNet, inpainting, presets, and real-time agent progress tracking - Real-time agent progress tracking - Support for multiple GPU environments via MLX and CUDA - Integration with DeepSeek Harness, allowing seamless workflow - Automatic updates and installation management - Support for various image editing techniques including reference-based editing, LoRA, ControlNet, inpainting, presets, and real-time agent progress tracking - Real-time agent progress tracking - Support for multiple GPU environments via MLX and CUDA - Integration with DeepSeek Harness, allowing seamless workflow - Automatic updates and installation management - Support for various image editing techniques including reference-based editing, LoRA, ControlNet, inpainting, presets, and real-time agent progress tracking - Real-time agent progress tracking - Support for multiple GPU environments via MLX and CUDA - Integration with DeepSeek Harness, allowing seamless workflow - Automatic

Python 39 stars

monaccode/astromesh

Multi-model AI agent runtime. Define agents in YAML, route each role to a model, orchestrate with 7 patterns (ReAct, Plan & Execute, Fan-Out, Pipeline, Supervisor, Swarm, Glyph), and deploy as a REST/WebSocket API with RAG, memory, MCP tools, guardrails and OpenTelemetry observability.

### Overview: Astromesh is an open-source runtime designed for multi-model AI agents. It allows agents to be defined in YAML or Python, and then orchestrated through 7 patterns (ReAct, Plan & Execute, Fan-Out, Pipeline, Supervisor, Swarm, Glyph) to be deployed as REST/WebSocket APIs with RAG, memory, MCP tools, and guardrails. Astromesh also includes observability with OpenTelemetry. ### Key features: - **Define Agents**: Agents can be defined in YAML or Python. - **Orchestration Patterns**: 7 patterns (ReAct, Plan & Execute, Fan-Out, Pipeline, Supervisor, Swarm, Glyph) to route roles to models. - **Deploy as API**: Deploy agents as REST/WebSocket APIs with RAG, memory, MCP tools, and guardrails. - **OpenTelemetry Observability**: Includes observability with OpenTelemetry for better monitoring. ### Tech stack: - Python ### Why it matters: Astromesh provides a robust and flexible environment for developing and deploying AI agents, making it easier to integrate with various models and frameworks while ensuring security and performance through guardrails and observability tools. ### Repository: - **Repository**: monaccode/astromesh - **Stars**: 38 - **Description**: Multi-model AI agent runtime. Define agents in YAML, route each role to a model, orchestrate with 7 patterns, and deploy as a REST/WebSocket API with RAG, memory, MCP tools, guardrails, and OpenTelemetry observability. ### Primary language:

JavaScript 39 stars

phetsims/paper-land

Build and explore multimodal web interactives with pieces of paper!

Overview: Paper Playground is an open-source project that enables collaborative creation of multimodal web experiences using pieces of paper and physical space. Key features: - Program creation systems - Computer vision - Execution and display - Collaboration Tech stack: JavaScript Why it matters: Paper Playground combines physical interaction with digital programming, providing a unique approach to collaborative problem-solving and prototyping in codesigning technology.

Jupyter Notebook 39 stars

robinhad/voice-recognition-ua

Training scripts for Speech-To-Text models for Ukrainian language

### Overview: This repository contains training scripts for Speech-to-Text models specifically designed for the Ukrainian language. The primary language is Jupyter Notebook, and the source code is available in the repository alongside auto-deploy pipeline scripts. The project aims to apply various speech recognition models on the Ukrainian language, with models trained on Common Voice datasets and synthetic datasets for improved accuracy. ### Key Features: - **Model Training Scripts**: Guides for training are available in corresponding folders for each model. - **GitHub Repository**: Contains the source code and auto-deploy pipeline scripts. - **Model Licenses**: Models are licensed under MIT, CC-BY-NC 4.0, and CC-BY-NC 4.0. ### Tech Stack: - **Language**: Jupyter Notebook ### Why it Matters: This project is crucial for the Ukrainian language community, providing accessible tools for speech recognition that can improve the quality of communication and accessibility.

C++ 39 stars

royshil/cloudvocal

Cloud AI live transcription and translation service plugin

### Overview: CloudVocal is a professional-grade cloud transcription and translation service plugin designed for live streaming and recording. It leverages industry-leading cloud providers to deliver exceptional accuracy and real-time performance, making it suitable for both small and enterprise-level use cases. ### Key Features: - **Professional-grade transcription with industry-leading accuracy** - **Support for 100+ languages** - **Real-time translation using enterprise cloud translation services** - **Streaming-optimized performance with minimal latency** - **Caption output in multiple formats (.txt, .srt)** - **Sync'ed captions with OBS recording timestamps** - **Direct streaming to platforms (YouTube, Twitch) with embedded captions** - **Partial transcriptions for a streaming-captions experience** ### Tech Stack: C++ ### Why It Matters: CloudVocal offers professional-grade transcription and translation services, making it ideal for live streaming and recording applications. Its enterprise-level reliability and real-time performance ensure high-quality captions and translations, enhancing the overall streaming experience.

Unknown 39 stars

saskinosie/weaviate-claude-skills

Claude Skills for connecting Claude.ai to local Weaviate vector databases - manage collections, ingest data, and query with RAG

### Overview: This repository contains a set of Claude Skills designed to connect Claude.ai with local Weaviate vector databases. These skills enable seamless management, data ingestion, and querying of Weaviate instances running in Docker, offering benefits such as no network restrictions, free usage, and full data privacy. ### Key Features: - **Local Weaviate Setup**: Configures and manages a local Weaviate instance using Docker for development and testing. - **Connection Setup**: Establishes a connection to the local Weaviate instance. - **Collection Management**: Creates, views, updates, and deletes Weaviate collections with comprehensive schema management. - **Data Ingestion**: Uploads single objects, batch uploads, and multi-modal content to Weaviate collections. - **Query Agent**: Searches and retrieves data using semantic search, filters, RAG, and hybrid queries. ### Tech Stack: The repository uses Docker for local Weaviate setup and provides instructions for connecting, managing, and querying Weaviate databases. ### Why It Matters: This set of skills is crucial for developers who need to integrate Claude.ai with local Weaviate vector databases, offering a local-first, no-cloud-cost, and privacy-focused solution that works seamlessly with Claude.ai and Claude Desktop.

Java 39 stars

seehiong/micronaut-llama3

A high-performance Llama3 implementation using Micronaut and GraalVM Native Image

Overview: This project is a high-performance implementation of Llama3 using Micronaut and GraalVM Native Image, designed for developers looking to leverage the capabilities of Llama3 in a Java environment. Key features: - Micronaut-based application framework - Native Image compilation for improved performance - Support for generating text and handling chat interactions Tech stack: Java, Micronaut, GraalVM, Gradle Why it matters: This project provides a fast and efficient way to integrate Llama3 capabilities into Java applications, making it easier for developers to leverage advanced AI models without the need for complex model management or expensive frameworks.

Python 39 stars

sneezeparty/soupy

Fully-local Discord bot with chat, RAG memory, autonomous Discord posts, Bluesky integration, and image generation. Runs against LM Studio + Stable Diffusion OR Flux, with a FastAPI web control panel for live config and monitoring.

### Overview: Soupy is a fully-local Discord bot and web control panel that runs on hardware you own, providing a configurable personality, retrieval-augmented memory, autonomous Discord and Bluesky posts, web search, image understanding, and two independent image-generation backends. No cloud APIs, no per-token costs, and no data leaving your network. ### Key Features: - **Conversational Chat**: Chat with a configurable personality and a self-knowledge document. - **Retrieval-Augmented Memory**: Pulls from a per-guild SQLite archive of your server's history. - **User Profiles**: Builds tailored replies from each member's message history. - **Self-Reflection**: Nightly self-reflection at a configurable hour. - **Daily Posts**: Autonomous daily article posts and Bluesky presence. - **Bluesky Presence**: Replies, quote-posts, and original article posts on a randomized daily schedule. - **Thinking Out Loud**: Periodic musings in a configured channel. - **Web Search**: Reads articles and answers with cited sources. - **Stock Price Lookups**: Free Finnhub key required. - **Vision**: Routes Discord image attachments through LM Studio's vision-capable LLM. - **Image Generation**: Two independent image-generation backends, sharing a single serial queue. ### Tech Stack: Python ### Why It Matters: Soupy is a fully-local Discord bot and web control panel that runs entirely on hardware you own, providing a configurable personality, retrieval-augmented memory, autonomous Discord and Bluesky posts, web search, image understanding, and two independent

Jupyter Notebook 39 stars

sovse/base_rus_whisper_stt

Fine tuning of the base model from OpenAI Whisper in Russian language on the dataset Sber-golos

Overview: This project fine tunes the base model from OpenAI Whisper in Russian language using the Sber-golos dataset. Key features: - Fine-tuning of Whisper model for Russian language - Dataset: Sber-golos - Model size: 290.5 MB, 71M parameters - Accuracy: 20.96% on Farfield test (compared to 73.45% for standard "base" model) - Available for download: https://disk.yandex.ru/d/ogeKhA-PfNpf9w - Accuracy on Sber-golos test: 20.96% Tech stack: Jupyter Notebook Why it matters: This project showcases the fine-tuning of a language model for a specific language and dataset, demonstrating the potential for improving accuracy and functionality in natural language processing tasks.

Python 39 stars

umer-sheikh/bird-whisperer

[InterSpeech 2024] Official code repository of paper titled "Bird Whisperer: Leveraging Large Pre-trained Acoustic Model for Bird Call Classification" accepted in InterSpeech 2024 conference.

### Overview: Bird Whisperer is an innovative project that leverages a large pre-trained acoustic model to classify bird calls, improving the F1-score by 15% compared to the baseline. The project was accepted at the InterSpeech 2024 conference and showcases the potential of adapting large pre-trained models for broader bioacoustic classification tasks. ### Key features: - **Fine-tuning of Whisper Model**: The project fine-tunes the Whisper model, originally designed for human speech recognition, to classify bird calls. - **Enhanced Feature Extraction**: The adapted model extracts richer, more informative features from bird calls, improving classification accuracy. - **Class Imbalance Mitigation**: The project addresses class imbalance by applying audio and spectrogram augmentations to the BirdCLEF 2023 dataset. ### Tech stack: Python ### Why it matters: This project demonstrates the potential of adapting large pre-trained acoustic models for broader bioacoustic classification tasks, providing valuable insights for ecological studies and biodiversity monitoring.

JavaScript 39 stars

waifuai/waifu-companion

Browser-based AI companion with Live2D characters, voice chat and long-term memory.

### Overview: Waifu Companion is a browser-based AI companion that brings your favorite characters to life using Live2D technology. It offers advanced AI integration, interactive live2D models, smart AI persona, voice chat, and more. ### Key features: - **Advanced AI Integration**: Supports custom LLM models via OpenRouter API with fallback models. - **Interactive Live2D Models**: Seamlessly renders standard Live2D Cubism models, supports built-in models and custom model loading. - **Smart AI Persona**: Customizable identity, contextual memory, auto-summarization, and AI settings control. - **Voice & Audio**: Local text-to-speech engine, free TTS API, browser speech synthesis, real-time lip-sync, TTS preloading, and speech-to-text. - **Chat System**: Message queue, ambient mode, typing indicators, online/offline status, debug context, and conversation summary markers. ### Tech stack: JavaScript ### Why it matters: Waifu Companion provides a unique and customizable AI companion experience, blending advanced AI technology with interactive visuals to create a personalized and engaging user interface.

Python 39 stars

wananing/Smart-Health-Assistant

对标蚂蚁阿福的开源大健康 AI 助手：LangGraph 多智能体 + 实时语音问诊（开放麦、可打断，流式 ASR/TTS）+ RAG、报告解读、拍照问药与链路评估。Open-source AI health assistant inspired by Ant Afu: LangGraph multi-agent, realtime voice agent (open mic, barge-in, streaming ASR/TTS), RAG, multimodal vision, tracing and evals.

Overview: 对标蚂蚁阿福的开源大健康 AI 助手：LangGraph 多智能体 + 实时语音问诊（开放麦、可打断，流式 ASR/TTS）+ RAG、报告解读、拍照问药与链路评估。这是一个基于 LangGraph + FastAPI + React 的 AI 多智能体全栈参考实现，旨在为医疗和医保领域提供智能对话系统。 Key features: - 实时语音问诊（Realtime Voice Agent） - 服务端会话、预问诊子图与 Agent 转接 - 多模态图片识别 - 多模型配置 - RAG 知识库增强 - Agent Skills System - 开源可观测与评估 - 移动端设计规范 Tech stack: Python Why it matters: 该项目旨在提供一个对标蚂蚁阿福的开源大健康 AI 助手，通过 LangGraph 多智能体框架，结合实时语音问诊、流式 ASR/TTS、RAG 知识库增强、多模态图片识别等功能，为用户提供便捷、智能的健康咨询服务。

Python 39 stars

xie-lab-ml/IV-mixed-Sampler

[ICLR2025] IV-Mixed Sampler: Leveraging Image Diffusion Models for Enhanced Video Synthesis

Overview: IV-Mixed Sampler is an innovative algorithm designed to enhance video synthesis by leveraging image diffusion models (IDMs). It addresses the need for improved performance in video diffusion models (VDMs) by scaling up inference costs effectively. Key features: - Leverages image diffusion models to enhance the quality of each video frame. - Ensures temporal coherence during the sampling process. - Achieves state-of-the-art performance on various benchmarks, including UCF-101-FVD, MSR-VTT-FVD, Chronomagic-Bench-150, and Chronomagic-Bench-1649. Tech stack: Python Why it matters: IV-Mixed Sampler offers a significant improvement in video synthesis quality by utilizing the strengths of image diffusion models, leading to enhanced image editing and compositional generalization. This algorithm addresses the limitations of existing research in video diffusion models and demonstrates substantial performance gains, particularly in reducing the UMT-FVD score, which is crucial for applications requiring high-quality video synthesis.