AI Trending

Project directory

Projects

Open-source AI projects tracked by the leaderboard, ordered by GitHub stars.

Go 8 stars

loxilb-io/loxilb-inference-gateway

Inference-aware L4/L7 load balancer for LLM serving fleets, forked from loxilb

### Overview: LoxiLB is an inference-aware L4/L7 load balancer designed for LLM serving fleets. It leverages the same GoLang/eBPF data path as the upstream loxilb, but adds routing capabilities that understand how various AI models serve requests. This includes KV-cache locality, prefill/decode phases, and streaming responses. The gateway can be used to carry both classic cloud-native traffic and inference traffic, making it a versatile solution for serving fleets. ### Key Features: - **KV-cache-aware routing:** Sends each request to the endpoint whose KV cache already holds the longest prefix of the prompt, using prefix-hash affinity or engine-specific cache events. - **Prefill/Decode disaggregation:** Splits each request across prefill and decode pools, with KV-transfer coordination and session affinity. - **Model-name routing:** Routes requests by the requested model to per-model endpoint pools. - **SSE streaming:** Stream-aware proxying with idle-timeout suppression and a runaway cap. - **AI traffic governance:** Enforces API keys, JWT, model authorization, and request/token quotas before dispatch. - **MCP gateway:** Validates and selects requests explicitly, or routes using legacy session routing. - **Observability:** Provides Prometheus metrics and Grafana dashboards for L4, L7, and AI traffic. - **Classic load balancing:** Supports L4/NAT, Kubernetes service LB, SCTP/telco, and high availability (HA). ### Why It Matters: LoxiLB is designed to be an inference-aware load balancer for LLM serving fleets
Python 8 stars

lukaszliniewicz/easy_xtts_trainer

A command line utility to easily finetune XTTS models in a fully automated way. Developed for Pandrator.

Overview: Easy XTTS Trainer is a command-line utility designed to simplify the fine-tuning of XTTS models, making it accessible within the Pandrator ecosystem and supporting standalone usage. Key features: - Supports Pixi-first WhisperX runtime with fallback to Conda. - Optional support for DeepFilterNet and source-text alignment. - Python 3.13 baseline moved to updated Torch/Coqui compatibility. - Provides robust source-text correction flow for full-book text sources. Tech stack: Python 3.13 Why it matters: Easy XTTS Trainer streamlines the process of preparing speech datasets and fine-tuning XTTS models, making it easier and more efficient for developers working within the Pandrator ecosystem or for standalone use.
Python 8 stars

lxdyxdnt1/seedance-2-0-face-prompts-es

Dreamina — Seedance 2.0 prompts: tested image prompt recipes with example renders, aspect ratios, the model id (`seedance-2.0`) and the cost per image.

### Overview: Seedance 2.0 Prompts: 8 Tested Recipes with Outputs Seedance 2.0 Prompts offers 8 tested recipes with example renders, aspect ratios, the model ID (`seedance-2.0`), and the cost per image. This repository provides a comprehensive set of prompts for image generation, ensuring accurate and efficient use of the Seedance 2.0 model. ### Key Features: - **Model ID:** `seedance-2.0` - **Aspect Ratios:** 16:9, 1:1, 3:4, 2:3, 1:1.33, 1:2, 1:1.77, 1:1.5, 1:1.25, 1:1.17 - **Cost Per Image:** $0.04 per second, estimated cost of the whole set: $0.16 ### Tech Stack: Python Seedance 2.0 Prompts is built using Python, ensuring compatibility and ease of integration with other tools and platforms. ### Why It Matters: - **Accurate Rendering:** Seedance 2.0 Prompts provides precise and accurate renders, ensuring that the final output meets the desired specifications. - **Efficient Pricing:** The cost per image is straightforward and easy to calculate, making it a cost-effective solution for image generation. - **Versatile Models:** The repository includes prompts for various models, including Seedance 2.0, ensuring a wide range of
TypeScript 8 stars

m-abdelwahab/build-smarter-chatbots-workshop

Build an AI-powered chatbot that is able to access external data to provide the most accurate answer

Overview: This workshop is designed for developers with basic web development knowledge, focusing on building an AI-powered chatbot that can access external data to provide accurate answers. Key features: - Access external data - Use Postgres as a vector store - Implement Retrieval-Augmented Generation (RAG) Tech stack: TypeScript Why it matters: This workshop is ideal for developers looking to enhance their skills in building modern, AI-enhanced applications, particularly those interested in integrating AI capabilities into their chatbots.
TypeScript 8 stars

marvinified/voctar

Simple TypeScript library with RAG primitives for embeddings, chunking, storage, and semantic retrieval.

Overview: Voctar is a simple TypeScript library that provides RAG primitives for embeddings, chunking, storage, and semantic retrieval. Key features: - Supports multiple vector stores (SQLite, Qdrant, custom) - Automatic chunking for long documents with various strategies - Semantic search with score thresholds and metadata filtering - TypeScript-first implementation Tech stack: TypeScript Why it matters: Voctar simplifies the process of integrating vector-based search capabilities into applications, making it easier for developers to leverage advanced semantic search without the complexity of building custom solutions.
Python 8 stars

maurocanuto/mempack

MemPack is a blazing-fast, lightweight alternative to heavy vector databases: a knowledge pack with an ANN index for fast semantic search.

### Overview: MemPack is a blazing-fast, lightweight alternative to heavy vector databases. It transforms AI memory into a portable two-file format, delivering efficient semantic search and sub-second access across millions of text chunks. ### Key Features: - **Two-File Format**: Compressed text chunks, metadata, and integrity checks in `.mpack` container file, and a separate `.ann` index for fast semantic retrieval. - **Efficient Retrieval**: Sub-100ms vector search with HNSW indexing, ensuring enterprise-grade performance. - **Memory Efficiency**: Minimal memory footprint during queries, with block caching for optimal performance. - **Cold Start Handling**: Proper warm-up eliminates initialization overhead. ### Tech Stack: Python ### Why It Matters: - **38% Faster Queries**: MemPack outperforms other vector databases across all critical metrics. - **72% Smaller Disk Footprint**: Offers better memory usage and disk efficiency. - **75% Less Memory Usage**: Minimal memory footprint during queries. - **100% Overlap in Answers**: Ensures perfect answer consistency. - **Zero Infrastructure Complexity**: Works with just two files, no servers, no configuration, no vendor lock-in. - **3.2x Faster Queries**: Significantly faster than the next best system. - **2.1x Smaller Disk Footprint**: Lower disk usage compared to alternatives. - **Lowest Memory Usage**: Minimal memory footprint during queries. - **Perfect Answer Consistency**: 100% overlap in answers. - **Excellent Resource Efficiency**: Reduces
Shell 8 stars

maxsagt/lambda-instructor

Run text embeddings with Instructor-Large on AWS Lambda.

Overview: Lambda-Instructor is an experimental deployment of the text-embedding model Instructor-Large on AWS Lambda, designed to run text embeddings efficiently and cost-effectively. Key features: - Generates a large number of embeddings with Instructor-Large on your local machine. - Deployed in global AWS Regions for data residency. - AWS Lambda scales virtually unlimited, with low cost production inference. - Capable of generating embeddings up to 512 tokens (ca. 380 words) compared to OpenAI's text-embedding-ada-002's 8191 tokens (ca. 6000 words). Tech stack: Shell Why it matters: Lambda-Instructor offers a cost-effective and scalable solution for text embedding generation, making it ideal for scenarios requiring large-scale inference without the need for continuous server management.
Python 8 stars

mcolsen/amd-npu-stable-diffusion-linux

Stable Diffusion image generation on AMD Ryzen AI NPUs for Linux

### Overview: Stable Diffusion image generation optimized for AMD Ryzen AI XDNA 2 NPUs on Linux. ### Key features: - **Stable Diffusion 1.5 (sd15)**: 20 steps, ~5 seconds. - **SD-Turbo**: Single-step distilled, no guidance, ~2 seconds. - **SDXL-Base**: 30 steps, ~35 seconds. - **SDXL-Turbo**: Two text encoders, single-step, CFG mandatory, ~2 seconds. ### Tech stack: Python ### Why it matters: This project enables efficient image generation using AMD's AI hardware, potentially reducing processing time and improving performance for users.
Swift 8 stars

megasoft1978/strim-engine

Fast Swift/MLX inference engine and sandboxed agent harness for Qwen3.8-27B on Apple Silicon

### Overview: Strim Engine is a fast, lightweight Swift/MLX inference engine and sandboxed agent harness designed specifically for Apple Silicon. It provides a single Swift binary and a menu-bar application, ensuring that all operations run entirely on your Mac without leaving the system. ### Key Features: - **Fast Inference**: Utilizes the latest MLX engine for lightning-fast inference. - **Sandboxed Agent**: Runs in a secure environment, ensuring no data leaks or unauthorized access. - **Single Binary**: A single Swift binary for seamless integration into your projects. - **Menu-Bar App**: Includes a menu-bar application for easy access and management. ### Tech Stack: Swift ### Why It Matters: Strim Engine is crucial for developers who need a fast and efficient way to integrate AI capabilities into their projects without compromising security or performance. Its lightweight nature and seamless integration make it an ideal choice for both small and large-scale applications.
Unknown 8 stars

microsoft/frontiertour27-BRK231-trusted-ai-at-scale-foundry-to-agent-365

### Overview: Microsoft Frontiertour 2027 is a session exploring how Microsoft Foundry, GitHub Copilot, Copilot Studio, and Agent 365 work together to help organizations build, operationalize, govern, and scale AI agents across the enterprise with security, control, and trust. ### Key Features: - Microsoft Foundry: A platform for building and deploying AI agents. - GitHub Copilot: A productivity tool for developers. - Copilot Studio: A tool for building conversational agents. - Agent 365: A platform for governing and scaling AI agents securely. ### Tech Stack: Microsoft Foundry, GitHub Copilot, Copilot Studio, and Agent 365. ### Why it Matters: This session provides a comprehensive overview of how these tools work together to help organizations build, operationalize, govern, and scale AI agents across the enterprise with security, control, and trust.
Python 8 stars

mkk8f7hh/wan-2-7-r2v-prompts

Wan 2.7 R2V prompts: tested image prompt recipes with example renders, aspect ratios, the model id (`wan2.7-r2v`) and the cost per image.

### Overview: This repository contains a set of 4 Wan 2.7 R2V prompts designed to generate high-quality images with specific attention to materials, lighting, and framing. The prompts are tested and provide accurate cost estimates, making them useful for developers and content creators. ### Key Features: - **4 Prompts**: Four distinct prompts for different scenarios. - **Effective Price**: $0.0664 per second, with an estimated cost of $0.2656 for the entire set. - **Model ID**: `wan2.7-r2v` - **Aspect Ratios**: 85mm, 24mm, macro, overhead flat lay - **Lighting**: Explicitly set with directions and qualities - **Framing**: Ratio and lens language determine the final image ### Tech Stack: Python ### Why It Matters: - **One Key, Entire Catalog**: The same base URL and `Authorization` header reach Wan 2.7 R2V and 300+ other models. - **$1 Minimum, Pay As You Go**: No subscription or prepaid plan required, and there's no free quota to burn through. - **Charge Backs**: The cost is reported in the response, so you know exactly how much you're spending per call. - **Async by Design**: Submit, get the `task_id`, and poll `GET /v1/tasks/{id}` for results, with retries and batching handled by the queue. ### Wan 2.7 R2V Prompts: - **
Python 8 stars

mmilunovic/m2vdb

vector db built by someone with no idea how to build a vector db

Overview: This project aims to understand vector search and databases from first principles, focusing on building a real vector database. Key features: - Brute Force (Python) - Brute Force (Rust) - Product Quantization (PQ) - Inverted File (IVF) Tech stack: Python Why it matters: It's an educational project aimed at demystifying vector databases and databases in general, providing a practical understanding through implementation. Repository: mmilunovic/m2vdb Primary language: Python GitHub stars: 8 GitHub description: vector db built by someone with no idea how to build a vector db README excerpt: This project is designed to help understand how vector databases work, focusing on building a real database from scratch. It includes implementations of various index types and APIs for interacting with the database.
Jupyter Notebook 8 stars

mongodb-developer/quickstart-rag-python

This Python project demonstrates semantic search using MongoDB and two different LLM frameworks: LangChain and LlamaIndex. The goal is to load documents from MongoDB, generate embeddings for the text data, and perform semantic searches using both LangChain and LlamaIndex frameworks.

Overview: This Python project showcases semantic search capabilities using MongoDB and two LLM frameworks: LangChain and LlamaIndex. It involves loading documents from MongoDB, generating embeddings, and performing searches using both frameworks. Key features: - Loading documents from MongoDB - Generating embeddings using LangChain and LlamaIndex - Performing semantic searches using both frameworks Tech stack: Python, Jupyter Notebook Why it matters: This project demonstrates practical applications of semantic search and LLM frameworks, making it valuable for developers looking to integrate advanced search functionalities into their projects. Repository: mongodb-developer/quickstart-rag-python GitHub stars: 8 GitHub description: This Python project demonstrates semantic search using MongoDB and two different LLM frameworks: LangChain and LlamaIndex. The goal is to load documents from MongoDB, generate embeddings for the text data, and perform semantic searches using both LangChain and LlamaIndex frameworks.
TypeScript 8 stars

mshojaei77/ollama-desktop

A powerful desktop application for interacting with local AI models through Ollama, featuring advanced capabilities and a modern interface.

Overview: Ollama Desktop is a powerful desktop application designed for interacting with local AI models through Ollama, featuring advanced capabilities and a modern interface. Key features: - Real-time streaming responses - Multi-model support - Vision model integration - Context-aware conversations - Model context protocol (MCP) - Specialized AI agents - Smart session management - Automated model discovery - Local-first design - Persistent storage - Extensible backend - Modern frontend Tech stack: TypeScript Why it matters: Ollama Desktop provides developers with a seamless and efficient way to integrate AI assistance into their workflows, researchers with powerful document analysis tools, content creators with specialized agents, and enterprise users with secure and local AI processing capabilities. Repository: mshojaei77/ollama-desktop Primary language: TypeScript GitHub stars: 8 GitHub description: A powerful desktop application for interacting with local AI models through Ollama, featuring advanced capabilities and a modern interface.
Python 8 stars

mt-upc/iwslt-2022

Systems submitted to IWSLT 2022 by the MT-UPC group.

Overview: This project is a submission to the IWSLT 2022 offline speech translation task by the UPC Machine Translation group. The system uses large pretrained speech and text models for end-to-end training and explores the use of adapter modules for non-trainable layers. Key features: - Speech Translation systems trained end-to-end - Use of HuBERT, wav2vec 2.0, and mBART models - Efficient fine-tuning technique - Exploration of knowledge distillation from a Machine Translation model - Improved BLEU scores on MuST-C and IWSLT 2020 test sets Tech stack: Python Why it matters: This project showcases advancements in speech translation systems, particularly in the use of pre-trained models and efficient fine-tuning techniques, which can significantly improve translation quality and efficiency. Repository: mt-upc/iwslt-2022 Primary language: Python GitHub stars: 8 GitHub description: Systems submitted to IWSLT 2022 by the MT-UPC group.
Python 8 stars

mu-hashmi/mlx-moe

Run large Mixture-of-Experts LLMs that exceed system RAM on Apple Silicon by loading only router-selected experts from SSD with MLX. Includes OpenAI/Anthropic-compatible serving for local agentic coding.

### Overview: MLX-MoE is a cutting-edge model that enables the efficient deployment of large Mixture-of-Experts language models on memory-constrained Macs, specifically targeting Apple Silicon devices. By leveraging SSDs and optimized routing, MLX-MoE ensures that only the necessary experts are loaded, significantly reducing memory usage and improving performance. ### Key Features: - **Efficient Model Loading**: MLX-MoE optimizes the loading of only the selected experts from SSDs, reducing memory footprint and improving performance. - **High Performance**: The model runs at 6-23 tokens per second, with coherent output through 1000+ tokens, making it suitable for high-throughput applications. - **Versatile Architecture**: Supports multiple MoE architectures, including Qwen, Mixtral, GLM, DeepSeek, Hunyuan, PhiMoE, Jamba, OLMoE, MiniMax, and GraniteMoE families, ensuring compatibility with a wide range of models. ### Tech Stack: Python ### Why It Matters: MLX-MoE is particularly valuable for developers and researchers who need to deploy large language models on memory-limited Macs, such as those equipped with Apple Silicon. By optimizing model loading and routing, MLX-MoE ensures that only the necessary experts are loaded, leading to faster performance and reduced memory usage, making it an ideal solution for applications requiring high throughput and low memory constraints.
TypeScript 8 stars

multimindlab/multimind-sdk-js

JavaScript for MultiMindSDK – bring advanced context transfer and multi-LLM orchestration, Node.js apps, and web tools

Overview: The MultiMind SDK for JavaScript provides a comprehensive interface for JavaScript developers to access advanced AI features such as AI agent orchestration, RAG, and fine-tuning, without requiring backend code management. This SDK enhances JavaScript applications with capabilities like model routing, evaluation, and fine-tuning, making it a valuable tool for developers looking to integrate advanced AI into their projects. Key features: - 🤩 AI Agent Orchestration: Create intelligent agents for complex task execution. - 🔎 RAG (Retrieval-Augmented Generation): Combine your data with AI reasoning for knowledge systems. - 🎯 Model Fine-tuning: Customize AI models for specific use cases and domains. - 🔄 Model Routing: Automatically select the best AI model for each task. - 📊 Model Evaluation: Assess and compare AI model performance. - 🔧 Adapter Management: Enhance models with specialized capabilities. - 🛠️ Advanced Workflows: LoRA fine-tuning, document processing, compliance monitoring, and more. Tech stack: The SDK is built using TypeScript, Node.js, and TypeScript, ensuring compatibility and ease of integration. Why it matters: This SDK simplifies the integration of advanced AI capabilities into JavaScript applications, making it easier for developers to leverage cutting-edge AI technologies without the need for complex backend setups.
Unknown 8 stars

mxslr/mlcraft

A research-first AI/ML research-engineer workflow for Claude Code

Overview: A research-first AI/ML engineering plugin designed to make your AI coding assistant work like a senior AI/ML research engineer. It leverages the latest research papers, selects the best methods, trains rigorously, evaluates honestly, and iteratively improves accuracy across various domains including computer vision, medical imaging, NLP, tabular data, time-series, audio, video, and anomaly detection. Key features: - Automatically selects the right skills based on the task description. - Supports multiple domains including computer vision, medical imaging, NLP, tabular data, time-series, audio, video, and anomaly detection. - Detects the desired deliverable format (Kaggle notebook, Google Colab notebook, or local GPU run). - Uses short, professional research tone in generated notebooks. - Integrates with Claude Code or Antigravity harnesses. Tech stack: Python, Jupyter Notebook, Git, and Claude Code or Antigravity harness. Why it matters: It ensures that your AI/ML assistant works at the highest research standards, providing accurate and honest results without data leakage or inflated metrics, making it a valuable tool for researchers and engineers alike.
Python 8 stars

myit9z1a/doubao-seedance-2-5-prompts

Seedance 2.5 prompts: tested image prompt recipes with example renders, aspect ratios, the model id (`seedance-2.5`) and the cost per image.

### Overview: Seedance 2.5 Prompts: A Comprehensive Guide Seedance 2.5 Prompts provides a collection of tested image prompt recipes, offering detailed examples, aspect ratios, and the model ID (`seedance-2.5`). This guide is designed for developers looking to optimize image generation costs and achieve high-quality renders efficiently. ### Key Features: 1. **Model ID**: `seedance-2.5` 2. **Example Renders**: Detailed images showcasing various prompts. 3. **Aspect Ratios**: 16:9, 4:3, 1:1, etc. 4. **Cost Per Image**: $0.0961 per second (480P), $0.216 per second (720P) 5. **Model Compatibility**: OpenAI-compatible endpoint at `https://api.apimart.ai/v1` ### Tech Stack: Python Seedance 2.5 Prompts is built using Python, ensuring seamless integration with other Python-based projects. ### Why It Matters: - **Efficient Pricing**: The cost per second is transparent and straightforward, allowing for precise budgeting. - **Model Flexibility**: Switches between different models without changing the client, simplifying integration. - **Real-Time Billing**: The cost is reported immediately after each call, eliminating guesswork. - **Scalability**: No subscription required; top up from $1 and spend it as needed. ### Repository: myit9z1a/doubao-seedance-2-5-prompts
Python 8 stars

natelalor/AI_report_generator

A tool that converts long audio files into a thorough, summarized report. Leverages OpenAI and its API (ChatGPT backend), Langchain for text processing, and Pinecone for vector database facilitation.

### Overview: AI Report Generator is a tool designed to convert long audio files into a summarized report. Leveraging OpenAI's ChatGPT backend, Langchain for text processing, and Pinecone for vector database facilitation, it provides a seamless conversion process. ### Key Features: - Converts audio files into summarized reports. - Supports both text-to-report and audio-to-text-to-report functionalities. - Outputs a .txt file containing the summarized report. - Provides a user-friendly interface for easy integration into projects. ### Tech Stack: Python ### Why it Matters: This tool simplifies the process of generating insightful reports from audio content, making it particularly useful for developers and project managers who need to quickly summarize large audio files into concise, actionable reports. ### Repository: natelalor/AI_report_generator GitHub Stars: 8 GitHub Description: A tool that converts long audio files into a thorough, summarized report. Leverages OpenAI and its API (ChatGPT backend), Langchain for text processing, and Pinecone for vector database facilitation.