Project directory

Projects

Open-source AI projects tracked by the leaderboard, ordered by GitHub stars.

Python 23 stars

nuhmanpk/text2imagebot

A Telegram bot that is capable of running stable diffusion Models (Support all model from huggingface) and generate images based on user prompts. Loaded with 20 Models .

### Overview: A Telegram bot that converts textual descriptions into visually appealing images using advanced text-to-image generation techniques. It supports various models from Hugging Face and can generate images based on user prompts. ### Key features: - Utilizes Diffusion models from Hugging Face for text-to-image generation. - Supports 20 models, including stable diffusion. - Generates images based on textual prompts. - Easy to use, just send a text message with the desired image description. ### Tech stack: Python ### Why it matters: The bot provides a powerful tool for creating compelling visual content directly from text, making it useful for designers, marketers, and anyone looking to enhance their visual communication skills.

Clojure 23 stars

phronmophobic/usearch.clj

A clojure wrapper for usearch, a fast open-source search & clustering engine for vectors.

### Overview: `usearch.clj` is a Clojure wrapper for `usearch`, a fast open-source search & clustering engine for vectors. It provides a convenient interface for integrating `usearch` functionality into Clojure applications. ### Key Features: - Integrate `usearch` for efficient vector search and clustering. - Support for various architectures (Linux x86-64, Darwin aarch64, Darwin x86-64). - Examples and usage instructions in the `examples` directory. - Save and load indices for persistence. ### Tech Stack: Clojure ### Why it Matters: This library simplifies the integration of `usearch` into Clojure applications, making it easier to implement fast and efficient vector search and clustering functionalities. It is particularly useful for developers looking to leverage `usearch` for their projects. Repository: `phronmophobic/usearch.clj` GitHub Stars: 23

Rust 23 stars

pinecone-io/pinecone-rust-client

The official Rust client for the Pinecone vector database

### Overview: The official Rust client for the Pinecone vector database provides a comprehensive set of tools for interacting with Pinecone's vector database, making it easier for developers to integrate vector search capabilities into their Rust projects. This client simplifies the process of managing indexes and accessing data efficiently. ### Key features: - **Rust Integration**: Built specifically for Rust developers, ensuring seamless integration with Rust projects. - **Pinecone API Key Management**: Automatically reads the API key from environment variables, making it easy to manage sensitive information. - **Index Creation**: Supports both serverless and pod indexing methods, allowing for flexible and scalable solutions. - **Client Configuration**: Provides a simple configuration structure to initialize the client, making it easy to set up and use. ### Tech stack: Rust ### Why it matters: This client is essential for developers who need to incorporate vector search capabilities into their Rust applications, providing a robust and efficient solution for vector database management.

TypeScript 23 stars

pochenai/llm-inference-calculator

A static performance model for LLM inference: given (model, GPU, interconnect, workload), instantly answer VRAM feasibility, latency(TTFT, TPOT, ITL), and throughput.

Overview: This project is a static performance model designed to quickly assess the feasibility, latency, and throughput of Large Language Model (LLM) inference given the model, GPU, interconnect, and workload parameters. It provides a ready-to-use hardware and model library, along with a two-phase Roofline performance model, automatic solver, and zero backend cost. Key features: - 403 models and 110+ NVIDIA GPUs organized by parameter tiers - 5 KV cache variants, including DeepSeek V3, Jamba, and Gemma 3 - Interconnect presets for NVLink, PCIe, InfiniBand, and more - Two-phase Roofline performance model for prefill and decode - Full parallelism support with TP/PP/EP/DP/PD disaggregation and automatic layout solver - Runs entirely in the browser with TypeScript, Vite, and React - VRAM breakdown, throughput, latency, and memory utilization visualization - Batch sweep chart for throughput and latency trends - Zero backend cost and client-side computation Tech stack: TypeScript Why it matters: This tool simplifies the complex calculation of LLM inference performance, making it easier for developers to evaluate hardware and model configurations without needing a backend setup. Repository: pochenai/llm-inference-calculator GitHub stars: 22

C# 23 stars

qdrant-operator/qdrant-operator

Qdrant operator creates and manages Qdrant clusters running in Kubernetes

Overview: The Qdrant Operator for Kubernetes automates the management of Qdrant clusters in a Kubernetes environment. Key features: - Deploy Qdrant clusters - Manage Qdrant collections and field indexes - Use Helm charts for easy installation - Support for persistence and replicas Tech stack: C# Why it matters: This operator simplifies the deployment and management of Qdrant clusters, making it easier for developers to scale their machine learning workloads.

Java 23 stars

redis/redis-vl-java

Redis Vector Library (RedisVL) -- the AI-native Java client for Redis.

Overview: Redis Vector Library (RedisVL) is the Java client designed for AI-native applications, providing fast information retrieval and vector similarity search capabilities. Key features: - Lightning-fast information retrieval - Real-time RAG pipelines - LLM semantic caching - Hybrid search combining vectors and metadata Tech stack: Java 17+ Why it matters: RedisVL simplifies AI-native applications by providing a high-performance, AI-native client for Redis, making it ideal for applications requiring vector-based search and semantic processing.

Unknown 23 stars

regolo-ai/awesome-regolo-ai

A collection of awesome tools and projects you can use with regolo.ai or that are built around it.

Overview: This repository is a collection of tools and projects that integrate with Regolo.ai, focusing on seamless and easy integration for developers. Key features: - Python client for interacting with Regolo.ai's LLM-based API - TypeScript SDK and CLI for Laravel AI integration - Integrations with Cheshire Cat Plugin, MaxKB, Dify, Gebo.ai, opencode-configs, opencode-regolo, n8n-nodes-regolo, maki, TokenHub, xsai, ebook-to-mindmap, collective-intelligence, OmniRoute, and Omnesis Tech stack: Python, TypeScript, Laravel, AI, OpenAI, and various AI models and plugins. Why it matters: These tools and integrations make it easier and more efficient for developers to harness the power of Regolo.ai in their projects, enabling them to build AI-driven applications and solutions that leverage Regolo's capabilities.

Unknown 23 stars

scf4/PreTTI

Improving Text-to-Image Models with Large Language Models

### Overview: The PreTTI project aims to leverage large language models to enhance the text-to-image model landscape, focusing on improving the efficiency and effectiveness of text prompts for generating high-quality images. ### Key features: - **Prompt Optimization**: Utilizing large language models to refine and optimize text prompts for better image generation. - **Model Training**: Developing a "Unsimplification" model to reduce the complexity of prompts, making them more intuitive and easier to write. - **Community Engagement**: Encouraging community feedback and discussion on potential proposals. ### Tech stack: Python with libraries such as Hugging Face Transformers and PyTorch. ### Why it matters: By addressing the inefficiencies in text-to-image model prompts, PreTTI aims to streamline the process, making the creation of high-quality images more accessible and less time-consuming, ultimately benefiting both users and developers.

Python 23 stars

semasuka/Talk-to-your-PDF

An application that enable the users to upload PDF files and ask questions regarding their content using Retrieval Augmented Generation (RAG)

Overview: "Talk to your PDF" is an application that allows users to upload PDF files and ask questions about their content using Retrieval Augmented Generation (RAG). This app extracts text from PDFs, generates embeddings, and provides relevant responses without manual navigation. Key features: - PDF text extraction and embedding generation - User query processing with intent detection - Database storage of PDF embeddings - Integration with Google Drive - Clean and intuitive interface Tech stack: Python 3.11+ Why it matters: This app leverages advanced NLP techniques to efficiently extract and generate relevant responses from PDFs, making it a valuable tool for users seeking information from their documents without the need for manual navigation. Repository: semasuka/Talk-to-your-PDF Primary language: Python GitHub stars: 23 GitHub description: An application that enable the users to upload PDF files and ask questions regarding their content using Retrieval Augmented Generation (RAG)

C 23 stars

shyringo/qwen3.8-27b-in-c

Qwen3.8-27B on one laptop CPU: up to 2.52 token/s, 8 GB tested, no runtime accuracy loss. Native OpenAI-compatible API with function tools; no GPU or Python. | 单颗笔记本 CPU 运行 Qwen3.8-27B：最快 2.52 token/s，最低 8 GB 内存可运行，推理加速不牺牲准确性。原生 OpenAI 兼容接口支持函数工具；无需 GPU 或 Python。

### Overview: Qwen3.8-27B on one laptop CPU achieves up to 2.52 token/s with no accuracy loss, making it a highly efficient and accurate model for text-based interactions. ### Key Features: - **Native OpenAI-compatible API**: Supports function tools and is compatible with the OpenAI API. - **No GPU or Python Required**: Utilizes a native C engine for inference, making it ideal for single-core laptops. - **8 GB RAM Minimum**: Requires at least 8 GB of RAM to run optimally, ensuring smooth performance. - **No Runtime Accuracy Loss**: The model maintains high accuracy even with performance optimizations. ### Tech Stack: The project leverages a native C engine for inference, ensuring no GPU, CUDA, Python, or external runtime is required. ### Why It Matters: Qwen3.8-27B on one laptop CPU offers a highly efficient and accurate solution for text-based interactions, making it suitable for applications requiring minimal computational resources and high performance.

C 23 stars

shyringo/qwen3.8-flash-next-in-c

The best model under 200B at near 10 token/s exact batch throughput on a single laptop CPU, with an automatic 8 GB RAM path. Native C, no GPU or Python. | 200B 以下最强模型，单颗笔记本 CPU 精确批处理接近 10 token/s，最低 8 GB 内存自动运行。原生 C 语言，无需 GPU 或 Python。

### Overview: Qwen3.8-Flash-Next is a state-of-the-art model optimized for performance on a single laptop CPU. It achieves an exact batch throughput of nearly 10 tokens per second, making it the best model under 200 billion parameters with an 8 GB RAM minimum path. ### Key Features: - **Performance**: Achieves 9.89 token/s exact batch throughput, verifying the model's performance. - **Model Size**: Includes the 125B-A6B main model and 51B PLE, totaling 176B parameters. - **Memory Usage**: Utilizes an 8 GB RAM minimum path, selected automatically. - **Chat Interface**: Supports terminal-based chat and seamless integration with OpenAI-compatible APIs. - **Optimizations**: Optimized paths preserve the selected GGUF's results, ensuring accuracy. ### Tech Stack: - **Language**: Native C - **No GPU or Python**: Utilizes a CPU-based solution without the need for GPU, CUDA, or external libraries. ### Why It Matters: - **Performance Efficiency**: The model achieves high performance on a single CPU, making it ideal for resource-constrained environments. - **Memory Efficiency**: The 8 GB RAM path ensures minimal memory usage, making it suitable for systems with limited RAM. - **Accuracy**: Preserves the results of the selected GGUF, ensuring high accuracy in the chat interface. ### GitHub Stars: - **Stars**: 23 ### Repository: - **Primary Language**: C - **GitHub Description**: The

C 23 stars

siris9476/pulsarforge

### Overview: PulsarForge is a CPU-only LLM inference engine developed by Qwen3-0.6B, designed to validate every milestone against an independent oracle. It features five validated architectures and expert streaming, pushing a 744-billion-parameter model through a 2018 laptop with 32GB of RAM and a USB SSD. ### Key Features: - **Five Architectures**: GLM-5.2, Qwen3, Qwen3moe, Deepseek2, Olmoe, and Gmldsa. - **Expert Streaming**: Pure CPU, 202GB GGUF, cross-layer predictive prefetch, arrival-order compute/IO overlap, and priority queue. - **Measurement Discipline**: Every number is backed by a public engineering retrospective, including failed experiments. ### Tech Stack: C ### Why It Matters: PulsarForge is a research engine designed to validate the performance and correctness of large language models, pushing the boundaries of inference efficiency and model validation.

Python 23 stars

siwooyong/Codalab-Microsoft-COCO-Image-Captioning-Challenge

🥉 Codalab-Microsoft-COCO-Image-Captioning-Challenge 3rd place solution(06.30.21)

### Overview: This repository contains the 3rd place solution for the Codalab-Microsoft-COCO-Image-Captioning-Challenge. It utilizes various image captioning models and architectures to achieve state-of-the-art performance. The primary language used is Python, and the repository has received 23 GitHub stars. ### Key Features: - **Encoder**: CLIP (Contrastive Language-Image Pre-Training) - **Decoder**: GPT-2 with a base model architecture - **Model Training**: Fine-tuning and ensemble techniques to improve performance - **Evaluation Metrics**: BLEU, CIDEr, and other quality metrics ### Tech Stack: - Python ### Why it Matters: This model demonstrates the effectiveness of using pre-trained models and fine-tuning techniques to achieve high performance in image captioning tasks. The use of CLIP as an encoder and GPT-2 for the decoder showcases the importance of leveraging pretrained models for efficient and effective training. The inclusion of beam search, self-critical sequence training, and ensemble methods further highlights the importance of these techniques in improving model performance.

Python 23 stars

ski-net/lipnet

LipNet with gluon

### Overview: LipNet is a Gluon implementation of the Lipreading model that processes sentences from audio and video streams, aiming to achieve end-to-end sentence-level lipreading. ### Key Features: - **End-to-End Sentence-Level Lipreading**: LipNet processes sentences from audio and video streams, providing a comprehensive solution for lipreading. - **Gluon Implementation**: Utilizes Gluon, a deep learning framework, for efficient and scalable model development. - **Grid Audio-Visual Sentence Corpus**: The model is trained on the GRID corpus, a large dataset of audio-visual sentences, to enhance its performance. ### Tech Stack: Python ### Why it Matters: LipNet offers a powerful solution for lipreading, leveraging Gluon to streamline the development process and the GRID corpus for robust training and validation.

Python 23 stars

steinathan/telephony-server

Telephony Server is a powerful bridge that connects telephony providers (Twilio, Vonage, Plivo, etc.) with real-time communication platforms (LiveKit, Jay.so, Pipecat, etc.). It enables seamless call routing, robust metrics collection, and observability features for enhanced telephony operations.

### Overview: Telephony Server is a powerful bridge that connects telephony providers with real-time communication platforms, enabling seamless call routing, robust metrics collection, and enhanced telephony operations. ### Key Features: - **Call Routing**: Intelligent call forwarding and routing between providers and communication platforms. - **Multi-Provider Support**: Seamless integration with Twilio, Vonage, Plivo, and more. - **Real-time Communication**: Connects with services like LiveKit, Jay.so, and Pipecat. - **Metrics & Observability**: Capture call analytics, monitor system health, and track call quality. - **Scalability**: Built for high-performance and scalable architectures. - **Webhooks & Event Handling**: Customizable event-driven architecture. ### Tech Stack: Python ### Why it Matters: Telephony Server simplifies the complex integration of telephony providers with communication platforms, enhancing operational efficiency and providing robust monitoring and observability features.

Kotlin 23 stars

stevan-milovanovic/LiteRT-for-Android

Image Classification, Image Captioning and LLM inference with LiteRT

### Overview: This project showcases how to integrate LiteRT and MediaPipe into an Android application for image classification, image captioning, and large language model (LLM) inference. It demonstrates modern Android development practices and serves as a practical reference for developers interested in on-device AI integration. ### Key Features: - **Image Classification**: Utilizes LiteRT for image classification tasks. - **Image Captioning**: Integrates MediaPipe for image captioning. - **LLM Inference**: Enables LLM inference directly on the device. ### Tech Stack: - Kotlin - Jetpack Compose - LiteRT - MediaPipe - Jetpack Core - MVVM architectural pattern - Dependency injection with Hilt - Retrofit for REST API communication - Moshi for JSON parsing - Coil for image loading ### Why It Matters: This project provides a comprehensive guide on how to integrate LiteRT and MediaPipe into Android apps, focusing on image classification, image captioning, and LLM inference. It serves as a practical reference for developers looking to leverage on-device AI capabilities in their applications.

Python 23 stars

vienneraphael/batchling

Save 50% off GenAI costs in two lines of code

### Overview: batchling is a simple and efficient tool that allows you to convert GenAI async functions or scripts into batch jobs, saving up to 50% on costs. It integrates with most providers and all frameworks, making it a versatile solution for various GenAI tasks. ### Key Features: - **Simple**: A single line of code or two lines of code gets you 50% off your GenAI bill. - **Transparent**: Your code remains the same, no added behaviors. Easy tracking of sent batches. - **Global**: Integrates with most providers and all frameworks. - **Safe**: Provides a complete breakdown of cost savings before launching a single batch. - **Lightweight**: Very few dependencies. ### Tech Stack: Python ### Why It Matters: batchling is a frictionless, batteries-included plugin that simplifies the process of converting GenAI async functions or scripts into batch jobs, thereby saving significant costs. It offers better cost efficiency, higher rate limits, and large-scale support, making it a valuable tool for various GenAI tasks.

C++ 23 stars

ybubnov/metalchat

Pure C++23 LLM inference for Apple Silicon chips

Overview: MetalChat is a Pure C++23 LLM inference framework and command-line interpreter designed for Apple Silicon chips, optimized for performance and ease of use. Key features: - Metal-accelerated inference for Meta Llama, Google Gemma, and Liquid models. - Active development with known issues and missing features listed in the issues tab. - Installation via Homebrew or Conan package manager for easy integration into projects. Tech stack: Pure C++23. Why it matters: MetalChat offers a high-performance solution for LLM inference on Apple Silicon, making it an ideal choice for developers looking to leverage cutting-edge AI technology on this platform.

Metal 23 stars

zhangshuibai/molten

Run a coding model at 300+ tok/s on your Mac. Native Rust + Metal inference engine for LiquidAI LFM2.5-24B-A2B (MLX 4-bit) on Apple Silicon.

Overview: Molten is a native Rust + Metal inference engine designed for LiquidAI's LFM2.5-24B-A2B model, running at 300+ tok/s on Apple Silicon. Key features: - Fast decoding performance - Single-binary, no runtime dependencies - Ready-to-use CLI, chat interface, and OpenAI-compatible server - Faithful to LiquidAI's MLX 4-bit checkpoint Tech stack: Rust + Metal Why it matters: Molten offers a high-performance, model-specific solution for running LiquidAI's LFM2.5-24B-A2B model on Apple Silicon, significantly outperforming general-purpose frameworks like MLX-lm and LLaMA.

C++ 22 stars

200lz/llm-inference-optimization-lab

Reproducible llama.cpp CPU inference profiling and a deterministic LLM serving simulator with continuous batching, KV cache, prefix caching, and workload-driven latency analysis.

### Overview: This project is a comprehensive examination of LLM inference optimization and serving, focusing on both CPU profiling and a deterministic LLM serving simulator. ### Key Features: - **CPU Profiling**: Measures llama.cpp CPU inference performance on an AMD Ryzen 7 5800H environment. - **Deterministic Serving Simulator**: Simulates LLM serving behavior with a deterministic metadata model. - **Continuous Batching**: Handles continuous requests with batch-based KV allocation. - **KV and Prefix-Cache**: Implements KV and prefix caching for efficient memory usage. - **Workload Analysis**: Analyzes latency, throughput, and SLO goodput under various workloads. ### Tech Stack: C++ ### Why It Matters: This project provides a detailed analysis of LLM inference optimization and serving, offering insights into both CPU performance and deterministic serving behavior. The results are based on real measurements and simulated deterministic models, ensuring accurate and reliable performance evaluation.