Project directory

Projects

Open-source AI projects tracked by the leaderboard, ordered by GitHub stars.

C++ 268 stars

mgonzs13/llama_ros

llama.cpp (GGUF LLMs) and llava.cpp (GGUF VLMs) for ROS 2

### Overview: This repository provides a set of ROS 2 packages to integrate [llama.cpp](https://github.com/ggerganov/llama.cpp) and [llava.cpp](https://github.com/ggerganov/llava.cpp) into ROS 2 projects. It allows you to easily incorporate the powerful optimization capabilities of these models into your ROS 2 projects by running GGUF-based LLMs and VLMs. The repository also offers an OpenAI-compatible chat completions interface with features like speculative decoding and decision models. ### Key features: - Integrate [llama.cpp](https://github.com/ggerganov/llama.cpp) and [llava.cpp](https://github.com/ggerganov/llava.cpp) into ROS 2 projects. - Use GGUF-based LLMs and VLMs for optimization. - Provide an OpenAI-compatible chat completions interface. - Support speculative decoding and decision models like OpenJEV, Lev, Kev, Nimble, Laya, and Clef. - Enable features from llama.cpp such as GBNF grammars and real-time LoRAs modification. ### Tech stack: C++ ### Why it matters: This repository is essential for developers looking to integrate advanced language models into their ROS 2 projects, offering a powerful and flexible solution for natural language processing and chat completions.

TypeScript 268 stars

oldjs/web-code-agent

Browser-local AI code Q&A engine. Chat with your codebase, ensure privacy, and optimize LLM collaboration.

### Overview: Folda-Scan is a revolutionary intelligent project Q&A tool that runs entirely locally in your browser, ensuring absolute privacy and seamless collaboration with your codebase. ### Key Features: - **Local Execution**: All data processing happens locally in your browser, ensuring your code never leaves your machine. - **"Chat" with Code**: Ask questions about your codebase as if talking to a colleague and get precise answers. - **Deep Semantic Understanding**: Grasps the true intent and complex logic within your code. - **Pinpoint Information Instantly**: Quickly locate relevant code snippets and files, even with vague descriptions. - **LLM Collaboration Accelerator**: Generates context-aware Markdown with one click, perfectly "feeding" AI assistants. - **Token Optimization**: Optimize LLM interactions, significantly reducing API call expenses and latency. - **Smart Config Generation**: Assists in creating project configuration files like `Dockerfile`. - **Instant Onboarding**: Clear guidance to kickstart your code exploration journey quickly. ### Tech Stack: Folda-Scan is built with cutting-edge web technologies and AI algorithms, including Next.js 14, File System Access API, and advanced Semantic Vector Analysis, Natural Language Processing Algorithms. ### Why It Matters: Folda-Scan transforms your codebase into a conversational partner, making code comprehension and AI collaboration unprecedentedly simple and secure. It optimizes LLM interactions, significantly reducing token costs, and provides smart configuration assistance, all while ensuring absolute privacy and seamless local execution.

Python 268 stars

scub-france/docling-Studio

Visual document analysis studio powered by Docling — configure the extraction pipeline, inspect text, tables and bounding boxes in the browser, then chunk, embed and index into OpenSearch and Neo4j.

Overview: Docling Studio is a powerful visual document analysis tool that leverages Docling to extract and analyze text, tables, images, and bounding boxes from PDFs. It provides a user-friendly interface for configuring the extraction pipeline, inspecting results in the browser, and seamlessly ingesting and indexing the extracted content into OpenSearch and Neo4j. Key features: - Configurable Docling pipeline for OCR, table extraction, code/formula enrichment, image classification, and bounding box visualization. - Bounding box overlay directly on the PDF for easy visualization. - Per-page results panel that syncs with the current PDF page. - Chunking functionality to split extracted content into semantic chunks. - Ingestion pipeline that connects Docling → chunking → embedding → OpenSearch vector indexing (one-click from Studio). - Graph storage in Neo4j, mirroring DoclingDocument tree with various relations. - Markdown and HTML export of extracted content. - Document management features including upload, list, delete, and search capabilities. - Analysis history and ability to re-open past analyses. - Upload limits and rate limiting configurable options. Tech stack: Python, FastAPI, Docling, SQLite, Vue 3, TypeScript, Vite, Pinia, Cytoscape.js. Why it matters: Docling Studio simplifies the process of extracting and analyzing documents, making it easier for developers to quickly identify and understand the content of PDFs. The integration with Docling and Neo4j provides a comprehensive solution for visual document analysis, enhancing the efficiency and accuracy of document management workflows. Repository: scub

Python 268 stars

xuiltul/animaworks

Organization-as-Code for autonomous AI agents. Brain-inspired memory that grows, consolidates, and forgets. Multi-model (Claude/Codex/Gemini/Cursor/Ollama).

Overview: AnimaWorks is an organization-as-code framework that enables autonomous AI agents to manage a production SaaS product, integrating GitHub events into a continuous workflow. Key features: - Continuous integration and deployment (CI/CD) - Multi-model reviews and collaboration - Automated task conversion from GitHub events - Real-time organization dashboard and pixel-art office visualization Tech stack: Python Why it matters: AnimaWorks provides a scalable and efficient way to manage a production environment with AI-driven teams, ensuring seamless integration and collaboration across various models like Claude, Codex, Gemini, Cursor, and Ollama. Repository: xuiltul/animaworks GitHub stars: 268 GitHub description: Organization-as-Code for autonomous AI agents. Brain-inspired memory that grows, consolidates, and forgets. Multi-model (Claude/Codex/Gemini/Cursor/Ollama).

TypeScript 267 stars

XimilalaXiang/DeLive

System audio capture + multi-provider ASR + local-first AI review workspace. Floating live captions, 12 ASR backends, 60+ languages, AI summary/chat/mindmap, Open API, MCP server, and Agent Skill.

### Overview: DeLive is a desktop transcription workspace that captures system audio, supports multiple ASR providers, and offers a local-first AI review desk with features like AI transcript correction, chat, and mind maps. It includes real-time transcription, bilingual captions, and multi-speaker identification. ### Key features: - **System-audio capture**: Captures audio from any source like browser video, live streams, meetings, courses, podcasts, or any other playback. - **Fourteen ASR backends**: Supports Soniox, Volcengine, Groq, SiliconFlow, Mistral AI, Deepgram, AssemblyAI, ElevenLabs, Gladia, Cloudflare Workers AI, 60db, OpenAI-compatible local services, and local FunASR / SenseVoice. - **Local-first AI review desk**: Transcripts are converted into searchable history with AI correction, chat, and mind maps. - **File transcription**: Transcribes uploaded audio/video files for offline processing. - **MCP integration**: Supports external AI tools via the MCP protocol. - **Skill invocation**: Invokes skills for additional functionality. ### Tech stack: TypeScript ### Why it matters: DeLive provides a comprehensive solution for system audio transcription, offering both real-time and offline transcription capabilities, and integrates seamlessly with AI tools for enhanced functionality.

Python 267 stars

chairc/Integrated-Design-Diffusion-Model

IDDM (Industrial, landscape, animate, latent diffusion), support LDM, DDPM, DDIM, PLMS, webui and distributed training. Pytorch实现扩散模型，生成模型，分布式训练

### Overview: Integrated Design Diffusion Model (IDDM) is a comprehensive framework that supports the diffusion models LDM, DDPM, DDIM, PLMS, and webUI, as well as distributed training. This project is implemented using PyTorch and aims to replicate and enhance diffusion models, trainers, and generators. It is actively maintained and has been featured in several academic papers and GitHub statistics. ### Key Features: - Replication and optimization of diffusion models. - Support for LDM, DDPM, DDIM, PLMS, and webUI. - Distributed training capabilities. - PyTorch implementation for diffusion models. ### Tech Stack: - Python - PyTorch ### Why It Matters: IDDM is a valuable resource for researchers and developers interested in improving diffusion models and their applications in various fields, including image synthesis, generative modeling, and more. Its comprehensive approach and active maintenance make it a go-to tool for anyone working on diffusion-based projects.

Python 267 stars

efeslab/fiddler

[ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration

### Overview: Fiddler is a fast inference system designed for large Mixture-of-Experts (MoE) models, specifically the Mixtral-8x7B model, that can run on a single 24GB GPU with high throughput. This system leverages both CPU and GPU resources to achieve significant speed improvements over traditional offloading methods. ### Key Features: - **High Token/Sec Performance**: Fiddler can process over 3 tokens per second on a single 24GB GPU, making it suitable for real-time applications. - **Cross-Device Communication Optimization**: It optimizes communication between CPU and GPU to reduce latency, especially beneficial for small batch sizes. - **Support for Multiple Models**: The system is designed to support a variety of MoE models, including the Mixtral-8x7B, which is currently the best-performing open-source model in the LMSys Chatbot Arena. ### Tech Stack: - **Python**: The primary programming language used for developing Fiddler. ### Why It Matters: - **Performance Improvement**: Fiddler significantly reduces the latency of CPU-GPU communication, making it ideal for applications requiring high throughput and low latency. - **Cost-Efficiency**: By utilizing CPU resources alongside GPU, Fiddler is more cost-effective, especially for models that have high computation power but limited memory. - **Model Scalability**: The system is designed to run models that can grow infinitely large, making it particularly useful for scenarios where more GPUs are not available or cost-effective. ### Benchmarks: - **Performance Comparison

Python 267 stars

lazyFrogLOL/llmdocparser

A package for parsing PDFs and analyzing their content using LLMs.

### Overview: LLMDocParser is a Python package designed to parse PDFs and analyze their content using Large Language Models (LLMs). It builds upon the concept of gptpdf, improving upon the parsing process by incorporating a layout analysis model to identify and categorize different regions within the PDFs. This package is particularly useful for applications that require detailed and accurate text extraction from PDFs. ### Key Features: - **Layout Analysis Model:** Identifies and categorizes regions such as Text, Title, Figure, Figure caption, Table, Table caption, Header, Footer, Reference, and Equation. - **Image Input:** Converts the images of the corresponding regions into text blocks that are friendly to Retrieval-Augmented Generation (RAG) solutions. - **Multimodal Model Integration:** Integrates a multimodal model like GPT-4o or Qwen-VL to directly obtain text blocks from the images. ### Tech Stack: - Python ### Why it Matters: LLMDocParser offers a more precise and efficient way to parse PDFs compared to traditional methods, making it particularly useful for applications that require high accuracy in text extraction from PDFs. Its integration with LLMs ensures that the extracted text is highly relevant and contextually accurate, enhancing the overall quality of the analysis. ### Repository: - **Repository:** lazyFrogLOL/llmdocparser - **GitHub Stars:** 267 - **GitHub Description:** A package for parsing PDFs and analyzing their content using LLMs. ### Installation: - **Using pip:** ```command

Python 267 stars

lmxyy/sige

[NeurIPS 2022, T-PAMI 2023] Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models

### Overview SIGE, or Spatially Incremental Generative Engine, is a novel approach to efficient spatially sparse inference for conditional GANs and diffusion models. This method significantly reduces computation and latency while maintaining high image quality, making it highly efficient for applications requiring real-time processing. ### Key Features - **Efficient Computation**: SIGE selectively performs computations on edited regions, reducing computation time by up to 18x compared to traditional methods. - **Model Compatibility**: SIGE supports Stable Diffusion and Mac MPS backend, enabling efficient inference on modern GPUs and CPUs. - **Benchmark Results**: SIGE achieves up to 7.2x speedups on NVIDIA RTX 3090 and 38x speedup on M1 Pro CPU when combined with GAN Compression. ### Technical Stack - **Python**: The core implementation is written in Python, leveraging its flexibility and ease of integration. - **Neural Networks**: SIGE is built on top of popular deep learning frameworks like PyTorch and TensorFlow. ### Why It Matters SIGE is a significant advancement in the field of generative models, offering substantial improvements in computational efficiency without compromising image quality. Its ability to reduce computation time by up to 18x and achieve 38x speedup on GauGAN when combined with compression techniques makes it a valuable tool for researchers and practitioners working on real-time image editing and generation tasks. ### Repository and Stars The repository `lmxyy/sige` has been starred by 267 users, indicating its growing interest and usefulness

Jupyter Notebook 267 stars

numindai/nuextract

### Overview NuExtract3 is a unified 4B vision-language reasoning model designed for document understanding. It combines strong structured information extraction with high-quality image-to-Markdown conversion, making it suitable for extraction pipelines, OCR, and RAG preprocessing for various types of documents such as scans, receipts, forms, invoices, contracts, or tables. ### Key Features - **Structured Extraction**: Input (text/images) + JSON template + instructions → JSON output - **Markdown Conversion**: Input (text/images) → Markdown - **Multimodal Inputs**: Text, images, or text + images - **Multilingual Documents** - **Reasoning and Non-Reasoning Inference Modes** - **Template Generation for Structured Extraction from Natural Language or Input Document** ### Tech Stack The NuExtract3 model is built using Jupyter Notebook. ### Why It Matters NuExtract3 is designed to handle complex document types and provide accurate structured extraction, OCR, and reasoning capabilities. It is particularly useful in scenarios where documents are scanned, contain multiple images, or have large amounts of text and data to extract. The model's ability to handle diverse use cases and provide high-quality results makes it a valuable tool for developers and researchers in the field of document understanding and natural language processing.

Python 267 stars

qdrant/skills

Agent skills for Qdrant vector search: scaling, performance optimization, search quality, monitoring, deployment, model migration, version upgrades, and SDK usage across Python, TypeScript, Rust, Go, .NET, Java

Overview: This repository contains agent skills specifically designed for Qdrant vector search, covering topics such as scaling, performance optimization, search quality, monitoring, deployment, model migration, and SDK usage across multiple programming languages including Python, TypeScript, Rust, Go, .NET, and Java. Key features: - Qdrant Advisor: Provides the freshest and most relevant Qdrant guidance, automatically updating with changes in the Qdrant ecosystem. - Scaling: Techniques for optimizing Qdrant performance and resource utilization. - Performance Optimization: Strategies for enhancing search quality and reducing latency. - Search Quality: Guidance on diagnosing and improving search results. - Monitoring: Tips for effectively monitoring Qdrant's performance and health. - Deployment: Best practices for deploying Qdrant in various environments. - Model Migration: Strategies for transitioning from one vector search model to another without downtime. - SDK Usage: Practical examples and best practices for using Qdrant's various SDKs. Tech stack: Python Why it matters: These skills provide a hands-on, practical approach to Qdrant vector search, focusing on the "when?" and "why?" aspects rather than the "how?" of implementation, making them invaluable for developers looking to optimize their vector search solutions. Repository: qdrant/skills Primary language: Python GitHub stars: 265

Python 266 stars

CIntellifusion/MultiWorld

Official Implementation of MultiWorld: Scalable Multi-Agent Multi-View Video World Models

Overview: MultiWorld is an official implementation of a unified framework for multi-agent multi-view world modeling, enabling accurate control of multiple agents while maintaining multi-view consistency. This project is designed to be scalable and supports autoregressive inference to generate beyond the training context length. Key features: - Multi-Agent Condition Module (Sec. 3.2) employs Agent Identity Embedding and Adaptive Action Weighting for multi-agent controllability. - Global State Encoder (Sec. 3.3) uses a frozen VGGT backbone to extract implicit 3D global environmental information from partial observations, enhancing multi-view consistency. - MultiWorld scales effectively across varying agent counts and camera views, supporting autoregressive inference. Tech stack: Python Why it matters: MultiWorld provides a comprehensive solution for multi-agent multi-view world modeling, making it valuable for applications requiring accurate control and consistent multi-view representation in complex environments. Repository: CIntellifusion/MultiWorld GitHub stars: 264 GitHub description: Official Implementation of MultiWorld: Scalable Multi-Agent Multi-View Video World Models

Unknown 266 stars

Hao0321/ai-media-generator

Zero-skill cinema. Senior-director prompts on autopilot. A Claude Code Skill for high-quality AI image / video / music prompt crafting and browser-based execution across 14+ generative platforms.

Overview: This project is a zero-skill AI tool designed to help anyone create high-quality AI-generated media across 14+ platforms, including cinema, photography, and music, without needing to understand complex technical details. Key features: - Automatically selects the right platform for the task. - Writes prompts tailored to each platform's specific requirements. - Allows direct browser-based execution of prompts across multiple generative models. Tech stack: The project uses a combination of open-source models and community guidelines to ensure compatibility with various platforms. Why it matters: This skill simplifies the process of creating professional-quality AI-generated media, making it accessible to anyone without needing to know the intricacies of each platform's specific requirements.

Python 266 stars

alibaba-damo-academy/MedEvalKit

MedEvalKit: A Unified Medical Evaluation Framework

Overview: MedEvalKit is a comprehensive evaluation framework designed for Large Medical Models (LMMs/LLMs) in the healthcare domain. It provides a unified evaluation platform for various medical benchmarks and models. Key features: - MultiModal Medical Benchmarks - Text-Only Medical Benchmarks - HuggingFace Exclusive Models: BiMediX2, BiomedGPT, HealthGPT, Janus, Med_Flamingo, MedDr, MedGemma, NVILA, VILA_M3 - HF + vLLM Compatible Models: HuatuoGPT-vision, InternVL, Llama_3.2-vision, LLava, LLava_Med, Qwen2_5_VL, Qwen2_VL Tech stack: Python Why it matters: MedEvalKit offers a standardized way to evaluate and compare LMMs across different medical domains and datasets, ensuring fair and accurate assessments.

Python 266 stars

ankanbhunia/Handwriting-Transformers

Handwriting-Transformers (ICCV21)

### Overview: Handwriting-Transformers is a novel transformer-based approach designed to generate realistic styled handwritten text images. The project aims to capture both global and local style patterns within the input examples, enabling the generation of high-quality styled handwriting in various scenarios. ### Key features: - **Transformer-based Generation**: Utilizes a transformer architecture for generating styled handwritten text. - **Style-Content Entanglement**: Captures long and short-range relationships within style examples, encoding both global and local style patterns. - **Self-Attention Mechanism**: Implements a self-attention mechanism to gather style representation of each query character. - **Realistic Results**: Generates realistic styled handwritten text images, significantly outperforming state-of-the-art methods. - **Versatile Application**: Handles arbitrary length of text and any desired writing style in a few-shot setting. ### Tech stack: - Python 3.7 - PyTorch >=1.4 ### Why it matters: Our proposed HWT approach is the first to introduce a transformer-based generative network for styled handwritten text generation. It significantly improves the quality of generated styled handwriting, making it a valuable tool for various applications such as text-to-image generation, style transfer, and more.

Python 266 stars

gabeur/mmt

Multi-Modal Transformer for Video Retrieval

Overview: This repository contains the code for training a Multi-Modal Transformer (MMT) for video retrieval, described in the paper "Multi-modal Transformer for Video Retrieval" [[arXiv](https://arxiv.org/abs/2007.10639), [webpage](http://thoth.inrialpes.fr/research/MMT/)]. Key features: - Multi-modal Transformer architecture for video retrieval - State-of-the-art performance on MSRVTT, ActivityNet, and LSMDC datasets - Aggregates multi-modal features (appearance, motion, audio, OCR) from videos - Embeds aggregated features into a shared space for text-based retrieval Tech stack: Python Why it matters: The MMT approach offers a novel and effective way to integrate video and text information, leading to improved video retrieval performance compared to traditional methods.

Python 266 stars

nttcslab-sp/kaldiio

A pure python module for reading and writing kaldi ark files

Overview: Kaldiio is a pure Python module designed for reading and writing Kaldi's ark and scp file formats. Key features: - Supports reading and writing of Kaldi's binary and text formats for matrices and vectors. - Handles compressed matrix formats for efficient loading. - Can read and write via pipes for easy integration with Unix commands. - Supports reading of wav.scp and wav.ark files. Tech stack: Python Why it matters: Kaldi is a widely used speech recognition toolkit, and its file formats (ark and scp) are crucial for processing large amounts of audio data. This module simplifies the process of reading and writing these files, making it easier for developers to work with Kaldi's formats in Python.

Python 266 stars

sakuranew/BERT-AttributeExtraction

USING BERT FOR Attribute Extraction in KnowledgeGraph. fine-tuning and feature extraction. 使用基于bert的微调和特征提取方法来进行知识图谱百度百科人物词条属性抽取。

### Overview: BERT-Attribute-Extraction is a project that uses BERT for attribute extraction in knowledge graphs, employing fine-tuning and feature extraction methods. This project aims to improve the accuracy of attribute extraction in knowledge graphs by leveraging the capabilities of BERT. ### Key features: - Fine-tuning of BERT for attribute extraction - Feature extraction using BERT - Dataset construction based on Baidu Encyclopedia character entries - Prerequisites for running the project - Pre-trained models available for Chinese language - Installation instructions for TensorFlow and scikit-learn ### Tech stack: Python ### Why it matters: BERT-Attribute-Extraction provides a robust solution for attribute extraction in knowledge graphs, enhancing the accuracy and efficiency of attribute extraction processes. By fine-tuning BERT and extracting features, this project addresses the limitations of traditional attribute extraction methods, leading to improved results.

Python 266 stars

zjunlp/KnowAgent

[NAACL 2025] KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents

### Overview: KnowAgent is a knowledge-augmented planning framework designed for LLM-based agents. It leverages extensive action knowledge to enhance the agents' ability to plan and execute tasks effectively. By integrating action knowledge from an action knowledge base and employing a knowledgeable self-learning mechanism, KnowAgent improves its understanding and application of action knowledge over time. ### Key Features: - **Action Knowledge Base:** An extensive database of action planning knowledge tailored to specific tasks. - **Action Knowledge Conversion:** Converts action knowledge into text, enabling the model to deeply understand and utilize this knowledge. - **Knowledgeable Self-Learning:** Continuously improves the model's understanding and application of action knowledge through iterative processes. ### Tech Stack: - Python ### Why It Matters: KnowAgent enhances the planning capabilities of LLM-based agents, making them better suited for complex tasks and applications in various domains. Its ability to learn and apply knowledge effectively makes it a valuable tool for improving the performance and reliability of AI systems. ### Repository: - **Repository:** zjunlp/KnowAgent - **GitHub Stars:** 266 - **GitHub Description:** [NAACL 2025] KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents - **GitHub URL:** [NAACL 2025] KnowAgent: [NAACL 2025] KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents ### Table of Contents: - **Table of Contents** - **News** - **Installation** - **World Map: Planning Path Generation** - **📝 Knowledge

Python 265 stars

WindVChen/DiffAttack

An unrestricted attack based on diffusion models that can achieve both good transferability and imperceptibility.

### Overview: DiffAttack is an unrestricted attack based on diffusion models that achieves both good transferability and imperceptibility. It is designed to be easily adaptable and can be applied to various diffusion models, including those from the `diffusers` library. ### Key features: - **Adversarial Generation**: Generates imperceptible and transferable adversarial examples. - **Versatile Models**: Works with diffusion models from the `diffusers` library. - **Robustness**: Ensures robustness across different normal-trained models. ### Tech stack: Python ### Why it matters: DiffAttack provides a practical and effective solution for generating imperceptible adversarial examples, making it valuable for applications requiring high transferability and low detectability.