Project directory

Projects

Open-source AI projects tracked by the leaderboard, ordered by GitHub stars.

Jupyter Notebook 87 stars

SkalskiP/SoM

Unofficial implementation and experiments related to Set-of-Mark (SoM) 👁️

Overview: This is an unofficial implementation of Set-of-Mark (SoM) tools, inspired by the paper "Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V". Key features: - Unofficial SoM implementation - Colab notebook for easy experimentation - Load and label images with specified area percentages Tech stack: Jupyter Notebook Why it matters: This repository provides a practical way to use SoM tools for visual grounding, showcasing the potential of the method in practical applications. Repository: SkalskiP/SoM Primary language: Jupyter Notebook GitHub stars: 87 GitHub description: Unofficial implementation and experiments related to Set-of-Mark (SoM) 👁️

Python 87 stars

ZeamoxWang/pdmd

Official Training and Benchmarking Implementation of PDMD: Projected Distribution Matching Distillation for Video Diffusion Models

Overview: This repository contains training, inference, and evaluation code for PDMD, a method for reducing video diffusion models to a few network evaluations (NFE) to improve stability and quality. Key features: - 🚀 Pre-trained checkpoints at 4 NFE and 2 NFE - 🚀 Inference scripts for running on a single 24GB or 80GB GPU - 🏋️ Multi-node training of student and critic LoRAs - 📊 Video evaluation metrics: VBench quality and Qwen semantic scoring - 🔊 Audio evaluation metrics: PQ, CE, CU, IS, IB, and DeSync Tech stack: Python Why it matters: PDMD reduces video diffusion models to a few network evaluations, improving stability and quality, and reaches a high VBench total score of 83.73 at 4 NFE and a best score on all six audio metrics among the compared 4-NFE models. GitHub stars: 82 Repository: ZeamoxWang/pdmd

Python 87 stars

aim-uofa/OIR

[ICLR 2024] Official PyTorch/Diffusers implementation of "Object-aware Inversion and Reassembly for Image Editing"

Overview: This project is an official implementation of the "Object-aware Inversion and Reassembly for Image Editing" paper presented at ICLR 2024. The code is written in Python and uses PyTorch and Diffusers for image editing tasks. Key features: - Implementations of object-aware inversion and reassembly for image editing using PyTorch and Diffusers. - Uses Stable Diffusion 1.4 as the model. - Provides a dataset called OIR-Bench for benchmarking the image editing results. - Includes a video showcasing the results of the search metric. Tech stack: Python, PyTorch, Diffusers. Why it matters: This project is significant as it provides a practical solution for image editing tasks, particularly for object-aware manipulation, which is not commonly addressed in the current state of the art. Repository: The project is hosted on GitHub at https://github.com/aim-uofa/OIR. GitHub stars: The repository has received 87 stars from the community, indicating its popularity and interest in the project. Primary language: The project is written in Python, making it accessible to developers familiar with Python. Why it matters: The project's popularity suggests that it is well-received and valuable to the developer community.

PowerShell 87 stars

ali-hv/comsu

A CLI tool for generating commit messages using Google Generative AI

Overview: Comsu is a CLI tool that automates the generation of high-quality commit messages using Google Generative AI, making it easier to write meaningful commit descriptions in Git. Key features: - Generates commit messages based on staged changes - Automatically commits changes with suggested commit messages - Supports conventional commit types - Easy setup with one-command installation Tech stack: PowerShell Why it matters: Comsu streamlines the process of writing commit messages, ensuring consistency and quality across projects. Repository: ali-hv/comsu GitHub stars: 87 GitHub description: A CLI tool for generating commit messages using Google Generative AI

JavaScript 87 stars

aymenfurter/smartrag

Deep Research through Multi-Agents, using GraphRAG

### Overview: SmartRAG: Elevating RAG with Multi-Agent Systems SmartRAG is a demonstration app showcasing various concepts to improve Retrieval-Augmented Generation (RAG) applications. It leverages multi-agent systems to enhance the effectiveness and accuracy of RAG-based applications. ### Key Features: 1. **Multiple Query Approaches**: Explore different data ingestion and querying methods, from simple and fast Azure OYD to the advanced GraphRAG approach. 2. **Voice Mode**: Utilize Azure OpenAI's Text-to-Speech (TTS) and Whisper for Speech-to-Text, enabling natural voice conversations. 3. **Advanced Querying**: Use Langchain summarizer or GraphRAG for complex queries in the "Ask" section. 4. **Advanced Indexing**: Enhance retrieval accuracy through multi-modal indexing techniques. 5. **Multi-Agent Research**: - **Multi-Agent**: Combine different indexes with agents (critic and researcher) working together for extended durations to find answers. - **AutoGen Integration**: Utilize Microsoft's AutoGen framework to create an ensemble of AI agents for collaborative research on complex topics. - **Time-Bounded Research**: Specify research duration to balance depth of analysis with response time. - **Citation and Verification**: Responses include citations for accuracy verification. ### Tech Stack: JavaScript SmartRAG is built using JavaScript, leveraging the latest advancements in AI and research methodologies. ### Why It Matters: SmartRAG demonstrates the potential of multi-agent systems in enhancing RAG applications, providing a more comprehensive and accurate user experience.

EJS 87 stars

hoaphamduc/jd-assistant

Luyện phỏng vấn vui vẻ nhé mọi người

Overview: A fun and engaging interview practice tool for developers. Key features: - Interactive interview scenarios - Customizable questions and answers - Real-time feedback and analysis Tech stack: EJS Why it matters: This tool helps developers improve their interview skills in a fun and engaging way, making the practice process more enjoyable and effective.

Python 87 stars

huggingface/optimum-graphcore

Blazing fast training of 🤗 Transformers on Graphcore IPUs

Overview: 🤗 Optimum Graphcore is a library that simplifies the training and inference of 🤗 Transformers models on Graphcore IPUs, making it accessible to anyone using the library. Key features: - 🤗 Transformers model loading and training on IPUs - 🤗 Hub compatibility - Model parallelization and fine-tuning - Compatibility with all tasks supported by 🤗 Transformers Tech stack: Python Why it matters: 🤗 Optimum Graphcore streamlines the process of training and inference on Graphcore IPUs, making it easier for developers to leverage the power of these specialized processors without needing to dive into the complexities of low-level hardware programming.

Jupyter Notebook 87 stars

iamycy/diffwave-sr

### Overview: This repository contains the source code for the paper "Conditioning and Sampling in Variational Diffusion Models for Speech Super-Resolution" by Yoyo LoliIcon. The code includes training scripts, evaluation commands, and pre-trained checkpoints for various speech super-resolution models. ### Key features: - **Training Scripts**: - `train.py`: Train a UDM (Unconditional Diffwave) model. - `vctk_dsp_baseline.py`: Baseline training script for VCTK dataset. - `vctk_infer.py`: Inference script for VCTK dataset. - `vctk_infer.py` with `nuwave` and `nuwave2` models. - `vctk_infer.py` with `nuwave2` model for NU-Wave 2+. - `vctk_infer.py` with `nuwave` model for NU-Wave 2+. - `vctk_infer.py` with `nuwave2` model for NU-Wave 2+. - **Evaluation Commands**: - `rate`: Upscaling ratio. - `downsample-type`: Downsampling filter type. - `infer-type`: Upscaling method. - `lr`: Learning rate. - **Pre-trained Checkpoints**: - 48 kHz: `ckpt/vctk_48k_udm/saved/training_checkpoint_500000.pt` - 16 kHz: `ckpt/vctk_

Python 87 stars

khanld/chunkformer

ChunkFormer: Masked Chunking Conformer For Long-Form Speech Transcription

### Overview: ChunkFormer is an ASR model designed for processing long audio inputs effectively on low-memory GPUs. It uses a chunk-wise processing mechanism with relative right context and employs the Masked Batch technique to minimize memory waste due to padding. The model is scalable, robust, and optimized for both streaming and non-streaming ASR scenarios. ### Key Features: - **Transcribing Extremely Long Audio**: ChunkFormer can transcribe audio recordings up to 16 hours. - **Chunk-wise Processing**: Utilizes a chunk-wise processing mechanism to handle large audio files efficiently. - **Relative Right Context**: Incorporates relative right context to improve model performance. - **Masked Batch Technique**: Minimizes memory waste by using the Masked Batch technique. - **Scalability**: Optimized for both streaming and non-streaming ASR scenarios. - **Robustness**: Robust against various speech characteristics and noise conditions. ### Tech Stack: Python ### Why It Matters: ChunkFormer is designed to handle extremely long audio inputs efficiently, making it suitable for applications requiring high processing speed and low memory usage, such as transcribing long audio recordings in real-time or offline scenarios.

Python 87 stars

linto-ai/linto-stt

An automatic speech recognition API

### Overview: LinTO-STT is an Automatic Speech Recognition (ASR) API designed for both standalone transcription and integration into microservices environments. It supports multiple engines including NVIDIA NeMo, OpenAI Whisper, Kaldi, Kyutai, and Whisper CTranslate, offering flexibility and performance across various use cases. ### Key Features: - **Multiple Engines Support**: Supports NeMo, Whisper, Kaldi, Kyutai, and Whisper CTranslate for comprehensive ASR capabilities. - **Real-Time and Offline Transcriptions**: Capable of both real-time streaming and offline transcription. - **Message Broker Integration**: Integrates with popular message brokers like RabbitMQ or Kafka for asynchronous processing. - **Standalone and Microservices Deployment**: Available as a standalone service or integrated into a microservices architecture. ### Tech Stack: - Python ### Why It Matters: LinTO-STT simplifies the process of integrating ASR capabilities into applications, making it an essential tool for developers working on speech recognition projects. Its versatility and support for multiple engines ensure it can meet the needs of a wide range of applications, from real-time transcription to complex microservices setups. ### Repository: linto-ai/linto-stt ### GitHub Stars: 87 ### GitHub Description: An automatic speech recognition API

Python 87 stars

lujiarui/esmdiff

Codebase of paper "Structure Language Models for Protein Conformation Generation" (ICLR'25)

Overview: Structure Language Models for Protein Conformation Generation (ICLR'25) is an open-source project that provides a framework for efficient protein conformation generation using a discrete variational auto-encoder and conditional language modeling. It introduces ESMDiff, an enhanced version of ESM3 with mask diffusion fine-tuning, as a state-of-the-art solution within the SLM framework. Key features: - ESMDiff: A discrete diffusion implementation of ESM3, optimized for protein conformation generation. - Discrete Variational Autoencoder: Encodes protein structures into a compact, discrete latent space. - Conditional Language Modeling: Captures sequence-specific conformational distributions for effective protein conformation generation. - PyTorch and PyTorch Lightning: Utilizes PyTorch and PyTorch Lightning for efficient model training and inference. Tech stack: Python, PyTorch, PyTorch Lightning, Hydra, ESM3. Why it matters: This project offers a practical solution for protein conformation generation, leveraging advanced language modeling techniques to efficiently generate accurate protein structures. The open-source nature of the codebase allows researchers and developers to fine-tune and adapt these models for various applications, making it a valuable resource in the field of computational biology and drug discovery. Repository: lujiarui/esmdiff GitHub stars: 87 GitHub description: Codebase of paper "Structure Language Models for Protein Conformation Generation" (ICLR'25) GitHub URL: [https://github.com/lujiarui/esmdiff](https://github.com/lu

Python 87 stars

mathpluscode/ImgX-DiffSeg

A JAX-based deep learning framework for image segmentation using diffusion models.

### Overview: ImgX-DiffSeg is a JAX-based deep learning framework for biomedical image segmentation, utilizing diffusion models for efficient and accurate segmentation of medical images. ### Key features: - Supports supervised and diffusion-based segmentation techniques. - Implements recycling training strategies for better performance. - Utilizes efficient attention mechanisms for handling 2D and 3D images. - Includes data augmentation techniques such as random affine transformations and gamma adjustment. - Supports multi-device training and mixed precision training for improved efficiency. - Incorporates early stopping for preventing overfitting during training. ### Tech stack: JAX and Flax ### Why it matters: ImgX-DiffSeg offers a powerful and efficient solution for biomedical image segmentation, leveraging advanced deep learning techniques and cutting-edge diffusion models to achieve high accuracy and speed. Its modular design and comprehensive features make it suitable for various applications in medical imaging research and development.

Python 87 stars

mazpie/genrl

[NeurIPS 2024] GenRL: Multimodal-foundation world models enable grounding language and video prompts into embodied domains, by turning them into sequences of latent world model states. Latent state sequences can be decoded using the decoder of the model, allowing visualization of the expected behavior, before training the agent to execute it.

Overview: GenRL is a groundbreaking project that leverages multimodal-foundation world models to enable generalization in embodied agents. By turning language and video prompts into sequences of latent world model states, it allows for the visualization of expected behavior before training the agent to execute it. This approach addresses the challenge of transferring knowledge across different domains and tasks. Key features: - Multimodal-foundation world models for generalization in embodied agents - Language and video prompts translated into latent world model states - Visualization of expected behavior before training - Training agent to execute the expected behavior Tech stack: Python Why it matters: GenRL offers a novel method for transferring knowledge across domains and tasks, providing a significant advancement in the field of embodied AI. Its ability to visualize expected behavior before training and its potential to generalize across different modalities make it a promising approach for future research and applications.

Unknown 87 stars

moosl/awsome-gpt-image-2-prompts

### Overview: This repository contains curated GPT Image 2 prompts collected from Twitter and X, organized into categories for easy access. Each prompt is verified with a rendered image, making it a valuable resource for designers and developers looking to create visually appealing content quickly. ### Key Features: - **Categories Overview**: Explore different categories such as Portrait & People, Product & E-commerce, Brand & Logos, Character Design, Posters & Typography, and more. - **All Prompts**: Access a comprehensive list of 1,791 prompts, each with a verified render. - **Mobile UX**: Fully responsive mobile design for easy access. - **One-Click Try**: Directly open each prompt on [aiistudio.com] for quick experimentation. ### Tech Stack: The repository is organized using a masonry grid layout for a visually appealing gallery. It supports full-text search and filters for easy navigation. ### Why It Matters: - **Efficiency**: Skip the blank-canvas syndrome by finding proven and verified prompts. - **Versatility**: Suitable for a wide range of design projects, from product shots to character design. - **Community**: Credited original authors ensure the quality and relevance of the prompts. ### GitHub Stars: The repository has been starred by 87 developers, indicating its popularity and usefulness in the community. ### Browse the Gallery: - **Web Gallery**: Explore the full collection on [aiistudio.com](https://aiistudio.com). - **README**: Learn more about the features and categories of the prompts. ### 🤔 What is This?

Python 87 stars

project-etalon/etalon

LLM Serving Performance Evaluation Harness

### Overview: etalon is a tool designed to benchmark and evaluate LLM inference systems, focusing on the performance of LLMs in various scenarios. It provides a comprehensive framework for testing and optimizing LLMs, making it an essential resource for developers and researchers in the field of natural language processing. ### Key Features: - **Benchmarking**: etalon allows users to test the performance of different LLMs under various conditions, such as different client configurations, request intervals, and metrics tracking. - **Open Source Compatibility**: etalon can be run with any open-source LLM inference system, making it versatile and adaptable to a wide range of models. - **Performance Analysis**: It provides detailed performance metrics and analysis, helping users understand the bottlenecks and areas for improvement in their models. ### Tech Stack: Python ### Why it Matters: etalon serves as a crucial tool for developers and researchers to evaluate and optimize the performance of LLMs. By providing a standardized benchmarking framework, etalon helps identify the strengths and weaknesses of different models, enabling developers to make informed decisions and improve the overall performance of their systems.

HTML 87 stars

svilupp/Julia-LLM-Leaderboard

Provides a platform for the Julia community to compare AI models' abilities in generating syntactically correct Julia code, featuring structured tests and automated evaluations for easy and collaborative benchmarking.

### Overview: A platform for the Julia community to compare AI models' abilities in generating syntactically correct Julia code, featuring structured tests and automated evaluations for easy and collaborative benchmarking. ### Key features: - Structured test cases - Automated evaluation methodology - Comparison of various AI models' code generation capabilities ### Tech stack: Julia ### Why it matters: Provides a practical benchmark for Julia language generation, guiding users in choosing the best model for their needs.

Python 87 stars

vincentkoc/airgapped-offfline-rag

Secure, locally-run Retrieval-Augmented Generation system for document-based question-answering, utilizing Llama 3, Mistral, and Gemini models with a user-friendly Streamlit interface.

Overview: This project implements a secure, locally-run Retrieval-Augmented Generation (RAG) system for document-based question-answering, utilizing Llama 3, Mistral, and Gemini models with a user-friendly Streamlit interface. Key features: - Local inference with LLaMA c++ - Model support for LLaMA 3 (3 and 3.1) - Model support for Mistral - Model support for Gemini - Support for quantized models - Document upload and processing - Question-Answering system - Sentence Transformer Model for Vector Storage - Integration with Streamlit for UI - Support for streaming responses Tech stack: Python Why it matters: This project provides a robust and secure solution for document-based question-answering, leveraging cutting-edge models and a user-friendly interface, making it valuable for developers and researchers in the field of natural language processing and document analysis. Repository: vincentkoc/airgapped-offline-rag GitHub stars: 87

Python 87 stars

zengxianyu/PPD-examples

Code and Models for the paper NeuralRemaster with Phase-Preserving Diffusion

Overview: Phase-Preserving Diffusion (PPD) is a novel technique that preserves the phase of images during diffusion processes, enabling consistent re-rendering of geometry while allowing for controlled appearance variation. It works seamlessly with various diffusion models without modifications. Key features: - Preserves image phase during diffusion. - Works with SD 1.5, FLUX, and Wan models. - Enables geometry-consistent re-rendering. - Controlled by a single parameter `r` for appearance variation. Tech stack: Python Why it matters: PPD addresses the limitations of traditional diffusion models by preserving phase information, leading to more realistic and consistent re-rendering of images, especially in games, videos, and simulators. Repository: zengxianyu/PPD-examples Primary language: Python GitHub stars: 87 GitHub description: Code and Models for the paper NeuralRemaster with Phase-Preserving Diffusion Repository: zengxianyu/structured-noise Primary language: Python GitHub stars: 87 GitHub description: Models and code for structured noise, used in PPD.

Python 86 stars

KR-HappyFace/KoDALLE

🇰🇷 Text to Image in Korean

### Overview: KoDALLE is a Korean-specific Text-to-Image model that leverages the PLMs' token embedding layer and position embedding layer to generate fashion designs from textual descriptions. ### Key Features: - **Text-to-Image Generation**: Converts textual descriptions into fashion designs. - **Korean Language Support**: Utilizes Korean-specific PLMs for better performance. - **Efficient Training**: Uses a smaller dataset of 0.8 million pairs for training. ### Tech Stack: Python ### Why it Matters: - **Efficient Design Generation**: Produces high-quality fashion designs from concise textual descriptions. - **Versatile Application**: Suitable for various fashion design tasks, including pattern creation and color combinations. - **Resource-Efficient**: Requires significantly less computational resources compared to OpenAI's DALLE, making it ideal for smaller datasets and limited computing power.

Python 86 stars

LearnedVector/Wav2Letter

Speech Recognition model based off of FAIR research paper built using Pytorch.

Overview: This project is a straightforward implementation of the Wav2Letter speech recognition model, built using PyTorch, based on the research paper from Facebook AI Research (FAIR). It includes features like CTC Loss and Greedy Decoder, and is designed for easy readability. Key features: - Uses CTC Loss - Uses Greedy Decoder Tech stack: Python Why it matters: This implementation provides a simple yet effective way to understand and implement the Wav2Letter model, making it accessible for developers and researchers alike. Repository: GitHub - https://github.com/LearnedVector/Wav2Letter Primary language: Python GitHub stars: 86 GitHub description: Speech Recognition model based off of FAIR research paper built using Pytorch.