Projects

### Overview: This beginner's guide is designed to help developers explore and master the NVIDIA Jetson Orin platform, focusing on AI computing and robotics applications. It covers software stack, tool coverage, hands-on projects, and step-by-step tutorials. ### Key Features: - **From Beginner to Master**: Progress from basic to advanced AI applications. - **Comprehensive Tool Coverage**: Master NVIDIA technologies like CUDA, JetPack SDK, TensorRT, and Deepstream. - **Hands-on Projects**: Build end-to-end AI networks, autonomous robots, and large language models. - **Step-by-Step Tutorials**: Clear, incremental instructions for developing complex AI applications. ### Tech Stack: Python ### Why It Matters: This guide is essential for developers looking to innovate in AI and robotics, providing a comprehensive approach to harnessing Jetson Orin's powerful capabilities. ### GitHub Stars: 189 ### Repository: Seeed-Projects/reComputer-Jetson-for-Beginners ### Primary Language: Python ### GitHub Description: Beginner's Guide to reComputer Jetson ### Why It Matters: This guide is essential for developers looking to innovate in AI and robotics, providing a comprehensive approach to harnessing Jetson Orin's powerful capabilities. ### Primary Language: Python ### GitHub Description: Beginner's Guide to reComputer Jetson

### Overview: ServeGen is a framework designed to generate realistic large language model (LLM) serving workloads, powered by analyzing billions of inference requests from Alibaba Cloud Model Studio. It aims to replicate the complexity of real-world LLM workloads, including bursty request arrivals, shifting input/output length distributions, and heterogeneous data composition. ### Key Features: - **Bursty Request Arrivals**: Models beyond simple Poisson distributions. - **Shifting Input/Output Length Distributions**: Diverse input/output length distributions over days and weeks. - **Heterogeneous Data Composition**: Multimodal workloads with varying data compositions. - **Bimodal Reasoning Length Distribution**: Reasoning workloads with bimodal reasoning lengths. ### Tech Stack: Python ### Why it Matters: ServeGen serves as a data-driven bridge between research and production, enabling more accurate and efficient design and deployment of new LLM serving systems. ### Repository: alibaba/ServeGen ### Primary Language: Python ### GitHub Stars: 180 ### GitHub Description: A framework for generating realistic LLM serving workloads. Powered by the analysis of billions of inference requests across 12 production models on Alibaba Cloud Model Studio. Aimed at replicating real-world LLM workloads, including bursty request arrivals, shifting distributions, and heterogeneous data composition. See examples for detailed usage and customization. GitHub: [https://github.com/alibaba/ServeGen](https://github.com/alibaba/ServeGen)

### Overview: HOT-Step-CPP is a desktop application for local AI music generation powered by GGML, built on the AceStep.cpp engine and featuring four native C++ backends: ACE-Step 1.5, MiniMax-Music3, YuE2, and Stable Audio 3. The app allows users to describe a song with a style caption and lyrics, generating stereo audio entirely on their own hardware without cloud services or external tools. It supports Windows, Linux, and macOS, and can be run in Colab or from a phone using a RunPod image. ### Key features: - **Local AI Music Generation**: Generates music entirely on the user's hardware. - **Four Native C++ Backends**: ACE-Step 1.5, MiniMax-Music3, YuE2, and Stable Audio 3. - **GPU-Hungry Training Studio**: Trains style adapters for different engines without Python or external tools. - **Stable Audio 3 Previews**: Renders audio previews at checkpoints, improving performance. ### Tech stack: Built on AceStep.cpp engine and powered by GGML. ### Why it matters: HOT-Step-CPP offers a powerful and efficient way to create music locally, bypassing cloud services, API keys, and subscription fees. It leverages the latest C++ engine and GPU training methods to generate high-quality audio without the need for external tools or cloud infrastructure.

### Overview: This project aims to address the multimodal retrieval challenge in a mobile e-commerce platform by using a state-of-the-art model, MCAN, and VisualBERT. The dataset consists of querying phrases and image features, and the goal is to rank the candidate products based on their image features. ### Key Features: - Implementing MCAN and VisualBERT models for multimodal retrieval. - Using RoBERTa embeddings for text and black box function for image features. - Preprocessing text and image features to improve model performance. - Negative sampling for efficient training. ### Tech Stack: - Jupyter Notebook ### Why it Matters: This project is significant because it demonstrates the effectiveness of multimodal retrieval in e-commerce platforms, showcasing advancements in deep learning techniques for real-world applications.

Overview: Trove AI is an open-source, self-hostable knowledge-management agent designed specifically for the Chinese internet. It helps users save and re-read articles, connect with various online platforms, and manage their personal knowledge base. Key features: - Captures articles from multiple platforms including WeChat Channels, WeChat, TikTok, Douyin, Sina Weibo, Bilibili, Medium, CSDN, and more. - Uses a private library to answer questions, review recent saves, find source material, compare viewpoints, and cite articles. - Supports semantic search and Q&A using RAG technology. - Automatically grows a knowledge graph based on semantic distance. - Provides learning paths based on one sentence inputs. - Integrates with WeChat Bot and Feishu Bot for easy sharing and management. - Supports Obsidian sync and multi-tenant, production-grade features like JWT auth and revocable sync tokens. Tech stack: Python Why it matters: Trove AI offers a self-hosted solution for knowledge management, allowing users to control their own data and knowledge base, ensuring it's not dependent on third-party servers.

### Overview: This repository contains the code for the CVPR 2023 paper "SINE: Single Image Editing with Text-to-Image Diffusion Models". The paper presents a novel approach to single image editing using text-to-image diffusion models, showcasing significant improvements in image editing tasks. ### Key features: - **Text-to-Image Diffusion Models**: Utilizes diffusion models to generate high-quality images from textual prompts. - **Fine-tuning**: The repository includes scripts for fine-tuning the model on specific datasets and experiments. - **Data Preparation**: Provides instructions on how to prepare and use the data used in the paper. - **Fine-tuning with Patch-based Training Scheme**: Offers a detailed guide for fine-tuning the model using a patch-based training scheme. - **Model-based Image Editing**: Demonstrates how to edit images using guidance from multiple models. - **Diffusers Library Example**: Provides an example of how to use the Diffusers library, which is still under development. ### Tech stack: Python ### Why it matters: This project is significant because it showcases a novel approach to single image editing using advanced diffusion models, potentially leading to more efficient and effective image editing techniques.

Overview: This project is an application of GraphRAG that focuses on improving its ability to process Chinese content. It includes methods to replace the OpenAI model, modify prompts, and use custom configurations for better Chinese handling. Key features: - Custom prompt translation for Chinese - Use of a Chinese tokenizer for document splitting - Custom model selection for better Chinese support - Custom configuration for indexing and querying Tech stack: Python Why it matters: This project demonstrates how to enhance the performance of GraphRAG in handling Chinese content, making it more suitable for Chinese language applications. Repository: Airmomo/graphrag-practice-chinese Primary language: Python GitHub stars: 188 GitHub description: GraphRAG的应用实例，项目特点在于提供了替换OpenAI模型的方法，并通过修改原有提示和切分文档的方法，提高了GraphRAG处理中文内容的能力。

Overview: This Python-based data ingestion service processes various document types into Azure AI Search content, including PDFs, images, spreadsheets, transcripts, and SharePoint content. Key features: - Chunks documents - Generates embeddings - Writes Azure AI Search content - Runs Blob, SharePoint, NL2SQL, and maintenance jobs - Exposes ingestion APIs - Provides an operator dashboard - Offers a separately gated hosted retrieval API Tech stack: Python Why it matters: This service is crucial for preparing and indexing documents for use in Azure AI Search, enhancing the accessibility and relevance of information for agents. Repository: Azure/agent-app-ingestion GitHub stars: 188

### Overview This repository contains a multi-gpu PyTorch implementation of the paper "Denoising Diffusion Autoencoders are Unified Self-supervised Learners" by Weilai Xiang, Hongyu Yang, Di Huang, and Yunhong Wang. The implementation includes frameworks for various diffusion models, networks such as the basic 35.7M DDPM UNet and a larger 56M DDPM++ UNet, and datasets for CIFAR-10 and Tiny-ImageNet. The code also includes feature quality evaluation, noise-conditional classifier training, and evaluation for ImageNet-256 pre-trained [DiT-XL/2] checkpoint. ### Key Features - **Framework**: Supports DDPM and DDIM frameworks. - **Networks**: Implements the basic 35.7M DDPM UNet and a larger 56M DDPM++ UNet. - **Datasets**: Includes CIFAR-10 and Tiny-ImageNet datasets. - **Evaluation**: Provides feature quality evaluation, linear probing and grid searching, contrastive metrics, fine-tuning, and evaluation for ImageNet-256 pre-trained [DiT-XL/2] checkpoint. ### Tech Stack - Python - PyTorch - PyTorch-FID - EMA-PyTorch ### Why It Matters This implementation is a simplified version of the official codebase and may differ in some details such as network initialization and hyperparameters. The repository provides a simplified DDPM / EDM implementation and showcases the gener

### Overview: DuQuant is a novel method for enhancing the performance of quantized language models by distributing outliers using dual transformation. This approach significantly improves the accuracy and robustness of 4-bit weight-activation quantization across various model types and downstream tasks. ### Key Features: - **Dual Transformation**: Utilizes rotation and permutation transformations to eliminate both massive and normal outliers. - **State-of-the-Art Baselines**: Establishes new benchmarks for 4-bit weight-activation quantization across different model types and tasks. - **NeurIPS 2024 Oral Presentation**: Accepted for an Oral presentation at NeurIPS 2024, showcasing its significance in the field of quantized language models. ### Tech Stack: - Python ### Why It Matters: DuQuant provides a practical solution for improving the performance of quantized language models by effectively handling outliers, thereby enhancing the overall accuracy and robustness of the models.

Overview: This project presents a research prototype of an LLM chatbot designed to assist hiring managers in resume screening. The chatbot leverages RAG (Revised Adaptive Journal) and RAG Fusion to dynamically retrieve and process resumes, offering a more effective alternative to keyword-based screening methods. Key features: - Adaptive Retrieval: Utilizes RAG/RAG Fusion for similarity-based and keyword-based resume retrieval. - Generation: Retrieves resumes to augment the LLM generator for cross-comparisons and decision-making. Tech stack: Jupyter Notebook Why it matters: The project aims to integrate LLM-based methods into resume screening, addressing the limitations of existing keyword-based approaches and providing a more reliable and effective solution for the recruitment domain. Repository: Hungreeee/Resume-Screening-RAG-Pipeline Primary language: Jupyter Notebook GitHub stars: 189 GitHub description: An LLM chatbot that dynamically retrieves and processes resumes using RAG to perform resume screening. README excerpt: The project is part of the author's graduating thesis, aiming to present a POC of an LLM chatbot for resume screening. The chatbot uses hybrid retrieval methods to enhance the LLM agent with relevant resumes, improving accuracy and effectiveness. The demo interface showcases the chatbot's capabilities with synthetic and job description datasets.

Overview: FineR is a cutting-edge system that democratizes fine-grained visual recognition by leveraging large language models to interpret visual attributes as text, enabling it to identify subtle differences between species or objects without needing expert annotations or category names. Key features: - Fine-grained Semantic Category Reasoning (FineR) - Democratizes fine-grained visual recognition - Leverages large language models for category identification - Outperforms current FGVR methods Tech stack: FineR is built using Python, PyTorch, and OpenAI API for discovering semantic concepts via large language models. Why it matters: FineR provides a powerful solution for fine-grained visual recognition, making it accessible to researchers and practitioners without the need for extensive annotation or prior knowledge of category names. This democratizes access to advanced visual recognition capabilities, potentially revolutionizing applications in fields like healthcare, security, and autonomous systems.

### Overview: The **awesome-rag** repository is a curated collection of resources and tools related to Retrieval-Augmented Generation (RAG), a technique that enhances language models with external context to improve their performance. This list serves as a one-stop catalogue for developers looking to build enterprise-grade RAG applications. ### Key features: - **Tools and Libraries**: A diverse range of open-source tools and libraries for RAG, including SDKs for building custom applications. - **Embedding Models & Libraries**: Resources for embedding models and libraries that enhance RAG capabilities. - **Vendor Examples**: Examples of RAG applications from various vendors. - **Vector DBs & Search Engines**: Information on vector databases and search engines used in RAG. - **Research Papers and Surveys**: Comprehensive lists of research papers and surveys related to RAG. - **RAG Approaches and Architectures**: Detailed information on different RAG approaches and architectures. - **Frameworks**: Resources for building RAG frameworks. - **RAG Techniques and Methodologies**: Information on various techniques and methodologies used in RAG. - **Multimodal RAG**: Resources specifically focused on multimodal RAG. - **Graph-based RAG**: Resources for graph-based RAG approaches. - **Retrieval Methods**: Detailed information on different retrieval methods used in RAG. - **Prompting Strategies**: Strategies for prompting in RAG systems. - **Chunking & Pre-processing**: Techniques for chunking and pre-processing inputs. - **Embeddings Models**: Resources for embeddings models used in RAG. - **Instruction Tun

Overview: This repository contains recipes, engine patches, and benchmark harnesses for LLM inference on Intel Arc Pro B60/B70 (Battlemage, Xe2). It includes MoE 35B at 160 t/s decode / 7.5K t/s prefill single-stream, 27B at 50~ t/s decode / 1.7K t/s prefill single stream. The recipes are unlocked for vLLM XPU MTP and feature a Muse Glimmer recipe. Key features: - MoE 35B inference - 27B inference - vLLM XPU MTP unlocked - Muse Glimmer recipe Tech stack: Python Why it matters: This repository provides detailed recipes, engine patches, and benchmark data for efficient LLM inference on Intel Arc Pro B60/B70, making it a valuable resource for developers and researchers working with this hardware. Repository: SergiioB/intel-arc-pro-b70-inference-cookbook GitHub stars: 188

### Overview: This repository contains the source code for the CVPR 2021 paper "Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAE". The project focuses on developing a method to generate diverse structures for image inpainting using a hierarchical VQ-VAE model. ### Key features: - **Hierarchical VQ-VAE Model**: The project utilizes a hierarchical VQ-VAE architecture to generate diverse inpainting structures. - **Image Inpainting**: The method is applied to image inpainting tasks, where missing regions in an incomplete image are filled with diverse structures. - **Training and Testing**: The code includes training scripts for VQ-VAE, structure generator, and texture generator, as well as a testing script to apply the generated structures to test images. ### Tech stack: - **Python**: The primary programming language used for the implementation. - **TensorFlow**: Used for training the VQ-VAE and structure generator models. ### Why it matters: This project addresses the challenge of generating diverse inpainting structures for images, which is crucial in various applications such as photo restoration and image editing. The hierarchical VQ-VAE model used in this project offers a novel approach to handling complex inpainting tasks, potentially improving the quality and diversity of the generated inpaintings.

### Overview: 基于DeepSpeed，本项目旨在突破硬件限制，实现大模型高效训练。 ### Key features: - 收录到[PytorchNetHub](https://github.com/bobo0810/PytorchNetHub) - 分布式数据并行DDP的训练pipeline - MoE用法 - 学习率调度器的配置 - ZeRO零冗余优化器的配置 - 流水并行的训练pipeline - 流水模型的保存、加载、指标评估 - TensorBoard可视化 ### Tech stack: Python ### Why it matters: 通过使用DeepSpeed，开发者可以轻松地实现分布式训练、模型并行、零冗余优化器等高级功能，从而显著提高大模型训练的效率和效果。

Overview: pdfLLM is a proof-of-concept Retrieval-Augmented Generation (RAG) microservice designed for processing, embedding, and querying documents. It integrates document parsing, semantic search, knowledge graph construction, and LLM-based generation to provide advanced document intelligence. Key features: - Semantic Search using OpenAI models - Graph-Based Search using Dgraph - LLM-Powered Responses using OpenAI chat models - Hybrid Retrieval combining vector similarity and entity relationships Tech stack: Python Why it matters: pdfLLM is a highly scalable and cost-effective solution for document processing and retrieval, leveraging cutting-edge AI technologies to provide advanced document intelligence.

Overview: Image Lens 是一个 Chrome 浏览器插件，用于快速反推出网页上图片的提示词，并通过实时生成图片验证这些提示词的有效性。它支持 Gemini 和 OpenAI Compatible 两种模型，用户可以自定义 API Key，实现更灵活的模型选择。 Key features: - 反推出网页图片的提示词 - 实时验证提示词的有效性 - 支持 Gemini 和 OpenAI Compatible 两种模型 - 自定义 API Key 选项页 - 跨平台支持（中文 / 英文切换） Tech stack: JavaScript, TypeScript, React, CSS, HTML Why it matters: Image Lens 提供了一种便捷的方式来测试和验证网页上的提示词，帮助开发者快速优化他们的设计和内容。通过实时生成图片验证提示词，可以显著提高开发效率和用户体验。

### Overview: This repository is a curated collection of research papers, benchmarks, frameworks, and tools across 24 domains focused on agentic AI. It serves as a one-stop resource for researchers and builders looking to explore and implement agentic AI solutions. ### Key Features: - **Research Papers**: Over 300 papers covering AI, cognition, reasoning, testing, healthcare, software engineering, automation, networking, vision, robotics, foundation models, hardware design, and more. - **Frameworks and Orchestration**: 65 frameworks and tools for building and orchestrating agents. - **Security and Safety**: 55 resources dedicated to security, red teaming, defenses, and ethical considerations. - **Benchmarks and Evaluation**: 27 resources for benchmarking and evaluating agentic AI systems. - **Domain Applications**: Resources for healthcare, security, IoT, smart cities, transportation, supply chain, and more. ### Tech Stack: The repository is built using Markdown and hosted on GitHub. ### Why It Matters: This comprehensive collection helps researchers and builders quickly find the resources they need to develop and implement agentic AI solutions, saving time and effort.

### Overview: Pokémon Champion is an expert-level minimax language agent designed for battle simulations. The project aims to showcase the capabilities of language models in competitive environments. ### Key Features: - **Language Model Agent (LLM Player)**: Implements a minimax language agent using various backend models like OpenAI GPT, Meta LLaMA, and Google Gemini. - **Battle Engine (poke_env)**: Independent of the LLM backend, ensuring flexibility and modularity. - **Bayesian Prediction System (bayesian)**: Supports Bayesian prediction for team and live battle outcomes. - **Battle Execution Scripts (scripts)**: Includes scripts for local 1v1 battles and evaluation tools. - **Training Dataset Processing (training)**: Handles the preprocessing of battle data for training. ### Tech Stack: - Python ### Why It Matters: PokéChamp demonstrates the potential of language models in competitive environments, showcasing their ability to strategize and make decisions effectively. The project aims to push the boundaries of what language models can achieve in complex, real-time simulations. ### Repository: sethkarten/pokechamp ### GitHub Stars: 188 ### GitHub Description: Official repository of the spotlight ICML 2025 paper, PokeChamp: an Expert-level Minimax Language Agent.