Projects

Overview: The official repository for the GUI-R1 project, a generalist R1-style Vision-Language Action Model for GUI agents. This model uses reinforcement learning to improve the execution capabilities of LVLMs in real-world GUI agent tasks. Key features: - A Generalist R1-style Vision-Language Action Model for GUI Agents - Reinforcement Learning based on a unified action space rule modeling approach - High-quality dataset of 800K samples across multiple platforms (Windows, Linux, MacOS, Android, and Web) - Performance improvement of 10x over previous state-of-the-art methods like OS-Atlas Tech stack: Python Why it matters: The GUI-R1 project demonstrates the potential of reinforcement learning in improving the execution capabilities of LVLMs for real-world GUI agent tasks, showcasing the immense potential of unified action space rule modeling in real-world applications. Repository: ritzz-ai/GUI-R1 GitHub stars: 260

Overview: This repository contains code, models, and related resources for the paper "Language Models Can See: Plugging Visual Controls in Text Generation". It provides a framework called MAGIC that enables language models to perform zero-shot image captioning. Key features: - MAGIC (MA-guided text generation with CLIP) framework - Combines a language model (e.g., GPT-2) with an image-text matching model (e.g., CLIP) - Enables LMs to perform multimodal tasks (e.g., image captioning) in a zero-shot manner - Simple yet efficient plug-and-play solution Tech stack: Python Why it matters: This framework opens up new possibilities for integrating visual information into text generation, making language models more versatile and capable of handling a wider range of tasks.

### Overview: HyCanvas is an open-source design editor that allows you to create graphics, slides, videos, whiteboards, and documents all in one app. It runs on your own server with Docker, and is completely free with no watermark. The app features 140 pre-built templates, including social posts, posters, presentations, and more. ### Key Features: - **Editing**: Tools open from a rail on the left, properties panel follows your selection, and a floating toolbar sits above whatever you pick. - **Presentations**: Slide layouts and themes, transitions including Magic Move, present mode with a presenter view, live audience polls and Q&A, and PPTX in and out. - **Templates**: 140 built-in templates across social, print, events, business, and presentations, plus your own saved templates and PowerPoint imports. - **Whiteboards**: Sticky notes, frames, connectors, and ink, workshop templates, a shared timer, and dot voting, and convert a board into a deck. - **Design**: Shapes, frames, tables, 9 chart types, QR codes, icons, illustrations, text effects, font pairings, image filters, crop and masks, background removal, Magic Resize. - **Video**: Multi-track timeline with trims, transitions, keyframes, captions (SRT/VTT), green-screen keying, screen and webcam recording. - **Docs and sheets**: A block editor with DOCX and PDF export, and spreadsheets with a formula engine. - **Together**: Realtime co-editing with presence cursors,

### Overview: Multimodal Co-Attention Transformer (MCAT) is a novel approach designed to predict survival outcomes in gigapixel whole slide images (WSIs) by integrating histology and genomic data. This method leverages the co-attention mechanism to learn pairwise interactions between histology patches and genomic embeddings, effectively fusing information from both modalities. ### Key Features: - **Multimodal Fusion**: MCAT integrates histology and genomic data through a co-attention mechanism, enabling the learning of interactions between histology patches and genomic embeddings. - **Transformer Attention**: The Transformer Attention is adapted for WSIs to handle long-range dependencies, enhancing survival outcome prediction. - **Genomic Signatures**: The method utilizes gene families from MSigDB as genomic signatures, specifically focusing on IDH1 mutation to distinguish GBM from LGG. ### Tech Stack: - **Python**: Utilized for implementing the ML models and data processing. - **Jupyter Notebook**: Used for detailed documentation and experimentation. - **OpenSlide**: For processing and extracting patches from WSIs. - **PyTorch**: For implementing the Transformer-based model. - **TensorFlow**: For training the model on GPU-accelerated systems. - **Captum**: For providing insights into the model's behavior. - **Shap**: For interpreting the model's predictions. ### Why It Matters: MCAT offers a significant improvement in survival prediction accuracy compared to traditional methods, demonstrating its potential in personalized medicine for cancer diagnostics and treatment. ### Repository: The repository is hosted on GitHub at [mahmoodlab/MCAT

Overview: This repository offers a beginner's guide to self-study Machine Learning and Artificial Intelligence, inspired by renowned AI/ML educator Andrej Karpathy's wisdom. Key features: - Comprehensive collection of resources for Machine Learning and Artificial Intelligence - Evolving and collaborative nature of the repository - Breakdown of the fundamental difference between Machine Learning and Artificial Intelligence - Focus on mathematics, Python, machine learning, advanced topics, data processing, specializations, and generative AI - Unclassified learning resources for additional exploration - Words of wisdom from experienced AI/ML experts - Changelog for updates and improvements Tech stack: Python, machine learning libraries, and other relevant tools Why it matters: This roadmap serves as a starting point for anyone looking to learn Machine Learning and Artificial Intelligence, providing a structured approach to mastering the fundamentals and expanding into specialized areas.

### Overview: ROCm™ Hyperloom is an agentic system designed to autonomously optimize Large Language Models (LLMs) on AMD Instinct™ GPUs. It profiles each workload, searches for framework and kernel optimizations, validates every candidate end-to-end, and integrates proven results into a knowledge base without requiring manual tuning. ### Key Features: - **Auto-Optimization**: Profiles and optimizes LLM workloads on AMD GPUs. - **Framework and Kernel Optimization**: Searches for the best framework and kernel configurations. - **End-to-End Validation**: Validates every candidate solution from start to finish. - **Recipe Knowledge Base**: Integrates proven results into a knowledge base. ### Tech Stack: Python ### Why It Matters: By automating the optimization process, Hyperloom reduces the time and effort required to tune LLM workloads, making it more efficient and cost-effective. This system ensures that LLMs run at optimal performance on AMD GPUs without the need for manual tuning, thereby maximizing hardware utilization and reducing latency.

### Overview: ShipGenAI is a curated collection of 50 complete, production-ready AI SaaS products that you can launch under your own brand. Each app ships with Stripe checkout, Google OAuth, 100+ AI models via MuAPI, Vercel-ready deployment, and MIT licensing. ### Key features: - 🚀 Production-ready AI SaaS apps - 💳 Stripe checkout + webhooks - 🔐 Google OAuth - 🤖 100+ AI models via MuAPI - 🌐 Vercel-ready deployment - 🗄️ Prisma + PostgreSQL - 🆓 MIT licensed ### Tech stack: JavaScript ### Why it matters: These AI SaaS apps are complete, sellable products that can be launched under your brand, generating significant revenue. The open-source version is available for anyone to use and customize, making it a valuable resource for developers and businesses looking to build AI-powered SaaS applications.

Overview: DAT (Data Ask Tool) is an enterprise-level AI tool designed to enable business users to interact with databases using natural language, making it possible to query data without writing complex SQL scripts. It leverages pre-built semantic layers to ensure AI’s accuracy and completeness. Key features: - 🚀 Enterprise-level architecture with SPI (Software Plugin Interface) for flexible database, LLM, and embedding model integration. - 🚀 Factory pattern implementation for standardized component creation and management. - 🚀 Modular design for clear responsibilities and easy maintenance and extension. - 📃 Multi-database support with MySQL, PostgreSQL, Oracle, and more through SPI. - 🚀 Intelligent SQL generation with natural language understanding and SQL dialect conversion. - 🚀 Semantic model binding to ensure query accuracy through pre-defined models. - 📃 Rich semantic modeling with entities, dimensions, measures, and YAML configuration. - 🚀 Enhanced vectorized retrieval with SQL answer pairs, synonyms, and business knowledge embedding. - 🚀 Multi-storage backend support with DuckDB, Weaviate, and PGVector. Tech stack: Java 17, Maven 3.6, AskDeepWiki, Apache License 2.0. Why it matters: DAT aims to bridge the last mile of data querying for businesses by enabling them to interact with databases using natural language, thereby reducing the need for complex SQL scripts. By leveraging pre-built semantic layers, it ensures that AI can express itself accurately and confidently, making it a valuable tool for businesses looking to improve their data querying capabilities.

### Overview: This project, `r1-ktransformers-guide`, provides a guide on running DeepSeek-R1 GGUFs on KTransformers, showcasing improved performance compared to LLaMA.cpp. It supports both CPU and GPU inference, with a focus on optimizing memory usage and performance on high-end systems. ### Key Features: - **DeepSeek-R1 GGUFs**: Optimized quantized versions of the DeepSeek-R1 model. - **KTransformers**: A lightweight and efficient model serving framework. - **Performance Optimization**: Demonstrates faster inference speeds on systems with sufficient VRAM and RAM, making it suitable for production use. ### Tech Stack: - Python - KTransformers ### Why It Matters: The project showcases how to leverage KTransformers to achieve better performance with the DeepSeek-R1 model, making it an excellent choice for developers looking to enhance their model inference capabilities on modern hardware. ### Repository: - **Repository**: `ubergarm/r1-ktransformers-guide` - **GitHub Stars**: 258 - **Primary Language**: Python - **GitHub Description**: Run DeepSeek-R1 GGUFs on KTransformers ### Quick Start: 1. **Download GGUFs**: - Choose a quantized model size based on your preference. - Use the `DeepSeek-R1-GGUF` model for optimal performance. - Ensure the entire model size fits into RAM or use `mmap()` for faster loading. 2. **Install KTransformers**: - Clone the repository and install the

### Overview: WorldLens is a comprehensive benchmark designed to evaluate driving world models across various dimensions, including visual realism, geometric consistency, functional reliability, and perceptual alignment. This unified benchmark aims to provide a balanced evaluation of models' performance in real-world driving scenarios. ### Key features: - **Visual Realism**: Evaluates models' ability to capture realistic visual details. - **Geometric Consistency**: Ensures models maintain consistent geometric properties. - **Functional Reliability**: Assesses models' performance in terms of functional correctness. - **Downstream Task**: Measures models' effectiveness in real-world applications. - **Human Preference**: Evaluates models' performance based on human preferences. ### Tech stack: Python ### Why it matters: WorldLens provides a comprehensive evaluation framework that helps researchers and practitioners understand the strengths and weaknesses of different driving world models. By covering a wide range of dimensions, it encourages balanced progress towards physically and behaviorally realistic world modeling, ensuring models can effectively simulate real-world driving scenarios.

### Overview: 这个站点把五百多个大模型搬进了像素小镇，每个模型都是住在厂商小屋里的角色。你一进来就能看到最聪明、最会写代码、最划算、最便宜、记性最好的人，以及国内和国外厂商的最新动态。 ### Key features: - **实时排行榜**：查看当前最聪明、最会写代码、最划算、最便宜、记性最好的人。 - **厂商广场**：国外和国内厂商的最新动态，每家厂商住一间能看见内部的小屋。 - **时间线**：几百个模型按发布日期排成一条长河，展示整个AI编年史。 - **排行榜**：综合智力、性价比、上下文、价格等多维度，展示参赛模型数和数据来源。 - **搜索功能**：输入模型名、厂商名或能力，快速找到所需信息。 ### Tech stack: 前端使用 Next.js 静态导出，后端使用 TypeScript。 ### Why it matters: 这个站点帮助你快速了解当前AI模型的最新动态，从最聪明到最划算，再到最新发布，一目了然。它不仅提供了排行榜和厂商广场，还通过时间线展示了整个AI的发展历程。通过搜索功能，你可以轻松找到特定模型的信息，如多模态模型。站点的数据取自多个数据源，保证了信息的准确性和实时性。

### Overview: ReNovel-AI 是一个本地化 AI 小说精修工作台，内置 RAG 长时记忆、三模态协作和卡片流式编辑器等功能，支持酒馆卡导入、全书自动化精修与扩写，旨在为小说作者提供智能化的改写和校对服务。 ### Key features: - **导入TXT文件并进行章节分析与记忆保存** - **查看小说内容/分章节或全文** - **根据需求进行精修** - **针对AI生成内容的精校** - **动态的记忆保存功能** - **通过*聊天*的形式获取小说内容** - **支持酒馆（sillytavern）的预设与角色卡** ### Tech stack: Python ### Why it matters: ReNovel-AI 通过智能化的改写、校对和扩写功能，大大提高了小说创作的效率和质量，使得作者能够专注于创意和故事构思，而无需担心繁琐的修改工作。其强大的记忆和协作功能，使得团队合作更加高效，大大提升了作品的整体质量。

Overview: MagicPIG is a project that explores the possibility of using a GPU-CPU system powered by Locality-Sensitive-Hashing (LSH) for efficient Large Language Model (LLM) generation. This project aims to improve the decoding throughput and downstream accuracy in various tasks compared to the state-of-the-art baseline. Key features: - GPU-CPU hybrid system for LLM generation - Locality-Sensitive-Hashing (LSH) sampling for efficient LLM generation - Improved decoding throughput and downstream accuracy Tech stack: Python Why it matters: MagicPIG represents a significant advancement in the field of LLM generation by leveraging the strengths of both GPUs and CPUs, leading to faster and more accurate results compared to existing methods.

Overview: Silo 是一个基于 SiliconCloud API 的纯前端多模型对话和文生图应用，支持多个大模型同时进行对话和生成图片，响应迅速，无需服务器。 Key features: - 多模型同时对话 - 文生图功能 - 支持多个模型参数调整 - 自动更新模型余额 - 简单配置密钥 - 夜间和禅定模式 - 支持多种语言 - 纯前端实现 - 不依赖服务端 Tech stack: JavaScript Why it matters: Silo 提供了一个简单且高效的解决方案，适用于需要多模型对话和文生图功能的场景，特别适合需要快速响应和低成本的项目。

Overview: RLP is an innovative reinforcement learning pretraining objective designed for improving model reasoning during the pre-training phase. This approach teaches models to think "before predicting" by rewarding their actions based on the information gain they provide on the next token. Key features: - RLP combines reinforcement learning with pre-training to enhance reasoning capabilities. - It rewards actions taken during pre-training to improve overall performance. - The method is verified-free and dense, making it suitable for various pre-training tasks. Tech stack: PyTorch Why it matters: RLP provides a novel way to enhance reasoning during pre-training, potentially leading to more robust and accurate models without additional computational resources.

### Overview: This repository is a curated list of papers focusing on LLM Ensemble, a survey paper published at IJCAI 2026. The project aims to provide a comprehensive overview of various ensemble methods used in large language models, including their applications, benefits, and potential challenges. ### Key Features: - **Survey Paper**: Contains a detailed survey of LLM Ensemble techniques. - **Author Contributions**: Contributions from multiple researchers including Zhijun Chen, Xiaodong Lu, Jingzheng Li, and others. - **Arxiv Publication**: The survey paper has been published on Arxiv with the identifier 2502.18036. - **Website**: A dedicated website for the project. - **Blog**: A Chinese-language blog with information on the project. - **GitHub Stars**: The repository has garnered 256 GitHub stars, indicating interest and engagement from the developer community. - **GitHub Forks**: The project has received 100 forks, suggesting contributors are actively using and contributing to the project. ### Tech Stack: The repository is primarily written in HTML, making it accessible and easy to read. ### Why It Matters: LLM Ensemble is a critical area in the field of natural language processing, aiming to improve the performance and robustness of large language models by combining multiple models. This project serves as a valuable resource for researchers, developers, and practitioners looking to understand and implement ensemble techniques effectively. ### Repository: Repository: junchenzhi/Awesome-LLM-Ensemble Primary Language: HTML GitHub Stars

### Overview: The Upstash RAG Chat SDK simplifies the development of retrieval-augmented generation (RAG) chat applications, providing easy setup and minimal configuration. This project is designed to be a versatile tool for developers looking to prototype RAG chat applications quickly. ### Key features: - **Next.js Compatibility**: The SDK integrates seamlessly with Next.js, enabling developers to build chat applications with streaming support. - **Ingestibility**: It can ingest entire websites, PDFs, and more, making it highly versatile. - **Built-in Vector Store**: A vector store is included for a knowledge base, enhancing the chat application's capabilities. - **Optional Built-in Redis Compatibility**: For fast chat history management, the SDK supports Redis. - **Optional Built-in Rate Limiting**: To ensure smooth chat performance, rate limiting can be enabled. - **Optional DisableRag Option**: This allows the SDK to be used as a standalone LLM chat application. - **Optional Analytics**: Provides analytics via Helicone, Langsmith, and Cloudflare AI Gateway for better insights. ### Tech stack: TypeScript ### Why it matters: The Upstash RAG Chat SDK simplifies the development of RAG chat applications, making it easier and faster to prototype these applications. It is designed to be a versatile tool for developers looking to build chat applications quickly and efficiently.

### Overview: CacheRoute is an innovative LLM scheduling scheme designed to improve task performance and system efficiency by enabling flexible KV cache reuse across LLM systems, particularly for knowledge-intensive applications like browser AI and knowledge QA. ### Key Features: - **Flexible KV Cache Reuse:** CacheRoute dynamically selects between text-based injection and KVCache-based injection based on task queues, compute load, and network load, optimizing resource utilization. - **Cross-System KV Cache Reuse:** Knowledge is shared across LLM systems through dedicated KDN servers, reducing redundant prefill computation and improving system efficiency. - **Compute-Network Coordination:** CacheRoute shifts knowledge-injection cost between compute and network resources, reducing latency and improving system throughput. ### Tech Stack: Python CacheRoute is built on Python, leveraging the vLLM, LMCache, FastAPI, Redis, and Docker technologies. ### Why It Matters: CacheRoute addresses the issue of redundant prefill computation and improves system performance by enabling flexible KV cache reuse across LLM systems, particularly for knowledge-intensive applications.

Overview: VenusFactory2 is an AI agent platform designed for protein engineering, offering a comprehensive suite of tools and resources for life science professionals. It includes features such as multi-step analysis, model fine-tuning, and a user-friendly web interface, making it accessible and efficient for both beginners and advanced users. Key features: - Multi-step analysis - Model fine-tuning - Comprehensive database support Tech stack: Python Why it matters: VenusFactory2 simplifies the process of protein engineering by providing a user-friendly interface and a variety of tools, making it an essential resource for life science professionals. Repository: ai4protein/VenusFactory2 GitHub stars: 255 GitHub description: AI agent platform with skills for protein engineering, the noob-friendly AI tutorial tool for life science professionals. (ACL Demo2025, arXiv 2026) Documentation: [docs/README.md](docs/README.md), [docs/wiki/Home.md](docs/wiki/Home.md), [docs/manual/](docs/manual/) --- ## What is it? VenusFactory2 is an AI agent platform designed for protein engineering. It offers a wide range of features, including multi-step analysis, model fine-tuning, and a user-friendly web interface. The platform is accessible and efficient, making it suitable for both beginners and advanced users. Key features: - Multi-step analysis - Model fine-tuning - Comprehensive database support Tech stack: Python Why it matters: VenusFactory2 simplifies the process of protein engineering by providing a user-friendly interface and

### Overview: FMBench is a Python package designed to benchmark any Foundation Model (FM) deployed on any AWS Generative AI service, including Amazon SageMaker, Amazon Bedrock, Amazon EKS, and Amazon EC2. It allows for flexibility in using various instance types, inference containers, and parameters to optimize performance and accuracy. FMBench can also be run on any AWS platform, ensuring that internet round trip times are not included in the end-to-end response time latency. ### Key features: 1. **Highly Flexible**: Supports any combination of instance types, inference containers, and parameters. 2. **Benchmark Any Model**: Can be used for open-source, third-party, and proprietary models trained on enterprise data. 3. **Run Anywhere**: Runs on any AWS platform where Python can be executed, including Amazon EC2, Amazon SageMaker, and AWS CloudShell. ### Tech Stack: Python ### Why it Matters: FMBench helps organizations optimize the price and performance of their generative AI workloads by benchmarking models on AWS. It provides a panel of LLM evaluators to ensure accurate model evaluations, making it a valuable tool for developers and data scientists.