Projects

### Overview: This repository is a curated collection of papers and resources focused on On-Policy Distillation for Large Language Models. It serves as a companion to the survey paper "A Survey of On-Policy Distillation for Large Language Models" by [Arxiv](https://arxiv.org/abs/2604.00626). ### Key features: - Papers on On-Policy Distillation for Large Language Models - Papers on various aspects of the student's learning process - Papers on different strategies for supervision and guidance - Papers on multiple agents learning together ### Tech stack: Python ### Why it matters: This collection provides a comprehensive overview of the current research in the field of On-Policy Distillation, offering insights into the latest techniques and strategies for improving the performance of large language models. ### GitHub stars: 587 ### Primary language: Python ### Repository: nick7nlp/Awesome-LLM-On-Policy-Distillation ### GitHub description: A curated collection of papers and resources on On-Policy Distillation for Large Language Models. ### Explore OPDHub: <a href="https://nick7nlp.github.io/OPDHub/"><strong>Explore OPDHub ↗</strong></a> ### Browse papers: <a id="browse-the-literature"></a> <a id="papers-4"></a> ### Objectives and update rules: - **[Residual Advantage: Student-Relative Teacher Guidance for RL with Verifiable Rewards](https://arxiv.org/abs

### Overview: This repository is a collection of awesome resources for parameter-efficient fine-tuning, focusing on transfer learning. It aims to provide a comprehensive set of tools and techniques for optimizing model parameters to achieve better performance with minimal computational resources. ### Key Features: - A curated list of papers and resources on parameter-efficient fine-tuning. - Tools and techniques for fine-tuning models without significantly increasing the model size. - Resources for improving the efficiency of model training and inference. ### Tech Stack: - Python - Jupyter Notebooks ### Why it Matters: This repository serves as a valuable resource for researchers and practitioners looking to optimize their model training processes, reducing computational costs while maintaining or improving model performance.

### Overview: NIMA is a PyTorch implementation of the paper "Neural Image Assessment" by Hossein Talebi and Peyman Milanfar. This project aims to evaluate image quality using neural networks, focusing on aesthetic assessment. ### Key Features: - **Model Training**: The model was trained on the AVA dataset, a large collection of 255,500+ images. - **Dataset Split**: The dataset is divided into 229,981 for training, 12,691 for validation, and 12,818 for testing. - **Base Network**: An ImageNet pretrained VGG-16 is used as the base network. - **Learning Rate**: The learning rate settings differ from the original paper, but the paper suggests that better results can be achieved with hyperparameter tuning. - **Training Details**: The project includes detailed implementation details, including the dataset split and the use of ImageNet pretrained models. ### Tech Stack: - **Language**: Python - **Dependencies**: PyTorch 1.8.1, CUDA 11.1 ### Why It Matters: NIMA provides a valuable tool for evaluating image quality, particularly in the context of aesthetic assessment. The project's focus on neural networks and the use of a large dataset make it a significant contribution to the field of image quality assessment.

Overview: Trace Anything is a 4D video representation method that maps each pixel across frames to a continuous, parametric 3D trajectory. The Trace Anything model efficiently estimates trajectory fields for any video, image pair, or unstructured image set with a single forward pass. Key features: - Efficiently estimates trajectory fields for any video, image pair, or unstructured image set with a single forward pass. - Maps each pixel across frames to a continuous, parametric 3D trajectory. - Provides an interactive 3D viewer for exploring trajectory fields. Tech stack: Python, PyTorch, Hugging Face Model Why it matters: Trace Anything offers a compact and efficient way to represent videos and images, making it useful for applications that require real-time processing or require minimal computational resources. The ability to estimate trajectory fields for any input type makes it versatile and applicable to a wide range of scenarios, such as video summarization, image-based video retrieval, or even as a preprocessing step for other computer vision tasks.

### Overview: Groma is an MLLM (Multimodal Large Language Model) designed for exceptional region understanding and visual grounding. It excels in taking user-defined region inputs and generating long-form responses that are grounded to visual context. ### Key features: - **Grounded Multimodal Large Language Model (Groma)** - **Region Understanding and Visual Grounding Capabilities** - **Exceptional Performance on RefCOCO Benchmark** ### Tech stack: - Python ### Why it matters: Groma stands out by providing a novel paradigm of grounded MLLMs, combining the strengths of localizing models like Kosmos-2, Shikra, and external modules like Lisa, with a dedicated visual tokenization approach called Groma, to achieve state-of-the-art performance in referring expression comprehension benchmarks. ### Repository: - **Repository:** FoundationVision/Groma - **Primary language:** Python - **GitHub stars:** 587 - **GitHub description:** [ECCV2024] Grounded Multimodal Large Language Model with Localized Visual Tokenization ### Contents: - **Install** - **Model** - **Data** - **Training** - **Inference** - **Evaluation** ### Performance: - **State-of-the-art performance on RefCOCO benchmarks among multimodal large language models.** - **Table showing performance metrics on RefCOCO, RefCOCO+, and Average metrics.** - **Comparative performance with other models:** - **Shikra:** 87.01,

### Overview: GraphBit is the world's first enterprise-grade Agentic AI framework built on a Rust core with a Python wrapper for unmatched speed, security, and scalability. It enables reliable multi-agent workflows with minimal CPU and memory usage, making it production-ready for real-world enterprise environments. ### Key Features: - **Rust Core with Python Wrapper**: GraphBit is built on Rust, providing unmatched performance and security, while its Python wrapper ensures ease of use and integration with existing Python-based systems. - **Enterprise-Grade Reliability**: GraphBit is designed to handle the most demanding enterprise environments, ensuring reliability and uptime. - **Minimal CPU and Memory Usage**: The framework is optimized to use minimal resources, making it highly efficient and scalable. ### Tech Stack: Rust ### Why It Matters: GraphBit is the future of AI development, offering unmatched performance, security, and scalability. Its Rust core ensures top-notch performance and security, while its Python wrapper makes it easy to integrate with existing systems. The framework is designed to handle the most demanding enterprise environments, making it the ideal solution for real-world applications.

### Overview: The complete syllabus of Computer Science and Engineering (UG) is structured as a practical checklist, covering four main parts: Digital Electronics, Computer Organization and Architecture, Theory of Computation and Automata, and Compiler Design. ### Key Features: - **Digital Electronics**: Includes topics like Boolean Algebra, Number Systems, Arithmetic Operations, Logic Gates, and Memory Hierarchy. - **Computer Organization and Architecture**: Covers Address Bus, Memory Hierarchy, PLDs, and Input / Output Devices. - **Theory of Computation and Automata**: Includes Automata and Languages, Regular Expressions, Context-Free Grammars, Pushdown Automata, Turing Machines, and Decidability. - **Compiler Design**: Topics include Lexical Analysis, Syntax Analysis, Type Checking, Intermediate Code Generation, Optimization, Code Generation, Assembly & Linking, and Memory Management. ### Tech Stack: - **GitHub Repository**: apeman/awesome_computer_science - **GitHub Stars**: 586 ### Why It Matters: This syllabus serves as a comprehensive guide for students in Computer Science and Engineering, providing a structured approach to learning and understanding the fundamental concepts and practical applications of computer science.

Overview: AppBuilder-SDK 是百度智能云千帆AppBuilder SDK，提供了一站式开发平台，帮助开发者灵活、快速地搭建AI原生应用。 Key features: - 调用大模型 - 调用能力组件 - 调用AI原生应用 - 配置知识库 - 编排工作流 - 监控 - 部署 Tech stack: Python Why it matters: AppBuilder-SDK 提供了开发AI原生应用所需的全部功能，帮助开发者快速构建和部署应用，提高了开发效率和应用质量。 Repository: baidubce/app-builder Primary language: Python GitHub stars: 587

Overview: SemanticDraw is an interactive text-to-image framework designed for real-time content creation using image diffusion models. It allows users to draw with semantic brushes, enabling semantic control over the generated images. Key features: - Interactive drawing with semantic brushes - Prompt separation for better control - Real-time editing capabilities Tech stack: Jupyter Notebook Why it matters: SemanticDraw simplifies the process of creating interactive and contextually meaningful images, making it valuable for applications requiring real-time content creation and semantic control. Repository: ironjr/semantic-draw GitHub stars: 588 GitHub description: Official code for the CVPR 2025 paper "SemanticDraw: Towards Real-Time Interactive Content Creation from Image Diffusion Models."

### Overview: Blended Diffusion for Text-driven Editing of Natural Images Blended Diffusion is an official implementation of the paper "Blended Diffusion for Text-driven Editing of Natural Images" presented at CVPR 2022. This project aims to provide a solution for performing local edits in generic natural images using a natural language description and an ROI mask. The method leverages a pretrained language-image model (CLIP) to steer the edit towards a user-provided text prompt and a denoising diffusion probabilistic model (DDPM) to generate natural-looking results. The key feature is the spatial blending of noised versions of the input image with the local text-guided diffusion latent at a progression of noise levels, ensuring seamless fusion of the edited region with the unchanged parts of the image. ### Key features: - **Text-driven Editing**: The ability to perform local edits in natural images using a text prompt. - **Image Generation**: Generates realistic natural images based on a text description. - **Spatial Blending**: Blends noised versions of the input image with the local text-guided diffusion latent to ensure seamless editing. - **Adversarial Mitigation**: Adds augmentations to the diffusion process to mitigate adversarial results. ### Tech stack: Jupyter Notebook ### Why it matters: Blended Diffusion offers a practical solution for performing text-driven image editing, providing a natural and intuitive interface for users. By combining a pretrained language-image model with a denoising diffusion probabilistic model, it achieves high realism and the ability to preserve the background and match the text,

### Overview: Adrian is an open-source runtime security monitoring and control engine designed to detect and prevent malicious, misaligned, or out-of-restrictions behavior in AI agents. It analyzes both agent activity logs and reasoning traces to ensure AI agents operate within defined parameters and catch potential threats in real-time. ### Key Features: - **Real-Time Monitoring:** Adrian watches what the agent does and intervenes before it acts. - **Enhanced Detection:** It detects up to 35% more nuanced attacks compared to behavior-only monitoring. - **Open Source:** Adrian is available for free and open-source, with no licensing fees. - **SDKs:** Available for Python, TypeScript, and a Claude Code plugin for seamless integration into AI toolchains. ### Tech Stack: - Python ### Why It Matters: Adrian is crucial for ensuring the security of AI agents by catching malicious tool use, prompt injection, and policy drift in real-time. Its ability to monitor and control AI agents makes it an essential tool for developers and security professionals alike. ### GitHub Stars: 587 ### Primary Language: Python ### GitHub Description: Open-source runtime AI agent security tool - monitors and controls AI agents, catching malicious tool use, prompt injection, and policy drift in real time, before the agent acts. ### Why Adrian is Different: Most agent monitoring stops at activity logs. Adrian enhances this by also analyzing the agent's reasoning, ensuring AI agents operate within defined parameters and catch potential threats in real-time.

Overview: This repository is a curated list of awesome GitHub instructions, prompts, skills, MCPs, and custom agent markdown files designed to enhance your GitHub Copilot AI experience. Key features: - Instructions - Boilerplates & Templates - Language & Stack - Framework / Library - Tools - Workflows - Prompts - AI Development Tasks - Custom Agents - AI Development Mode - Agent Skills - MCPs - How to Use Tech stack: TypeScript Why it matters: These custom instructions, prompts, skills, MCPs, and agents help guide GitHub Copilot by providing specific context about your repository, such as coding standards, frameworks, and project-specific rules. Repository: Code-and-Sorts/awesome-copilot-agents GitHub stars: 586

Overview: FastT5 is a Python library that optimizes T5 models by reducing their size by 3x and increasing inference speed by up to 5x, making it 30% faster and 20% smaller than the original T5 models. Key features: - Optimizes T5 models for faster inference. - Reduces model size by 3x. - Increases inference speed by up to 5x. - Converts pretrained T5 models to ONNX format. - Quantizes the models for faster inference and reduced size. - Supports custom output paths for storing models. Tech stack: Python Why it matters: FastT5 is essential for developers looking to optimize their T5 models for faster inference and reduced model size, making them more efficient and faster to deploy in production environments. Repository: Ki6an/fastT5 Primary language: Python GitHub stars: 586 GitHub description: ⚡ boost inference speed of T5 models by 5x & reduce the model size by 3x. README excerpt:  <h1 style="text-align:center; font-weight:bold; font-size:1.875rem">Reduce T5 model size by 3X and increase the inference speed up to 5X. </h1> <p align="center"> <a href="https://github.com/Ki6an/fastT5/blob/master/LICENSE"> <img alt="GitHub" src="

### Overview: The EDGE project is an official PyTorch implementation of a state-of-the-art method for editable dance generation, introduced at CVPR 2023. EDGE uses a transformer-based diffusion model paired with Jukebox, a powerful music feature extractor, to generate realistic, physically-plausible dances from input music. The method offers joint-wise conditioning and in-betweening capabilities, and includes a new metric for physical plausibility, making it a significant improvement over previous state-of-the-art methods. ### Key features: - Joint-wise conditioning - In-betweening capabilities - New metric for physical plausibility ### Tech stack: Python ### Why it matters: EDGE provides a powerful tool for generating editable dances from music, making it easier and faster to create new dance routines. The method's ability to generate realistic, physically-plausible dances while preserving the input music's essence makes it a valuable addition to the dance generation field.

### Overview: BiFormer is a state-of-the-art vision transformer model introduced in the CVPR 2023 paper "BiFormer: Vision Transformer with Bi-Level Routing Attention". This implementation is part of the official PyTorch code repository and has been rigorously tested and validated. ### Key Features: - **Bi-Level Routing Attention**: BiFormer employs a novel attention mechanism that combines both spatial and positional routing for improved performance. - **Vision Transformer Architecture**: The model is based on a Vision Transformer (ViT) framework, making it highly effective for image classification tasks. - **Efficient Implementation**: The code is optimized for both memory and computation efficiency, making it suitable for large-scale applications. ### Tech Stack: The primary language used for this implementation is Python. ### Why It Matters: BiFormer stands out due to its innovative attention mechanism and efficient implementation, making it a valuable tool for researchers and practitioners in computer vision and image classification tasks. Its performance is significantly better than previous models, as evidenced by the impressive results reported in the paper and the excellent performance metrics in the evaluation section. ### Repository: rayleizhu/BiFormer - **Primary Language**: Python - **GitHub Stars**: 586 - **GitHub Description**: Official PyTorch implementation of **BiFormer**, from the following paper: [BiFormer: Vision Transformer with Bi-Level Routing Attention](https://arxiv.org/abs/2303.08810). CVPR 2023. ### Installation: For installation instructions, refer to the `INSTALL.md

### Overview: 全息闪卡通用 Agent Skill，不挑宿主、多模态模型都能用。一句话生成可拖转、会流光、带层次景深的3D闪卡网页，附可编辑Blender工程。 ### Key features: - **一句话出卡**：描述或参考图 → 分层图 → 配置 → 流水线 → 网页 - **不挑宿主、不挑模型**：只要是多模态模型，配任何读 `SKILL.md` 的宿主都能跑通 - **真3D层次景深**：主体前凸、背景后缩，层与层随视角错开 - **视点流光**：镭射彩虹的相位跟着视角走，转到哪闪到哪 - **浏览器里随便玩**：拖拽旋转、翻面、滑块、手机布局 - **出厂自带验收**：验证报告和截图，比对真实画面帧 - **免装 Blender**：官方便携版自动下载、SHA-256校验后装进项目目录 - **网页零依赖请求**：查看器打包成单文件，广告拦截插件无从下手 - **可编辑交付**：`card.blend` 真工程 + 透明分层PNG + `card-config.json`，想改哪层改哪层 - **纯文本 skill**：本体只有代码和文字，一键打包ZIP交付，无二进制、无

### Overview: 信使 Web Builder 是一款基于 TailwindCSS 的低代码平台，专为 AI 驱动的 UI 生成和应用开发而设计。它支持快速构建现代化响应式 UI、动态自定义组件、多主题和多语言网站应用。通过 AI 助推，用户可以轻松创建和发布网站，无需编写代码。 ### Key features: - **AI 助推的 UI 生成**：利用 AI 技术快速生成现代化响应式 UI。 - **多主题、多语言支持**：提供多种主题和语言选项，满足不同用户需求。 - **动态组件编辑**：支持 JSON 编辑和组件数据的动态调整。 - **媒体库管理**：批量上传、查看和更新媒体库。 - **页面历史版本**：覆盖操作时自动记录历史版本。 - **页面和组件管理**：通过 Builder CMS 管理小程序页面和组件。 - **快速页面生成**：根据规则从组件库生成页面。 - **切换全宽布局**：方便大屏编辑，减少干扰。 - **动画效果**：支持通用 AOS 页面滚动动画和高细节 GSAP 动画。 - **多主题预览**：预览组件在多主题下的显示情况。 - **暗黑风格**：支持切换浅色和暗黑风格，专注于内容创作。 - **页面预览**：在新窗口查看真实的页面。 - **响应式预览**：查看页面在不同设备尺寸下的响应式排版

### Overview: 开源SFT数据集整理项目旨在提供大量高质量的SFT（Simultaneous Transfer Learning）数据集，涵盖多种语言、任务和数据类型，帮助开发者和研究人员进行自然语言处理（NLP）模型的训练和评估。 ### Key features: - **多语言支持**: 包括中文、英文等多种语言。 - **广泛任务覆盖**: 支持通用指令、数学推理、对话、翻译、价值观指令、考试、论文元数据、中文摘要、图文多模态等任务。 - **多样数据来源**: 来自文本、人工标注、自动工具等多渠道，确保数据的丰富性和准确性。 - **社区贡献**: 项目持续补充新数据集，方便用户随时获取最新资源。 ### Tech stack: 使用Python、Hugging Face库进行数据整理和管理。 ### Why it matters: 该项目为NLP研究和应用提供了丰富的数据资源，有助于加速模型训练和评估，推动技术发展和应用创新。

Overview: RT-2 is an all-new robotics model that leverages the power of vision, language, and action to transform industries with its intelligence and efficiency. Key features: - RT-2 is a multi-modality model that integrates vision, language, and action representations. - It uses the PALM-E backbone with a unified multi-modal representation. - RT-2 is easy to architect but lacks deep understanding of individual modality representations. - RT-2 is designed to democratize robotics by making it accessible to developers. Tech stack: Python, PyTorch, PALM-E, Vision encoder, Language backbone. Why it matters: RT-2 aims to revolutionize robotics by providing a unified multi-modal representation that can be easily integrated into existing robotics systems, making it accessible and efficient for developers.

Overview: Augraphy is a Python library designed to create multiple copies of original documents through an augmentation pipeline that simulates various paper printing, faxing, scanning, and copy machine processes. This pipeline generates realistic old or noisy documents for training neural networks to remove distortions. Key features: - Randomly distorts each copy of the original document - Simulates realistic paper printing, faxing, scanning, and copy machine processes - High-configurable pipelines for creating realistic old or noisy documents - Training data for AI/ML processes to learn how to remove distortions Tech stack: Python Why it matters: Augraphy addresses the lack of readily available datasets with both a clean and noisy version of target documents, allowing for the simulation of training data with dirty documents for neural networks. This approach enables the training of models to remove distortions and restore documents to their original form, making it a powerful tool for enhancing neural network capabilities in document restoration and enhancement tasks. Repository: sparkfish/augraphy Primary language: Python GitHub stars: 584 GitHub description: Augmentation pipeline for rendering synthetic paper printing, faxing, scanning and copy machine processes