Projects

### Overview: This project focuses on ablation techniques for text-to-image diffusion models, specifically targeting the removal of copyrighted materials and memorized images. The method aims to prevent the generation of specific concepts without retraining the model from scratch, ensuring a seamless transition in image generation. ### Key Features: - **Concept Ablation**: The ability to remove copyrighted materials and memorized images from pretrained models. - **Anchor Concept Matching**: The model learns to match the image distribution for a given target style, instance, or text prompt to the distribution corresponding to an anchor concept. - **Efficiency**: The method is designed to be efficient and minimally affect surrounding concepts that should be preserved. ### Tech Stack: The project uses Python as the primary programming language. ### Why It Matters: This technique is crucial for maintaining the integrity of copyrighted material and avoiding the replication of memorized images, ensuring that the generated images remain original and free from unauthorized content.

**Overview:** Bournemouth Forced Aligner (BFA) is a powerful tool designed to extract phoneme-level timestamps from speech audio recordings. It provides precise alignment of phonemes and words, making it an essential resource for linguists, speech researchers, and language enthusiasts. **Key features:** - Extracts phoneme-level timestamps from speech audio recordings. - Supports 82 languages, including English, Spanish, French, German, Italian, Dutch, Russian, Ukrainian, Polish, Turkish, Mandarin, Cantonese, Japanese, Korean, Vietnamese, Thai, Arabic, Persian, Hindi, Urdu, Greek, Swahili, and more. - Works offline, requiring only Python 3.8 or newer. - Provides a time-based representation of phonemes and words, written in Praat TextGrid or JSON format. - Aligns recordings with or without a pronunciation dictionary, using the text provided. **Tech stack:** Python 3.8 or newer. **Why it matters:** BFA simplifies the process of aligning speech recordings with text, eliminating the need for pronunciation dictionaries or extensive training. This tool is invaluable for researchers and language enthusiasts who need to analyze speech data without the constraints of manual transcription. **Repository:** The repository is hosted on GitHub, with a total of 170 GitHub stars. It includes detailed documentation, including language support, evaluation metrics, and a developer guide. **Primary language:** Python 3.8 or newer. **GitHub description:** Extract phoneme-level timestamps from speech audio recordings. **Getting started:** -

### Overview: HiveMtk 是一个开源私域 AI 营销系统，专为中文私域运营场景设计，提供多端触达、自主智能体、本地知识库和数据零出域等核心功能，助力团队高效运营。 ### Key features: - **多端触达**:支持抖音、快手、小红书、微信、WhatsApp、Telegram等主流社交媒体渠道，通过扩展桥接和协议直连，实现统一收件箱和客户视图管理。 - **ReAct 自主智能体**:采用45个原子工具自主组合，无需写死的工作流，适应各种复杂场景，包括感知、规划、调用工具和反思。 - **本地知识库 RAG**:使用pgvector混合检索、混合粗排和BGE-RERanker-V2-M3精排，提供强大的知识库支持。 - **数据零出域**:采用本地推理栈和私域部署，确保数据安全，实现零数据出域。 ### Tech stack: Go ### Why it matters: HiveMtk 是一个面向中文私域运营场景的开源AI营销系统，提供多端触达、自主智能体、本地知识库和数据零出域等核心功能，助力团队高效运营。 ### Repository: xiaofang142/hivemtk ### Primary language: Go ### GitHub stars: 155 ### GitHub description: 开源私域 AI 营销 SCRM 系统，Martech 获客引流利器！全渠道

### Overview: This comprehensive course, "No Shortcuts," provides a deep dive into the math, code, systems, and security of AI engineering. It includes 173 lessons and 166 runnable labs, ensuring that you understand the full process from start to finish. The course is designed to equip you with the skills necessary to excel in data engineering, machine learning, deep learning, and AI engineering. ### Key Features: - **Math and Code**: Every piece of code in the labs is executed by CI before landing, ensuring reliability. - **Security and Systems**: The course covers security and systems, including how to find and fix vulnerabilities. - **Real-World Applications**: The course includes real-world scenarios, such as handling large datasets, security concerns, and real-time systems. ### Tech Stack: - Python ### Why It Matters: Completing this course will prepare you to confidently take on a Data Engineer, Data Scientist, Machine Learning Engineer, Deep Learning Engineer, or AI Engineer role. It equips you with the skills to handle messy business questions, build robust data platforms, and deploy cutting-edge AI systems. The course also includes practical labs that teach you how to build and train deep networks, pretrain models, and build RAG systems, making you a well-rounded AI engineer.

Overview: VideoDPO aims to improve the alignment of preferences in video diffusion models by incorporating an OmniScore that considers both visual quality and semantic alignment between text and videos. This project is a significant contribution to the field of generative diffusion models, particularly in the area of video generation. Key features: - VideoDPO: A pipeline for preference alignment in video diffusion models. - OmniScore: A preference score that integrates visual quality and semantic alignment. - Omni-Preference Alignment: Ensures no preference aspect is neglected in the generation process. Tech stack: Python Why it matters: VideoDPO addresses the issue of preference alignment in video diffusion models, which is crucial for generating videos that better match user preferences. By improving both visual quality and semantic alignment, VideoDPO enhances the overall user experience and the effectiveness of video generation models. Repository: CIntellifusion/VideoDPO GitHub stars: 169 GitHub description: Official Implementation of VideoDPO

### Overview: AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolation is a survey that provides a detailed overview of the state-of-the-art in video frame interpolation (VFI). This survey covers over 250+ papers across various approaches, including kernel-, flow-, hybrid-, phase-, GAN-, Transformer-, Mamba-, and more recently diffusion model-based methods. The survey aims to serve as a unified reference for both newcomers and experts seeking a deep understanding of modern VFI landscapes. ### Key Features: - **Papers**: Over 250+ papers covering various VFI approaches. - **Approaches**: Kernel-, flow-, hybrid-, phase-, GAN-, Transformer-, Mamba-, and diffusion model-based methods. - **Core Principles**: Systematically organized methodologies, design assumptions, and technical characteristics. - **Datasets**: Standard datasets and loss functions. - **Evaluation Metrics**: Evaluation metrics for VFI tasks. - **Applications**: Applications including event-based, cartoon, medical image VFI, and joint VFI with other Low-Level Vision (LLV) tasks. ### Tech Stack: - **GitHub Repository**: CMLab-Korea/Awesome-Video-Frame-Interpolation - **GitHub Stars**: 169 ### Why It Matters: This survey serves as a comprehensive reference for researchers and practitioners seeking to stay updated on the latest advancements in VFI. It provides a structured understanding of the methodologies, challenges, and applications of VFI, making it invaluable for anyone looking to deepen their knowledge in this field. ### Repository: -

### Overview: The repository "Awesome Code Agents" is a curated collection of frontier research papers and technical reports on code agents. It serves as a living collection of cutting-edge research in autonomous code agents, spanning from the code we build to the real-world applications we act in. The repository is actively tracking the latest advancements in this field, providing a comprehensive overview of the current state of research. ### Key Features: - **Foundation Models**: Explore foundational research on code agents. - **Surveys & Empirical Studies**: Dive into surveys, empirical studies, and empirical research on code agents. - **Code as Artifact: Building the Digital World** - **General Software** - **Debugging & Issue Resolution** - **Code Generation & Completion** - **Environment Setup & CI/CD** - **Maintenance & Evolution** - **Code Review** - **Security** - **Comprehension & Documentation** - **Testing & Verification** - **Feature Development** - **3D & CAD** - **Web Applications** ### Tech Stack: The primary programming language used in this repository is Python. ### Why It Matters: This repository is essential for developers and researchers interested in the future of software and artificial intelligence. It provides a wealth of knowledge on the latest research in code agents, helping to shape the digital and real-world applications of autonomous code.

### Overview: This repository is a curated list of recent diffusion-based high-resolution image and video synthesis works. It includes papers and codes that showcase advancements in generating high-resolution images and videos using diffusion models. The repository aims to provide a comprehensive overview of the latest research in the field, encouraging discussions and contributions. ### Key Features: - **Papers and Codes**: Contains papers and codes related to various aspects of high-resolution synthesis, including tuning-free algorithms, fine-tuning algorithms, training from scratch, super-resolution, and metrics. - **Tuning-Free Algorithms**: Includes papers and codes for algorithms that do not require further tuning, such as FreeScale, I-Max, HiPrompt, and MegaFusion. - **Metrics**: Provides evaluation metrics for the generated images and videos. ### Tech Stack: The repository uses Markdown, GitHub, and various badges to enhance its visibility and engagement. ### Why It Matters: This repository is crucial for developers and researchers in the field of computer vision and image synthesis. It provides a platform to explore the latest advancements in high-resolution image and video synthesis, fostering innovation and collaboration in the community. ### Repository Details: - **Repository Name**: GuoLanqing/Awesome-High-Resolution-Diffusion - **GitHub Stars**: 170 - **GitHub Description**: 🔥🔥🔥A curated list of papers on recent diffusion-based high-resolution image and video synthesis works. Questions and discussions are most welcome! Upcoming works will be updated on a regular basis, feel free to contact me to add... :thumbsup: ### Primary Language:

Overview: ImageCritic is an official implementation of the ImageCritic model, designed to correct inconsistencies in generated images using reference-guided attentive alignment. This project is based on the CVPR 2026 paper "The Consistency Critic: Correcting Inconsistencies in Generated Images via Reference-Guided Attentive Alignment". Key features: - Corrects inconsistencies in generated images - Uses reference-guided attentive alignment for effective correction - Supports both local and global inpainting techniques Tech stack: Python Why it matters: ImageCritic is a valuable tool for image generation models, ensuring the consistency of generated images by leveraging reference images for guidance. This project is particularly useful in applications where image consistency is crucial, such as in image editing, restoration, and enhancement tasks. Repository: HVision-NKU/ImageCritic GitHub stars: 170 GitHub description: Official implementation of ImageCritic (CVPR 2026) README excerpt: This repository contains the official implementation of the ImageCritic model, designed to correct inconsistencies in generated images using reference-guided attentive alignment. It includes dependencies, installation instructions, quick inference methods, and more. The project is based on the CVPR 2026 paper and is available on GitHub for download and usage.

### Overview: LongCodeZip is a novel two-stage long code compression method designed specifically for code language models. This method introduces a plug-and-play framework that can be seamlessly integrated with existing code LLMs to achieve significant compression ratios while maintaining or improving task performance. ### Key Features: 1. **Coarse-grained Compression**: Function-based chunking and ranking using conditional perplexity with respect to the query to select the most relevant functions. 2. **Fine-grained Compression**: Entropy-based block detection combined with 0/1 knapsack optimization to maximize relevance within adaptive token budgets. ### Tech Stack: - Python 3.9.7 ### Why It Matters: LongCodeZip addresses the challenge of compressing long code contexts efficiently without compromising the quality of the generated code, making it particularly useful for code language models aiming for better performance and resource efficiency.

### Overview: ETO (Exploration-based Trajectory Optimization) is an agent learning framework inspired by the "trial and error" process of human learning. It allows an LLM agent to iteratively collect failure trajectories and update its policy by learning from contrastive failure-success trajectory pairs. ETO significantly outperforms strong baselines on three datasets and demonstrates superior performance in generalization, task-solving efficiency, and handling extreme scenarios. ### Key Features: - 🕹️ **Learning by Trial and Error**: ETO allows agents to learn from their exploration failures, contrary to previous approaches that exclusively train on successful expert trajectories. - 🎭 **Contrastive Trajectory Optimization**: ETO applies [DPO](http://arxiv.org/abs/2305.18290) loss to perform policy learning from failure-success trajectory pairs. - 🌏 **Iterative Policy Learning**: ETO can be expanded to multiple rounds for further policy enhancement. ### Tech Stack: Python ### Why It Matters: ETO is a significant advancement in the field of agent learning, particularly in the realm of language models. By allowing agents to learn from their failures, ETO provides a more realistic and effective learning process, leading to superior performance across various datasets and scenarios.

Overview: ClearML Serving is a command-line utility designed for easy model deployment and orchestration, supporting various models including Machine Learning and Deep Learning. Key features: - Supports Machine Learning Models (Scikit Learn, XGBoost, LightGBM) - Supports Deep Learning Models (Tensorflow, PyTorch, ONNX) - Allows customization of REST APIs for serving - Enables on-line and on-line endpoint model/version deployment - Supports per model standalone preprocessing and postprocessing - Scalable with multi-model per container, multi-model per serving service, multi-service support, and multi-cluster support - Efficient with multi-container resource utilization, support for CPU & GPU nodes, and auto-batching for DL models - Automatic deployment with automatic model upgrades and programmable API for deployment - Supports Canary A/B deployment - Provides model monitoring with usage metric reporting, model performance metric, and model performance dashboard Tech stack: Python Why it matters: ClearML Serving simplifies the process of deploying and managing machine learning models, making it easier to deploy models in production environments, and provides comprehensive monitoring and performance insights.

### Overview: InstructABSA is an official implementation of the paper "Instruction Learning for Aspect-Based Sentiment Analysis" presented at NAACL 2024. It focuses on the instruction tuning of language models to improve the efficacy of aspect-based sentiment analysis. ### Key features: - **Instruction Tuning**: The model is fine-tuned using instruction tuning to enhance its performance. - **Subtask Expansion**: The task list has been expanded to include aspect-based opinion extraction (ABPE), aspect opinion pair extraction (AOPE), and aspect opinion sentiment triplet extraction (AOSTE). - **Revised Paper**: The paper has been updated to include more systematic ablation studies. ### Tech stack: Jupyter Notebook ### Why it matters: This repository showcases the effectiveness of instruction tuned language models in surpassing previous state-of-the-art (SOTA) on downstream Aspect-Based Sentiment Analysis (ABSA) subtasks. The approach is particularly effective in improving the performance of aspect-based sentiment classification and opinion extraction tasks.

### Overview `fstalign` is an efficient tool for calculating Word Error Rate (WER) and aligning NLP-formatted references with CTM hypotheses using OpenFST and lazy algorithms. It significantly improves performance by optimizing composition graph traversal and handling punctuation and case sensitivity. ### Key Features - **WER Calculation**: Computes the word error rate between two sequences. - **Alignment**: Aligns references with CTM hypotheses, providing a flexible way to measure errors and analyze them. - **Efficiency**: Utilizes OpenFST for fast composition graph traversal, especially beneficial for long sequences. ### Tech Stack - **Language**: C++ - **GitHub Stars**: 170 ### Why It Matters `fstalign` is crucial for developers working on NLP applications, particularly in speech recognition and text-to-speech systems, where precise alignment and error measurement are essential. Its optimized performance and flexibility make it an indispensable tool for quality assurance and continuous improvement in NLP tasks. ### Installation - **Dependencies**: Ensure you have OpenFST installed. - **Build**: Follow the build instructions provided in the README. - **Docker**: Use Docker to easily run `fstalign` locally. ### Documentation - **Documentation**: Available in the `docs` directory, including NLP format documentation. ### What's New in 2.0 - **Improved Composition Traversal**: Dramatically speeds up alignment for long sequences, especially with adapted composition. - **Punctuation and Case Sensitivity**: Enhanced handling of punctuation and case insensitivity, reducing errors and confusion. ### Example Comparison

Overview: ViDiT-Q is an efficient and accurate quantization method for diffusion transformers, specifically designed for image and video generation tasks. It achieves W8A8 quantization without noticeable degradation in quality. Key features: - ViDiT-Q is specialized for diffusion transformers. - It supports efficient and accurate quantization without metric degradation. - It supports W4A8 quantization without notable visual quality degradation. Tech stack: The code is written in Python, utilizing CUDA kernels for acceleration. Why it matters: ViDiT-Q provides a practical solution for resource-efficient quantization of large-scale models used in image and video generation, offering significant improvements in both performance and accuracy.

### Overview: Promptbook is a tool that turns your company's scattered knowledge into actionable AI agents. It leverages the Agents Server to create persistent, context-aware AI assistants that can perform tasks and drive business outcomes. ### Key Features: - **Persistent AI Agents**: Create AI assistants that remember context and can collaborate with other agents. - **Agents Server**: The central hub for all AI agents, where they live, remember context, and can be integrated into teams. - **Real-World Goals**: Agents can work on real business goals, enhancing productivity and decision-making. ### Tech Stack: TypeScript ### Why it Matters: Promptbook is essential for businesses looking to automate processes and improve efficiency without the need for complex coding. By turning knowledge into action, it helps teams achieve their goals faster and more effectively.

### Overview: This repository contains a collection of resources and tutorials for Diffusion Large Language Models (LLMs). It includes a regularly updated paper list, a tutorial, and a nano code snippet. The primary language used is Jupyter Notebook, and the repository has been starred by 170 users, indicating its popularity. ### Key Features: - **Paper List**: A comprehensive list of papers related to Diffusion LLMs. - **Tutorial**: Detailed instructions and explanations for understanding and implementing Diffusion LLMs. - **Nano Code Snippet**: A simple implementation of a Continuous Diffusion Language Model and a Mask-based Discrete Diffusion Language Model, presented as Jupyter notebooks. ### Tech Stack: - Jupyter Notebook ### Why It Matters: This repository serves as a valuable resource for developers and researchers interested in understanding and implementing Diffusion LLMs. It provides a practical way to grasp the fundamental principles of these models and can serve as a starting point for further exploration and experimentation.

### Overview: DiTo is an official PyTorch implementation of "Diffusion Autoencoders are Scalable Image Tokenizers," designed for efficient and scalable image tokenization. ### Key features: - DiTo-XL: A scalable image tokenizer using diffusion autoencoders. - DiTo-L: A lightweight version for faster inference. - DiTo-B: A basic model for demonstration purposes. ### Tech stack: Python ### Why it matters: DiTo is a powerful tool for image tokenization, making it easier and faster to process and analyze images for various applications such as image captioning, image retrieval, and more.

### Overview: BLINK is a benchmark designed to evaluate the visual perception abilities of multimodal language models. It consists of 3,807 multiple-choice questions paired with images and visual prompts, challenging current models to perform tasks that humans can complete quickly and accurately. ### Key Features: - 3,807 multiple-choice questions - Images and visual prompts for each question - Tasks include relative depth estimation, visual correspondence, forensics detection, and multi-view reasoning - Human accuracy of 95.70% ### Tech Stack: Python ### Why It Matters: BLINK benchmarks the capabilities of multimodal language models against classic computer vision tasks, highlighting the limitations of current models in visual perception. It encourages further research into improving these models to match human-level visual understanding.

Overview: LLM-API is a project that simplifies the use of Large Language Models (LLMs) by providing a unified API for running various LLMs on different hardware configurations. It includes a client reminiscent of OpenAI's approach and supports LangChain integration. Key features: - Runs different LLMs on Docker containers or local machines - Streamlines the process with a YAML configuration file - Supports LangChain integration for additional capabilities - Automatically downloads and executes models based on configurations - Supports multiple LLMs, including Llama, Llama 2, Mistral, and others Tech stack: Python Why it matters: This project makes LLMs more accessible and easier to integrate into various applications, making them more useful for developers, researchers, and enthusiasts alike. Repository: 1b5d/llm-api Primary language: Python GitHub stars: 169 GitHub description: Run any Large Language Model behind a unified API