Projects

Overview: HPSv3++ and HPDv3++ are advanced models designed for scaling reward models across the full spectrum of diffusion model capabilities. These models are specifically tailored for modern text-to-image reward modeling, providing a benchmark dataset and training pairs for researchers and developers. Key features: - HPSv3++ as a reward model checkpoint - HPDv3++ aesthetic and text-following training pairs - Ready-to-use training and test splits for independent dataset use - Two-stage training code and configurations for reproducible capability- and iteration-conditioned reward modeling Tech stack: Python Why it matters: HPSv3++ and HPDv3++ are essential tools for researchers and developers looking to scale reward models across the full spectrum of diffusion model capabilities. They provide a benchmark dataset and training pairs, making it easier to evaluate and improve image preference prediction. Repository: PlantPotatoOnMoon/HPSv3-PlusPlus GitHub stars: 38 GitHub description: [ACM MM 2026] HPSv3++: Scaling Reward Models Across the Full Spectrum of Diffusion Model Capabilities

Overview: DiffTAD is an official PyTorch implementation for "DiffTAD: Denoising Diffusion Probabilistic Models for Vehicle Trajectory Anomaly Detection". The framework aims to address the anomaly detection challenge in traffic video surveillance, autonomous driving, and taxi fraud detection by utilizing diffusion models. Key features: - Formalizes anomaly detection as a noisy-to-normal process. - Reconstructs near-normal trajectories from corrupted ones. - Integrates Transformer-based temporal and spatial encoders. - Utilizes interval sampling for efficient inference. Tech stack: Python Why it matters: DiffTAD provides a novel approach to anomaly detection in vehicle trajectories, leveraging diffusion models to formalize the problem and improve the accuracy and efficiency of anomaly detection compared to existing methods.

Overview: RAMen is a fast in-memory data store designed specifically for AI applications and agents, built on top of Redis. It offers a drop-in Redis protocol, native vector search, semantic caching, and a built-in MCP server, making it a versatile tool for AI-driven systems. Key features: - Drop-in Redis protocol compatibility - Native vector search and semantic caching - Built-in MCP server for AI agents Tech stack: Go Why it matters: RAMen is a practical alternative to Redis, tailored for AI applications, offering features like vector search, semantic caching, and a built-in MCP server, making it ideal for AI-driven systems and agents. Repository: GitHub - Rohit-Dnath/RAMen GitHub stars: 38 GitHub description: RAMen is a fast in-memory data store, like Redis, but built for AI: drop-in Redis protocol, native vector search, semantic caching, and a built-in MCP server for agents. Single Go binary, BSD-3.

Overview: This repository implements Collage Diffusion, an enhancement of Stable Diffusion that offers layer-based controls for compositional image generation. Key features: - Layer-based controls for compositional image generation - Implementation of Collage Diffusion - Python notebook for easy experimentation Tech stack: Jupyter Notebook Why it matters: This implementation provides a practical tool for generating complex image compositions with customizable layers, making it useful for artists, designers, and anyone looking to enhance their image generation capabilities.

Overview: IECDF is a multi-domain multi-modal fake news detection framework designed to address the challenges of domain imbalance, modality imbalance, and class imbalance. Key features: - Expert collaboration module for domain imbalance - Dynamic fusion module for modality imbalance - DLINEX loss for class imbalance Tech stack: Python Why it matters: IECDF provides a comprehensive solution for detecting fake news in complex and imbalanced environments, making it valuable for researchers and practitioners in the field of information retrieval and data science.

Overview: TDPO is an official PyTorch implementation of a paper that addresses the issue of reward overoptimization in diffusion models. The repository includes detailed installation instructions and training scripts for evaluating cross-reward generalization using different datasets. Key features: - Training on Aesthetic Score and PickScore for cross-reward generalization - Utilizing out-of-domain reward functions - Detailed hyperparameter configuration for 8 x NVIDIA A100 GPUs Tech stack: Python Why it matters: This project provides a practical solution for mitigating reward overoptimization in diffusion models, offering valuable insights into the biases and challenges in reinforcement learning. Repository: ZiyiZhang27/tdpo GitHub stars: 38

Overview: This project aims to apply the DALL-E concept to Korean optical character recognition (OCR) by encoding images as sequences of visual tokens and generating text-like descriptions from them. Key features: - Vector-quantization for encoding images as sequences. - Image-to-text captioning using GPT-2. - Dynamic shape encoding for OCR tasks. Tech stack: Python. Why it matters: This project explores the potential of DALL-E for OCR, demonstrating how image sequences can be used to generate text-like descriptions, which could lead to more accurate OCR results.

Overview: LatentSync是一款由字节跳动开源的唇形同步软件，旨在提供高质量的视频对口型效果，无需复杂的安装和配置。 Key features: - 一键启动整合包，快速上手体验 - 支持Windows 10/11系统 - 软件运行路径中不包含非英文字符和空格 - 支持英伟达显存不低于12G的显卡 - 可以在项目文件夹内的temp文件夹内找到合成视频 Tech stack: C#、Python、HTML、JavaScript Why it matters: 该软件简化了安装部署过程，减少了对电脑配置的要求，使得更多用户能够轻松使用，同时也提供了详细的教程和注意事项，帮助用户更好地理解和使用软件。

Overview: This project is an official implementation of the "Divide & Bind Your Attention for Improved Generative Semantic Nursing" paper presented at BMVC 2023 Oral. Key features: - Generate images using the Divide & Bind pipeline - Compatible with Diffusers 0.21.4 - Updated environment compatible with Stable Diffusion v1.5 Tech stack: Jupyter Notebook Why it matters: This project provides a practical implementation of the proposed method for generating images, which is crucial for the research and potential applications in the field of generative semantic nursing.

### Overview: Carve3D is a code repository for improving multi-view reconstruction consistency in diffusion models through reinforcement learning finetuning. This project addresses a significant challenge in the field of computer vision and image synthesis by proposing a novel approach to enhance the consistency of multi-view and sparse-view reconstruction in diffusion models. ### Key Features: - **Multi-View Reconstruction Consistency (MRC) Metric**: Implementing a metric to evaluate the consistency of multi-view and sparse-view reconstruction. - **Sparse-View Reconstruction Model**: Utilizing an open-sourced sparse-view reconstruction model such as OpenLRM, GRM, or LGM to improve the pipeline. - **SDXL-based Multiview Diffusion Model**: Replacing Instant3D's finetuned SDXL with a multi-view diffusion model, which is a key component of the Carve3D pipeline. ### Tech Stack: Python ### Why it Matters: Carve3D introduces a novel method to improve the consistency of multi-view and sparse-view reconstruction in diffusion models, which is crucial for applications in computer vision and image synthesis. By enhancing the consistency, it can lead to better quality and more realistic results in various tasks such as image synthesis, scene reconstruction, and more.

Overview: FREEDOM is a PyTorch implementation for the paper "A Tale of Two Graphs: Freezing and Denoising Graph Structures for Multimodal Recommendation" presented at ACM MM'23. It is integrated into the MMRec framework and can be run with minimal setup. Key features: - PyTorch implementation - Integrated into MMRec framework - Data integration from Google Drive Tech stack: Python Why it matters: This model addresses the challenge of freezing and denoising graph structures for multimodal recommendation, offering a valuable solution in the field of recommendation systems. Repository: enoche/FREEDOM Primary language: Python GitHub stars: 38

Overview: This course aims to provide a comprehensive understanding of machine learning principles and techniques, focusing on supervised, unsupervised, and deep learning methods. It covers topics such as regression, classification, clustering, and generative models, with practical programming assignments to enhance hands-on experience. Key features: - Supervised Learning: Regression, Classification, Linear Models, Support Vector Machines, Kernel Methods, Ensemble Methods, Sparse Modeling, Clustering, Dimensionality Reduction, Generative Models, Deep Learning. - Unsupervised Learning: Clustering, Dimensionality Reduction, Matrix Factorization, GAN, Autoencoders. - Practical Programming Assignments: Hands-on experiments with various learning algorithms. Tech stack: Jupyter Notebook Why it matters: This course is essential for students aiming to advance in the field of machine learning, providing a solid foundation in theory and practical skills necessary for research and industry applications.

Overview: This repository provides a hands-on case study for demonstrating the stages involved in a machine learning project, from Exploratory Data Analysis (EDA) to production. Key features: - EDA (Exploratory Data Analysis) - Model Development - Model Evaluation - Production Deployment Tech stack: Python programming language. Why it matters: This workflow serves as a concise guide to the minimal steps required in a machine learning project, ensuring each step is executed correctly to avoid common pitfalls and provide a comprehensive view of the project's journey. Repository: hamedonline/ml-workflow Primary language: Jupyter Notebook GitHub stars: 38 GitHub description: A case study for demonstrating the stages involved in a machine learning project, from EDA to production. README excerpt: This repo aims to study and apply the least minimal steps involved in machine learning workflow the right way. It was compiled during the first cohort of ["Machine Learning Zoomcamp"](https://datatalks.club/courses/2021-winter-ml-zoomcamp.html) course instructed by [@alexeygrigorev](https://github.com/alexeygrigorev). The dataset used is a tabular one, containing 188,318 samples with 130 features, including "claim id" and "loss" columns. The main intention is to provide a quick look at the minimal required steps in a machine learning project, avoiding common slips and showcasing a complete sub-steps approach. Pull requests are welcome.

Overview: This tool automates the process of generating text descriptions for images using Ollama vision models, including LLaVA, Qwen3-VL, and Llama Vision. It can be used as a web application or CLI, making it versatile for AI image generation training. Key features: - Fully customizable system prompts to precisely control the output format. - Built-in presets optimized for Stable Diffusion, Z-Image, Midjourney, Flux, and more. - Automatically generates detailed, consistent descriptions tailored to your target model. Tech stack: Python Why it matters: This tool streamlines the captioning process for image datasets, making it easier to train LoRA models on image generation AI like Stable Diffusion, Flux, Z-Image, or other diffusion models. Repository: hydropix/AutoDescribe-Images GitHub stars: 38

Overview: PipelineLLM 是一个系统性的大语言模型后训练学习项目，涵盖从监督微调到偏好优化、强化学习再到持续学习的完整技术栈。 Key features: - 系统性地覆盖后训练全流程 - 包括从基础到前沿的理论与实践 - 每个模块配套可运行代码 - 全中文文档，适合中文开发者 - 对比分析 SFT/DPO/RL 的适用场景 - 专门两天讲解 CL 及其在 LLM 中的应用 Tech stack: Python Why it matters: 本项目通过环环相扣的7天课程，帮助开发者系统地理解并实现大语言模型后训练的全流程，包括从基础到前沿的理论与实践，以及如何选择技术路线和持续学习的策略。 Repository: iBacklight/PipelineLLM GitHub stars: 38 GitHub description: PipelineLLM 是一个系统性的大语言模型（LLM）后训练学习项目，涵盖从监督微调（SFT）到偏好优化（DPO）、强化学习（RLHF/PPO/GRPO）再到持续学习（Continual Learning)的完整技术栈。

### Overview: This repository is a curated collection of state-of-the-art video generation models, research papers, datasets, and creative workflows. It provides a comprehensive overview of the latest advancements in AI video generation, including both commercial and open-source models. ### Key Features: - **Runway Gen-3 Alpha**: Industry-leading motion brush, director mode, and character consistency. - **Luma Dream Machine**: High-speed generation, realistic physics, and complex camera movements. - **Kling AI**: Long-form video support, native 4K, and superior human movement. - **OpenAI Sora**: High fidelity, 60-second clips with high physical consistency. - **Google Veo 3**: Integration with Google Vids and Workspace. - **Pika 1.5**: Specialized in "Pikaffects" (physics-defying creative transformations). ### Tech Stack: The repository is built using Python and relies on various open-source libraries and frameworks. ### Why It Matters: This repository serves as a valuable resource for developers and enthusiasts interested in exploring and utilizing cutting-edge AI video generation techniques. It provides a platform to experiment with state-of-the-art models and datasets, contributing to the ongoing development of this field.

### Overview: This project predicts Body Mass Index (BMI) using a single image of a human face, achieving state-of-the-art results. ### Key features: - Predicts BMI with a single image of a human face. - Achieves superior performance compared to benchmarks set by state-of-the-art methods on the VisualBMI dataset. ### Tech stack: Python ### Why it matters: This model is designed to accurately predict BMI using facial images, offering a practical solution for health monitoring and analysis. ### Repository: liujie-zheng/face-to-bmi-vit ### GitHub stars: 38 ### GitHub description: Predicts Body Mass Index (BMI) with one image of a human face, with state-of-the-art results.

Overview: This project focuses on building an end-to-end speech recognition system using QuartzNet, wav2vec2.0, and a CTC decoder. It supports Vietnamese speech recognition and is designed to be simple and easy to understand. Key features: - Vietnamese speech recognition pipeline - QuartzNet model for speech recognition - wav2vec2.0 model for feature extraction - CTC decoder for sequence-to-sequence tasks - Beam search algorithm for language modeling - Customizable dataset using VLSP2020 ASR competition data - Streamlit-based demo for speech recognition Tech stack: Jupyter Notebook Why it matters: This project provides a comprehensive solution for Vietnamese speech recognition, making it accessible and easy to use for developers and researchers. Repository: manhph2211/ViSR GitHub stars: 38 GitHub description: This repo builds an end-to-end deep learning application that supports speech recognition system. It's simple to use and understand :smile:

Overview: The "The Invisible EgoHand: 3D Hand Forecasting through EgoBody Pose Estimation" project aims to predict 3D hand positions using ego-body pose estimation, providing valuable insights into human body movements and hand gestures. Key features: - Real-time 3D hand prediction - Ego-body pose estimation for accurate predictions - Open-source code release Tech stack: Python Why it matters: This project offers a practical solution for analyzing human body movements and hand gestures, with applications in various fields such as robotics, gaming, and medical diagnostics. Repository: masashi-hatano/EgoH4 Primary language: Python GitHub stars: 38

Overview: This repository fine-tunes a state-of-the-art PII detection system and enhances performance with synthetic PII data generation. Key features: - Synthetic PII datasets with BIO formatting - PII Entity Detection Systems using Masked Language Modeling (MLM) with Hugging Face Trainer for Domain Adaptation - Fine-tuning of a Microsoft Deberta-V3-Large model using Hugging Face Token Classification Tech stack: Python Why it matters: This repository provides solutions that can improve the privacy and security of individuals by enhancing PII detection systems, helping to comply with legal and regulatory requirements, and preventing identity theft and fraud.