Projects

Overview: Lightning Generative Models is a framework designed for implementing and training various generative models using PyTorch Lightning and Weights & Biases. Key features: - Implement and train different types of generative models - Leverage the simplicity of PyTorch Lightning and the comprehensive tracking capabilities of Weights & Biases - Seamlessly log and visualize experiments for efficient and impactful machine learning development Tech stack: Python Why it matters: This framework provides a user-friendly and robust solution for working with generative models, making it easier to implement and analyze different types of models. Repository: seungjunlee96/lightning-generative-models GitHub stars: 36

### Overview: VisionTeX is a fine-tuned Qwen2-VL-7B model designed for LaTeX OCR, leveraging LoRA and Unsloth for improved performance and efficiency. The project includes a comprehensive pipeline for training, evaluation, and deployment, focusing on metrics such as Character Error Rate (CER), Token F1, BLEU-4, and Exact Match. ### Key Features: - Fine-tuned Qwen2-VL-7B model for LaTeX OCR - Uses LoRA and Unsloth for optimization - Augmentation pipeline with rotation, noise, contrast jitter, and blur - LoRA rank sweep across r=8, r=16, r=32 for F1 comparison - Data augmentation pipeline for error rate comparison - Evaluation metrics: CER, Token F1, BLEU-4, and Exact Match ### Tech Stack: - Jupyter Notebook ### Why It Matters: The project provides a robust and efficient solution for digitizing mathematical expressions from handwritten or scanned documents, with a focus on accuracy and performance metrics. The fine-tuning approach ensures that every design choice is backed by empirical results, making it a valuable tool in the field of OCR and mathematical expression recognition. ### GitHub Stars: 34 ### Repository: - shlokchorge/Fine-Tuning-QWEN2vl ### Primary Language: - Jupyter Notebook ### GitHub Description: - Fine-tuned Qwen2-VL-7B for LaTeX OCR using Unsloth and LoRA. Includes training, evaluation, and a Gradio demo. Deployed

Overview: This project employs deep learning techniques to detect floods from images, utilizing a fine-tuned MobileNet architecture for image classification. Key features: - Fine-tuned MobileNet architecture for flood detection - Utilized Keras library for deep learning - Data includes both "Flooding" and "No Flooding" images Tech stack: Jupyter Notebook Why it matters: The project showcases the effectiveness of deep learning in real-world applications, demonstrating the potential of image classification for detecting environmental hazards like floods. Repository: sohailahmedkhan/Flood-Detection-from-Images-using-Deep-Learning Primary language: Jupyter Notebook GitHub stars: 36 GitHub description: Flood detection from images using deep learning. Deep learning library KERAS was employed and MobileNet architecture was fine-tuned for image classification task.

### Overview: ROSECHO is a Chinese voice-based conversational module designed for the ROS ecosystem. It supports the ROS 2.0 Go system and is available on GitHub. ### Key features: - **ROS 2.0 Go Support**: ROSECHO is compatible with the ROS 2.0 Go system, making it easily integrable into existing ROS projects. - **Wake-Up Word**: ROSECHO has a wake-up word of "波弟波弟", allowing it to be activated by simply saying this phrase. - **Automatic Sound Recognition**: ROSECHO can recognize and respond to voice commands, providing a natural conversational interface. - **TTS (Text-to-Speech)**: ROSECHO supports text-to-speech functionality, enabling it to read out commands and responses. - **WiFi Configuration**: ROSECHO can be configured to connect to WiFi networks, enhancing its connectivity capabilities. - **Simple Demo**: ROSECHO is designed to be easily integrated into ROS environments, making it accessible to developers. ### Tech Stack: C ### Why it Matters: ROSECHO is an essential tool for developers working with ROS systems, providing a robust and user-friendly voice-based interaction module. Its compatibility with ROS 2.0 Go and automatic wake-up capabilities make it a valuable addition to any robotics project involving voice-based communication.

### Overview: PsyChat 是一款基于范例引导与Agentic RAG技术的智能AI心理咨询助手。它通过高度匹配的Few-shot与学习笔记，应用理性情绪行为疗法（REBT）帮助用户识别并调整非理性信念，缓解负面情绪。 ### Key features: - **ReAct 查询优化**：轻检索获取锚点 → 分析匹配度 → 改写或直接检索，动态提升检索准确性。 - **完整对话上下文扩展**：突破 chunk 片段限制，回溯完整咨询案例，让模型看到咨询师的完整工作节奏。 - **咨询师话术分析**：主动提炼参考案例中咨询师的共情方式、提问策略、引导节奏和语言特点，作为显式约束指导回答。 - **交互策略规划**：回答前先判断对话所处阶段，规划下一步引导方向，而非被动应答。 - **对话持续监控**：连续多轮无 RAG 时自动触发强制检索，保持专业性。 - **语音合成播放**：基于阿里云百炼 TTS，将回答转换为温暖的语音。 ### Tech stack: Python ### Why it matters: PsyChat 通过高度匹配的 Few-shot 与学习笔记，结合理情行为疗法（REBT），帮助用户识别和调整非理性信念，缓解负面情绪，提供专业的心理健康支持。

### Overview: PageTalk 是一款开源的 Chrome 浏览器扩展，它允许你在任何网页的文本框中通过语音进行输入，彻底解放你的双手。无需鼠标，只需点击悬浮按钮或按下快捷键即可开始说话，文字便会自动输入。 ### Key features: - **语音输入**: 在任何网页的文本框中通过语音进行输入。 - **便捷操作**: 界面上显示的可拖动悬浮麦克风按钮，单击即可开始/停止录音。 - **快捷键支持**: 完全可自定义的全局快捷键，无需鼠标也能高效操作。 - **双引擎驱动**: 内置一个开箱即用的免费识别服务，支持配置阿里云百炼 API Key，使用性能更强、更稳定的 `qwen3-asr-flash` 模型。 - **多语言支持**: 支持自动检测语言，也可手动选择包括中文、英语、日语在内的十几种常用语言。 - **文件转录**: 可以通过双击悬浮按钮打开转录面板，支持拖拽本地音频/视频文件进行离线转录。 - **智能格式化**: 开启“逆向文本标准化”（ITN）后，能将语音中的数字、日期、单位等自动转换为书面语格式。 - **上下文优化**: 可设置特定领域的专业词汇作为上下文，显著提升复杂场景下的识别准确率。 - **高度可定制**: 自由调整悬浮按钮及相关 UI 的大小，选择隐藏悬浮按钮，完全通过快捷

### Overview: This repository contains the training and inference code for the Rich Human Feedback (RAHF) model, designed to generate high-quality text-to-image (T2I) images. The model was developed as a close replica of the original RAHF model presented in the CVPR 2024 paper, aiming to replicate the state-of-the-art results in text-to-image generation. ### Key features: - Replicates the results from the CVPR 2024 paper. - Uses the same architecture as the original RAHF model. - Reproduces most of the testing metrics from the paper. - Utilizes the same ViT and T5 models as the original RAHF model. ### Tech stack: - Python 3.9.18 - PyTorch 2.0.0 - torchvision 0.15.1 - Transformers 4.32.0 - Datasets 2.16.1 - Scipy 1.11.4 - Tensorboard 2.18.0 - Scikit-learn 1.3.2 - TensorFlow 2.18.0 ### Why it matters: The Rich Human Feedback model is crucial for generating high-quality images that are both plausible and aesthetically pleasing, as evaluated by the metrics presented in the CVPR 2024 paper. The model's performance is comparable to the original RAHF model, making it a valuable tool for researchers and practitioners in the field of text-to-image generation.

### Overview: EdgeJev 是一个离线可用的本地类型化决策模型，支持 laya、kev、NanoJev 和 PlayJev。它通过导出 ONNX、INT8 量化和部署到官方协议兼容的本地服务，实现 CPU 上的推理，无需依赖 torch。模型在 4 核 CPU 上运行时，单题时间为 15.6 ms，显著优于其他版本，节省了网络往返和 API 费用。 ### Key features: - **离线运行**：无需联网，数据不离开本地。 - **转换 → 量化 → 部署**：从 Hugging Face 拉取权重，转换为 INT8 量化，部署为本地服务。 - **性能优越**：单题时间 15.6 ms，比官方 Jev 托管 API 快 30 倍。 - **精度高**：经过多次验证，准确率在 90% 以上。 - **适用场景**：适用于数据不能出内网的场景、air-gapped 环境和不想为每次决策付网络往返的实时链路。 ### Tech stack: - **Python**：开发语言。 - **ONNX + INT8量化**：模型转换和部署技术。 - **CPU推理**：在本地CPU上运行，无需外网。 ### Why it matters: - **性能优越**：单题时间显著缩短，比其他版本快 30 倍。 - **成本节约**

Overview: This repository is a curated list of awesome Milvus projects and resources, focusing on vector database for scalable similarity search and AI applications. Key features: - Towhee - Haystack - NVTabular - Milvus Bootcamp - pymilvus - milvus-sdk-go - milvus-sdk-java - milvus-sdk-node - qmilvus - attu - milvus_cli - milvus-operator - milvus-helm Tech stack: Python, Golang, Java, Node.js Why it matters: This repository provides a comprehensive collection of tools and resources for developers working with Milvus, a powerful open-source vector database.

Overview: The Personal AI Assistant is a cutting-edge tool designed to efficiently extract and synthesize information from web and PDF sources, transforming complex data into actionable insights. Key features: - Multi-Source Information Retrieval - Semantic Data Processing - Dynamic Response Generation - PDF Summarization Tech stack: Python Why it matters: This innovative tool streamlines data retrieval and comprehension, making it indispensable for researchers and professionals across various fields. Repository: Abhi0323/RAG-Powered-AI-Assistant-Transforming-Data-Retrieval-and-Analysis-Across-the-Web-and-PDFs Primary language: Python GitHub stars: 35 GitHub description: Harness the power of Retrieval-Augmented Generation with the Personal AI Assistant, an AI-driven tool designed to efficiently extract and synthesize information from web and PDF sources, transforming complex data into actionable insights.

### Overview UMA-ASR is an official implementation of unimodal aggregation (UMA) for automatic speech recognition (ASR). It consists of two works: one for non-autoregressive offline ASR and another for streaming ASR. The repository is based on ESPnet2 and focuses on improving feature representations for text tokens through unimodal aggregation. ### Key features 1. **UMA for Non-autoregressive Offline ASR**: Segments and integrates feature frames for better text token representation using unimodal weights derived from an encoder. 2. **Mamba for Streaming ASR**: Implements a streaming-style unimodal aggregation (UMA) method that automatically detects token activity and streamingly triggers token output, improving recognition latency. ### Tech stack The repository uses Python 3.9 and PyTorch 1.13. ### Why it matters UMA-ASR offers improved feature representations for text tokens in both non-autoregressive and streaming ASR scenarios, demonstrating significant performance enhancements over traditional approaches.

Overview: AgentHub is a production-ready multi-agent platform that enables orchestration, session governance, and audit capabilities. It provides self-hosted, multi-agent, and observable features. Key features: - Agent orchestration - Session governance - Audit capabilities Tech stack: Go/Python/Rust microservices, multi-type data storage, Next.js admin UI, automated CI/CD pipelines, and AI code review triggered via PR comment mentions. Why it matters: AgentHub simplifies the creation of AI agent teams, offering full orchestration, IAM, sandbox execution, and search capabilities, eliminating the need for complex multi-agent flows. It provides real-time observability and is self-hosted, making it ideal for personal and local-first setups.

Overview: qr-sampler is an engine-agnostic framework that integrates true randomness from external sources into LLM token sampling, replacing standard pseudorandom token sampling with entropy from quantum random number generators, hardware noise, and other sources. Key features: - Integrates entropy from quantum random number generators (QRNGs), hardware noise, and other sources. - Provides infrastructure for weak-signal integration experiments, including entropy purity and integrity verification. - Enables the creation of new profiles for inference engines, entropy sources, sampling methods, or consciousness signal amplification algorithms. - Offers thin engine adapters that integrate it with vLLM, vLLM-Metal, or any engine that supports logits processing. Tech stack: Python Why it matters: qr-sampler provides a research tool for studying the effects of tiny statistical biases in physical entropy sources on LLM token selection, enabling the study of entropy purity and integrity verification.

Overview: Uni-ViGU aims to unify video generation and understanding by extending a video generator as the foundation, addressing the imbalance in computational costs between the two tasks. Key features: - Unified flow method for continuous video and discrete text matching - Modality-driven MoE-based framework for text generation - Bidirectional training mechanism for knowledge recall and capability refinement Tech stack: Python Why it matters: This project proposes a novel approach to unify video generation and understanding, addressing the fundamental challenge of the imbalance in computational costs between the two tasks, and validating generation-centric architectures as a scalable path toward unified multimodal intelligence. Repository: Fr0zenCrane/Uni-ViGU GitHub stars: 35 GitHub description: Uni-ViGU: Towards Unified Video Generation and Understanding via A Diffusion-Based Video Generator

### 🔗 Key Features - **First latent diffusion autoencoder framework for 3D medical images** - LDAE learns semantic representations without supervision, suitable for classification, manipulation, and interpolation of brain scans. - **20x faster inference** than voxel-space DAEs while improving reconstruction fidelity. - **Semantic control** - Enables manipulation of clinically meaningful features (e.g., Alzheimer's atrophy patterns), smooth interpolation, and scan synthesis from pure noise. - **Excellent generalizability** - Representations are effective in downstream tasks such as AD classification (ROC-AUC: 89.48%) and age prediction (MAE: 4.16 years). ### 🔗 Model Architecture LDAE consists of four main components: 1. **AutoencoderKL** - Compresses 3D MRI scans (1×128×160×128) to a low-dimensional latent space (3×16×20×16). 2. **Latent Diffusion Model (LDM)** - A DDPM trained on compressed latents, modeling p(z) with a 3D UNet backbone. 3. **Semantic Encoder (`Encϕ`)** - A 2.5D network that processes slices with a 2D CNN and attention aggregation, producing a semantic vector y_sem ∈ R768. 4. **Gradient Estimator (`Gψ`)** - A conditional decoder approximating ∇z log p(y|z), guiding the reverse diffusion. The framework operates by first compressing the

### Overview: Aozora Trainer is a GUI-based fine-tuning toolkit designed for users who want more control over their model training process on single consumer GPUs. It simplifies the process by keeping important training controls visible and editable, while still utilizing modern VRAM-saving techniques and custom optimizers. ### Key Features: - **SDXL UNet Models:** Supports epsilon (EPS), v-prediction (VPRED), and rectified flow. - **Anima DiT Models:** Compatible with the Anima DiT model. - **Customizable Training:** Allows users to adjust learning rates and timesteps for fine-tuning. - **GPU Memory Efficiency:** Combines modern VRAM-saving techniques with custom optimizers to enable full-quality model training on GPUs with as little as 12 GB of VRAM. ### Tech Stack: Python ### Why It Matters: Aozora Trainer simplifies the process of fine-tuning large models on GPUs, making it practical for users with limited VRAM, such as those on consumer-grade GPUs. It ensures that users can train models without sacrificing quality and can easily manage their training process with the help of a GUI. ### GitHub Stars: 35

Overview: This project introduces a method to improve the performance of diffusion models by bridging the gap between training and inference, achieving better results in text generation tasks. Key features: - Post-train diffusion model acceleration for "accelerate the inference" speed and "achieve better performance". - Extensive experiments showing that our method can generate full sequences in just 4 denoising steps. Tech stack: Python Why it matters: This project offers a practical solution for improving diffusion model performance, making it easier to generate high-quality text sequences in various applications. Repository: LCM-Lab/Bridge_Gap_Diffusion GitHub stars: 35 GitHub description: Diffusion Model Improvement Method

### Overview: 无需预算，使用你的个人数据克隆自己——赛博飞升！Clone yourself by tuning a LLM using your own data. ### Key features: - 使用你的个人数据训练聊天模型 - 无需任何硬件设备，仅需一台电脑 - 提供详细的教程和示范代码 ### Tech stack: Python, Jupyter Notebook, GitHub ### Why it matters: 通过使用个人数据训练模型，可以创建一个高度个性化和互动性强的聊天机器人，无需高昂的硬件成本或复杂的技术知识。这使得技术对于普通人来说更加可及和易于使用。

Overview: Gomaa is an autonomous agent memory system designed for AI agents, integrating Obsidian Markdown vaults with PostgreSQL or SQLite for persistent, human-like, and transhuman long-term memory. It includes features like Obsidian integration, hybrid RRF search, knowledge graphs, security gates, MCP server, and more. Key features: - Persistent memory system - Obsidian Markdown vault integration - Hybrid RRF search - Knowledge graphs - Security gates - MCP server - Cross-agent fleet sharing - Asynchronous cloud synchronization Tech stack: Python Why it matters: Gomaa addresses the common issues of AI memory systems by providing a 4-quadrant cognitive architecture, HippoRAG 2 associative retrieval, and biological sleep consolidation, ensuring human-readable and auditable memory. It also includes features like human-readable Markdown notes and cross-agent fleet sharing, making it a robust solution for AI agents. Repository: M4F-S/gomaa Primary language: Python GitHub stars: 35 GitHub description: Gomaa — Autonomous Agent Memory OS. Persistent memory system for AI agents with Obsidian vault integration, hybrid RRF search, knowledge graphs, security gates, and MCP server.

### Overview: An improved and bug-free implementation of LLaVA-UHD, based on the official repo, with cleaner code and additional features. ### Key features: - Cleaned up file structure and code. - Fixed bugs. - Supplemented missing procedures. - Improved implementation of important functions. - Modified architecture definition of the Resampler. ### Tech stack: Python ### Why it matters: This repo aims to address the issues and improve the quality of the LLaVA-UHD implementation, following the spirit of the paper. It provides a cleaner and more reliable version of the code for the community to use and review, aiming to contribute to the advancement of the field.