Projects

Overview: This project introduces CycleNet, a novel approach to text-guided image manipulation that redefines cycle consistency in diffusion models. It aims to revolutionize image manipulation tasks by leveraging text guidance. Key features: - CycleNet: A text-guided image manipulation model. - NeurIPS 2023: Official submission to the NeurIPS 2023 conference. - ManiCups Dataset: A specific dataset used for testing and evaluation. - Models: Available on Hugging Face. Tech stack: Python. Why it matters: CycleNet represents a significant advancement in the field of image manipulation, offering new possibilities for text-guided image editing and manipulation. Repository: sled-group/CycleNet GitHub stars: 96 GitHub description: Official implementation of CycleNet: Rethinking Cycle Consistent in Text-Guided Diffusion for Image Manipulation.

### Overview: Feluda is a flexible engine designed for analyzing multi-lingual and multi-modal content. It is built to process data from various sources, including images, text, and videos, and supports different languages. Feluda is particularly useful for applications that require advanced search capabilities, such as reverse image search engines and analyzing social media data. ### Key Features: - **Modular Operators**: Feluda is built around operators, which are plugins that can be mixed and matched to perform specific analyses on your data. - **Vector-based Embeddings**: Supports ResNet models and Sentence Transformers for vector-based embeddings. - **Hash-based Search**: Uses pHash for efficient search operations. - **Text Extraction from Images**: Extracts text from images and indexes it for analysis. - **Entity Extraction**: Extracts entities from both text and images and indexes them. - **Support for Multiple Languages**: Handles data in various languages. ### Tech Stack: Feluda is built using Python, with support for vector-based embeddings using ResNet models and Sentence Transformers, and hash-based search using pHash. ### Why It Matters: Feluda is crucial for applications that require advanced search capabilities and the ability to analyze data from various sources and languages. Its modular design allows for easy configuration and swapping of operators, making it highly adaptable to different use cases and trade-offs. ### Repository: The repository for Feluda is located at [tattle-made/feluda](https://github.com/tattle-made/feluda). ### Primary Language: The primary language used in Feluda is Python. ### GitHub Stars: The repository has

Overview: A vectorless, reasoning-based retrieval-augmented generation model designed for efficient and accurate text generation. Key features: - Vectorless input processing - Reasoning-based model for context-aware generation - Integration of retrieval for improved relevance Tech stack: Python Why it matters: This model enhances text generation by leveraging reasoning and context, making it particularly useful in applications requiring high accuracy and relevance.

### Overview: 本项目是一个基于LLama.cpp的分布式语言模型推理程序，允许您利用区域网络中的多台计算机协同进行大型语言模型的分布式推理。使用Electron制作跨平台桌面应用程序的用户界面。 ### Key features: - 🖥️ 现代化界面：支持深色/明亮模式切换 - 🌐 智能节点发现：自动检测+手动添加，支持定时广播 - ⚡ 高效分布式推理：本机API服务器+远程RPC节点协同计算 - 📁 智能模型管理：自动扫描GGUF格式模型，支持GPU层数调整 - 📊 实时监控：多标签日志系统，自动滚动，状态实时更新 - 🔧 专业工具：节点连接检查、本机IP显示、手动节点管理 - 🚀 跨平台：支持Windows、macOS和Linux，统一体验 ### Tech stack: JavaScript ### Why it matters: 通过分布式推理，您可以利用多台计算机的并行计算能力，加速大型语言模型的训练和推理过程，从而提高效率和性能。同时，跨平台的Electron应用程序确保了跨设备的无缝体验，使得开发和部署变得更加简单。

Overview: PPFlow is an advanced tool for target-aware peptide design, utilizing torsional flow matching to enhance the accuracy of peptide sequences. This project is part of the ICML 2024 conference and is designed to improve the design of peptides for specific targets. Key features: - Target-Aware Peptide Design: Enhances the design of peptides for specific biological targets. - Torsional Flow Matching: Utilizes flow matching to improve the accuracy of peptide sequences. - Official Code Source: The repository contains the official code for PPFlow, including installation instructions and training methods. Tech stack: Python Why it matters: PPFlow offers a significant improvement in peptide design accuracy, making it essential for researchers and developers working in the field of bioinformatics and drug discovery. Repository: EDAPINENUT/ppflow GitHub stars: 95

Overview: MMC: Advancing Multimodal Chart Understanding with LLM Instruction Tuning introduces a large-scale MultiModal Chart Instruction dataset and a Multi-Modal Chart Benchmark to evaluate chart understanding capabilities. The dataset and benchmark are used to assess the limitations of existing Large Language Models (LLMs) in interpreting charts. Key features: - MMC-Instruction dataset: A large-scale MultiModal Chart Instruction dataset with 300,000 question-answer pairs synthesized with arXiv data. - MMC-Benchmark: A comprehensive Human-annotated Benchmark with nine distinct tasks evaluating reasoning capabilities over charts, including chart QA and chart summarization. - MMC-Alignment: Chart-text alignment data for chart-text alignment training, containing 250,000 samples and associated images. Tech stack: Python Why it matters: The dataset and benchmark provide a comprehensive evaluation framework for chart understanding, highlighting the limitations of existing LLMs in interpreting charts. The results can guide the development of more accurate and effective chart understanding models.

### Overview: 同传字幕是一款支持日语、英语视频与直播的实时同传悬浮字幕工具，适用于Android、macOS和Windows平台。它通过本地离线语音识别和自备LLM接口翻译（BYOK）技术，提供无缝的字幕体验，无需依赖网络或外部服务。 ### Key features: - **不挑播放器，全局悬浮**：支持多种播放器，包括B站、YouTube、Twitch、浏览器或本地播放器。 - **边听边出，句尾定稿**：实时刷新草稿字幕，停顿或句末自动转为正式字幕；切换为「说完再出」节省电量与Token。 - **语音识别完全在本地完成**：内置ReazonSpeech与Silero VAD离线模型，音频不会离开设备。 - **自备LLM接口（BYOK）**：支持任意OpenAI兼容接口，按实际用量付费，可保存多套配置。 - **支持任意目标语言**：内置中文、英语、日语、韩语，或根据背景一键生成常用术语。 - **暂停与字幕文本**：随时暂停、继续，无需重新加载模型，每句定稿的原文和译文按时间存成纯文本。 ### Tech stack: Kotlin ### Why it matters: 同传字幕为实时字幕提供了一种便捷且高效的方式，尤其适用于需要即时翻译的场景，如视频会议、直播翻译、B站VTuber直播等

Overview: This repository is a work in progress that demonstrates how to align video diffusion models with epipolar geometry constraints using preference-based optimization. It aims to improve the 3D consistency in generated videos by quantifying the error of epipolar alignment and using it as a reward signal in a Flow-DPO framework. Key features: - Flow-DPO Training: Adaptation of DPO for flow-matching video diffusion models. - Epipolar Geometry Constraints: Classical computer vision as reward signals. - 3D Consistency Metrics: Epipolar error, motion dynamics, perspective fields, and depth estimation. - Modular Pipeline: Complete four-step workflow from generation to evaluation. - LoRA Adaptation: Efficient fine-tuning of large video models. Why Epipolar Geometry? Epipolar geometry provides fundamental mathematical constraints for 3D consistency in videos. By computing the fundamental matrix between frame pairs and measuring Sampson distance for matched keypoints, we quantify how well generated videos adhere to rigid scene structure. Lower epipolar error indicates better 3D consistency and more realistic camera motion. Tech stack: Python Why it matters: This approach leverages classical computer vision techniques to enhance modern video generation models, leading to videos with improved visual quality, enhanced 3D consistency, and significantly improved motion stability.

### Overview: Dream-in-4D is an innovative PyTorch implementation that addresses the challenge of text-to-4D synthesis, offering a unified approach for text-to-4D, image-to-4D, and personalized 4D generation tasks. ### Key features: - **Text-to-4D**: Learns a high-quality static 3D asset using 3D and 2D diffusion guidance. - **Image-to-4D**: Implements zero123 and deep-floyd-if guidance for faster convergence. - **Personalized 4D Generation**: Offers a unified approach for text-to-4D, image-to-4D, and personalized 4D generation tasks. ### Tech stack: PyTorch ### Why it matters: Dream-in-4D provides a unified solution for 4D scene generation, enabling efficient and effective text-to-4D, image-to-4D, and personalized 4D generation tasks. The method's motion-disentangled representation allows for easy adaptation to controllable generation, making it highly versatile and practical for various applications.

Overview: 本项目提供了一个详细的大型语言模型(LLM)课程的代码部分整理，包括基础概念、工具介绍、系统架构等内容，旨在帮助开发者快速入门并实践。 Key features: - LLM基础改念 - LLM相关工具 - LLM系统架构 Tech stack: Python Why it matters: 本项目为开发者提供了一个简单、免费、易实作的代码示例，帮助他们逐步了解如何基于Python构建LLM流程、部署功能等。 Repository: PipiHi0926/llm-course-zh Primary language: Jupyter Notebook GitHub stars: 95

### Overview: Tylogi/TyloQuant is a project focused on enhancing the intelligence of models by optimizing their quantization and inference processes. This project aims to achieve better quantization formats and smarter calibration, allowing larger, stronger models to run smoothly on the hardware you already own. ### Key Features: - Fine-grained mixed-precision calibration - Compact weight formats - Native CUDA/Metal runtimes - Co-design of fine-grained mixed-precision calibration, compact weight formats, and native CUDA/Metal runtimes ### Tech Stack: C++ ### Why It Matters: Tylogi/TyloQuant improves the quality and efficiency of models by optimizing their quantization and inference processes, making them run smoothly on the hardware you already own. This project is essential for developers looking to enhance the performance and efficiency of their models.

Overview: VISII is a framework that enables the inversion of visual prompts into editing instructions, allowing users to perform image editing tasks by providing visual examples. This approach leverages text-to-image diffusion models to replicate the desired editing style. Key features: - Inverting visual prompts into editing instructions - Applying learned instructions to new images for the same edit - Utilizing hybrid instruction for more complex effects - Optimized for NVIDIA RTX 3090, Python 3.7, and PyTorch 1.13.0 Tech stack: Python Why it matters: VISII simplifies the process of applying visual editing styles to new images, making it accessible to a wide range of users, including artists, designers, and anyone looking to enhance their digital creations with specific visual styles.

Overview: PyAtlas is an interactive map that visualizes the top 10,000 Python packages on PyPI, helping users discover similar tools and alternatives. Key features: - Interactive visualization using React and Three.js - Vector embeddings and dimensionality reduction using Sentence Transformers, UMAP, and HDBSCAN - Cluster labels generated by OpenAI's GPT-5-mini Tech stack: TypeScript Why it matters: PyAtlas provides a fun and useful way to explore and discover Python packages, making it easier to find similar tools and alternatives. Repository: fpgmaas/pyatlas GitHub stars: 95 GitHub description: Browse the top 10,000 packages on PyPI with the help of vector embeddings

### Overview: HaloFPX is a high-performance model server and zoo designed for AMD Strix Halo GPUs and high-end Radeon GPUs. It leverages the latest RDNA 3.5 Wave64/ROCmFP4 kernels and silicon-tuned profiles, offering a cutting-edge developer experience that surpasses standard Lemonade. ### Key Features: - **Next-Gen MoE Models**: Supports Ling-3.0-Flash (124B), Qwen 3.8 Flash Next (125B), DeepSeek V4 Flash (284B), and Ornith 1.5 (35B). - **Compute Kernels**: Uses Mesa RADV Wave64 cooperative matrices for RDNA 3.5 + Dual-Backend ROCm/Vulkan. - **Quantization Formats**: Implements custom ROCmFP4 / ROCmFP4_FAST for direct hardware block mapping, reducing size by 13.5% and improving decode speed by 13.5%. - **Silicon-Tuned Profiles**: Provides hardware-benchmarked `run_config` for optimized performance and zero flag guessing. - **Long Context Scaling**: Supports 262K context with TurboQuant KV in unified memory, avoiding OOM issues on APUs. - **CLI & Workflow Parity**: Command parity with Lemonade, including 100% command compatibility and seamless model sharing. - **Lemonade Cache Sharing**: Shares weights between Lemonade and HaloFPX models, resolving aliases seamlessly. ### Tech Stack: Python ###

### Overview: Indic-Subtitler is an open-source platform designed for transcribing and translating videos and audio files in Indic languages, including Tamil, Hindi, Malayalam, and others. It utilizes machine learning models to facilitate the process of creating subtitles in various languages. ### Key Features: 1. **Transcription and Translation**: Transcribes audio and video files in Indic languages and translates subtitles into other languages using ML models. 2. **SeamlessM4T Model**: Utilizes Meta's Seamless Communication technology, which supports over 12 Indic languages out of the box. 3. **Fine-Tuned Whisper Models**: Fine-tuned Whisper models for Malayalam and other Indic languages for better performance. 4. **Generative UI**: Novel architecture for a Generative UI that works with any ASR models. 5. **SeamlessM4T Model**: Supports almost 12 Indic languages, including Assamese, Bengali, English, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, Telugu, and Urdu. 6. **Seamless Communication Repository**: Provides detailed documentation and tutorials on using the SeamlessM4T model. ### Tech Stack: - **SeamlessM4T Model**: Supports 12 Indic languages, including Malayalam, Tamil, and Telugu. - **faster-whisper**: A reimplementation of OpenAI's Whisper model, which is fine-tuned for various Indic languages. ### Why It Matters: Indic-Sub

### Overview: The MultiCaRe dataset is an open-source clinical case dataset consisting of 98,000 de-identified clinical cases and 139,000 medical images from PubMed Central. It covers a wide range of specialties and includes a hierarchical image taxonomy with 140+ classes. ### Key features: - **98,000+** clinical cases from 72,000+ PubMed Central case reports. - **139,000+** medical images across multiple specialties. - Comprehensive image taxonomy with logical constraints. - Fully open-source under a CC0 license. ### Tech stack: Jupyter Notebook ### Why it matters: The MultiCaRe dataset provides a rich resource for medical image classification and multimodal AI applications, offering a diverse and detailed dataset for training and evaluating models.

Overview: Local-GenAI-Search is a generative search engine built on Llama 3, Langchain, and Qdrant, designed to answer questions based on local files on a 32GB laptop or computer. It uses MS MARCO embeddings for semantic search and can run locally using a 8B parameter model when NVIDIA API credits are not available. Key features: - Local generative search engine - Uses Llama 3 model for semantic search - Supports indexing of local files with extensions `.pdf`, `.txt`, `.docx`, and `.pptx` - Runs on 32GB laptops or computers - Can be accessed via HTTP endpoints for postman or direct requests - Streamlit user interface for easy integration Tech stack: Python, Langchain, Transformers, Qdrant, PyPDF2, MS MARCO IR embeddings Why it matters: This tool simplifies the process of finding answers to local file-based questions, making it accessible even on smaller devices, and provides a convenient way to interact with the generated search results.

### Overview: PersonalForge is a tool that allows users to fine-tune any language model on their own data, without the need for cloud services or coding. It streamlines the process by providing a 100% offline experience, including data cleaning, model training, and offline chat capabilities. ### Key Features: - **Data Sources**: Supports file uploads, HuggingFace datasets, web search, URLs, Google Drive, Dropbox, AWS S3, GitHub Gist, and JSON APIs. - **HuggingFace Datasets**: 26 verified datasets for various categories including coding, chat, reasoning, general, medical, legal, science, finance, and multilingual. - **Universal Model Support**: Automatically detects and supports over 58 popular models, including Qwen, Llama, DeepSeek, and others. - **Data Cleaning Pipeline**: A 17-step process to clean and prepare data for model training. ### Tech Stack: Python ### Why It Matters: PersonalForge provides a free, offline, and no-coding solution for fine-tuning language models, making it accessible to everyone. The GGUF file generated is directly tied to the user's knowledge, eliminating the need for internet access or cloud storage. This tool simplifies the process of creating a personalized AI model and allows for offline chat capabilities, making it ideal for remote or low-bandwidth environments.

Overview: This repository is a Pytorch implementation of the paper "Ladder Fine-tuning approach for SAM integrating complementary network," focusing on the SAM (Self-Attention Module) fine-tuning method. The project aims to integrate complementary networks to improve model performance. Key features: - Pytorch implementation - Ladder Fine-tuning approach for SAM - Complementary network integration Tech stack: Python Why it matters: This project is valuable for developers looking to enhance their models with complementary networks and fine-tuning techniques, potentially improving performance and accuracy. Repository: 11yxk/SAM-LST Primary language: Python GitHub stars: 94 GitHub description: Pytorch implementation of paper [Ladder Fine-tuning approach for SAM integrating complementary network](http://arxiv.org/abs/2306.12737).

Overview: TextAtlas5M is a large-scale dataset designed for evaluating models' abilities in dense text image generation, specifically focusing on the TextAtlasEval benchmark. Key features: - Contains high-resolution images with dense text annotations. - Includes detailed information such as image paths, prompts, bounding boxes, font sizes, and more. - Supports evaluation of various models using TextAtlasEval. Tech stack: Python Why it matters: This dataset is crucial for benchmarking and evaluating models' performance in generating text-rich images, providing a comprehensive resource for researchers and developers in the field of text image generation. Repository: CSU-JPG/TextAtlas GitHub stars: 94 GitHub description: [ICML 2026] A Large-scale Dataset for Dense Text Image Generation