Projects

### Overview: Waifu On The Web is a web-based Artificial Intelligence with Natural Language Processing and a Live2D model (PoC). Created by Chiai, it aims to showcase the capabilities of AI through a live chat interface. ### Key Features: - **Chat in Multiple Languages**: Supports English, Bahasa Indonesia, and Japanese. - **Live2D Model**: Provides a visually engaging interface with Live2D animations. - **Speech Recognition**: Compatible with updated Google Chrome browsers, allowing users to send messages via voice input. ### Tech Stack: TypeScript ### Why It Matters: This project serves as an introduction to the development of AI, showcasing the integration of NLP, Live2D, and speech recognition. It highlights the importance of maintaining a balance between functionality and user experience. ### Repository: Repository: sglkc/waifu-otw GitHub Stars: 68 GitHub Description: ❤️ Waifu On The Web: A web-based Artificial Intelligence with Natural Language Processing and a Live2D model (PoC) ### Primary Language: TypeScript ### GitHub Description: A client-sided Artificial Intelligence with Natural Language Processor, Live2D, and Speech Recognition. A project for Introduction to Computer Science. Chiai, the developer, is a teenage girl whose AI model is based on Live2D samples and internet assets. The project aims to demonstrate the capabilities of AI through a live chat interface.

### Overview: This repository is a collection of lessons exploring image diffusion models, focusing on understanding and education. It is heavily inspired by Andrej Karpathy's "Zero to Hero" series and is designed to be run in real time with minimal cost, making it ideal for beginners and advanced learners alike. ### Key Features: - **Zero-to-Hero Series**: Lessons that cover the basics of diffusion models, including introduction, deep unsupervised learning, generative modeling, and more. - **PyTorch and Python 3.9**: All lessons are built using PyTorch and Python 3.9, ensuring compatibility and ease of use. - **Colab and GitHub Code**: Each lesson includes an explanatory video, a Colab notebook, and runnable code on GitHub, allowing for interactive learning. - **Historical Order**: Lessons are ordered chronologically, making it easy to trace the development of research and historical progress. - **MNIST Dataset**: Most lessons are restricted to training on the MNIST dataset, which is quick to train and easy to visualize. - **Unified Modeling Framework**: The repository serves as a unified modeling framework for image, audio, and video diffusion modeling, accessible through the [xdiffusion](https://github.com/swookey-thinky/xdiffusion) repository. ### Tech Stack: - **Python**: 3.9 - **PyTorch**: 2.0.0 ### Why It Matters: This series is a valuable resource for anyone interested in understanding and exploring image diffusion models. It provides a comprehensive educational experience through videos

Overview: This repository contains the PyTorch implementation of SketchKnitter, a model that uses diffusion models to generate vectorized sketches. The model is based on the ICLR 2023 Spotlight paper "SketchKnitter: Vectorized Sketch Generation with Diffusion Models". Key features: - Vectorized sketch generation - Diffusion models for data distribution learning - Integration of recognizability into the reverse time diffusion process - Conditional generation framework for better visual appeal and recognizability Tech stack: Python 3.7, PyTorch 1.6, MIT License Why it matters: This project showcases the potential of using diffusion models for generating vectorized sketches, which can be particularly useful in applications requiring high-quality visualizations and recognition, such as in augmented reality or interactive design tools. Repository: wangqiang9/SketchKnitter GitHub stars: 68 GitHub description: [ICLR 2023 Spotlight] About PyTorch implementation of SketchKnitter: Vectorized Sketch Generation with Diffusion Models README excerpt: # SketchKnitter    [](https://github.com/X

Overview: This repository contains code and data for the IJCAI 2024 paper "Continual Multimodal Knowledge Graph Construction". It provides a dataset and implementation for the task of continual multimodal relation extraction. Key features: - Continual Learning Scenario - Dataset Segmentation - Multimodal Named Entity Recognition (MNER) - Multimodal Relation Extraction (MRE) - Implementation in Python Tech stack: Python Why it matters: This repository offers a comprehensive solution for continual learning in relation extraction, demonstrating the effectiveness of the proposed dataset and model in handling large and evolving datasets.

### Overview: This repository hosts a curated catalog of public datasets in Kazakh language, focusing on various tasks such as NLP, LLM, speech, and vision. The datasets are verified and accessible, providing valuable resources for researchers and developers. ### Key features: - **Datasets**: Over 100 datasets ranging from pretraining corpora to legal documents and QA benchmarks. - **Task Coverage**: Includes tasks like language modeling, low-resource NLP, speech recognition, and vision tasks. - **Open Access**: All datasets are freely accessible and open to use. - **Storage**: Datasets are stored in various formats, including text, audio, and images. ### Tech stack: Python ### Why it matters: These datasets are crucial for researchers and developers working on NLP, LLM, speech recognition, and vision tasks in the Kazakh language. They provide a rich resource for training models, evaluating performance, and advancing research in the field.

Overview: OBS-Diff is an official implementation of "OBS-Diff," a novel one-shot pruning framework designed specifically for large-scale text-to-image diffusion models. This method aims to address the computational challenges of these models while maintaining high visual quality, achieving state-of-the-art results in one-shot pruning. Key features: - OBS-Diff revitalizes the classic Optimal Brain Surgeon (OBS) framework, adapting it to modern diffusion models. - It supports various pruning granularities, including unstructured, semi-structured (N:M), and structured (MHA heads and FFN neurons) sparsity. - It incorporates a novel timestep-aware Hessian construction that mitigates error accumulation during pruning. - A computationally efficient group-wise sequential pruning strategy is proposed to speed up the process. Tech stack: Python Why it matters: OBS-Diff offers a practical solution for compressing large-scale text-to-image diffusion models while maintaining high visual quality, making it highly relevant for researchers and practitioners working on efficient and accurate model compression techniques.

### Overview: Blurt is an open-source macOS dictation tool that translates speech into polished text directly within your preferred application. Powered by AssemblyAI, it offers a seamless, subscription-free experience without requiring any middlemen or subscriptions. ### Key Features: - **Works Anywhere You Type:** Transcripts are pasted into the focused app via a synthesized ⌘V, preserving clipboard history and saving the text in the focused app's clipboard. - **One Key, No Chords:** Dictation is triggered by a single modifier key (right ⌘ by default), allowing for quick and efficient speech-to-text conversion. - **Polished in One Step:** Each utterance is sent to AssemblyAI's dictation API with minimal context, resulting in a polished text output. No second request or model downloads. - **Styles:** Save up to four custom styles for different writing styles, and switch between them from the **Style** menu. ### Tech Stack: Swift 6 ### Why It Matters: Blurt is a powerful tool for anyone looking to streamline their workflow by integrating speech-to-text capabilities directly into their preferred macOS applications. Its native implementation and AssemblyAI-powered dictation API make it a versatile and efficient solution for dictation needs.

Overview: VideoGen-of-Thought is an official implementation of the VideoGen-of-Thought project, a step-by-step framework for generating multi-shot videos with minimal manual intervention. This project aims to address the challenges of cohesive multi-shot narratives in video generation, particularly in the context of movie-like content. Key features: - Dynamic storyline modeling - Identity-aware cross-shot propagation - Adjacent latent transition mechanisms Tech stack: Python Why it matters: VideoGen-of-Thought offers a solution that automates multi-shot video synthesis, improving visual quality and narrative coherence compared to existing methods, making it a significant advancement in the field of video generation. Repository: DuNGEOnmassster/VideoGen-of-Thought Primary language: Python GitHub stars: 66 GitHub description: Official Implementation of VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual intervention. This project was accepted by NeurIPS 2025 NextVid Workshop as an Oral presentation.

### Overview: ComfyUI Custom Nodes for HunyuanImage-3.0, featuring BF16 and NF4 quantized options, allowing for efficient local execution with ComfyUI. ### Key Features: - Nodes to run Hunyuan Image 3 locally with BF16 and NF4 quantized options in Comfyui. - Instruct INT8 now working, with five bugs fixed. - v2 pre-quantized models available on Hugging Face. - Experimental latent/image input nodes for base models. - Unified Generate V2 node for all base-model generate variants. - High-Resolution Generation node for 3MP–4K+ generation on 96GB GPUs. - NF4 Low VRAM Loader and Low VRAM Budget generator verified on 24-32 GB cards. - INT8 Budget loader verified on Hugging Face. ### Tech Stack: Python ### Why It Matters: This project integrates the HunyuanImage-3.0 model developed by Tencent Hunyuan Team and uses their official system prompts. The model and original code are licensed under Apache 2.0. The integration code is separately licensed under CC BY-NC 4.0 for non-commercial use. ### Repository: - Repository: [EricRollei/Comfy_HunyuanImage3](https://github.com/EricRollei/Comfy_HunyuanImage3) - GitHub stars: 67 - Primary language: Python - GitHub description: Nodes to run Hunyuan Image 3 locally with BF16

### Overview: Kuaipedia is the world's first large-scale multi-modal short-video encyclopedia, developed by Knowledge Engineering Group in Kuaishou. It consists of items, aspects, and short videos, providing a comprehensive resource for knowledge discovery and sharing. ### Key features: - **Items**: Entities and concepts, such as Shiba Inu, Moon, and Galileo Galilei, with detailed information. - **Aspects**: Keywords or keyphrases attached to items, describing specific aspects like "selection", "food-protecting", "color" for Shiba Inu, or "formation", "surface conditions", "how-to-draw" for Moon. - **Videos**: Short videos of up to 5 minutes, focusing on knowledge videos detected by the OECD. ### Tech stack: Python, JavaScript, and Hugging Face Transformers. ### Why it matters: Kuaipedia offers a unique platform for users to explore and share knowledge through multimedia content, making it an invaluable resource for education, research, and personal growth.

### Overview: PropMolFlow is a property-guided molecule generation model that leverages geometry-complete flow matching to generate molecules that match specified properties. ### Key Features: - **Property-Guided Generation**: PropMolFlow generates molecules based on specific property requirements. - **Geometry-Complete Flow Matching**: Utilizes a flow matching approach to ensure the generated molecules are consistent with the input properties. - **SE(3) Equivariant Architecture**: The model is designed to be invariant under rigid transformations, making it versatile for different molecular structures. ### Tech Stack: - **Python**: The primary programming language used for development and training. - **CUDA**: Used for GPU acceleration, ensuring efficient computation. ### Why it Matters: PropMolFlow offers a powerful tool for generating molecules that adhere to specific properties, making it valuable in fields such as drug discovery, materials science, and chemical engineering where precise molecular design is crucial.

Overview: CausalMM is a method designed to mitigate modality prior-induced hallucinations in multimodal large language models, achieving a maximum score improvement of 65.3% on 6 VLind-Bench indicators and 164 points on MME Benchmark compared to conventional methods. Key features: - Deciphering attention causality in multimodal large language models. - Improves hallucination mitigation in MLLMs. - Maximum score improvement of 65.3% on 6 VLind-Bench indicators. - 164 points improvement on MME Benchmark. - Full preprint available at [arxiv.org/abs/2410.04780](https://arxiv.org/abs/2410.04780). Tech stack: Python. Why it matters: CausalMM provides a practical solution for mitigating hallucinations in multimodal models, enhancing their performance and reliability across various benchmarks.

Overview: R1-Track is an innovative project that leverages Multi-Modal Large Language Models (MLLMs) to directly apply Multi-Object Tracking (MOT) tasks, specifically focusing on Visual Object Tracking (VOT). This approach eliminates the need for traditional fragmentation and subtask modeling, aiming to improve the efficiency and accuracy of tracking. Key features: - Direct application of MLLMs for end-to-end VOT tasks - Support for flexible initialization from text descriptions or bounding boxes - Two types of prompts: SFT or RL_wo_Think, and RL_with_Think Tech stack: Python Why it matters: By utilizing MLLMs, R1-Track aims to simplify and enhance the tracking process, potentially leading to more efficient and accurate results compared to traditional methods. This project showcases the potential of leveraging advanced language models for complex tasks like tracking, demonstrating the integration of AI and machine learning in real-world applications. GitHub stars: 67 Repository: Wangbiao2/R1-Track

### Overview: This project introduces the "Optimal Transport-Guided Conditional Score-Based Diffusion Model" (OTCS), a novel conditional score-based model designed for unpaired and partially paired data settings. The code is based on the NeurIPS 2023 paper by Xiang Gu et al., and it is designed to be easily understandable and implementable. ### Key features: - Unpaired super-resolution on CelebA dataset - Semi-paired image-to-image translation on animal dataset ### Tech stack: Jupyter Notebook ### Why it matters: This model is the first conditional score-based model designed for unpaired or partially paired data settings, making it a significant advancement in the field of conditional score-based diffusion models. ### Repository: [XJTU-XGU/OTCS](https://github.com/XJTU-XGU/OTCS) ### Primary language: Jupyter Notebook ### GitHub stars: 67 ### GitHub description: Official code for "Xiang Gu, Liwei Yang, Jian Sun, and Zongben Xu. Optimal Transport-Guided Conditional Score-Based Diffusion Model. NeurIPS, 2023."[\[OpenReview Version\]](https://openreview.net/forum?id=9Muli2zoFn&referrer=%5Bthe%20profile%20of%20Xiang%20Gu%5D(%2Fprofile%3Fid%3D~Xiang_Gu1)) [\[ArXive Version\]](https

### Overview: This repository contains a collection of multimodal fake news datasets, focusing on misinformation detection across various languages, domains, platforms, and modalities. It serves as a comprehensive resource for researchers studying fake news and related topics. ### Key Features: - **33 Datasets**: 21 core datasets and 12 closely related benchmarks. - **Classic Datasets**: Includes popular datasets like Fakeddit, FakeNewsNet, Weibo, Weibo21, MuMiN, and FakeSV. - **Recent Datasets**: Continuously updated with datasets such as ROM, MDSM, FineFake, AMG, MMFakeBench, MFND, MM-Health, VLDBench, DriftBench, DeceptionDecoded, ReMMDBench, and FakeVE. - **Settings**: Covers different settings including multi-domain, multilingual, social context, external evidence, fine-grained labels, multi-image, audio-video, and generative AI. - **Distinguishing Features**: Distinctively differentiates between real-world data, curated real-world data, out-of-context construction, synthetic manipulation, mixed real + synthetic data, and GenAI-diversified data. ### Tech Stack: Python ### Why It Matters: This dataset collection is crucial for researchers interested in developing and evaluating multimodal fake news detection models, making it a valuable resource for advancing the field of misinformation detection and related topics. ### Repository: Repository: Yuchen-zh/Multimodal-Fake-News-Datasets GitHub Stars: 67 ### Primary Language

Overview: AvatarGAN is a project that generates cartoon images using a Generative Adversarial Network (GAN) architecture. It utilizes advanced techniques such as Replacing pooling layers with strided convolutions and removing fully connected layers to create a more effective model. The project is based on the DC-GAN architecture and includes detailed explanations and a dataset for training. Key features: - Generate cartoon images using a Generative Adversarial Network - Use advanced techniques for better model performance - Detailed explanation and dataset for training Tech stack: Jupyter Notebook Why it matters: AvatarGAN is a significant project that showcases the potential of Generative Adversarial Networks in generating high-quality cartoon images, demonstrating the effectiveness of advanced GAN architectures in image generation. Repository: aakashjhawar/AvatarGAN GitHub stars: 67 GitHub description: Generate Cartoon Images using Generative Adversarial Network

Overview: CLIP as a service allows developers to easily integrate OpenAI's innovative CLIP model into their projects, enabling image and sentence embedding, object recognition, visual reasoning, and more. Key features: - Encode: Transform text or images into meaningful embeddings for tasks like neural search and custom ranking. - Rank: Perform zero-shot image classification, providing versatile ways to sort and categorize data. - Visual Reasoning: Offer reasoning about visual scenarios, such as counting objects, identifying colors, and understanding motion. Tech stack: BentoML Why it matters: This service simplifies the integration of CLIP, making it accessible and easy to use for developers without the need for extensive setup.

### Overview: ArcSub is an end-to-end subtitle translation workstation that supports both cloud and local OpenVINO models, providing a unified interface for media intake, speech to text, subtitle translation, and video playback. ### Key Features: - **Media Intake**: Import local media files or download online media. - **Speech to Text**: Use either cloud ASR services or local OpenVINO ASR models. - **Subtitle Translation**: Choose cloud or local translation models. - **Video Playback**: Watch the finished subtitles with the video and adjust styling for the viewing page. ### Tech Stack: TypeScript ### Why It Matters: ArcSub simplifies subtitle translation workflows by treating cloud and local models equally, making it versatile and adaptable to different environments and needs. ### GitHub Stars: 67 ### Repository Scope: This repository contains the application source code and public documentation. It does not include local runtime data, downloaded models, or portable bootstrap runtimes. ### Main Capabilities: - Import local media or download online media. - Run speech to text with cloud ASR services or local OpenVINO ASR models. - Use word alignment, VAD, and diarization-related helpers. - Translate subtitles with cloud translation services or local OpenVINO translation models. - Watch subtitle results with the video and tune styling for the viewing page. ### Cloud and Local Models: ArcSub allows each project to choose the most practical model path across cloud services and local OpenVINO runtimes. Cloud ASR and translation models are configured in `Settings`, while local ASR and translation models are installed

### Overview: HotVoice is a project that adds speech recognition support to AutoHotkey, enabling users to control their scripts via voice commands. This feature enhances the automation capabilities of AutoHotkey by integrating voice recognition technology, making it easier to script and automate tasks. ### Key Features: - **Voice Command Integration**: Allows users to control scripts using voice commands. - **Language Support**: Supports multiple languages through language packs, including English, Spanish, French, German, and more. - **Scripting Enhancements**: Enhances the functionality of AutoHotkey scripts by adding voice recognition capabilities. ### Tech Stack: AutoHotkey ### Why It Matters: HotVoice provides a powerful and versatile tool for automating tasks in AutoHotkey, making it easier to script and automate complex tasks with voice commands. This feature is particularly useful for users who prefer to interact with their systems through voice, offering a new level of flexibility and accessibility.

Overview: Vowel Practice is an iOS application designed to help users practice and improve their pronunciation of vowels, including monophthongs and diphthongs, by providing real-time feedback on formants. Key features: - Monophthong vowel recognition - Recording and playback of vowel sounds - Configuration page for better results - Integration with a Hillenbrand 1995 data set for test cases - Support for donations of test data Tech stack: Swift Why it matters: Vowel Practice is designed to help individuals and professionals improve their pronunciation, particularly in speech therapy and accent reduction, by providing a practical tool for practicing vowel sounds and receiving immediate feedback on formants. Repository: fulldecent/vowel-practice Primary language: Swift GitHub stars: 67 GitHub description: iOS application for finding formants in spoken sounds