Projects

Overview: This project implements a document classification model using Hierarchical Attention Networks, as described in the paper "Hierarchical Attention Networks for Document Classification" by Yang et al. (2016). Key features: - Uses Hierarchical Attention Networks for effective document classification. - Implements a TensorFlow-based solution. - Includes a pre-trained English model for spaCy for quick model loading. Tech stack: Python Why it matters: The project showcases a practical application of Hierarchical Attention Networks in document classification, demonstrating its effectiveness and potential for real-world applications.

### Overview: This repository contains papers and codes related to various computer vision tasks presented at CVPR 2022. The papers cover topics such as detection, segmentation, image processing, estimation, and more. The repository includes implementations and analyses of cutting-edge methods in these areas, aiming to provide insights and practical tools for researchers and developers. ### Key features: - **Detection**: Papers on object detection, including methods like OW-DETR, AdaMixer, and QueryDet. - **Segmentation**: Papers on image segmentation techniques. - **Image Processing**: Papers on image processing and computer vision techniques. - **Estimation**: Papers on estimation methods. - **Image & Video Retrieval/Video Understanding**: Papers on image and video retrieval and understanding. - **Face**: Papers on face recognition and analysis. - **3D Vision**: Papers on 3D vision techniques. - **Object Tracking**: Papers on object tracking methods. - **Medical Imaging**: Papers on medical imaging techniques. - **Text Detection/Recognition**: Papers on text detection and recognition. - **Remote Sensing Image**: Papers on remote sensing image processing. - **GAN/Generative/Adversarial**: Papers on generative adversarial networks. - **Image Generation/Image Synthesis**: Papers on image generation and synthesis techniques. - **Scene Graph**: Papers on scene graph modeling. - **Visual Localization**: Papers on visual localization methods. - **Visual Reasoning/VQA**: Papers on visual reasoning and question answering. - **Image Classification**: Papers on image classification techniques. - **Neural Network Structure Design**:

### Overview: ACTalker is an end-to-end video diffusion framework designed for talking head synthesis. It supports both single and multi-signal control, including audio and expression. The project is aimed at improving the naturalness and realism of synthesized talking heads, making it particularly useful in applications requiring high-quality video generation. ### Key Features: - **Stable Video Diffusion**: The core technology that enables the synthesis of talking heads. - **Multi-Signal Control**: Supports audio and expression inputs for enhanced realism. - **End-to-End Framework**: Integrates all necessary components for video generation. ### Tech Stack: Python ### Why It Matters: ACTalker is significant because it addresses the need for high-quality video generation, especially in scenarios where natural and realistic talking heads are crucial. The ability to control audio and expressions makes the generated videos more engaging and lifelike, which is valuable in applications such as virtual assistants, video games, and more. Repository: harlanhong/ACTalker Primary language: Python GitHub stars: 467 GitHub description: ICCV 2025 ACTalker: an end-to-end video diffusion framework for talking head synthesis that supports both single and multi-signal control (e.g., audio, expression). README excerpt: ## :book: Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generation (ICCV 2025) > [[Paper](https://arxiv.org/abs/2504.02542)]   [[

Overview: KSS is a comprehensive suite of Korean string processing tools designed for various applications in natural language processing, data preprocessing, and data analysis. Key features: - Augment: Enhances text by adding or removing specific characters. - Collocate: Finds co-occurring words and phrases. - G2P: Translates Korean words into their phonetic representation. - Hangulize: Converts text to Korean Hangul characters. - Split Hanja: Splits Korean Hanja characters into their base characters. - Is Hanja: Determines if a character is a Hanja. - Hanja2Hangul: Converts Hanja characters to Hangul. - H2J: Converts Hangul characters to their corresponding Korean words. - H2HCJ: Converts Hangul characters to their corresponding Hanja. - J2H: Converts Korean words to Hangul characters. - J2HCJ: Converts Korean words to Hanja. - HCJ2H: Converts Hanja characters to Hangul. - HCJ2J: Converts Hanja characters to Korean words. - Is Jamo: Determines if a character is a Jamo. - Is Jamo Modern: Determines if a character is a modern Jamo. - Is Hcj: Determines if a character is a Hcj. - Is Hcj Modern: Determines if a character is a modern Hcj. - Is Hangul Char: Determines if a character is a Hangul character. - Select Josa: Identifies and selects Josa in Korean sentences. - Combine Josa: Combines Josa in Korean sentences

Overview: Gemini is an open-source implementation of the Gemini model, designed to "eclipse ChatGPT" by Google. It processes all modalities (texts, audio, images, and videos) into tokens and uses a transformer with special decoders for text and image generation. Key features: - Supports multiple modalities (texts, audio, images, and videos) - Uses a transformer with special decoders for text and image generation - Processes inputs into tokens and then processes them through a transformer - Utilizes conditional decoding for image generation - Architecture bears resemblance to Fuyu's but expanded for multiple modalities - Uses image embeddings directly in the transformer instead of a visual transformer - Token inputs will likely be indicated by special modality tokens such as [IMG], <img>, [AUDIO], or <audio> Tech stack: Python Why it matters: Gemini aims to revolutionize language models by processing all modalities directly, potentially surpassing current models like ChatGPT. Its architecture and processing methods make it a significant development in the field of AI. Repository: kyegomez/Gemini GitHub stars: 467 GitHub description: The open source implementation of Gemini, the model that will "eclipse ChatGPT", it seems to work by directly taking in all modalities all at once into a transformer with special decoders for text or img generation!

### Overview: FluxRT is a real-time stream editing pipeline powered by the FLUX.2-klein-4B model, optimized for consumer GPUs. It enables low-latency transformation of webcam streams or videos with interactive prompt updates and reference image conditioning support. ### Key Features: - **Real-time Stream Editing**: Transforms webcam streams or videos in real-time with interactive prompt updates and reference image conditioning. - **Instruct Image Editing Model**: Uses the FLUX.2-Klein model for precise control and reference image conditioning. - **Low-Latency Performance**: Achieves frame rates of 20-40 FPS on RTX 5090 and 15-30 FPS on RTX 4090. - **GUI and OBS Integration**: Includes an interactive GUI that supports creating Virtual Web Cameras and integrating with OBS, Zoom, Chrome, TouchDesigner, Resolume, and other apps that use web cameras. - **Daydream Scope Integration**: Native implementation of the Daydream Scope plugin. - **Reference Image Support**: Natively supports all FLUX.2 reference image features, including interactive paint-style apps with iterative image updates. - **System Requirements**: Minimal (int8 only) on RTX 4090, Recommended on RTX 5090. ### Tech Stack: Python ### Why It Matters: FluxRT offers a powerful and flexible solution for AI Virtual Try-On, creative coding, and other applications requiring real-time stream editing and interactive prompt updates. Its optimized performance and integration capabilities make it suitable for a wide

### Overview: An AI-powered Sublime Text plugin that integrates with OpenAI models, including GPT-5, LLaMA, and Ollama, providing a cursor-level assistant for developers. ### Key Features: - **Chat Mode**: Powered by various models. - **gpt-5 Support**: Built-in support for GPT-5. - **LLaMA's Server**: Compatible with LLaMA's server. - **Ollama and LLMMan**: Support for other OpenAI-compatible APIs. - **Dedicated Chats Histories**: Projects-specific settings. - **File Context Expansion**: Sends file parts as context. - **Phantoms**: Inline answers without disrupting the view. - **Markdown Syntax Highlight**: Chat mode only. - **Server Side Streaming**: Supports streaming. - **Status Bar**: Model name, mode, tokens. - **Proxy Support**: Available. ### Tech Stack: Python ### Why It Matters: This plugin enhances Sublime Text with advanced AI capabilities, making it more productive and efficient for developers working with various OpenAI models.

### Overview: This repository contains detailed implementations and comparisons of language models for the Russian language, focusing on the Russian version of the Alpaca model. Key features include a dataset of instruction-based prompts and fine-tuning configurations for various models. The project aims to provide insights and benchmarks for language modeling in Russian. ### Key Features: - **Dataset**: A collection of instruction-based prompts for the Russian language. - **Models**: Fine-tuning configurations for LLaMA-7B and LLaMA-13B models. - **Training Configurations**: Detailed training configurations for LLaMA-7B and LLaMA-13B models. - **Base Models**: Supported by Hugging Face models like Huggyllama/llama-7b and Huggyllama/llama-13b. - **Training Details**: The project includes a separate README for detailed training configurations. ### Tech Stack: Jupyter Notebook ### Why it Matters: This project offers valuable insights into language modeling in Russian, providing benchmarks and comparisons for the Russian version of the Alpaca model. The detailed training configurations and fine-tuning features make it a useful resource for researchers and developers in the field of language modeling.

Overview: ColorFlow is an advanced image sequence colorization method that addresses the challenge of preserving character and object identity while automatically converting black-and-white images to color. It is designed for industrial applications, particularly in the field of cartoon or comic series colorization. Key features: - **Retrieval-Augmented Colorization Pipeline:** A novel approach that leverages relevant color references to colorize images, making it robust and generalizable. - **Dual-Branch Design:** One branch for color identity extraction, another for colorization, utilizing the strengths of diffusion models for strong in-context learning and color identity matching. - **ColorFlow-Bench Benchmark:** A comprehensive benchmark for reference-based colorization that evaluates the model's performance across multiple metrics. Tech stack: Python Why it matters: ColorFlow provides a practical solution for industrial applications, enhancing the quality and consistency of image sequences, especially in the art industry, where maintaining character and object identity is crucial. Repository: [TencentARC/ColorFlow](https://github.com/TencentARC/ColorFlow) GitHub stars: 466 GitHub description: The official implementation of paper "ColorFlow: Retrieval-Augmented Image Sequence Colorization".

Overview: This project is a Retrieval Augmented Generation (RAG) Streamlit application backed by [txtai](https://github.com/neuml/txtai), designed to help generate factually correct content by limiting the context in which a language model can generate answers. Key features: - Vector RAG: Context supplied via a vector search query - Graph RAG: Context supplied via a graph path traversal query Tech stack: Python Why it matters: This application combines search and language models to find insights with your own data, making it a valuable tool for developers looking to enhance their applications with improved accuracy and context-awareness.

Overview: SILENTCHAIN AI™ Community Edition is an AI-powered vulnerability scanner extension for Burp Suite, utilizing advanced language models for intelligent passive analysis of HTTP traffic. It identifies OWASP Top 10 vulnerabilities, security misconfigurations, and potential attack vectors. Key features: - Passive AI Analysis - Professional UI with live findings panel and task management - Multi-AI Support (Burp AI, Ollama, OpenAI, Claude, Gemini, Azure) - Smart Reporting with CWE and OWASP mappings - Data Privacy (DataSanitizer) Tech stack: Java Why it matters: SILENTCHAIN AI™ Community Edition leverages AI models to provide intelligent passive analysis, reducing noise and focusing on real vulnerabilities, making it a valuable tool for web application security testing. Repository: silentchainai/SILENTCHAIN GitHub stars: 466

### Overview: This repository is a curated list of awesome resources for image alignment and stitching, including tutorials, papers, and projects. It aims to provide developers with valuable information and tools for their projects. ### Key Features: - **Image Registration**: Resources for image registration, including tutorials and papers. - **Medical Image Registration**: A practical review on medical image registration. - **Image Matching**: Feature detection methods, including key points and line detection. - **Geometric Pre-processing**: Techniques for rectifying fisheye images and calibrating cameras. - **Image Alignment and Stiching**: Projects and tutorials for image alignment and stitching. ### Tech Stack: Python, OpenCV, TensorFlow, Theano, and GitHub. ### Why it Matters: This list serves as a comprehensive resource for developers looking to enhance their image processing and alignment skills. It covers a wide range of topics, from basic tutorials to advanced research papers, ensuring that users can find the information they need to improve their projects.

Overview: SoundNet is a state-of-the-art model that learns rich natural sound representations from unlabeled video data, presented in the NIPS 2016 paper "SoundNet: Learning Sound Representations from Unlabeled Video". This project focuses on the implementation and usage of the SoundNet model for sound recognition and feature extraction. Key features: - SoundNet model for learning sound representations from unlabeled video data - Pre-trained models available for 8 and 5 layers - Script for recognizing categories and extracting features from audio files - Feature extraction capabilities using Torch - Fine-tuning option for custom datasets Tech stack: Lua, Torch7, Audio (SOX), HDF5 Why it matters: SoundNet offers a powerful approach to learning sound representations from unlabeled video data, providing a significant advancement in the field of audio and visual data fusion. Its ability to recognize categories and extract features makes it highly valuable for applications in audio classification and feature extraction, making it a valuable tool for researchers and developers working in audio and visual systems.

### Overview: RLMRec is a PyTorch implementation of a model-agnostic framework for enhancing recommendation systems with large language models (LLMs). This paper, presented at WWW2024, proposes RLMRec, which integrates representation learning with LLMs to capture intricate semantic aspects of user behaviors and preferences. The framework is designed to be model-agnostic, making it applicable across various recommendation models. ### Key Features: - **Model-Agnostic Framework**: Enhances existing recommenders with LLM-empowered representation learning. - **User/Item Profiling**: Empowered by LLMs, it generates high-quality text descriptions for users and items. - **Cross-View Alignment**: Aligns the semantic space of LLMs with the representation space of collaborative relational signals. - **Auxiliary Textual Signals**: Incorporates auxiliary textual signals to improve recommendation accuracy. ### Tech Stack: - **Python**: The primary language used for development. - **PyTorch**: Framework used for implementing the model. - **Torch-scatter, Torch-sparse**: Libraries for efficient computation of large-scale sparse matrices. ### Why It Matters: RLMRec offers a novel approach to recommendation systems by leveraging the semantic understanding capabilities of large language models. This method enhances the representation learning process, making it more effective in capturing user preferences and item characteristics, thereby improving the accuracy and relevance of recommendations.

Overview: Maccarone is an AI-driven Python extension that automates the management of code blocks, making your projects more efficient and dynamic. Key features: - AI-assisted code block management - Updates code sections based on changes - VS Code extension available Tech stack: Python Why it matters: Maccarone streamlines the development process by delegating repetitive tasks to AI, reducing manual effort and improving code quality. Repository: bsilverthorn/maccarone GitHub stars: 464 GitHub description: AI-managed code blocks in Python ⏪⏩

Overview: Instant-angelo is a project that aims to build high-fidelity digital twins within a remarkably short timeframe, specifically within 20 minutes, using the state-of-the-art UniSDF renderer. Key features: - High-fidelity digital twin creation - Rapid photorealistic neural 3D reconstructions - Integration of UniSDF for improved reflective surface reconstruction Tech stack: Python Why it matters: Instant-angelo offers a significant improvement in the speed and quality of 3D surface reconstruction, making it ideal for applications requiring real-time or near-real-time rendering, such as augmented reality, virtual reality, or interactive simulations. Repository: hugoycj/Instant-angelo Primary language: Python GitHub stars: 464 GitHub description: Instant-angelo: Build high-fidelity Digital Twin within 20 Minutes!

Overview: TPA: Tensor ProducT ATTenTion Transformer (T6) is a state-of-the-art transformer model that leverages Tensor Product Attention (TPA) mechanisms to enhance performance and reduce KV cache size. This repository provides tools for data preparation, model pretraining, and evaluation to facilitate research and development using the T6 architecture. Key features: - Tensor Product Attention: Implements advanced attention mechanisms for improved model performance. - Scalability: Efficient training procedures optimized for large-scale datasets and multi-GPU setups. - Flexible Data Support: Compatible with popular datasets like [Fineweb-Edu-100B](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu/) and [OpenWebText](https://openwebtext2.readthedocs.io/en/latest/). - Comprehensive Evaluation: Integrated with [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) for standardized benchmarking. - FlashTPA Decoding: see Algorithm 2 and 3 in [Paper](./Tensor_Product_Attention.pdf) for FlashTPA Decoding algorithms and [./decode](./decode) for Python and Triton implementations. Tech stack: Python Why it matters: TPA Transformer (T6) is a significant advancement in transformer models, offering enhanced performance and reduced memory usage through Tensor Product Attention mechanisms. The repository provides essential tools for researchers and developers to experiment with and improve upon this cutting-edge model.

### Overview: gptAssist is a simple WebView wrapper for ChatGPT, designed to be used in applications that require web browsing capabilities. ### Key Features: - **WebView Integration**: Allows developers to easily integrate ChatGPT into their applications. - **Personal Data Submission**: The app is designed to work on certified Android devices, but developers must submit personal identity details to Google. - **Customizable Icons**: Supports various Android packages with customizable icons. ### Tech Stack: Java ### Why it Matters: The app is essential for developers who need to integrate ChatGPT into their applications but are concerned about privacy and compliance with Google's requirements. It provides a straightforward solution without the need to submit personal data to Google, ensuring compliance and maintaining user privacy.

Overview: Recent advancements in generative diffusion models have enabled text-controlled synthesis of realistic and diverse images. However, their application in computer vision for standard visual recognition tasks remains limited. This project introduces InstructCV, an official PyTorch implementation, which abstracts away task-specific design choices and enables execution through natural language instructions. Key features: - Instruction-tuned text-to-image diffusion models - Unified language interface for computer vision tasks - Multi-modal and multi-task training dataset - Use of a large language model to create a training dataset Tech stack: Python Why it matters: By providing a unified interface for computer vision tasks, InstructCV simplifies the process of training and executing tasks, making it more accessible and efficient for researchers and practitioners in the field. Repository: AlaaLab/InstructCV GitHub stars: 463

### Overview: Yap is a free, open-source voice dictation tool for macOS that works without requiring an account, API keys, or cloud storage. It uses Apple's Speech framework for on-device transcription, providing a fast and efficient solution for dictating text. ### Key Features: - **On-Device Transcription**: Yap transcribes speech directly to text on your Mac, without needing an external model or API. - **Global Shortcut**: A customizable shortcut to trigger the dictation process. - **No Model or API Key**: Yap does not use any external models or require API keys, ensuring privacy and security. - **No Network or Browser Engine**: Yap runs entirely on the local machine, without requiring any network access or browser engine. ### Tech Stack: - **Swift**: The primary programming language used to build Yap. - **Apple's Speech Framework**: The underlying technology used for speech-to-text transcription. ### Why It Matters: - **Privacy and Security**: Yap ensures that your audio and transcripts remain on your machine, providing complete privacy and security. - **Efficiency**: Yap is significantly faster and more efficient than many other dictation tools, with a word error rate of 2.12% on clean audio and 4.56% on noisy audio. - **Ease of Use**: Yap is simple to use, with a global shortcut that can be easily redefined, and a small, lightweight app that runs without any additional resources. ### Repository: - **GitHub Repository**: [FrigadeHQ/yap](https://github.com/FrigadeHQ/yap)