Projects

### Overview: Customize-It-3D is an innovative project that aims to create high-quality 3D models from a single image using subject-specific knowledge. This approach leverages multi-modal information to ensure the 3D object's coherence with respect to a particular identity, leading to more realistic and view-consistent results. ### Key features: - **Subject-Specific Knowledge Prior**: The project incorporates a subject-specific knowledge prior to enhance the 3D model's coherence with the subject's identity. - **NeRF for Geometry Reconstruction**: A NeRF model is used to reconstruct the geometry of the reference image in a shading-aware manner. - **Enhanced Texture from Coarse Stage**: Point clouds are generated with enhanced texture from the coarse stage, and this stage is further optimized to generate realistic and view-consistent textures. ### Tech stack: The project is implemented using Python, leveraging libraries such as PyTorch, PyTorch 3D, and Contextual Loss PyTorch for the neural network components. ### Why it matters: This project is significant because it addresses the challenge of creating high-quality 3D models from a single image, which is crucial for applications such as augmented reality, virtual reality, and interactive 3D modeling. By incorporating subject-specific knowledge, it ensures that the 3D models are more aligned with the subject's identity, leading to more realistic and contextually relevant results.

Overview: This repository contains three examples of recommendation system architectures utilizing NVIDIA Merlin and Redis, providing cloud deployment instructions for production usage. Key features: - Offline Batch Recommendations - Online Recommendation Systems - Large-Scale Recommender Models with HugeCTR Tech stack: PureBasic Why it matters: This repository offers practical insights into building recommendation systems with real-time capabilities, making it valuable for developers looking to enhance their recommendation engine solutions.

### Overview: This project explores the potential of language models (LLMs) to classify images, utilizing multimodal techniques to enhance traditional image classification methods. ### Key Features: - **Image Classification Testing with LLMs**: The project focuses on evaluating the performance of LLMs in image classification tasks. - **Multimodal LLMs**: It leverages LLMs that can both "see" images and generate language descriptions, offering a more advanced approach to image classification. - **LLaVA Model**: Utilizes a 7B parameter quantized version of LLaVA (Large Language and Vision Assistant) for this project, showcasing the model's capabilities in image description and context understanding. ### Tech Stack: The project employs Python as the primary programming language. ### Why It Matters: This project aims to explore the potential of multimodal language models in image classification, potentially replacing traditional methods by providing more advanced contextual understanding and zero-shot learning capabilities.

Overview: This project provides a Python library and command-line interface (CLI) for interacting with Speechmatics Realtime and Batch ASR v2 APIs. Key features: - Real-time transcription - Batch transcription - Diarization features - Speaker identification metrics Tech stack: Python Why it matters: This project is deprecated, but it serves as a reference for migrating to the official SDKs and provides a comprehensive documentation for the Speechmatics API and product documentation.

### Overview: BitBat is an AI-powered transcription app that automatically converts audio and video files into text, making it easier for users to manage their work efficiently. ### Key Features: - **Transcribe any media** to text: Supports transcribing from URLs and uploading files. - **Download transcriptions** in various formats: TXT, JSON, VTT, SRT, or copy the text to clipboard. - **Translate subtitles** to any language supported by LibreTranslate. - **Powerful subtitle editor**: Highlights transcriptions based on media position, provides CPS warnings, segment splitting, and language selection. - **100% local transcription, translation, and subtitle editing**: Transcription, translation, and subtitle editing happen on your machine. - **Fast and efficient**: Uses FasterWhisper backend and can run on CPU or GPU for faster transcription times. - **Quick and easy setup**: Use the quick start script or follow a few steps. - **GPU support**: Use NVIDIA GPU for even faster transcription times. - **CPU support**: Transcription can run on CPU as well. ### Tech Stack: Svelte ### Why It Matters: Inspired by his father's struggles with transcription, the developer created BitBat to simplify the work of transcribing audio and video files. The app's features, such as automatic transcription, translation, and powerful subtitle editing, make it a valuable tool for professionals and students alike.

Overview: This highly experimental app uses WebRTC and Web Speech APIs to provide real-time translations during video calls. It currently works only in Chrome and is based on the iTranslate app UI. Key features: - Real-time translation during video calls - WebRTC and Web Speech API integration - Experimental implementation of Web Speech Recognition API in Firefox Tech stack: JavaScript, WebRTC, Web Speech API Why it matters: The app offers a unique real-time translation experience, making it useful for language learners and remote communication. However, it is currently limited to Chrome and requires an experimental implementation in Firefox.

### Overview: HEISS-UI is a local front-end for ComfyUI, providing a calm and intuitive interface for workflows and prompt rendering. It simplifies the process of using ComfyUI by offering a prompt box, gallery, and settings tailored to the models you choose. ### Key Features: - **30 Model Families Ready to Go:** ComfyUI's 30 model families are pre-installed, ready for immediate use. - **Missing Parts, One Click Away:** Easily find what parts of a model are missing by clicking the "Download" button for each component. - **Native 2K and Sizes:** Choose 2K or 4K resolution, or set any size of your own, and save the image for every device. - **Smart Upscale:** Automatically upscale any picture to 2K or 4K with SeedVR2, with a countdown. - **Bring Your Own Workflow:** Import a ComfyUI workflow and it becomes a clean set of controls, with your graph running underneath. - **Inpainting (Beta):** Paint over part of a picture and only that part changes, working with 18 image families. - **Auto-Grouping:** Ten takes of one idea become one tile, making galleries of hundreds easy to scan. - **On Your Phone and Every Device:** A phone layout for one thumb, and a full studio on any laptop or tablet. - **Hidden:** A locked place for images you keep to yourself, opened with Touch ID, Windows Hello, or a password. - **Civitai-

### Overview: AdaptiveDiffusion is a novel adaptive inference paradigm that offers a third-order latent differential estimator to determine whether to reuse noise predictions from previous timesteps for denoising. This approach allows for training-free acceleration and adapts skipping strategies based on the optimal approximation for different prompts. ### Key Features: - **Training-Free Acceleration:** Reduces the step number of noise predictions without training. - **Unified Skipping Strategy:** Applies to both image and video generation models. - **Interchangeable Noise Schedulers:** Supports different diffusion speeds and output quality. ### Tech Stack: Python ### Why It Matters: This method enables faster inference times and better performance across various prompts, making it highly effective for applications requiring high-speed processing and diverse input types.

### Overview: Cortex-M-KWS 是由 OPEN AI LAB 开发的基于 Cortex-M4 核心的微控制通过 Tengine Lite 推理框架部署关键词识别（KWS）功能的示例项目。该项目旨在方便开发者进行技术评估。 ### Key Features: - 实现基于 Cortex-M4 的微控制板上部署关键词识别功能。 - 使用 Tengine Lite 推理框架进行关键词识别。 - 包含 11 个关键词，包括小智小智、打开窗帘、关闭窗帘、打开空调、关闭空调、加热模式、制冷模式、降低温度、调高温度、开启扫风、启动空调。 ### Tech Stack: C ### Why it Matters: Cortex-M-KWS 项目为开发者提供了一个简单易用的框架，用于评估和测试基于 Cortex-M4 的微控制板上的关键词识别功能。通过使用 Tengine Lite 推理框架，该项目简化了关键词识别的过程，提高了开发效率。

### Overview: This repository chronologically traces the development of deep learning, highlighting key papers and their contributions to the field. It covers breakthroughs from the 1980s to the present, emphasizing the evolution of neural network architectures and algorithms. ### Key Features: - **78 Selected Papers**: Covering all deep learning applications and research areas. - **Image Recognition**: Including seminal works like the first neural network paper and the first CNN. - **Machine Translation**: Describing the wake-sleep algorithm and its impact. - **Speech Recognition**: Highlighting the development of LSTM networks. - **Optimization and Meta-Learning**: Discussing the universal function approximation and meta-learning techniques. ### Tech Stack: - **Neural Networks**: Feed-forward and recurrent neural network architectures. - **Machine Learning Algorithms**: Backpropagation, error backpropagation, and other optimization techniques. ### Why It Matters: Understanding the historical development of deep learning provides context for the current research landscape and helps in appreciating the significant advancements made in AI.

Overview: This project demonstrates how to deploy generative AI models from Amazon SageMaker JumpStart using AWS CDK. Key features: - Deploy generative AI models from Amazon SageMaker JumpStart using AWS CDK - Use foundation models for various tasks like content creation and code generation - Access pre-trained models, solution templates, and examples through Amazon SageMaker Studio Tech stack: AWS CDK, Amazon SageMaker, Amazon SageMaker JumpStart Why it matters: This project showcases the integration of generative AI with AWS CDK, enabling developers to quickly incorporate cutting-edge ML techniques into their production workflows.

Overview: Cactus Kotlin is a Kotlin Multiplatform library designed for running AI models locally in your Kotlin apps, specifically for deploying language models (LLMs) and speech-to-text capabilities. This library is built on top of the Cactus framework, which is optimized for local deployment and supports both iOS and Android platforms. Key features: - Local deployment of LLMs and speech-to-text models - Supports both iOS and Android platforms - Built on Kotlin Multiplatform, ensuring cross-platform compatibility Tech stack: Kotlin Multiplatform Why it matters: This library simplifies the process of deploying AI models locally, allowing developers to integrate advanced AI functionalities directly into their apps without needing to deploy them as separate services. This is particularly useful for creating apps that require real-time AI capabilities without the need for external cloud services, ensuring faster response times and better performance.

### Overview: Mask Forcing is an innovative approach to improving the quality and realism of autoregressive video diffusion models. By introducing a Dual-Noise Masking Rollout strategy, it addresses the limitations of traditional DMD, leading to more diverse and visually appealing results. ### Key Features: - **Dual-Noise Masking Rollout**: Injects cleaner tokens into noisy rollout inputs through random masks, diversifying student rollout trajectories to cover more teacher modes. - **Improved Intermediate Predictions**: Guides the denoising of noisier tokens, reducing error accumulation and enhancing intermediate predictions. - **No Real-Video Supervision Required**: The method does not require real-video supervision or additional post-training stages, making it highly practical. ### Tech Stack: - Python ### Why It Matters: Mask Forcing simplifies the process of generating high-quality video sequences by eliminating the need for real-video supervision and additional post-training stages. It achieves better visual quality and realism without compromising the inference process, making it a significant advancement in the field of video diffusion models. ### Repository: - **delaprada/Mask-Forcing** - **GitHub Stars**: 74 - **GitHub Description**: Official Implementation for Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout ### Demo: - **Pipeline Visualization**: The Mask Forcing pipeline is illustrated in the provided image, showcasing how cleaner tokens are injected into noisy rollout inputs to improve intermediate predictions. ### News: - **[2026-09]**: Paper and project page are

Overview: This repository contains the official implementation of the paper "PuzzleFusion++: Auto-agglomerative 3D Fracture Assembly by Denoise and Verify" presented at ICLR 2025. The project aims to solve 3D fracture assembly problems using an auto-agglomerative approach inspired by human puzzle-solving methods. Key features: - Auto-agglomerative 3D Fracture Assembly - Denoise and Verify process - Diffusion model for alignment denoising - Transformer model for alignment verification Tech stack: Python Why it matters: This project showcases a novel method for 3D fracture assembly that significantly outperforms existing techniques in terms of accuracy and distance metrics, demonstrating the potential of auto-agglomerative approaches in solving complex spatial puzzles.

### Overview: otto-m8 is an open-source platform designed to automate deep learning workflows using a flowchart-like interface. It allows developers to build AI/ML workflows without writing code, making it accessible for beginners and advanced users alike. ### Key Features: - **Visual Workflow Building**: Users can create workflows by connecting pre-built blocks and custom blocks. - **Customization**: Users can modify and extend the pre-built blocks through code. - **Incremental Development**: Workflow development can be done incrementally with real-time feedback. - **Docker Deployment**: Deploy workflows as a RESTful API, allowing for easy integration and management. - **Traceability**: Keeps a record of every workflow run. ### Tech Stack: JavaScript ### Why It Matters: otto-m8's approach of building workflows through visual flowcharts makes it accessible to a wide range of users, including those without extensive coding experience. This platform aims to overcome the limitations of low-code platforms by providing a fully open-source solution that can scale and integrate with various AI frameworks.

### Overview: gLLM is an efficient and versatile inference engine designed for distributed LLM serving. It supports a wide range of models, including dense, MoE, multimodal/vision-language, and hybrid-attention architectures from HuggingFace. The engine also supports offline/online inference and interactive chat scenarios. Key features include continuous batching, paged attention, chunked prefill, prefix caching, cuda graph, token throttling, pipeline parallelism, expert parallelism, and tensor parallelism, delivering superior inference speed while keeping a minimal code base. ### Key Features: - **Broad Model Support**: Supports dense, MoE, multimodal/vision-language, and hybrid-attention architectures from HuggingFace, including FP8 checkpoints. - **Flexible Parallelism**: Supports pipeline, tensor, and expert parallelism for single- or multi-node deployments. - **High-Performance Execution**: Continuous batching, paged attention, chunked prefill, prefix caching, and cuda graph. - **Balanced Scheduling**: Token throttling for smoother pipeline utilization across prefill and decode. - **Versatile Serving**: Supports offline batch inference, online serving, and interactive chat. ### Tech Stack: Python ### Why It Matters: gLLM is an efficient and versatile inference engine that supports a wide range of models and deployment scenarios. It integrates advanced features like continuous batching, paged attention, and cuda graph, delivering superior inference speed while keeping a minimal code base. This makes it an ideal choice for developers looking to optimize LLM inference performance in distributed environments.

### Overview: Whisper is an automatic speech recognition (ASR) system implemented as a Gradio Web UI. It offers a user-friendly interface for processing audio files and converting them into text. ### Key features: - **Gradio Web UI**: The project provides a web-based interface for the Whisper ASR system. - **Multiple Model Sizes**: Available in five model sizes, including both English-only and multilingual versions. - **English-Only Models**: Perform better for English-only applications, with `tiny.en` and `base.en` models showing significant performance improvements. - **Screencast**: Available for reference and support. ### Tech stack: Python ### Why it matters: Whisper simplifies the process of automatic speech recognition, making it accessible and user-friendly through a Gradio interface. Its performance improvements for English-only applications make it particularly valuable for applications requiring high accuracy in English. ### Repository: innovatorved/whisper-openai-gradio-implementation ### GitHub stars: 74 ### GitHub description: Whisper is an automatic speech recognition (ASR) system Gradio Web UI Implementation

Overview: VectorSmuggle is a research framework designed to examine covert data leakage in AI/ML environments, specifically focusing on retrieval-augmented generation (RAG) systems. It demonstrates vector-based data exfiltration techniques and provides tools for both offensive analysis and defensive countermeasure development. Key features: - Steganographic techniques: Embedding obfuscation through noise injection, rotation, scaling, offset, fragmentation, and decoy interleaving. - Multi-format ingestion: Support for 15+ document formats (PDF, Office, email, structured data, databases). - Multi-backend evaluation: FAISS Flat / HNSW / PQ, Chroma HNSW, Qdrant float32 and int8 — used for the cross-backend bit-channel survival study. - Multi-model evaluation: OpenAI `text-embedding-3-large` plus four local Ollama embedding models. - Detection battery: Isolation Forest and One-Class SVM detectors used as defenders inside every empirical script. Tech stack: Python. Why it matters: VectorSmuggle provides a valuable tool for security researchers, red and blue team exercises, and academic study of attack and defense in AI/ML pipelines, offering insights into the risks of traditional Data Loss Prevention systems and the effectiveness of cryptographic provenance techniques. Repository: jaschadub/VectorSmuggle GitHub stars: 74 GitHub description: Research framework that quantifies how steganographic obfuscation of embeddings defeats off-the-shelf statistical detection in RAG pipelines, paired with the VectorPin cryptographic-provenance defense.

### Overview: Whisper-CPP-Server is a high-performance speech recognition service written in C++, designed to provide developers and enterprises with a reliable and efficient speech-to-text inference engine. This project implements technology from ggml to perform inference on the open-source Whisper model. It supports pure CPU-based inference operations, ensuring high-quality speech recognition services without the need for specialized hardware accelerators. ### Key Features: 1. Pure C++ Inference Engine 2. High Performance 3. Support for Multiple Languages 4. Docker Container Support 5. Easy Integration for Clients ### Tech Stack: C++ with ggml technology ### Why it Matters: Whisper-CPP-Server is suitable for applications that require fast and accurate speech recognition, including voice-driven interactive applications, transcription of meeting records, automatic subtitle generation, and automatic translation of multi-language content. It is designed to be easily integrated into various client applications and supports deployment using Docker containers, making it accessible on any platform that supports Docker, including Linux, Windows, and macOS.

Overview: Anthropic Laravel is a Laravel integration that provides access to the Anthropic API, including Message Batches, token counting, and fallbacks. It simplifies the use of Claude, the Anthropic companion, by offering a facade and other useful features. Key features: - Facade for the Anthropic API - Message Batches - Token counting - Rate-limit headers - Fallback with fake requests Tech stack: PHP Why it matters: Anthropic Laravel makes it easy to integrate Claude into Laravel projects, providing a streamlined interface and additional features that simplify the use of the Anthropic API. Repository: mozex/anthropic-laravel