Projects

### Overview: Jev Voice is a small floating bar for macOS that allows you to give voice and text commands to your Mac. It picks the next action from your Mac's live controls, executes it, and continuously loops until your request is fulfilled. ### Key features: - **Voice Command:** Speak a full request, pause, and give another. Commands queue while work is in progress. - **Text Command:** Click the bar, enter a command, and press Return. Text commands work with the mic off. - **Multiple Steps:** Each action is followed by a fresh observation and another Jev decision. Options come from the current interface and installed apps. - **Live Feedback:** The bar shows your command and current action. Settings → Jev activity shows the model’s inputs, choices, errors, and observed results. ### Tech stack: Swift ### Why it matters: Jev Voice simplifies the interaction with your Mac by providing a continuous loop of commands and actions, making it easier to manage tasks and workflows.

Overview: MemVanta is a low-memory C++20 LLM inference runtime designed for quantized GGUF models on CPUs with limited RAM. It uses mmap-backed weights, paged KV cache, Q4/Q8 kernels, and reproducible benchmarks against pinned `llama.cpp`. Key features: - Low-memory C++20 LLM inference runtime - Quantized GGUF models - mmap-backed model access - paged KV cache - Q4/Q8 kernels - Reproducible memory and throughput benchmarks Tech stack: C++ Why it matters: MemVanta explores the trade-off between resident memory and token throughput for CPU LLM inference, aiming to minimize memory usage while maintaining model correctness. This makes it suitable for experiments where RAM pressure is more critical than maximum token throughput, such as constrained developer machines, edge systems, and research into memory-aware local inference.

Overview: This project provides a JavaScript-based speech analyzer that can be used for labeling, feature extraction, and training neural networks for speech and music analysis. Key features: - Speech and music analysis - Feature extraction using formant analysis - Training neural networks with ml5js for speech emotion recognition - Support for different output modes (Segment Features, Syllable Features, etc.) Tech stack: JavaScript Why it matters: This tool offers a powerful and flexible solution for speech analysis and classification, making it useful for researchers, developers, and anyone interested in speech technology. Repository: tabahi/WebSpeechAnalyzer GitHub stars: 40 GitHub description: JS speech analyzer for fast speech analysis and labeling

Overview: Voce Browser is a Chromium-based voice-controlled web browser that integrates PyQtWebEngine for enhanced functionality and supports voice commands for various browser actions. Key features: - Supports voice commands for opening tabs, closing tabs, opening windows, closing windows, searching, navigating back and forward, going home, reloading pages, and exiting the browser. - Utilizes PyQtWebEngine for Chromium-based browsing. - Integrates Google Speech Recognition for voice command translation and execution. Tech stack: Python, PyQt5, PyQtWebEngine, SpeechRecognition, gTTS, PyAudio, Pydub Why it matters: Voce Browser offers a unique and convenient way to interact with the internet using voice commands, making browsing more accessible and user-friendly.

### Overview: Real-time Japanese speech recognition translator using Wav2Vec2 and Google Translate. This project leverages a finetuned version of Facebook's Wav2Vec2 models to translate Japanese speech into text in real-time. ### Key Features: - Real-time Japanese speech recognition and translation. - Uses finetuned Facebook Wav2Vec2 models for Japanese hiragana. - Detects speaker using WASAPI for output loopback and microphone using MME. - Fine-tuned on Common Voice Japanese test data with various datasets. ### Tech Stack: - Python - PyTorch - Transformers - Jupyter Notebook ### Why It Matters: This project offers a practical solution for real-time Japanese speech translation, which can be particularly useful in applications such as live captioning, voice-activated assistants, or language learning tools. The integration with Wav2Vec2 and Google Translate ensures high accuracy and real-time performance. ### Repository: - Repository: ttop32/wav2vec2-live-japanese-translator - GitHub Stars: 40 - GitHub Description: Real time Japanese speech recognition translator using Wav2Vec2 and Google Translate ### Required Environment: - Python 3.6 - Install necessary libraries: - `conda create -n torch python=3.6` - `conda activate torch` - `conda install pytorch==1.9.1 torchvision torchaudio==0.9.0 cudatoolkit=11.1 -c pytorch -c nvidia -c conda

### Overview: VietGPT VoiceBot is a Vietnamese speech recognition chatbot that utilizes the ChatGPT API for natural language interaction. ### Key features: - Automatic Vietnamese voice recognition - Integration with the ChatGPT API for seamless conversation - Record audio using PyAudio - Transcribe speech using PhoWhisper's transformers library - Connect with OpenAI for chat functionality - Convert text to speech using gTTS ### Tech stack: Python ### Why it matters: VietGPT VoiceBot is a powerful tool for Vietnamese language users, offering a natural and interactive experience through voice commands and text-to-speech capabilities.

### Overview: Cottontail DB is a column store vector database designed for multimedia retrieval, offering both classical boolean and vector-space retrieval methods. It simplifies the process of building and using a database system for efficient similarity search. ### Key Features: - **Vector Space Retrieval**: Supports nearest-neighbors lookup for multimedia data. - **Unified Data and Query Model**: Allows for a single model to handle both boolean and vector-based queries. - **Open Source**: Available as an open-source project with a comprehensive documentation and community support. ### Tech Stack: Kotlin ### Why it Matters: Cottontail DB provides a flexible and efficient solution for multimedia retrieval, making it suitable for applications requiring fast and accurate similarity searches. ### Repository: vitrivr/cottontaildb ### Primary Language: Kotlin ### GitHub Stars: 40 ### GitHub Description: Cottontail DB is a column store aimed at multimedia retrieval. It allows for classical boolean as well as vector-space retrieval, i.e., nearest-neighbours lookup, as used in similarity search.

Overview: mini-dalle is a lightweight implementation of the DALL·E model in OCaml, designed for developers who need a simple and efficient image generation tool. Key features: - Image generation capabilities - Lightweight and fast performance - Compatible with OCaml ecosystem Tech stack: OCaml Why it matters: mini-dalle provides a practical solution for developers looking to integrate advanced image generation capabilities into their projects without the need for extensive machine learning expertise.

### Overview: Chat Selfie is an AI-driven platform that transforms conversations into expressive self-portraits, enhancing the emotional connection between users and their AI agents. ### Key Features: - **Visualized Emotion:** AI agents generate fresh self-portraits or use mood-mapped stickers to reflect the conversation's mood. - **Heartbeat Presence:** Agents can share their current status through heartbeat pushes, indicating they are present even when silent. - **Self-Evolving Persona:** The agent continuously adapts its description to better match the bond between users, making the generated images more aligned with the relationship. - **Works Everywhere:** Compatible with various image stacks, APIs, and local models, ensuring versatility. ### Tech Stack: Python ### Why It Matters: Chat Selfie breaks the traditional boundaries between AI and humans, making interactions more emotionally engaging and personal. It transforms AI into a partner that you miss, enhancing the emotional connection and making conversations more meaningful and enjoyable.

Overview: This project provides a comprehensive guide for optimizing and deploying an LLM model to handle 100,000 parallel queries, focusing on weight quantization, caching, and Kubernetes deployment. Key features: - Weight quantization (e.g., W4A16, W4A8) - KV caching - Speculative decoding - Kubernetes deployment Tech stack: Jupyter Notebook Why it matters: This guide is essential for developers looking to optimize and efficiently host large-scale language models, ensuring high performance and scalability for applications requiring millions of parallel queries.

### Overview: AniStudio is an extendable generative media application built with C++ using ImGui, OpenCV, and ggml libraries. It supports a wide range of models for image, video, and edit diffusion, including SD1.x, SD2.x, SDXL, SDXL-Turbo, SD3/SD3.5, Flux-dev/Flux-schnell, Chroma, Wan2.1/Wan2.2, and PhotoMaker. The application features an Entity Component System, customizable views, cross-platform support, and optimized memory usage with Flash Attention and AVX/AVX2/AVX512 acceleration. ### Key Features: - Entity Component System for modular development - Dockable and customizable views using Dear ImGui - Cross-platform support for Windows and Linux - Standard media project management - Built-in UI editor for ease of development - Supported models: Image Models (SD1.x, SD2.x, SDXL, SDXL-Turbo, SD3/SD3.5, Flux-dev/Flux-schnell, Chroma, Wan2.1/Wan2.2), Video Models (Wan2.1/Wan2.2), Image Edit Models (FLUX.1-Kontext-dev), and PhotoMaker support - 16-bit, 32-bit float support, 2-bit, 3-bit, 4-bit, 5-bit, and 8-bit integer quantization support - Accelerated memory-efficient CPU inference - Flash Attention for memory usage optimization

Overview: CDGS is a conditional diffusion model designed for generating molecular graphs, specifically focusing on the QM9 and ZINC250k datasets. This project extends the original CDGS model to incorporate joint 2D and 3D diffusion models, making it suitable for complete molecule generation. Key features: - Conditional Diffusion Model for Molecular Graph Generation - Joint 2D & 3D Diffusion Models for Improved Generation - Pretrained Checkpoints Available for QM9 and ZINC250k Datasets Tech stack: The project uses PyTorch and PyG for the implementation, with dependencies including PyTorch 1.11 and PyG 2.1. Why it matters: CDGS offers a more comprehensive approach to molecular graph generation, improving the quality and diversity of generated molecules compared to traditional methods. Its ability to handle both 2D and 3D representations makes it particularly effective for complex molecule generation tasks. Repository: GRAPH-0/CDGS Primary language: Python GitHub stars: 39 GitHub description: Conditional Diffusion Based on Discrete Graph Structures for Molecular Graph Generation GitHub link: [https://github.com/GRAPH-0/CDGS](https://github.com/GRAPH-0/CDGS)

### Overview: LabLLM is a free, native macOS app that streamlines the entire process of training and fine-tuning small language models. It allows users to build a Transformer from scratch, train models on their own datasets, leverage Apple Silicon and MLX for local training, and monitor live training metrics and loss curves. The app also supports checkpoint saving and resuming, fine-tuning with LoRA and DPO, inspecting tokens and embeddings, and chatting with the model. ### Key Features: - **Build Transformer from Scratch:** Start with a blank slate and build your own Transformer. - **Train on Custom Datasets:** Use your own datasets to train the model. - **Local Training on Apple Silicon:** Train models locally with MLX acceleration for faster performance. - **Live Metrics and Loss Curves:** Monitor training progress in real-time. - **Save and Resume Checkpoints:** Save checkpoints and resume training from any point. - **Fine-Tuning with LoRA and DPO:** Fine-tune the model using LoRA and DPO for better performance. - **Inspect Tokens and Embeddings:** Analyze the model's tokens and embeddings. - **Chat with the Model:** Engage with the trained model for interactive sessions. - **Model Management:** Manage models, datasets, and experiments seamlessly. ### Tech Stack: Swift ### Why It Matters: Training a small language model should be a hands-on experience rather than a complex task. LabLLM simplifies the process, making it accessible and straightforward. It allows users to explore and experiment with language models directly, without the

Overview: This repository contains code for the paper "Implicit Search via Discrete Diffusion: A Study on Chess," which proposes a novel method called DiffuSearch to enhance the planning abilities of Large Language Models without relying on explicit search methods. Key features: - DiffuSearch: A novel method to enhance the planning abilities of Large Language Models. - By leveraging discrete diffusion modeling, DiffuSearch enables implicit search through future state prediction. - Demonstrated in the domain of chess. Tech stack: Python Why it matters: The paper presents a significant improvement in the planning abilities of Large Language Models, showcasing the potential of implicit search methods in various domains, including chess.

Overview: This repository provides comprehensive tutorials, examples, and production-ready code for building powerful AI agents using LLMs and multi-agent systems. Key features: - HuggingFace’s Agent Course - MCP with Anthropic - Building Vector Databases with Pinecone - Vector DBs: From Embeddings to Apps - Agent Memory - RAG Apps: Build & Evaluate - Browser Agents - LLMOps - Evaluating AI Agents - Computer Use with Anthropic - Practical Multi-Agent Use - Improving LLM Accuracy - Agent Design Patterns - Multi-Agent Systems Tech stack: Python, JavaScript, HTML, CSS, Markdown, and various LLMs and multi-agent frameworks. Why it matters: This repository offers essential resources for developers looking to build intelligent automation systems, focusing on advanced AI techniques and practical applications.

### Overview: This repository provides training scripts for fine-tuning HunyuanImage-3.0 models, a 80B unified understanding and generation model. Designed to work with the official model at the provided GitHub link, it enables the community to experiment with and customize this state-of-the-art model. ### Key features: - Training scripts for fine-tuning HunyuanImage-3.0 models. - Comprehensive pipeline built on Hugging Face Trainer, DeepSpeed, and PEFT for efficient experimentation and customization. - Quick start guide available on the project page for easy installation and setup. ### Tech stack: Python ### Why it matters: This repository empowers researchers and developers to experiment with and fine-tune the HunyuanImage-3.0 model, contributing to the advancement of multimodal AI.

Overview: This project aims to bring up and verify the first-generation AMD XDNA1 (Phoenix/Hawk Point) NPU on Linux, enabling the execution of models on the silicon itself. The project is open-source and fully based on 100% open source technologies, including XRT, driver, and firmware. Key features: - Custom vector kernels hand-authored in the AIE ISA - 100% open-source implementation - 512³ int16 matmul on the NPU - Bit-exact forward pass of a real model on the NPU - Non-bijunctive collapse (keep-strong / prune-weak) - Hand-authored AIE vector kernel - Fused `reduce_max`→dynamic-τ collapse in one kernel - `aie::reverse` (vec_perm / shuffle) on the NPU - Full collapse (reduce+threshold+compact) in one AIE kernel - Measured FFN net-win (NPU prune → dense down_proj) - Attention KV-prune (NPU evicts low-mass keys) - Vision pipeline (rgba2gray→3×3 conv→threshold) on the NPU - SigLIP/ViT patch-embed wired on the NPU - Quick start guide for running the project Tech stack: Shell Why it matters: This project provides a practical solution for running large language models (LLMs) on AMD's XDNA1 NPU, which is otherwise unsupported on Linux. By bringing up the NPU on Linux and verifying its

### Overview: This repository contains the official implementation of the paper "Interaction-based Retrieval-augmented Diffusion Models for Protein-specific 3D Molecule Generation" presented at ICML 2024. The project utilizes Python as the primary language and includes pre-trained models, data preparation, training, sampling, and evaluation scripts. ### Key features: - Pre-trained models for protein-specific 3D molecule generation - Interaction-based retrieval augmentation for diffusion models - Comprehensive documentation and scripts for reproducibility ### Tech stack: Python ### Why it matters: This project provides a practical implementation of the proposed method, making it accessible for researchers and developers interested in protein structure generation and molecular modeling. ### Repository: YangLing0818/IRDiff - Stars: 39 - Description: Official implementation for ICML 2024 paper - "Interaction-based Retrieval-augmented Diffusion Models for Protein-specific 3D Molecule Generation"

### Overview: This repository contains 8 tested recipes for Grok Imagine Video 1.5 prompts, showcasing various aspects of image and video generation. The prompts are designed to be effective and cost-efficient, suitable for both beginners and advanced users. ### Key Features: - **Model ID:** `grok-imagine-video-1.5` - **Cost per Image:** $0.064 per second - **Duration:** 5 seconds - **Prompt Structure:** The recipes are structured to include the subject, light, framing, setting, and finish, ensuring clear and effective prompts. ### Tech Stack: - Python ### Why It Matters: - **Cost Efficiency:** The cost per image is relatively low, making it an attractive option for users on a budget. - **Versatility:** The model supports a wide range of image and video prompts, catering to various creative and practical needs. - **User-Friendly:** The prompt structure is designed to be intuitive, making it easy for users to generate effective and cost-effective prompts. ### Repository: - **Repository:** a1udn/grok-imagine-1-5-video-prompts-pt - **GitHub Stars:** 39 - **GitHub Description:** Grok Imagine Video 1.5 prompts: tested image prompt recipes with example renders, aspect ratios, the model id (`grok-imagine-video-1.5`) and the cost per image. ### Primary Language: - **Primary Language:** Python ### GitHub Description: - **Description:** Grok Imagine Video 1.

### Overview: Audiolizr is a Python-based API powered by BentoML designed to transcribe audio and extract metadata such as keywords, topics, and named entities from YouTube videos. This service is deployed on AWS EC2 with a GPU instance for efficient processing, making it suitable for real-time applications. ### Key features: - Transcription of audio content - Extraction of keywords and topics using Yake algorithm - Generation of summaries using the T5 Transformer model - Extraction of named entities using spaCy ### Tech stack: Python ### Why it matters: This API provides valuable insights into video content, allowing users to understand the main points, key topics, and entities discussed in a YouTube video. It can be particularly useful for businesses looking to improve workplace transparency and fairness, or for anyone interested in gaining deeper understanding from video content.