Projects

### Overview: This project provides a comprehensive guide on how to utilize diffusion priors under sparse views for image processing and rendering. The code is based on NeurIPS 2024 and includes detailed installation instructions, pre-trained model preparation, data preparation, and training methods for the IPSM-Gaussian model. ### Key features: - **Installation**: Requires Ubuntu 22.04, CUDA 11.3, and PyTorch 1.12.1. - **Pre-trained Models Preparation**: Provides pre-trained models for StableDiffusion-v1.5, StableDiffusionInpainting-v1.5, MiDaS, and BLIP. - **Data Preparation**: Includes instructions for preparing data from LLFF and DTU datasets, with options for COLMAP and BLIP-based text results. - **Training & Rendering**: Offers scripts for training and rendering the IPSM-Gaussian model on the LLFF and DTU datasets with 3 views. - **Acknowledgement**: Acknowledges contributions from projects like gaussian-splatting, FSGS, and DNGaussian. ### Tech stack: Python ### Why it matters: This project is significant for researchers and developers interested in image processing, rendering, and diffusion models, providing a practical guide and code for utilizing diffusion priors under sparse views.

Overview: LLaMa 3.2 Multimodal Web UI is a user-friendly interface designed for interacting with the Ollama platform, supporting text, image, and code inputs, enhancing the accessibility of multimodal AI. Key features: - Multimodal Input Support: Accepts text and image inputs for context-aware responses. - Formatted Responses: Highlights code blocks and detailed explanations. - Copy-to-Clipboard: Easily copy code snippets for seamless integration. - Responsive UI: Optimized for both desktop and mobile devices. - Interactive Output: View text, images, and other media types generated by LLaMa 3.2. Tech stack: PHP Why it matters: This project provides a user-friendly interface for interacting with the Ollama platform, making multimodal AI accessible to all users. Repository: iamgmujtaba/llama3.2-webUI Primary language: PHP GitHub stars: 35 GitHub description: LLaMa 3.2 Multimodal Web UI is a user-friendly interface for interacting with the Ollama platform. README excerpt: LLaMa 3.2 Multimodal Web UI is a user-friendly interface for interacting with the Ollama platform. It effortlessly supports text and image inputs, allowing users to ask questions, submit prompts, and receive responses in text, code, and even visual outputs, making the power of multimodal AI accessible to all.

Overview: H3xAssist is a meeting assistant that integrates modern Python and Next.js, automating the recording, transcription, and summarization of online meetings. Key features: - Automatic meeting join - High-quality audio recording with PipeWire integration - WhisperX-powered speech recognition for transcription - Google Gemini-generated meeting summaries - Automatic Outlook calendar synchronization - Real-time dashboard built with Next.js and responsive design - Mobile-friendly interface optimized for all screen sizes - WebSocket-powered live status updates Tech stack: TypeScript, Python, Next.js, TypeScript, Playwright, FFmpeg, WhisperX, Google Gemini, PipeWire, WebSockets, Taskfile.dev Why it matters: H3xAssist streamlines meeting management by automating the recording, transcription, and summarization processes, enhancing productivity and efficiency in team collaboration. Repository: ih3xcode/h3xassist GitHub stars: 35 GitHub description: Automated meeting assistant with modern Python and Next.js backend, real-time dashboard, and AI-powered summarization.

Overview: This course is designed to teach you how to leverage open-source models available on the Hugging Face Hub for various tasks in NLP, audio, image, and multimodal domains. Key features: - Using the transformers library to build AI applications for NLP, audio, image, and multimodal tasks. - Packaging code into user-friendly apps that can run on the cloud using Gradio and Hugging Face Spaces. - Performing tasks like translation, summarization, similarity measurement, audio transcription, text-to-speech, audio classification, image narration, and object identification. Tech stack: Jupyter Notebook Why it matters: This course provides a practical guide on how to effectively utilize open-source models for various NLP, audio, image, and multimodal tasks, making it a valuable resource for developers looking to enhance their AI applications.

### Overview: Use Google Speech-to-Text API to caption live streams in Unity. This project allows for real-time captioning of live streams, enhancing the virtual character experience. It's particularly useful when combined with a virtual character. ### Key Features: - Real-time live stream captioning - Integration with virtual characters - Customizable connection port - Adjustable text printing settings ### Tech Stack: Python ### Why It Matters: This project is essential for enhancing the virtual character experience in live streams, making the interaction more engaging and accessible to a broader audience. The ability to caption in real-time and adjust settings on the fly makes it particularly valuable for virtual character creators and streamers. ### Repository: kwea123/Unity_live_caption ### GitHub Stars: 35 ### Primary Language: Python ### Important Notice: The speech-to-text API is not free. The pricing guide is available [here](https://cloud.google.com/speech-to-text/pricing). ### YouTube Demo: [YouTube Live Stream](https://www.youtube.com/watch?v=AZsUm_cuj9U) ### Pre-Requisite: - Google Cloud API account - Python 3.6 or higher - `pyaudio` library (download from [here](https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyaudio)) ### Installation: - Ensure Python is installed - Install dependencies via `pip install -r requirements.txt` ### Usage: 1. Test speech recognition in Python: - Change `key.json` location - Run `python goog

### Overview: BInD is an official implementation of the "BInD: Bond and Interaction-Generating Diffusion Model for Multi-Objective Structure-Based Drug Design" research paper. This model is designed to enhance the efficiency and accuracy of structure-based drug design by incorporating bond and interaction information into the diffusion process. ### Key features: - **BInD Model**: A diffusion model that incorporates bond and interaction information to improve drug design. - **Multi-Objective Optimization**: The model aims to optimize multiple objectives simultaneously, enhancing the accuracy and diversity of the design. - **Structure-Based Drug Design**: BInD is specifically tailored for structure-based drug design, focusing on the interactions between molecules and the target. ### Tech stack: Python ### Why it matters: BInD provides a powerful tool for structure-based drug design by leveraging advanced diffusion models and incorporating bond and interaction information, making it highly relevant for researchers and practitioners in the field of drug discovery and design. ### Repository: lee-jwon/BInD ### Primary language: Python ### GitHub stars: 35 ### GitHub description: The official repository for BInD, an implementation of the "BInD: Bond and Interaction-Generating Diffusion Model for Multi-Objective Structure-Based Drug Design" research paper.

### Overview: Weave CLI is a universal command-line tool designed for managing vector databases. It simplifies the process of interacting with popular vector databases like Weaviate, Qdrant, Milvus, Chroma, and others, making it easier to view, list, create, delete, and search collections and documents in collections for development, testing, and debugging purposes. ### Key Features: - **Vector Database Support**: Supports 11 vector databases including Weaviate, Qdrant, Milvus, Chroma, Supabase, Neo4j, MongoDB, Redis, Pinecone, and OpenSearch. - **Embedding Provider Support**: Compatible with 3 embedding providers, including 100% free local options like OpenAI, sentence-transformers, and Ollama. - **Command-Line Interface (CLI)**: Built in Go for performance and ease of use. - **Configuration**: Quick setup options like interactive configuration, health checks, and detailed database info. - **Embedding Model Comparison**: Available models can be listed and compared to determine the best fit for different use cases. ### Tech Stack: Go ### Why It Matters: Weave CLI simplifies the management of vector databases, making it easier for developers to work with various vector databases without needing to learn multiple APIs or configurations. The CLI's fast performance and ease of use make it a valuable tool for developers and researchers alike.

Overview: DuckBoard is a minimal server-rendered dashboard framework built with Node.js, designed for developers who want a simple and clean dashboard without any dependencies. Key features: - Zero dependencies - Server-rendered layout - Widget-based design - Escape-safe rendering Tech stack: Node.js Why it matters: DuckBoard simplifies the development of server-rendered dashboards, making it an ideal choice for developers looking to quickly build and deploy customizable dashboards without the hassle of managing dependencies.

Overview: A Python-based voice assistant designed for seamless interaction and task automation, featuring speech recognition, text-to-speech, and various APIs for web searches, email sending, alarm setting, and more. Key features: - Voice Interaction: Uses `pyttsx3` for text-to-speech and `SpeechRecognition` for voice input. - Natural Language Processing: Integrates `spaCy` for intent recognition. - Web and App Control: Opens websites and applications with voice commands. - Search Capabilities: Performs Google, YouTube, and Wikipedia searches. - Email and Messaging: Sends emails via Gmail SMTP and WhatsApp messages. - System Control: Manages system volume, locks the device, shuts down, and clears the recycle bin. - Productivity Tools: Sets alarms, takes notes, and remembers user inputs stored in text files. - News Updates: Fetches and reads latest news from various categories. - Mathematical Calculations: Solves math queries using Wolfram Alpha API. - Keyboard Shortcuts: Executes browser shortcuts via `pyautogui`. - Extensibility: Modular architecture allows easy addition of new features. Tech stack: Python, `pyttsx3`, `SpeechRecognition`, `spaCy`, `pywhatkit`, `wikipedia`, `pyautogui`, `pywhatkit`, `pyinstaller`, `pyinstaller`, `pyinstaller`, `pyinstaller`, `pyinstaller`, `pyinstaller`, `pyinstaller`, `pyinstaller`, `pyinstaller`, `pyinstaller`, `pyinstaller`, `pyinstaller`, `pyinstaller`, `pyinstaller`, `py

Overview: STACKn is a lightweight, cloud-native machine learning platform designed for collaboration and deployment of machine learning models and applications on Kubernetes. Key features: - Project-based multi-tenancy - Data science tools integration (Django framework) - Storage management - Environment configuration - Notebook support - Experiment tracking - Model registry and serving - Model catalog - Dashboards / apps deployment Tech stack: Python, Django, Kubernetes, Helm charts Why it matters: STACKn simplifies the process of deploying and managing machine learning models and applications in a cloud-native environment, making it accessible to data scientists and developers without requiring extensive technical knowledge. GitHub stars: 35 Repository: scaleoutsystems/stackn

Overview: Skeg is a multi-tenant vector database optimized for extreme RAM efficiency, designed for high-density deployments. Key features: - Tenants isolated by construction, vectors and key-value in one process - Lightweight, scalable, and optimized for high-density deployments - Recall 1.000, without the corpus in RAM Tech stack: Rust Why it matters: Skeg efficiently manages vector databases, ensuring that vector data is kept on SSD and only a small quantized working set sits in RAM, leading to far slower memory growth compared to serving the corpus.

### Overview: This repository contains re-measured speculative decoding results for the Qwen3.6-35B-A3B UD-Q4_K_XL model on a single RTX 3090. The data was collected and re-derived from the committed commit 3737e4137, ensuring accuracy and reliability. ### Key Features: - Re-measured speculative decoding results. - Detailed coverage reports on how many of the re-measured results are covered by the checker. - A list of retracted claims in the ERRATA.md file, including the "100% draft acceptance yet slower" retraction. ### Tech Stack: - Python for data processing and analysis. ### Why It Matters: This study provides a re-measured benchmark for speculative decoding, offering insights into the performance and accuracy of the model under different conditions. The findings are crucial for developers and researchers looking to optimize and improve the performance of large language models.

Overview: Haas-ASR is an inference code for Haas speech recognition models, designed for fast and accurate Japanese speech recognition. The model achieves a weighted CER strict of 6.12% on the asr-ja-bench v1.1, making it competitive with commercial APIs. Key features: - Fast and accurate Japanese speech recognition - Compatible with Linux with NVIDIA GPU support - Supports Python 3.11+ and NeMo 2.7 - Transcribes audio in real-time Tech stack: Python Why it matters: Haas-ASR is a valuable tool for developers looking to integrate fast and accurate Japanese speech recognition into their projects, offering comparable performance to commercial APIs while being more cost-effective. GitHub stars: 32 Repository: third-intelligence/Haas-ASR

### Overview: Cooking AGI is a comprehensive resource for anyone looking to learn about Artificial Intelligence (AI) and Machine Learning (ML). This bilingual (English / Chinese) guide includes worked examples, step-by-step figures, and coding practice, making it an ideal tool for both beginners and advanced learners. The site covers foundational concepts, interview preparation, engineering practice, and tech history, providing a holistic view of AI and ML. ### Key Features: - **Foundations:** How models work, learn, and are evaluated. - **Interview Prep:** ML / LLM review, ML coding, Python, and algorithm patterns. - **Engineering Practice:** Recommendation, RAG, and post-training systems, end to end. - **Tech History:** Power, chips, networks, software, and AI, with original sources. - **Career Notes:** What I tried in internship and new-grad searches. ### Tech Stack: Python ### Why It Matters: Free to read, this resource offers a deep dive into AI and ML, covering everything from foundational concepts to advanced engineering practices. Whether you're a beginner looking to learn the basics or an experienced developer seeking to enhance your skills, Cooking AGI provides valuable insights and practical examples.

### Overview: Hark is an Android application designed for hearing-impaired users, providing an on-device wake word detection feature. It uses PocketSphinx to detect when a configured word is spoken, triggering a heads-up notification immediately, even when the screen is off or the device is in airplane mode. ### Key Features: - **Offline Wake-Word Detection**: Detects wake words on the device without any network calls. - **No Network Calls**: The application runs entirely on the device, ensuring privacy and security. - **Accessibility Feature**: Alerts users when they are addressed, particularly useful for hearing-impaired individuals. ### Tech Stack: - **Language**: Kotlin - **UI**: Jetpack Compose + Material 3 - **Architecture**: MVVM, `ViewModel` + `StateFlow` - **DI**: Hilt - **Persistence**: DataStore ### Why It Matters: Hark is a significant improvement over cloud-based wake-word detection, offering a more reliable and accessible solution for users who rely on hearing aids or cochlear implants. It ensures that users can be alerted without the need for an internet connection, enhancing their independence and quality of life. ### GitHub Stars: 35 ### Repository: tuckercr/wakewordapp ### Primary Language: Kotlin ### Tech Stack: Kotlin, Jetpack Compose + Material 3, MVVM, Hilt, DataStore

### Overview: This repository contains an end-to-end Automatic Speech Recognition project developed by Victor369Basu. The project utilizes PyTorch for the neural network model and MLflow to manage the ML lifecycle, including experimentation, reproducibility, deployment, and a central model registry. ### Key Features: - **Neural Network Model:** The project employs a neural network architecture consisting of Residual-CNN blocks, BidirectionalGRU blocks, and fully connected layers. - **MLflow Integration:** MLflow is used to track and compare model performance, manage experiments, and deploy models. - **LibriSpeech Dataset:** The project utilizes the LibriSpeech dataset for training and validation, including a subset of 2000 files for faster training and validation. - **Feature Extraction:** Feature extraction is performed using Mel-frequency cepstrum coefficients (MFCC) and other techniques like MelSpectogram and Frequency Masking. - **Acoustic Model:** The model architecture includes Residual-CNN blocks, BidirectionalGRU blocks, and fully connected layers for classification. - **Loss Function:** The CTC (Connectionist Temporal Classification) loss function is used as the base loss function. - **Decoding:** Greedy Decoder is used for text decoding. - **ML Lifecycle Pipeline:** The project follows an ML lifecycle pipeline that includes initializing the MLflow server, creating experiments, starting runs, saving models, and registering them for deployment. ### Tech Stack: - **MLflow:** Used for managing the ML lifecycle, tracking, and comparing model performance. - **Pytorch:**

### Overview: AnySD is an official repository for the implementation and benchmark evaluation of AnyEdit, a unified high-quality image editing model that can be applied to various tasks. ### Key features: - AnySD: The official implementation and benchmark evaluation repository for AnyEdit. - AnyBench: A guide for evaluating AnyEdit using the AnyBench tool. - AnySD: The model implementation and benchmark evaluation repository for AnyEdit. ### Tech stack: Python ### Why it matters: AnySD serves as a comprehensive resource for developers and researchers interested in image editing and model benchmarking, providing both the model implementation and evaluation tools.

### Overview: 基于 Gitee AI 的免费 2K 图片生成器，提供 18000+ AI 提示词库和 12 种预设风格，每天可生成 100 张免费 2K 高清图片。内置快速生成功能、完全免费、支持多尺寸适配和提示词库搜索。 ### Key features: - 极速生成：基于 Z-Image-Turbo 模型，8 步快速生成 2K 高清图片。 - 完全免费：每天 100 张免费生成额度，2K 分辨率。 - 风格预设：12 种预设风格（公众号/小红书配图）。 - 多尺寸支持：1:1 / 4:3 / 3:4 / 3:2 / 2:3 / 16:9 / 9:16 七种比例，适配公众号、小红书等平台。 - 提示词库：内置 18000+ 精选 AI 提示词，按用途 / 风格 / 主体分类，支持关键词搜索。 - 历史记录：自动保存生成记录，支持单条删除、一键重新下载。 - 图床上传：一键把生成图上传到 GitHub 图床，外链自动复制，换设备不丢图。 - 隐私安全：API Key 仅保存在本地浏览器，不会上传

### Overview: ReNeg is a reward-guided approach that directly learns negative embeddings for text-to-image and text-to-video models, enhancing their visual output with strong generalization capabilities. ### Key Features: - **Re**ward-guided learning for negative embeddings. - **Neg**ative embeddings exhibit strong generalization capabilities. - Seamlessly adaptable to text-to-image and text-to-video models. - Highly effective yet simple, enhancing the visual appeal of outputs from base Stable Diffusion models. ### Tech Stack: Python ### Why it Matters: ReNeg amplifies the visual appeal of outputs from base Stable Diffusion models, making them more engaging and visually appealing.

### Overview: This project is a native inference provider plugin for the Hermes Agent, integrating Google Antigravity and Cloud Code Assist, enabling state-of-the-art Gemini and Claude models with full streaming support and native tool calling. ### Key features: - **Google Antigravity / Cloud Code Assist Integration**: Seamless integration of Google's Antigravity and Cloud Code Assist into the Hermes Agent. - **Streaming Support**: Full streaming support for Gemini and Claude models. - **Native Tool Calling**: Native tool calling capabilities. - **Reasoning/Thinking Tokens**: Includes reasoning and thinking tokens for enhanced functionality. ### Tech stack: Python ### Why it matters: This plugin solves the problem of authenticating with Google account via OAuth 2.0 PKCE, allowing subscription holders to use their quota without burning paid API tokens, thus unlocking the full potential of their subscription models in Hermes Agent.