Projects

Overview: This project serves as an umbrella for multiple implementations of model serving, encompassing various frameworks like Akka, Flink, Kafka, and Spark. Key features: - Akka Streams and Akka HTTP for Akkaserver - Flink for Flinkserver - Kafka Streams for Kafkaclient and Kafkaconfiguration - PMML and TensorFlow models for serving samples - Protobufs for all streaming frameworks - SparkML for machine learning and exporting results to PMML - Spark server for SparkML Tech stack: Scala Why it matters: This project provides a comprehensive solution for model serving across different frameworks and languages, making it a valuable resource for developers working on model serving systems.

Overview: Word Teacher AI 英语口语训练是一款通过 AI 对话、跟读练习、单词游戏、语音识别和智能评分的高效 Web 应用，旨在帮助英语学习者提高口语能力。 Key features: - AI 对话练习 - 英语跟读练习 - 单词游戏 - 语音识别 - 智能评分系统 Tech stack: TypeScript Why it matters: Word Teacher AI 英语口语训练提供了一个全面的解决方案，帮助英语学习者通过沉浸式对话和即时反馈提高口语技能，同时通过游戏化学习方法增加学习的趣味性和动力。 Repository: vancece/word-teacher Primary language: TypeScript GitHub stars: 46 GitHub description: Efficient AI English Learning: Read & Speak via Web | 英语学习，英语单词，通过 AI 学英语朗读，对话的高效 Web 应用 README excerpt: # Word Teacher AI 英语口语训练 **AI 英语口语训练 | AI English Speaking Practice** [English](README_EN.md) | 中文 通过 **AI 对话 + 跟读练习 + 单词游戏 + 语音识别 + 智能评分**，让英语口语练习更轻松高效。 Practice English speaking with **AI conversations + read-aloud + word games + speech recognition + smart scoring**. ## 📸 产品截图 学生登录  场景选择  AI 对话

### Overview: A PyTorch implementation of diffusion models built from scratch, featuring a clean, modular codebase, sampling methods including DDPM, DDIM, and more, and support for datasets like MNIST and CIFAR-10. ### Key Features: - 🎯 From-Scratch Implementation - 🚀 Sampling Methods: DDPM, DDIM, DiT, UViT - 🧠 Network Architectures: UNet, DiT, UViT - 📊 Datasets Support: MNIST, CIFAR-10 ### Tech Stack: Python ### Why it Matters: This project offers a comprehensive and detailed implementation of diffusion models, providing a solid foundation for further research and experimentation in the field of diffusion models and related techniques.

### Overview: Voice2Action is a Unity package designed to leverage Large Language Models (LLMs) for efficient real-time interaction in virtual reality environments. It integrates an "Expand" interaction technique to create innovative selection and manipulation mechanisms, aiming to enhance user experiences. ### Key Features: - **Voice2Action Unity Package**: A package that utilizes LLMs for analyzing user instructions and executing tasks through action and entity extraction. - **Embedding-Based Retrieval System**: Implemented in version 0.0.2, this system improves the accuracy and efficiency of task execution. - **"Expand" Interaction Technique**: This technique divides execution tasks into canonical interaction subsets, preventing errors from game engine feedback. ### Tech Stack: - **C#** - The primary programming language used. ### Why It Matters: - **Efficiency**: Voice2Action is highly efficient, with a cost of approximately 0.16 cents per interaction, making it cost-effective for real-time interactions. - **Customization**: It is highly customizable, allowing users to inherit runtime scripts for easy domain-specific manipulation. - **Scalability**: The package is designed to be scalable, with all property classes performing their tasks in parallel, making it highly parallelizable. ### Repository: The repository is hosted at [yang-su2000/Voice2Action](https://yang-su2000.github.io/Voice2Action/). It has been starred by 46 developers, indicating its growing interest and use in the community. ### Demos: - **Terrain Generation**: Demonstrates the ability to generate

Overview: This repository provides a comprehensive implementation of the BGE-M3 multilingual embedding model and tokenizer, converting it to ONNX format for use in multiple programming languages including C#, Java, and Python. The model supports dense, sparse, and ColBERT vector types, offering reduced latency and full control over the embedding pipeline. It is designed for offline use and cross-platform compatibility, with CUDA GPU acceleration for improved performance. Key features: - Generates all three BGE-M3 embedding types: dense, sparse, and ColBERT vectors - Reduced latency with local embedding generation - Full control over the embedding pipeline with no external dependencies - Works offline without internet connectivity requirements - Cross-platform compatibility (C#, Java, Python) - CUDA GPU acceleration support Tech stack: Jupyter Notebook Why it matters: This repository demonstrates the conversion of a complex model to ONNX format, showcasing how to integrate it into multiple programming languages and highlighting the benefits of reduced latency and full control over the embedding pipeline, making it ideal for applications requiring high performance and offline functionality. GitHub stars: 45

### Overview: This project introduces a modified version of CLIP (Contrastive Language-Image Pre-Training) that addresses the modality gap issue, enabling zero-shot accuracy and text encoders for generative AI applications. The key features include the addition of registers and gated MLPs, resulting in a model with +20 million parameters and a significantly reduced modality gap. ### Key Features: - **Registers and Gated MLPs**: Introduce registers and gated MLPs to improve the modality gap. - **+20M Params**: The model has 20 million more parameters than the original CLIP, making it more powerful. - **Zero-Shot Accuracy**: Achieves zero-shot accuracy comparable to or better than the original CLIP. - **Text Encoders for Generative AI**: Provides text encoders for applications like Text-to-Image and Text-to-Video. ### Tech Stack: The project uses Python for the implementation and utilizes the Hugging Face Transformers library for fine-tuning and evaluation. ### Why It Matters: This modification is crucial for applications requiring zero-shot capabilities and improved text encoder performance, making it highly relevant for researchers and developers working on generative AI and vision transformers.

### Overview: Helios Engine is a powerful and flexible Rust framework designed for building LLM-powered agents with tool support, chat capabilities, and easy configuration management. It allows users to create intelligent agents that can interact with users, call tools, and maintain conversation context. ### Key Features: - **Agent System**: Create multiple agents with different personalities and capabilities. - **Tool Builder**: Simplified tool creation with a builder pattern. - **Tool Registry**: Extensible tool system for adding custom functionality. - **Extensive Tool Suite**: 16+ built-in tools including web scraping, JSON parsing, timestamp operations, file I/O, shell commands, HTTP requests, system info, and text processing. - **RAG System**: Retrieval-Augmented Generation with vector stores (InMemory and Qdrant). - **Custom Endpoints**: Ultra-simple API for adding custom HTTP endpoints to your agent server. - **Streaming Support**: True real-time response streaming for both remote and local models with immediate token delivery. - **Local Model Support**: Run local models offline using llama.cpp with HuggingFace integration (optional `local` feature). - **HTTP Server & API**: Expose OpenAI-compatible API endpoints with full parameter support. - **Dual Mode Support**: Auto, online (remote API), and offline (local) modes. - **CLI & Library**: Use as both a command-line tool and a Rust library crate. - **Feature Flags**: Optional `local` feature for offline model support. - **Improved Syntax**: Cleaner, more ergonomic API for adding multiple tools and agents. ### Tech Stack

Overview: This project is a virtual assistant designed for visually impaired individuals, featuring models for face recognition, object detection, text-to-speech, and speech recognition. Key features: - Face recognition - Object detection - Text-to-speech - Speech recognition Tech stack: Python Why it matters: This project aims to provide a practical solution for visually impaired individuals by offering essential functionalities that enhance their daily lives. Repository: Deimos-M/DL-Virtual-Assistant Primary language: Python GitHub stars: 45

Overview: This project presents a novel score-based diffusion method for 6D object pose estimation, specifically designed for handling pose ambiguity in single RGB images. The method is applied to the SE(3) group and is the first application of diffusion models to SE(3) within the image domain. Extensive evaluations demonstrate its effectiveness in mitigating perspective-induced ambiguity and improving pose estimation accuracy. Key features: - Novel score-based diffusion method for SE(3) - First application of diffusion models to SE(3) - Mitigates perspective-induced ambiguity - Improves pose estimation accuracy - Extensive evaluations Tech stack: C++ Why it matters: This project introduces a promising strategy for 6D object pose estimation, particularly addressing pose ambiguity in single RGB images, which is a significant challenge in computer vision. The method's ability to handle pose ambiguity and improve computational efficiency makes it a valuable contribution to the field.

### Overview: Sustain-LC is a benchmarking environment designed for evaluating control strategies in high-performance computing (HPC) data centers. It supports traditional and reinforcement learning (RL) based control methods, including LLM-based control, and enables researchers, engineers, and practitioners to develop and test their control algorithms efficiently. ### Key Features: - **End-to-End Customizable Control**: The environment supports customizable, scalable, and multi-agent control setups. - **Real-Time Control Interfaces**: It provides real-time control interfaces via Gymnasium and ASHRAE G36-based traditional controllers. - **Granular Control**: Agents can control inlet setpoints, pump flow, and valve positions. - **Hybrid and Interpretable Policies**: The environment supports hybrid and interpretable policy support using decision tree distillation techniques. - **Ablation Studies and Scalability Testing**: Built-in tools for ablation studies and scalability testing. - **LLM-Based Explainability**: The environment supports LLM-based explainability. ### Tech Stack: - Python 3.10+ - Conda - PyFMI ### Why It Matters: Sustain-LC is ideal for researchers, engineers, and practitioners in RL, control systems, and sustainable computing infrastructure. It provides a comprehensive benchmarking environment for evaluating and testing control strategies in HPC data centers, ensuring energy-efficient cooling optimization across fine-grained server Blade-Groups, Cooling Towers (CT), and Heat Reuse Units (HRU). ### Repository: Repository: [HewlettPackard/sustain-lc](https://

Overview: MME-Finance is a comprehensive bilingual multimodal benchmark designed specifically for financial analysis, featuring 1,171 English and 1,103 Chinese open-ended questions covering diverse financial image types and various multimodal capabilities. It serves as the first benchmark for evaluating large financial multimodal models. Key features: - 1,171 English open-ended questions - 1,103 Chinese open-ended questions - Covers diverse financial image types - Various multimodal capabilities Tech stack: Python Why it matters: MME-Finance provides a valuable benchmark for evaluating large financial multimodal models, offering insights into the strengths and shortcomings of current models in financial applications.

### Overview: YouTube MCP Server is an AI-powered solution designed to revolutionize your YouTube experience. It empowers users to search for YouTube videos, retrieve detailed transcripts, and perform semantic searches over video content—all without relying on the official API. By integrating with a vector database, this server streamlines content discovery. ### Key features: - Search YouTube videos without using the official API - Retrieve video transcripts - Store video information and transcripts in a vector database - Perform semantic search over stored video transcripts ### Tech stack: Python ### Why it matters: This server provides a powerful toolset for users to access and search YouTube videos, transcripts, and metadata, all without the need for the official YouTube API. It leverages vector databases to enhance search capabilities, making it an invaluable resource for anyone looking to explore YouTube content efficiently.

Overview: This curated index of AI tools and models is designed for developers, data scientists, researchers, students, and AI practitioners seeking a structured starting point. It includes open-source projects with consistent categories and short descriptions, helping you discover, compare, and select tools for specific tasks. Key features: - Open-source projects prioritized - Consistent categories for easy navigation - Direct links to tools and descriptions - Annotations with license and pricing badges Tech stack: Python, JavaScript, and other relevant technologies. Why it matters: This index serves as a map for effective use of AI tools, providing a structured starting point for developers and AI practitioners. It helps in understanding the capabilities and limitations of various tools, making informed decisions.

Overview: NEKO is an open-source project aimed at building a "generalist" model capable of handling a wide range of tasks, including image, text, reinforcement learning, and robotics, inspired by the GATO model from DeepMind. The project is currently in progress and focuses on implementing a multimodal, multiobjective model that can excel across various domains. Key features: - Image processing - Text generation - Reinforcement learning - Robotics tasks Tech stack: Python Why it matters: NEKO represents a significant step towards developing a more versatile and capable AI model, capable of handling a variety of tasks and domains, which could have applications in fields such as robotics, gaming, and more. Repository: ManifoldRG/NEKO GitHub stars: 45 GitHub description: Implementation of GATO style Generalist Multimodal model capable of image, text, RL and Robotics tasks GitHub URL: https://github.com/ManifoldRG/NEKO

### Overview: OpenLayer is a free, open-source plugin for Adobe Photoshop that allows you to generate, inpaint, upscale, and split images into layers directly within Photoshop without leaving your computer. It leverages local AI models and runs on your own GPU, providing real, editable Photoshop layers. ### Key Features: - **Free, No Subscription**: No credits, no metering, no account. - **Local AI Layers**: Generated layers are named, positioned, and editable within Photoshop. - **Fourteen Generation Tools**: Supports text-to-image, image-to-image, image editing, sketch-to-image, and inpainting. - **Experimental Split Tool**: Splits a flat picture back into layers. ### Tech Stack: TypeScript ### Why It Matters: OpenLayer is a powerful tool for anyone looking to enhance their Photoshop workflow without the need for cloud storage or subscription services. It allows for seamless integration with ComfyUI, making it ideal for those who prefer to keep their projects local and secure.

Overview: AMG-RAG (Agentic Medical Graph-RAG) is a comprehensive framework designed to automate the construction and continuous updating of Medical Knowledge Graphs (MKGs), integrating reasoning and retrieving current external evidence for medical Question Answering (QA). Key features: - Enhanced Knowledge Graph Construction: Advanced entity extraction with confidence scoring - Bidirectional Relationship Analysis: Comprehensive relationship mapping with confidence scoring - Context-Aware Entity Processing: LLM-generated descriptions with medical context integration - Multi-source Evidence Retrieval: Integrates PubMed search, Wikipedia, and vector database retrieval - Chain-of-Thought Reasoning: Structured reasoning synthesis with evidence integration - Real-time Graph Updates: Dynamically incorporates latest medical literature and research - Entity Summarization: Enhanced entity understanding with relevance-based confidence scoring Tech stack: Python Why it matters: AMG-RAG addresses the challenge of rapidly evolving medical knowledge by dynamically linking new findings and complex medical concepts, providing superior performance on standard medical QA benchmarks compared to comparable models and those 10 to 100 times larger. Repository: MrRezaeiUofT/AMG-RAG GitHub stars: 44 GitHub description: AMG-RAG (Agentic Medical Graph-RAG) is a comprehensive framework that automates the construction and continuous updating of Medical Knowledge Graphs (MKGs), integrates reasoning, and retrieves current external evidence for medical Question Answering (QA).

### Overview: sflow is a declarative workflow orchestrator designed for massive GPU clusters. It simplifies the deployment and management of GPU clusters across various backends such as Docker, Slurm, and Kubernetes. sflow consolidates the infrastructure glue into a single portable YAML file, ensuring that the same `sflow.yaml` can run on Docker, Slurm, and Kubernetes, and provides topology-aware node and GPU placement, multi-node replicas and sweeps, readiness/failure probes, and batch submission. ### Key features: 1. **Backend Agnostic**: One recipe on `local`, `docker`, `slurm`, `kubernetes` (`k8s` / `k8s_mpi`) -- swap the backend fragment, keep the DAG. 2. **Modular Composition**: Reusable YAML fragments deep-merged by name at runtime (`sflow compose`, multi-file `sflow run -f`), with `required_by` reverse edges. 3. **Topology-aware GPU Allocation**: Automatic node/GPU placement and `CUDA_VISIBLE_DEVICES` slicing; pin devices with `gpus.indices`, UUID-based re-selection inside Slurm steps, and machine-local GPU reservation on Docker. 4. **Probes**: Readiness and failure gates -- TCP port, HTTP, log watch with pattern matching. 5. **Replicas & Sweeps**: Parallel/sequential replicas with Cartesian product variable sweeps. 6. **Batch Mode**: Generate sbatch scripts, CSV-driven bulk sweeps, parallel preflight validation. 7. **Expressions**: Jinja2 `${{ }}` syntax for variables, backend info,

Overview: Conformer is an implementation of Conformer: Convolution-augmented Transformer for Speech Recognition, a Transformer variant designed to achieve the best of both worlds by combining the strengths of transformers and convolution neural networks. It significantly outperforms previous Transformer and CNN-based models, achieving state-of-the-art accuracies in speech recognition tasks. Key features: - Conformer is a model that combines the benefits of transformers and convolutional neural networks to model both local and global dependencies in audio sequences. - It achieves state-of-the-art performance in speech recognition tasks. - It is implemented in TensorFlow, making it easy to integrate into existing projects. Tech stack: Conformer is built using TensorFlow, a popular open-source framework for machine learning. Why it matters: Conformer is significant because it provides a more efficient and effective way to model speech recognition tasks by leveraging the strengths of both transformers and convolutional neural networks. Its performance is particularly impressive, making it a valuable tool for researchers and developers in the field of speech recognition. Repository: The repository is maintained by Rishit Dagli and is hosted on GitHub. It has been starred by 45 users, indicating its popularity and usefulness in the community. GitHub stars: The repository has been starred by 45 users, indicating its popularity and usefulness in the community. GitHub description: This repository implements Conformer, a model that combines the benefits of transformers and convolutional neural networks to model both local and global dependencies in audio sequences. It significantly outperforms previous Transformer and CNN-based models, achieving state-of-the-art accuracies in speech recognition tasks.

### Overview: Voice to Speech is an in-browser client-side service that converts a user's voice into synthesized speech live via speech recognition, with hundreds of voices, text to speech (TTS), language auto-translation, and socket support. ### Key features: - Live speech recognition via Web Speech API (only on Chrome). - Low latency Voice-to-Speech by using interim speech recognition results. - Alternative Text-to-Speech input for fine control of output. - Auto translation from input voice language to output speech language. - Large selection of languages and dialects for speech recognition and voices. - Includes support for voices on your computer and built in to your browser. - Customize audio input and output devices. - Customize output volume, as well as pitch and rate for select voices. - Live speech output transcript with playback support for all transcript entries. - Selecting all text on the page will only select transcript text, allowing easy copy-paste. - Live diagnostics for speech recognition output, confidence, and errors. - Use a web socket to send speech output text directly to your PC. - Use two instances to allow bi-directional translation between you and someone else. ### Tech stack: JavaScript ### Why it matters: Voice to Speech is a powerful tool for real-time communication and translation, making it ideal for translators, interpreters, and anyone needing to quickly convert speech to text and vice versa. The service supports hundreds of voices, languages, and dialects, making it versatile for various communication needs. The low latency feature ensures smooth interaction, and the web socket allows for seamless bi-direction

### Overview: This repository contains tested image prompt recipes for the Grok Imagine Video 1.5 model, including example renders, aspect ratios, and the associated cost per image. The model ID is `grok-imagine-video-1.5`, and the pricing is $0.064 per second on the APIMart route. ### Key Features: - **Prompt Recipes:** 4 tested recipes with example renders, aspect ratios, and the model ID (`grok-imagine-video-1.5`). - **Cost Information:** The cost per image is $0.064, estimated for the entire set: $0.256. - **Model Compatibility:** Compatible with OpenAI endpoints, including 300+ models and video, and supports API keys. ### Tech Stack: - Python ### Why It Matters: - **Unified Endpoint:** A single API key for all models in the Grok Imagine Video 1.5 catalog, including 300+ models and video. - **Flexible Pricing:** No need for monthly subscriptions or additional fees; start with $1 and use as much as you need. - **Real-Time Billing:** Charges are reported in the response, and tasks that fail are refunded. - **Asynchronous Processing:** Tasks are processed asynchronously from the start, with `task_id` and polling for updates. ### Example Recipes: - **Product Ad:** A matte ceramic mug on a dark stone plinth with steam rising, soft key light from the left, shallow depth of field, commercial product photography. - **C