Projects

Overview: EasyDist is an automated parallelization system designed to simplify the process of parallelizing training and inference code across multiple ecosystems, including PyTorch, Jax, and TVM. Key features: - **Usability**: EasyDist allows parallelizing training or inference code with a single line of code, enhancing the ease of use. - **Ecological Compatibility**: It supports SPMD rules at the operator-level for PyTorch, Jax, and TVM, ensuring compatibility with various machine learning frameworks. - **Infrastructure**: EasyDist decouples auto-parallel algorithms from specific machine learning frameworks and IRs, enabling the development and benchmarking of different auto-parallel algorithms in a more flexible manner. Tech stack: Python Why it matters: EasyDist simplifies the parallelization process for machine learning models, making it easier to scale up training and inference tasks across multiple ecosystems, thereby accelerating the development and deployment of AI models.

Overview: This project is designed to help developers understand and implement various machine learning techniques, including MLOps, Generative AI, NLP, Computer Vision, and ML System Design. It provides hands-on projects and code algorithms from scratch, making it ideal for those looking to deploy models at scale and build applications with large language models. Key features: - Hands-on projects on MLOps, Generative AI, NLP, Computer Vision, and ML System Design - Code algorithms from scratch - Deploy models at scale - Build applications with large language models Tech stack: Jupyter Notebook Why it matters: This project is essential for developers looking to implement and deploy machine learning models effectively, understand the latest techniques in Generative AI, NLP, and Computer Vision, and design and build ML systems from scratch. Repository: hemansnation/Machine-Learning-MLOps-GenerativeAI-NLP-CV-MLSystem-Design Primary language: Jupyter Notebook GitHub stars: 81

Overview: This repository contains lecture materials on Diffusion models, focusing on advanced artificial intelligence topics at IE university. It includes slides and code for understanding and improving diffusion models. Key features: - Slides for lectures - Code for Diffusion models - Information on advancements and improvements Tech stack: Jupyter Notebook Why it matters: This repository provides essential resources for learning and teaching about Diffusion models, a crucial area in artificial intelligence, particularly in advanced artificial intelligence courses.

Overview: Qwen-Image 3.0 is a powerful API offering text-to-image generation and editing with outputs of 1K and 2K resolutions, providing a cost-effective alternative to traditional image creation methods. It supports a variety of models and resolutions, making it suitable for both small and large projects. Key features: - Supports 1K and 2K output resolutions - Offers a flexible model selection and resolution scaling - Provides a cost-effective pricing model - Supports both text-to-image generation and image editing Tech stack: Python Why it matters: Qwen-Image 3.0 simplifies the process of creating high-quality images, making it an essential tool for developers and designers looking for more efficient and cost-effective alternatives to traditional image creation methods.

Overview: Sherpa-NCNN是一个基于ncnn的新一代kaldi语音识别框架，支持多种语言、多种平台和多种模型。它目前尚未适配Unity环境，但通过移植实现了实时并准确的中英双语语音识别功能。 Key features: - 实现了实时并准确的中英双语语音识别功能 - 支持多种语言、多种平台和多种模型 - 目前尚未适配Unity环境 Tech stack: C# Why it matters: Sherpa-NCNN为实时语音识别提供了强大的工具，特别是在多语言和多平台的支持方面，这使得它成为开发实时语音应用的首选方案。

### Overview VIBE is an extensible benchmark for approximate nearest neighbor search methods, or vector indexes, using modern embedding datasets. It includes datasets for both in-distribution and out-of-distribution settings, and supports the most comprehensive collection of state-of-the-art vector search algorithms. VIBE also includes quantized datasets in 8-bit integer and binary precision and supports HPC environments with Slurm and NUMA, as well as GPU algorithms. ### Key Features - 📊 Modern vector index benchmark with embedding datasets - 🎯 Includes datasets for both in-distribution and out-of-distribution settings - 🏆 Includes the most comprehensive collection of state-of-the-art vector search algorithms - 💎 Support for quantized datasets in both 8-bit integer and binary precision - 🖥️ Support for HPC environments with Slurm and NUMA - 🚀 Support for GPU algorithms ### Tech Stack Python ### Why It Matters VIBE provides a comprehensive and extensible benchmark for evaluating vector index algorithms, making it a valuable resource for researchers and developers looking to compare and evaluate different methods. Its inclusion of both in-distribution and out-of-distribution datasets, support for quantized datasets, and compatibility with HPC and GPU environments make it a versatile tool for benchmarking and research. ### GitHub Stars GitHub stars: 80

Overview: Wan 3.0 API is a Python SDK and MCP server designed for developers working with AI video generation, including text-to-video, image-to-video, multimodal references, file uploads, and asynchronous job polling. It supports the Wan 3.0 API on MuAPI, providing a streamlined and efficient way to integrate AI-generated videos into projects. Key features: - Supports text-to-video, image-to-video, multimodal references, and uploads. - Synchronized audio capability. - Asynchronous job polling for better performance. Tech stack: Python Why it matters: This SDK and server provide a comprehensive solution for developers looking to integrate AI-generated videos into their projects, making it easier to create engaging and dynamic content using AI technology.

### Overview: Terra is a comprehensive multimodal spatio-temporal dataset designed to study Earth's spatial and temporal dynamics. This dataset is crucial for researchers and developers looking to advance their understanding of environmental changes and their impacts. ### Key Features: - **Multimodal Data**: Includes images, videos, and satellite data to capture various aspects of Earth's surface. - **Spatio-Temporal Coverage**: Extensive coverage of different regions and time periods to provide a comprehensive view of Earth's variability. - **Stable Storage**: The dataset is hosted on a more stable cloud storage service to ensure long-term accessibility. ### Tech Stack: - Jupyter Notebook ### Why It Matters: Terra serves as a foundational resource for researchers and developers in the fields of environmental science, climate studies, and data analysis. Its comprehensive nature makes it invaluable for developing and testing models that can predict and understand Earth's complex temporal and spatial phenomena. ### Repository: - **Repository**: CityMind-Lab/NeurIPS24-Terra - **GitHub Stars**: 80 - **GitHub Description**: [NeurIPS 2024] Terra: A Multimodal Spatio-Temporal Dataset Spanning the Earth ### Follow-Up Work: - **Datasets**: ClimateIQA, E2-CSTP, MM4TSA - **Methods**: E2-CSTP, ST-TTC, ST-TTC - **Surveys**: MM4TSA ### Citation: - **NeurIPS 2024**: Chen, W.,

Overview: This repository provides a PyTorch implementation of the Continuous Integrate-and-Fire (CIF) module for end-to-end automatic speech recognition (ASR), originally proposed in the paper "CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognition" at ICASSP 2020. Key features: - CIF module implementation - PyTorch framework - Supports various configurations for CIF module Tech stack: PyTorch Why it matters: This implementation is useful for researchers and developers looking to integrate the CIF module into their ASR models, providing a practical tool for end-to-end speech recognition tasks.

### Overview: Gemini Pro is a lightweight WebView-based application designed to access Google AI Studio with additional features to enhance your workflow. ### Key features: - **Instant Diagram Viewing**: Automatically detects and renders Mermaid diagrams from copied text. - **HTML Preview**: Instantly view and interact with generated HTML code snippets. - **Save to File**: Quick option to save copied text directly to a `.txt` file. - **Caffeine Mode**: Prevents the screen from turning off while you're deep in your workflow. - **Media Downloads**: Option to download images and videos directly from the page. - **Quick Access Tools**: Direct links to Google Docs and Google Flow (Fx) via the additional menu. - **Navigation & Sharing**: Share the current page, copy links, and navigate forward/backward easily. - **Customizable UI**: Choose between left or right side placement for the menu handle. - **Gestures**: Reload page with swipe down from the top center handle, open additional menu with swipe from the side handle. - **Update System**: Built-in notifications to keep you on the latest version. ### Tech stack: Kotlin ### Why it matters: Gemini Pro enhances your workflow by providing additional features directly within a lightweight WebView, making it a valuable tool for developers and AI enthusiasts alike.

### Overview: MuAPI ComfyUI Nodes is a comprehensive set of custom ComfyUI nodes designed to run 600+ AI models, including Seedance, Kling, Veo3, Flux, HiDream, GPT-image, Imagen4, and more. This project leverages the MuAPI API to provide seamless integration with ComfyUI, allowing for the creation of high-quality AI-generated content. ### Key Features: - **MuAPI API Key**: Set up a single API key to access all 600+ models. - **MuAPI Text-to-Image**: Flux, HiDream, GPT-image-1.5, Imagen4, Seedream. - **MuAPI Image-to-Image**: Flux Kontext, GPT-4o edit, Seededit, Wan edit. - **MuAPI Text-to-Video**: Seedance, Kling, Veo3, Wan, HunyuanVideo, Grok. - **MuAPI Image-to-Video**: 25+ I2V models, up to 4 reference images. - **MuAPI Extend Video**: Extend any generation via request_id. - **MuAPI Image Enhance**: Upscale, bg-remove, face-swap, Ghibli, colorize. - **MuAPI Video Edit**: Effects, dance, dress-change, upscale, lipsync. - **MuAPI Lipsync**: Sync.ai, Veed, Creatify. - **MuAPI Audio**: Suno create/remix/extend, mmaudio. - **MuAPI Generate (Generic)**:

### Overview: This project is a real-time AI ChatBot and voice-enabled AI VoiceBot that leverages Groq LLaMA3 for conversational intelligence, Deepgram for speech recognition and synthesis, and LangChain for integration. It is designed for easy setup, cross-platform compatibility, and real-time voice recognition using Deepgram. ### Key features: - Real-time voice recognition using Deepgram - AI-powered responses via Groq LLM - Easy setup with virtual environment - Cross-platform compatibility ### Tech stack: Python ### Why it matters: This project is a comprehensive solution for creating a real-time AI ChatBot and voice-enabled AI VoiceBot, utilizing cutting-edge technologies such as Groq LLaMA3, Deepgram, and LangChain. It is designed to provide fast, high-quality, and natural-sounding AI interactions, making it suitable for various applications requiring real-time voice-based communication.

### Overview: This project is a production-grade low-latency FPGA trading system, featuring a custom VHDL 10GBASE-R PHY, hardware NASDAQ ITCH 5.0 order book, and advanced clock domain crossing architecture. The system is designed for high-frequency trading and leverages the power of FPGA for efficient market data processing and order book management. ### Key Features: - **Custom VHDL 10GBASE-R PHY**: Zero vendor IP, full RTL implementation for minimal jitter. - **Hardware NASDAQ ITCH 5.0 Order Book**: Real-time NASDAQ ITCH 5.0 protocol parsing in FPGA fabric. - **Deterministic Order Book Execution Engine**: Ultra-low latency bitmask-based price level tracking. - **C++20 Kernel Bypass Network Stack**: High-throughput software data plane using DPDK and XDP for sub-microsecond packet processing. - **Zero-Copy PCIe DMA Subsystem**: Custom ring-buffer memory management for efficient data transfers. ### Tech Stack: - **VHDL**: Custom 10GBASE-R PHY and MAC implementation. - **C++20**: Software data plane using DPDK and XDP. - **XDP**: eBPF (Linux eBPF) for packet processing. - **DPDK**: Data Plane Development Kit for high-throughput data plane. ### Why It Matters: This system is designed for high-frequency trading and leverages the power of FPGA for efficient market data processing and order book management. The custom 10GBASE-R PHY and hardware NASDAQ ITCH

### Overview: This repository contains the code for CvT2DistilGPT2, an encoder-to-decoder model designed for chest X-ray report generation. The model leverages a Convolutional Vision Transformer (CvT) pre-trained on ImageNet-21K to generate detailed and accurate reports from chest X-ray images. ### Key Features: - **Model Architecture**: An encoder-to-decoder architecture where the encoder is the CvT pre-trained on ImageNet-21K, and the decoder is DistilGPT2. - **Task-Specific Checkpoints**: Available checkpoints for MIMIC-CXR and IU X-Ray tasks. - **Multi-Head Attention**: The model uses multi-head attention for efficient computation. - **Visualization**: A visual representation of the model architecture, including multi-head attention and subword separation. ### Tech Stack: - **Python**: The primary programming language used for the implementation. ### Why It Matters: This approach improves the accuracy and detail of chest X-ray report generation by leveraging pre-trained knowledge from ImageNet-21K, leading to more informative and detailed reports compared to traditional methods. ### Repository: - **Repository**: aehrc/cvt2distilgpt2 - **GitHub Stars**: 79 - **GitHub Description**: Improving Chest X-Ray Report Generation by Leveraging Warm-Starting ### Installation: - **Requirements**: Install the required packages using `pip` in a virtual environment. - **Checkpoints**: Available checkpoints for MIMIC-CXR and IU X-Ray tasks can be found on GitHub

Overview: This project is a comprehensive RAG (Reinforcement Learning with Attention) boilerplate system designed for developers. It includes features like chunking, vector database integration, LLM reranking, query enhancement, and session management. Key features: - Chunking with propositional model and late chunking - Vector database using Qdrant for hybrid search (BM25 + Dense) - LLM-based reranking - Query enhancement via LLMs - CrewAI for conversation and retrieval agents - Session management with Redis and MongoDB - Gradio UI for chat interface - Evaluation metrics for hit rate and MRR Tech stack: Python Why it matters: This project provides a robust and flexible framework for developers to build and test RAG systems, leveraging cutting-edge technologies like Qdrant, LLMs, and CrewAI. Repository: mburaksayici/RAG-Boilerplate Primary language: Python GitHub stars: 80 GitHub description: The project includes a RAG system with various features, including chunking, vector database integration, LLM reranking, query enhancement, and session management. It also includes a Gradio UI and evaluation metrics for hit rate and MRR. The project is designed to be educational and adaptable for different use cases. Read my blog on [RAG Systems](https://mburaksayici.com/blog/).

**Overview:** UNO-Bench is a comprehensive benchmark designed to evaluate the compositional law between uni-modal and omni-modal models in omni models. It includes 1250 human-curated omni-modal samples and 2480 enhanced uni-modal samples, covering 44 task types and 5 modality combinations. The dataset is well-suited for real-world scenarios, particularly in the Chinese context, and offers a 90% increase in speed while maintaining 98% consistency across 18 public benchmarks. **Key Features:** 1. **Unified Benchmark:** UNO-Bench is the first unified omni model benchmark, providing a comprehensive evaluation framework for both uni-modal and omni-modal capabilities. 2. **High-Quality and Diverse Dataset:** The dataset includes 1250 human-curated omni-modal samples and 2480 enhanced uni-modal samples, ensuring a wide range of scenarios and tasks. 3. **Cross-Modality Solvability:** The dataset is 98% cross-modality solvable, making it suitable for real-world applications. 4. **Multi-Step Open-Ended Questions:** The benchmark includes an innovative multi-step open-ended question format to assess complex reasoning, supporting 6 question types for automated evaluation with 95% accuracy. 5. **Compositional Law Demonstration:** Experimental results show the compositional law between omni-modal and uni-modal performance, highlighting the omni-modal capability as a bottleneck effect on weak models and a synergistic promotion on strong models. **Tech Stack

### Overview: The Diffusion Transformer is an implementation of the Diffusion Transformer model, designed to offer scalable performance using a single GPU. It is based on the paper "[Scalable Diffusion Models with Transformers](https://arxiv.org/abs/2212.09748)" and is available on GitHub under the MIT license. ### Key features: - Implementation of the Diffusion Transformer model. - Uses Pytorch for the technical stack. - Supports training on Imagenet-64 dataset. - Provides sample output demonstrating model performance. ### Tech stack: Python ### Why it matters: The Diffusion Transformer model is designed to offer efficient performance on a single GPU, making it suitable for real-time applications or systems with limited computational resources. ### Repository: - Repository: milmor/diffusion-transformer - Primary language: Python - GitHub stars: 80 - GitHub description: Implementation of Diffusion Transformer Model in Pytorch - README excerpt: Implementation of the Diffusion Transformer model in the paper: [Scalable Diffusion Models with Transformers](https://arxiv.org/abs/2212.09748). See [here](https://github.com/facebookresearch/DiT) for the official Pytorch implementation. ### Dependencies: - Python 3.9 - Pytorch 2.1.1 ### Training Diffusion Transformer: Use `--data_dir=<data_dir>` to specify the dataset path. Example command: `python train.py --data_dir=./data/` ###

Overview: Kim 是一款基于阿里云服务的开源智能家居语音助手，提供文本和语音交互，支持远程会话功能，可接入多种智能家居设备。 Key features: - 基于阿里云服务构建 - Docker 化快速安装部署 - 优化中文语义仲裁算法（KSM），精准理解中文语义 - 可选安装远程会话服务 - 跨平台支持 Raspberry Pi、macOS、Windows - 响应消息可通过 DingTalk 机器人推送到群 Tech stack: Python Why it matters: Kim 使您的私人语音助理更加智能、易扩展且充满魅力，与阿里云深度融合，提供云端能力触手可及。 Repository: mozhou-tech/kim-voice-assistant Primary language: Python GitHub stars: 80 GitHub description: Kim，您的个人语音助手，用于智能家居。它提供开源的语音、文本交互方案，并通过云端服务增强设备的远程会话能力。它已支持 HomeAssistant 和小米等主流智能家居系统。 ## 关联项目 - Kim 远程会话服务 (RemoteTalk) - Kim 插件集 ## 主要特性 1. 基于阿里云服务构建 1. Docker 化快速安装部署 1. 优化中文语义仲裁算法（KSM），精准理解中文语义 2. 可选安装 "远程会话服务" 2. 跨平台支持 Raspberry Pi、macOS、Windows 1. 响应消息可通过 DingTalk

Overview: This repository benchmarks fully-managed vector databases, focusing on their performance metrics such as Queries Per Second (QPS) and cost-performance ratios. Key features: - Vector database performance benchmarking - Fully-managed vector databases - Detailed results and cost-performance analysis Tech stack: Python Why it matters: This benchmark helps developers and data scientists evaluate the efficiency and cost-effectiveness of fully-managed vector databases, providing valuable insights into their performance and cost metrics. Repository: myscale/vector-db-benchmark Primary language: Python GitHub stars: 80

### Overview: VerifAI is an open-source project aimed at building an easy-to-deploy generative question-answering engine that can verify the correctness of answers. The project uses generative models to generate answers and then verifies them using a posteriori model. ### Key features: - Easy installation via a single script - Support for indexing documents in various formats (txt, md, docx, pptx, pdf) - Combination of lexical and semantic search - Use of any HuggingFace listed model for document embeddings - Compatibility with LLMs following OpenAI API standard - Support for large indexed documents (tested with over 200GB of data and 30 million documents) - Shows the closest sentence in the document to the generated claim - User registration and log-in - React.js-based user interface - Verification that generated text does not contain hallucinations - Future plans to support single-sign-on with AzureAD ### Tech stack: Jupyter Notebook ### Why it matters: VerifAI aims to address the issue of hallucinations in generative models, ensuring that the answers generated are verifiable and correct. By making the project open-source, anyone can contribute to and use the technology, potentially improving its accuracy and reliability. ### Repository: nikolamilosevic86/verifAI ### Primary language: Jupyter Notebook ### GitHub stars: 80 ### GitHub description: VerifAI initiative to build open-source easy-to-deploy generative question-answering engine that can reference and verify answers for correctness