Projects

### Overview: ReadyLLM is a powerful tool for deploying, monitoring, and tuning large language models. It optimizes performance by automatically selecting the best parameters for your hardware, resulting in a 50% speed increase and improved resource utilization. ### Key Features: - **Unified Target Management**: Supports local machines and remote LANs (SSH, key, or password), detecting OS and configuring inference engines and model directories. - **Hardware Detection & Model Recommendation**: Determines GPU, VRAM, CPU, and RAM, recommending models that fit your hardware. - **Model Store**: A curated catalog of text, image, and video models, with detailed information about versions, parameters, precision, resolution, and weight filenames. - **One-Click Deployment**: Installs and starts inference services with adjustable runtime parameters; auto-downloads and compiles the engine if missing on the target. - **Real-Time Monitoring**: Displays GPU, VRAM, throughput, and latency curves in real-time via WebSocket. - **Smart Tuning**: Two-phase parameter search and benchmark results for optimal performance. ### Tech Stack: ReadyLLM uses Python, FastAPI, React, Vite, and ComfyUI for its frontend. ### Why it Matters: ReadyLLM helps optimize the performance of large language models by automatically tuning the parameters, resulting in significant speed improvements and better resource utilization.

Overview: SummerAsr 是一个基于C++的可独立编译且几乎没有额外依赖库的本地中文语音识别器。 Key features: - 独立编译 - 无需网络连接 - 零依赖 - Eigen底层计算库 - 适用于MIT License Tech stack: C++ Why it matters: SummerAsr 是一个高效、独立且无需额外依赖的本地中文语音识别系统，适用于各种开发环境和平台。

Overview: This is a Java library designed to provide convenient utility methods and overloads for interacting with the Qdrant vector search engine. Key features: - Provides a high-level client for Qdrant - Supports TLS configuration for secure connections - Includes API key authentication for secure communication - Offers usage examples in the Qdrant documentation Tech stack: Java Why it matters: This library simplifies the interaction with Qdrant, making it easier to integrate vector search capabilities into Java applications securely and efficiently. Repository: qdrant/java-client GitHub stars: 100 GitHub description: Official Java client for Qdrant README excerpt: <p align="center"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://github.com/qdrant/qdrant/raw/master/docs/logo-dark.svg"> <source media="(prefers-color-scheme: light)" srcset="https://github.com/qdrant/qdrant/raw/master/docs/logo-light.svg"> <img height="100" alt="Qdrant" src="https://github.com/qdrant/qdrant/raw/master/docs/logo.svg"> </picture> <img height="150" width="100" src="./resources/java-logo-small.svg" alt="Java"> <p align="center"> <b>Java library for the &

Overview: This project provides a React Native speech recognition module utilizing the Vosk library, offering offline speech recognition capabilities for integration into native applications. Key features: - Offline speech recognition using Vosk library - Support for multiple models for different languages - Models available for download from Vosk official website - Android and iOS compatibility - Dynamic model loading for experimental purposes - Model folder structure for Android and iOS projects - Example project included in the repository for easy integration Tech stack: React Native, Objective-C++ Why it matters: This module simplifies integrating speech recognition functionality into native React Native applications, making it easier for developers to implement voice-based applications without the need for complex backend setups.

Overview: Fast-HuBERT is an efficient pretraining framework for self-supervised speech representation learning, optimized for fast training on large datasets like Librispeech. Key features: - Optimized front-end inputs and loss computation - Aggregates SOTA techniques including ILS and MonoBERT - Achieves 5.2X speedup over the original implementation with minimal performance drop - Pretrained in just 1.1 days on 8 V100 GPUs Tech stack: Python Why it matters: Fast-HuBERT simplifies the training process for speech representation learning, making it more accessible and efficient for academic research and real-world applications.

### Overview: AlignLedger is a human-feedback and data-governance infrastructure designed for RSI research, enabling traceable records and frozen dataset candidates from model outputs, human corrections, and second-pass decisions. ### Key features: - Per-record judgments, corrections, and unknown states - Separate second-pass decisions and final labels - Actor attribution, revisions, and idempotent writes - Frozen dataset candidates, not automatic training approval - Documented handoff and external adapter/split policy - Per-record annotation with correction fields and explicit unknown states ### Tech stack: Python and SQLite ### Why it matters: AlignLedger ensures accountability, traceability, and versioning in RSI research, providing a robust foundation for iterative model development and evaluation.

### Overview: LLMinator is a Python-based tool that allows developers to run and test large language models (LLMs) directly from the Hugging Face model hub, using Gradio for a user-friendly interface. It includes features like context-aware streaming chatbots, inbuilt code syntax highlighting, and support for both CPU and CUDA modes. ### Key Features: - **Context-Aware Streaming Chatbot:** Enables real-time text generation. - **Inbuilt Code Syntax Highlighting:** Enhances readability of code snippets. - **Direct Model Loading:** Supports loading models from Hugging Face. - **Supports CPU and CUDA Modes:** Allows seamless integration with different hardware configurations. - **LLM Inference with llama.cpp:** Enables inference using llama.cpp with parameters like n_gpu_layers, temperature, and max_tokens. - **Model Conversion:** Converts models from Safetensors to gguf format. - **Customizable Parameters:** Allows customization of LLM inference parameters. - **WebSocket Integration:** Enables real-time text generation via websockets, facilitating seamless integration with various frontend frameworks. ### Tech Stack: Python ### Why it Matters: LLMinator simplifies the process of running and testing large language models directly from the Hugging Face model hub, making it easier for developers to experiment with and integrate these models into their projects.

### Overview: This repository provides samples to help developers leverage the new Native Vector Support in Azure SQL Database, enabling the storage and querying of embeddings for enhanced AI capabilities in applications. ### Key features: - Embeddings: Fetch embeddings from OpenAI directly using Azure SQL. - Exact Vector Search: Implement vector similarity search within an SQL database, showcasing semantic search capabilities. - Approximate Vector Search: Utilize DiskANN to perform approximate nearest neighbor search in SQL Server 2025 or Azure SQL. - Hybrid Search: A Python sample demonstrating how to integrate keyword search with vector-based search for more precise results. ### Tech stack: Jupyter Notebook ### Why it matters: This repository showcases the integration of vector-based AI techniques into SQL databases, enhancing applications with semantic search capabilities and improving data retrieval efficiency.

### Overview: MagicTailor is an innovative framework designed to enable component-controllable personalization in text-to-image diffusion models. This task allows users to reconfigure and personalize specific components of concepts, overcoming semantic pollution and imbalance challenges. ### Key Features: - **Dynamic Masked Degradation (DM-Deg)**: Dynamically perturbs undesired visual semantics to enhance personalized concepts. - **Dual-Stream Balancing (DS-Bal)**: Establishes a balanced learning paradigm for visual semantics to address semantic imbalance issues. - **Component-Controllable Personalization**: A novel task that allows reconfiguring specific components of concepts, enabling precise control over visual elements. ### Tech Stack: Python ### Why It Matters: MagicTailor introduces a new paradigm for text-to-image diffusion models, enabling more nuanced and creative image generation. Its innovative approach to component-controllable personalization opens up new possibilities in personalized image generation, making it a significant contribution to the field of AI and image synthesis.

### Overview: This is an unofficial implementation of the SODA (SODA: Bottleneck Diffusion Models for Representation Learning) paper, focusing on the linear-probe classification performance. The implementation is based on PyTorch and uses a multi-gpu setup for efficient training. It addresses the limitations of the original paper by focusing on the specific task of linear-probe classification and includes some minor adjustments to the architecture and training process. ### Key Features: - **Linear-Probe Classification**: The implementation is specifically designed to evaluate the performance of the model on linear-probe classification tasks. - **Diverse Datasets**: The repo includes experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet, covering small and medium scale datasets. - **Diverse Training Configurations**: The README provides detailed configurations and experiments on different datasets, including resolution, epochs, and model parameters. ### Tech Stack: - **Framework**: PyTorch - **Language**: Jupyter Notebook ### Why It Matters: This implementation is a valuable resource for developers interested in the linear-probe classification performance of SODA models. It provides a starting point for further development and understanding of the model's capabilities. ### Repository: The primary language used for this project is Jupyter Notebook, and the repository is hosted on GitHub under the name `FutureXiang/soda`. It has received 99 GitHub stars, indicating its popularity and usefulness in the developer community. ### Requirements: - **Dependencies**: Install PyTorch, PyYAML, EMA-PyTorch

### Overview: MeanFlowNFT is an innovative method that leverages Forward-Process Reinforcement Learning (Forward-process RL) to optimize rewards in the instantaneous-velocity space while maintaining the model in the average-velocity space. This approach is particularly effective in average-velocity generators, where it outperforms other methods in various metrics. ### Key Features: 1. **Induced Instantaneous-velocity Optimization:** MeanFlowNFT optimizes rewards in the instantaneous-velocity space while keeping the model in the average-velocity space. 2. **Native Any-Step Generation:** The method preserves MeanFlow's efficient sampler without the need for reverse-trajectory backpropagation or likelihood estimation. 3. **Policy Improvement:** It inherits DiffusionNFT's strict improvement guarantee under the idealized pointwise optimum. ### Tech Stack: The implementation is built using PyTorch, leveraging the PyTorch Lightning framework for efficient training and inference. ### Why It Matters: MeanFlowNFT stands out by providing a practical solution that maintains the efficiency of MeanFlow's sampling while optimizing for forward-process RL. This makes it particularly useful in average-velocity generators where it outperforms other methods across multiple metrics, including image and video generation tasks.

### Overview: A zero-shot style transfer method that leverages pre-trained diffusion models and rearranges attention to fuse content and style information without retraining or tuning. ### Key features: - Zero-shot image style transfer method. - Rearranged attention mechanism for disentangling and fusing content/style information in the diffusion latent space. ### Tech stack: Python ### Why it matters: This project is an official implementation of the paper "Z∗: Zero-shot Style Transfer via Attention Rearrangement," providing a reliable source for researchers and developers interested in zero-shot style transfer methods.

Overview: This project involves adapting two advanced language models, Falcon-7B and LLAMA 2, to enhance their understanding of Indian law through fine-tuning. Key features: - Fine-tuning of Falcon-7B and LLAMA 2 - Utilization of QLoRA and PEFT for memory-efficient fine-tuning - Integration of a comprehensive dataset of 3300 Indian law instructions Tech stack: Jupyter Notebook Why it matters: This project aims to create an AI model that can proficiently handle Indian law, providing valuable insights and legal advice to users.

Overview: llm-inference is a platform designed for deploying and managing large language model (LLM) inference tasks, offering features such as centralized resource management, comprehensive monitoring, automatic scaling, serverless inference, and support for various frameworks and formats. Key features: - Centralized resource management with Ray technology - Comprehensive monitoring and management interface - Automatic scaling of inference tasks - Serverless inference for resource optimization - Support for multiple inference frameworks and formats - User-friendly configuration for model loading and execution parameters Tech stack: Python Why it matters: llm-inference provides a robust and scalable solution for deploying and managing LLM inference tasks, making it ideal for developers and teams looking to streamline their workflow and improve resource utilization. Repository: OpenCSGs/llm-inference GitHub stars: 99 GitHub description: llm-inference is a platform for publishing and managing LLM inference, providing a wide range of out-of-the-box features for model deployment, such as UI, RESTful API, auto-scaling, computing resource management, monitoring, and more.

### Overview: Seg2Sat is an innovative project that leverages diffusion models such as StableDiffusion and ControlNet to generate synthetic aerial images based on terrain segmentation data. The dataset originates from IGN's FLAIR (French Land cover from Aerospace Imagery) and aims to create photorealistic or stylized aerial imagery. ### Key Features: - **StableDiffusion and ControlNet Integration**: Utilizes advanced diffusion models to generate high-quality synthetic images. - **Terrain Segmentation Data**: Derived from IGN's FLAIR dataset, providing accurate land cover information. - **Photorealistic and Stylized Images**: Offers both photorealistic and stylized aerial views. - **Seed-Based Generation**: Generates images by varying the seed in the random number generator, allowing for diverse variations. ### Tech Stack: - **StableDiffusion**: A pretrained diffusion model used for generating images. - **ControlNet**: An extension to StableDiffusion that enhances image quality and realism. ### Why It Matters: - **Dataset Creation/Augmentation**: Useful for creating more diverse and realistic datasets. - **Map Creation**: Ideal for creating detailed maps for geospatial projects. - **Outpainting and Inpainting**: Enables realistic blending of images, enhancing the quality of generated aerial views. ### Repository: - **Repository**: RubenGres/Seg2Sat - **Stars**: 99 - **Language**: Jupyter Notebook - **GitHub Description**: Using StableDiffusion and ControlNet to generate synthetic aerial images based on terrain segmentation data. ### Example Photorealistic Images: The

### Overview: This repository contains material related to my new book "Synthetic Data and Generative AI," available on <a href="https://mltechniques.com/shop/">ML Techniques</a>. It also includes fast NoGAN code, a tabular data synthesizer, and DeepResampling code for generating synthetic data efficiently. ### Key features: - NoGAN code, a fast tabular data synthesizer. - DeepResampling code, a fast NoGAN based on resampling and distribution-free Hierarchical Bayesian Models. - NoGAN_Hellinger code, a blend of NoGAN and DeepResampling with the Hellinger model evaluation metric. ### Tech stack: Python ### Why it matters: This repository showcases cutting-edge techniques in synthetic data generation and generative AI, making it valuable for researchers and practitioners looking to improve their data synthesis and AI models.

### Overview: KeyMorph is a robust multimodal image registration framework that leverages keypoints for fast and accurate registration across different modalities and volumes. ### Key Features: - **Multi-modal Registration**: Supports registration between unimodal and multimodal images using rigid, affine, and nonlinear transformations. - **Keypoint Extraction**: Automatically extracts corresponding keypoints from input images. - **Flexible Transformations**: Utilizes various transformations including rigid, affine, and nonlinear to align images. - **Efficient Training**: Supports training on your own data using CSV files and supports training on IXI dataset for brain MRI registration. ### Tech Stack: Python ### Why it Matters: KeyMorph is essential for applications requiring accurate and efficient registration of medical images across modalities and volumes, ensuring robustness and flexibility in various imaging scenarios.

### Overview: This comprehensive toolkit provides a production-ready infrastructure for deploying generative AI on Amazon EKS, including components such as AI gateways, LLM serving engines, vector databases, and observability solutions. It simplifies the deployment process with a single command, making it ideal for teams running self-hosted LLMs and AI workloads on AWS. ### Key Features: - **AI Gateway**: Deploy AI gateways using LiteLLM or Kong AI Gateway OSS. - **LLM Serving Engines**: vLLM, SGLang, and TGI. - **Embedding Models**: Text Embedding Inference (TEI). - **Observability**: Langfuse and Phoenix. - **GUI App**: Open WebUI. - **Vector Databases**: Qdrant, Chroma, and Milvus. - **Workflow Automation**: n8n. - **AI Agent Frameworks**: OpenClaw and Strands Agents. - **MCP Server**: FastMCP 2.0. - **AI Agent**: OpenClaw. ### Tech Stack: Kubernetes. ### Why It Matters: This starter kit fast-tracks the deployment of generative AI infrastructure on Amazon EKS, making it easier for teams to start using AI without the hassle of setting up complex infrastructure manually.

Overview: This workshop is designed for developers interested in learning how to build generative AI applications on AWS using PartyRock, Amazon Bedrock, and Amazon Q. It covers practical skills such as prompt engineering and the use of different foundational models, including Amazon Bedrock, RAG, embeddings, and agents. The workshop also explores how to use Amazon Q Developer for coding and debugging. Key features: - Build Generative AI Applications with PartyRock - Use Foundation Models in Amazon Bedrock - Chat with your Documents using Amazon Bedrock Tech stack: Python Why it matters: This workshop is intended for developers looking to build generative AI applications on AWS, focusing on practical skills and foundational models to enhance their development workflow. Repository: build-on-aws/gen-ai-workshop Primary language: Python GitHub stars: 99 GitHub description: Learn how to build generative AI applications on AWS using PartyRock, Amazon Bedrock, and Amazon Q README excerpt: This workshop is designed for builders ready to learn about Generative AI on AWS. You'll learn to develop applications using PartyRock and Amazon Bedrock. You will focus on practical skills like prompt engineering and using different foundational models. We will also explore how to 'chat with your documents' through knowledge bases, retrieval augmented generation (RAG), embeddings, and agents. Additionally, you'll discover how to use Amazon Q Developer to assist in coding and debugging. This is a 300 level workshop, so some familiarity with using services such as AWS Lambda and writing Python code is helpful. No AI/ML

Overview: This repository contains fifty prompts for GPT-Image-2.5, each accompanied by the exact frame it produced. The prompts are categorized into 20 short ones and 30 full engineering briefs, along with a comparison between Flare and Sunburst. Key features: - GPT-Image-2.5 prompts - Exact frame output for each prompt - 20 short prompts - 30 full engineering briefs - Flare vs Sunburst comparison Tech stack: Unknown Why it matters: This repository provides a comprehensive set of prompts for generating images using GPT-Image-2.5, along with detailed results and a comparative analysis, making it valuable for developers looking to enhance their image generation capabilities.