Projects

Overview: This project provides a comprehensive model deployment solution for the Crucible ecosystem, integrating with vLLM, Ollama, TGI, Hugging Face, Kubernetes, and Noop adapters. It supports canary deployments, A/B testing, traffic routing, health checks, and rollback strategies, all utilizing Elixir-based ML workflows. Key features: - Adapters for various model targets (vLLM, Ollama, TGI, Hugging Face, Kubernetes, Noop) - Rollout strategies: Replace, Blue/Green, Canary, A/B - Format converters: GGUF, ONNX, TensorRT - Continuous health checks with auto-rollback signals - Crucible framework Stage integration (optional) - Telemetry events for deploy, promote, rollback, and health checks Tech stack: Elixir Why it matters: This solution simplifies the deployment and management of machine learning models across different environments, ensuring efficient and secure model updates and rollbacks, while also providing robust health monitoring and analytics. Repository: North-Shore-AI/crucible_deployment Primary language: Elixir GitHub stars: 0 GitHub description: ML model deployment for the Crucible ecosystem. vLLM and Ollama integration, canary deployments, A/B testing, traffic routing, health checks, rollback strategies, and inference serving for Elixir-based ML workflows.

### Overview: ImmoPrix is a comprehensive MLOps project designed to predict median house prices in California. The project includes a complete pipeline from model training to deployment via FastAPI, MLflow for tracking experiments, and monitoring for drift detection using Evidently. The project is built using Python 3.11, scikit-learn, pandas, and other modern ML tools, ensuring efficient and scalable operations. ### Key Features: - **Model Training and Tracking:** Utilizes MLflow for tracking experiments and model registry. - **Pipeline with CI/CD:** Includes CI/CD using GitHub Actions for continuous integration and deployment. - **Monitoring for Data Drift:** Uses Evidently for detecting model drift and ensuring model performance. - **API and Streamlit Interface:** Provides an API for prediction and a Streamlit interface for user interaction. - **Model Selection:** The Random Forest model has been selected and is registered in the MLflow Model Registry. - **Data Exploration and Visualization:** Includes an exploratory data analysis notebook and SHAP visualization. - **Deployment and Monitoring:** The project is designed to be easily deployed and monitored, ensuring robust and reliable predictions. ### Tech Stack: - **Python 3.11** - **scikit-learn** - **pandas** - **numpy** - **MLflow** for tracking experiments and model registry - **FastAPI** for the API - **Streamlit** for the user interface - **Docker** for containerization - **GitHub Actions** for CI/CD - **GitHub Container Registry

Overview: This project fine-tunes Qwen2.5-1.5B-Instruct with LoRA and serves the model using an OpenAI-compatible llama-server, achieving an F1 score of 0.8503 on the MASSIVE dataset with a 95% confidence interval of 0.8394 to 0.8600. The model is quantized to Q4_K_M format and optimized for 16 concurrent clients, achieving a peak GPU memory usage of 4859 MiB. Key features: - Fine-tuned Qwen2.5-1.5B-Instruct model with LoRA - Quantized to Q4_K_M format for efficient deployment - Served via an OpenAI-compatible llama-server - Achieves 16 concurrent clients with P95 latency of 1353 ms Tech stack: Python Why it matters: This project showcases the effectiveness of fine-tuning a large language model with LoRA and the practical benefits of quantization for real-world deployment, including improved performance and reduced resource usage.

### Overview: This project is a Dockerized FastAPI service that serves an XGBoost churn model, utilizing structured logging and tests. It is designed for the MLOps Practitioner course, focusing on packaging, logging, serialization, testing, and containerization. ### Key Features: - **Model Serving**: The service serves a churn classifier pulled live from an MLflow Model Registry. - **Model Switching**: Switching the served model requires zero code changes and zero rebuilds, allowing for easy updates. - **CI/CD**: The project includes CI/CD pipelines for Docker, MLflow, and DVC, ensuring consistent and reliable deployment. ### Tech Stack: - **Python**: The primary programming language used. - **Docker**: Containerization for easy deployment. - **MLflow**: Tracking and model registry for model management. - **DVC**: Data versioning and reproducible pipelines. - **XGBoost**: The primary model used for churn prediction. - **FastAPI**: The backend API for the service. ### Why It Matters: - **Model Switching**: The ability to switch models without code changes or rebuilds is crucial for maintaining a production environment. - **CI/CD Pipeline**: The CI/CD setup ensures that the service is consistently deployed and tested, ensuring high-quality production code. ### Repository: - **Owner**: OmarAmir2001 - **Stars**: 0 - **Description**: A Dockerized FastAPI service serving an XGBoost churn model, with structured logging and tests, designed for the MLOps Practitioner

Overview: Atlas AI is an open-source machine learning systems platform designed for distributed training, inference optimization, performance benchmarking, observability, and model serving. It explores modern AI system behavior under real-world constraints. Key features: - Distributed training infrastructure - Transformer systems engineering - Inference optimization - Performance benchmarking - Observability and diagnostics - AI infrastructure research Tech stack: Python Why it matters: Atlas AI provides a comprehensive solution for researchers and developers to explore and understand the behavior of modern AI systems under real-world constraints, focusing on memory scaling, communication overhead, synchronization cost, inference latency, throughput efficiency, and transformer cache growth. Repository: OmprakashSahani/atlas-ai Primary language: Python GitHub stars: 0 GitHub description: ML systems platform for distributed training, transformer inference, profiling, benchmarking, observability, and model serving.

Overview: This project is a Streamlit frontend for an academic MLOps project that predicts historical NYC taxi fares using a machine learning model served through a FastAPI backend deployed on Google Cloud Run. Key features: - Streamlit frontend for interactive fare prediction - FastAPI backend for model serving - Historical data training and prediction capabilities Tech stack: Python Why it matters: This project demonstrates an end-to-end machine learning workflow, showcasing data processing, model training, model versioning, API deployment, and a user-facing interface. It is designed for learning and portfolio purposes, not as a live commercial fare calculator.

Overview: Batch engagement prediction for RankShift Serving, a FastAPI service that scores up to 100 user-video interactions per call, with per-row fault isolation. Key features: - Batch engagement prediction - FastAPI service for scoring user-video interactions - Streamlit console for interactive scoring - Per-row fault isolation Tech stack: Python Why it matters: This service is crucial for optimizing engagement predictions in a real-time context, ensuring that only valid and reliable interactions are scored, thereby improving the accuracy and reliability of the predictions. Repository: PSCRedefine/BatchPrediction

Overview: This project transforms a short-video platform’s log data into an end-to-end ML system that provides calibrated probabilities for engagement predictions, improving the platform’s ability to identify and engage with users. Key features: - Single Prediction: A scalable and efficient system for real-time engagement predictions. - Batch Prediction: A batch processing system for large-scale predictions. - Model Info: Detailed information about the model’s performance and calibration. - Analytics Dashboard: A user-friendly interface for monitoring and analyzing engagement predictions. Tech stack: Python Why it matters: The system flags 23.9% of traffic and finds engaged users 1.40x more often than acting at random, with a precision of 39.3% on the flagged slice. It costs almost nothing to run, with a model size of 3 KB and a scoring time of a dot product, and it tells you what not to build, as 23 of 25 candidate features are statistically indistinguishable from noise. GitHub stars: 0

Overview: The model-canary-gate is a progressive delivery controller designed to ensure the safety and correctness of ML model services. It uses paired shadow analysis to detect and reject models that break behaviorally, and applies staged canary ramping with Wilson-bound gates and hysteresis rollback to manage traffic exposure and prevent false rollbacks. Key features: - Paired shadow analysis to detect and reject models that break behaviorally. - Staged canary ramping with Wilson-bound gates and hysteresis rollback to manage traffic exposure and prevent false rollbacks. - Measured safety with 0 false rollbacks in 10 identical-model rollouts and 100ms regression caught at 5% traffic. Tech stack: Python. Why it matters: The model-canary-gate ensures that new models ship safely in production, preventing silent failures that can lead to significant business impact and operational issues.

### Overview: This project benchmarks different model-serving configurations to determine their cost-effectiveness in terms of throughput and resource usage, specifically focusing on a 4.9 million parameter intent classifier. It measures performance metrics such as p50, p95, p99, request per second (req/s), peak RSS, and prices throughput against a dated AWS snapshot. ### Key Features: - **Dynamic Int8 Quantization**: Measures throughput and resource usage for dynamic integer quantization. - **Micro-Batching**: Evaluates performance with and without micro-batching. - **TorchScript Variants**: Compares FP32 and INT8 TorchScript configurations. - **Performance Metrics**: Tracks throughput, resource usage, and cost per million requests. ### Tech Stack: - Python ### Why It Matters: This project provides a concrete, measurable way to understand the cost implications of different model-serving configurations, helping teams make informed decisions based on actual performance and cost data rather than adjectives or assumptions.

### Overview: IncidentLab is an end-to-end machine learning project designed to detect and investigate unusual behavior in cloud services. It combines machine learning, cloud observability, and controlled fault experiments to provide a comprehensive solution for monitoring and troubleshooting. ### Key features: - **Multimodal ML:** Features from logs, metrics, and traces. - **CPU Serving:** Frozen checkpoints and FastAPI inference without an NVIDIA GPU. - **Interactive Dashboard:** Research replay, bounded HTTP demos, cluster evidence, and downloadable figures. - **Durable Telemetry:** SQLite ingestion, duplicate rejection, chronological replay, and restart state. - **Fault Experiments:** Frontend → checkout → inventory, with delay, HTTP-error, and unavailability trials. - **Evidence-Supported Investigation:** Request timing, status codes, dependency traces, and capture reports. - **Reproducible Evaluation:** Saved predictions, reports, source hashes, and held-out workloads. ### Tech stack: Python, PyTorch, scikit-learn, FastAPI, Docker, Kubernetes (kind), SQLite. ### Why it matters: IncidentLab connects the dots between detecting abnormal behavior and investigating it through captured telemetry, providing a robust and reproducible solution for cloud service monitoring and troubleshooting.

### Overview: PedroM2626/mlops-experiments is a portfolio of reproducible ML/MLOps experiments across various domains including NLP, time-series forecasting, causal inference, computer vision, recommender systems, feature selection, AutoML/RL, and model serving with MLflow. This repository serves as a comprehensive index of each experiment's academic README, providing detailed information on methodology, results, and reproducibility. ### Key Features: - Sentiment Analysis, Topics, and Representations - Wine Scoring from Text - Variations of the Sentiment Pipeline - Hierarchical 20 Newsgroups Classification - Twitter Entity Sentiment Analysis - Reinforcement Learning for AutoML - Time Series Forecasting - Sales Forecasting - Local equivalents of cloud AutoML/forecast pipelines - Computer Vision Methods (CIFAR-10) - Animal Multi-label Classification - Face Detection/Recognition - Boxing Pose Detector ### Tech Stack: Jupyter Notebook ### Why it Matters: This repository offers a detailed and reproducible exploration of various machine learning and MLOps techniques, providing a valuable resource for researchers and practitioners alike. The comprehensive README format ensures that each experiment is accessible and understandable, making it an essential tool for anyone looking to deepen their understanding of ML and MLOps.

Overview: This project is a cloud-native AI model serving platform built with FastAPI, Docker, and Kubernetes, featuring CI/CD, Docker, Kubernetes autoscaling, and security scanning. Key features: - Containerized model serving - Stateless API design - Health and reliability checks - Kubernetes-ready deployment structure Tech stack: Python, FastAPI, Docker, Kubernetes, Terraform Why it matters: This project demonstrates the infrastructure layer for ML systems, focusing on packaging, serving via API, containerization, monitoring, scalability, and infrastructure management.

Overview: 该项目旨在为大模型推理服务提供一个可审计的网关，确保参数版本切换时请求的正常处理，同时保证资源的高效释放和回滚功能。通过资格检查、灰度发布、排空和回滚机制，确保了模型参数的更新过程安全可控。 Key features: - 可审计的请求网关 - 通过资格检查的参数版本灰度发布 - 请求记录与耗时核对引擎计算与KV缓存释放 - 训练产物（LoRA或全参数）先过资格再发布 - 双节点K3s集群部署，支持A10 GPU节点 - 网关热路径优化 - 固定负载的双节点集群部署 - 受控发布与热回滚机制 - 训练产物接入与量化的质量门 - 分布式训练与恢复 Tech stack: Python Why it matters: 该项目通过提供一个安全、可控的推理网关，确保了大模型参数版本切换时请求的正常处理和资源的高效释放，同时保证了训练产物的发布流程安全可控，为大模型的持续更新和应用提供了坚实的基础。

Overview: This project is a high-performance, low-latency machine learning model serving framework built in Rust, designed for efficient model inference, concurrent request handling, and robust deployment for production AI applications. Key features: - Blazing Fast Inference: Leverages Rust's performance characteristics for minimal latency. - Concurrent Request Handling: Asynchronous architecture for handling multiple inference requests simultaneously. - Model Agnostic: Designed to serve models from various frameworks (e.g., ONNX, TensorFlow Lite, custom Rust models). - RESTful API: Provides a clean and well-documented REST API for model predictions. - Metrics & Monitoring: Integrated with Prometheus for real-time performance monitoring. - Hot Reloading (Planned): Future support for updating models without service interruption. Tech stack: Rust Why it matters: This project is essential for developers looking to build high-performance, low-latency machine learning models and applications, leveraging Rust's performance and robustness. Repository: Poin1961/ml-model-server Primary language: Rust GitHub stars: 0 GitHub description: A high-performance, low-latency machine learning model serving framework built in Rust.

Overview: This project provides an inference stack for Nvidia DGX Spark, a powerful AI supercomputer, on a desktop. It is designed to be accessible and affordable for those without a dedicated AI lab. Key features: - Single Spark cluster setup - VLLM (Versatile Language Learning Model) based inference - Shell-based technology stack Tech stack: Shell Why it matters: This project offers a practical solution for deploying AI models on a desktop, making advanced AI capabilities accessible to a broader audience.

Overview: Simplify the self-hosting of large language models with a user-friendly YAML-based system that integrates seamlessly with Git. Key features: - YAML-based model and profile configuration - CLI and TUI for a full lifecycle management - Git-friendly integration for version control Tech stack: Python Why it matters: This project offers a straightforward way to self-host large language models, making it accessible and easy to manage for developers and enthusiasts alike.

Overview: This project is a production-grade credit card fraud detection platform designed for reliability and efficiency. It leverages advanced techniques such as leakage-safe feature engineering, DuckDB feature storage, PR-AUC optimized model training, Kafka-based micro-batch scoring, and FastAPI serving with Docker and Kubernetes for a resilient and scalable system. Key features: - Leakage-safe feature engineering - DuckDB feature storage - PR-AUC optimized model training - Kafka-based micro-batch scoring - FastAPI serving - Streamlit UI - Docker & K8s containerization - CI/CD and automated testing Tech stack: Python, Docker, Kubernetes, FastAPI, Streamlit, DuckDB, Kafka, Docker, Kubernetes, FastAPI, Streamlit, Python, Docker, Kubernetes, FastAPI, Streamlit, Python, Docker, Kubernetes, FastAPI, Streamlit, Python, Docker, Kubernetes, FastAPI, Streamlit, Python, Docker, Kubernetes, FastAPI, Streamlit, Python, Docker, Kubernetes, FastAPI, Streamlit, Python, Docker, Kubernetes, FastAPI, Streamlit, Python, Docker, Kubernetes, FastAPI, Streamlit, Python, Docker, Kubernetes, FastAPI, Streamlit, Python, Docker, Kubernetes, FastAPI, Streamlit, Python, Docker, Kubernetes, FastAPI, Streamlit, Python, Docker, Kubernetes, FastAPI, Streamlit, Python, Docker, Kubernetes, FastAPI, Streamlit, Python, Docker, Kubernetes, FastAPI, Streamlit, Python, Docker, Kubernetes, FastAPI, Streamlit

### Overview: This project provides a scalable, production-ready microservice blueprint for deploying high-performance machine learning models, specifically Ultralytics YOLO models, using a containerized architecture. ### Key Features: - **Reusability:** The microservice is designed to be easily integrated with any model, including custom-trained YOLO models. - **Scalability:** It supports both CPU and GPU compute workers for efficient scalability. - **Modularity:** The architecture separates compute, API, and UI services, making it highly modular and easy to maintain. ### Tech Stack: Python ### Why It Matters: This setup showcases how to efficiently deploy and scale machine learning models using a microservice architecture, ensuring minimal overhead and optimal performance.

### Overview: OCR LAB is a Python-based image-to-text OCR service powered by PaddleOCR, providing structured text extraction with confidence scores and bounding boxes. It supports JSON and CSV export, and offers an authenticated inference API. ### Key Features: - **Projects**: Group photos by project for easy management. - **Upload & OCR**: Batch OCR uploads with real-time status updates. - **Results**: Thumbnails, text, bounding boxes, and confidence scores. - **Search**: Free text or regex search over OCR'd text. - **Export**: JSON and CSV exports with customizable fields. - **API**: Generate an API key for authenticated access. - **Quota & Usage**: Counter for photo processing, OCR runs, and API calls. - **Bilingual**: Toggle between English and Indonesian. ### Tech Stack: Flask + vanilla JS, no build step. ### Why It Matters: OCR LAB simplifies the process of extracting structured text from images and documents, making it accessible and efficient for developers and businesses alike.