Projects

Overview: This project is a production-style Go REST API designed to serve churn predictions from a logistic regression model trained in Python. The model is exported as a portable JSON artifact, and the Go service handles preprocessing and inference without calling Python at request time. Key features: - Portable JSON model artifact - Go inference API and tests - JSON validation - Batch prediction - Metrics tracking - Docker support Tech stack: Go Why it matters: This project focuses on the end-to-end serving workflow for reproducible, validated, and reliable ML services, making it useful for anyone working on similar projects. Repository: mobster1425/go-ml-inference-service Primary language: Go GitHub stars: 0 GitHub description: Production-style Go inference service for a Python-trained ML model, with JSON artifact export, validation, batch prediction, metrics, tests, and Docker support.

Overview: This project serves as a production-ready FastAPI/Uvicorn-based gateway for Google TimesFM's time-series forecasting. It is designed to efficiently handle incoming requests and route them to the appropriate models for accurate predictions. Key features: - High-performance FastAPI/Uvicorn backend - Integration with Google TimesFM's time-series forecasting models - Scalable architecture for handling large volumes of requests Tech stack: Python Why it matters: This project is crucial for developers looking to integrate Google TimesFM's forecasting models into their applications, ensuring accurate and timely predictions. The use of FastAPI and Uvicorn allows for efficient and scalable deployment, making it ideal for production environments.

### Overview: This project is a RoBERTa sentiment and sarcasm inference API service built using FastAPI, Docker, PostgreSQL, Terraform on AWS, and CI/CD. The service is designed for on-demand deployment, monitoring, and performance improvements. ### Key features: - **Model Serving**: FastAPI, pinned model versions, request validation, PostgreSQL prediction records, and labelled feedback. - **Package and Deploy**: Docker image with the model loaded offline, Terraform-managed AWS infrastructure, and GitHub Actions deployment with OIDC and live smoke tests. - **Observe and Improve**: Prometheus metrics, Grafana dashboards, reproducible CPU load benchmarks, and dynamic request batching. ### Tech stack: - Python ### Why it matters: This service is a fast, reliable, and scalable solution for sentiment and sarcasm classification, with reproducible model serving and performance improvements. It can be easily deployed and monitored, making it ideal for real-world applications. ### Repository: - Repository: mohammadabdalaziz241/review-classifier-service - Primary language: Python - GitHub stars: 0 - GitHub description: RoBERTa sentiment & sarcasm inference API: FastAPI, Docker, PostgreSQL, Terraform on AWS, CI/CD, Prometheus/Grafana monitoring, dynamic batching (+79% throughput). ### Measured outcomes: - Sentiment inference through the HTTP API: 0.8987 macro-F1 on 2,183 test texts, within 0.0005 of the training result. - S

Overview: This project is a microservice architecture designed for serving machine learning models on AWS EKS, featuring tenant isolation, resource governance, and path-based routing. It demonstrates a deployment pattern commonly used by ML platforms in regulated industries like healthcare and payer analytics. Key features: - **Tenant Isolation**: Kubernetes namespaces ensure different lines of business, model versions, or customer segments are cleanly separated on shared infrastructure. - **Resource Governance**: `ResourceQuota` caps compute and pod counts per tenant, enforcing service-level agreements across services. - **Path-Based Routing**: Envoy routes requests to the appropriate Flask service based on the URL path, allowing A/B testing and request type routing without exposing the internal topology. - **Job-Based Inference**: Kubernetes Jobs run the model inference for each request, reclaiming compute resources after each request, which is better suited for batch-style scoring workloads. - **Namespace-Scope RBAC**: Service accounts can only create resources in their own namespace, aligning with least-privilege requirements common in HIPAA-aware environments. Tech stack: Python 3.9, Flask, PyTorch, Amazon EKS, kubectl, eksctl, Docker, DockerHub registry, Envoy, Kubernetes Jobs, AWS Network Load Balancer, EC2 (t3.medium nodes), MNIST, KMNIST. Why it matters for ML / healthcare ML: This project implements the deployment scaffolding that real ML platforms need, particularly in regulated industries like healthcare and payer analytics. It ensures tenant isolation, resource governance, and path-based routing, aligning with least-priv

### Overview: This repository showcases a production-ready Python backend development project that integrates FastAPI for RESTful APIs, Pydantic v2 for data validation, and a machine learning pipeline using Scikit-learn. The project includes a patient management system API, an insurance premium classifier API, and a Streamlit frontend interface for real-time predictions. ### Key features: - **Patient Management System API**: Built with FastAPI, this API allows CRUD operations on patient records. - **Insurance Premium Classifier API**: Leverages FastAPI for serving predictions and integrates a Scikit-learn model pipeline. - **Streamlit UI**: Provides an interactive interface for users to input data and view predictions. ### Tech stack: - **Python**: 3.10+ - **FastAPI**: 0.100+ - **Pydantic v2**: v2 - **Streamlit**: 1.28+ - **Scikit-learn**: ML Pipeline ### Why it matters: This project demonstrates the integration of modern Python backend development, robust data validation, and a comprehensive machine learning pipeline, making it a valuable resource for developers looking to build scalable and user-friendly applications.

### Overview Apron is a decision and evidence system for LLM inference. It helps you choose the right model, API, GPU, and configuration for your task, ensuring the best performance and cost efficiency. ### Key features - **Model Selection**: Recommends the most suitable model based on your task requirements. - **GPU Configuration**: Suggests the optimal GPU configuration for your workload. - **Serving Configuration**: Provides the best serving configuration for efficient inference. - **Deployment Evidence**: Tracks and reports the performance of your deployment, ensuring it meets your quality and latency requirements. ### Tech stack - Python ### Why it matters Apron helps you make informed decisions about your LLM inference, ensuring that your model performs optimally and meets your performance and cost targets.

Overview: SmarterRouter is an intelligent LLM gateway and VRAM-aware router designed for local AI labs, featuring semantic caching, model profiling, and automatic failover. Key features: - Semantic caching - Model profiling - Automatic failover Tech stack: Python Why it matters: SmarterRouter optimizes performance and resource management for AI models, ensuring seamless operation in local AI labs.

### Overview: This project provides a production-ready machine learning model serving API that integrates with various MLOps tools including FastAPI, TensorFlow, Docker, Kubernetes, and Prometheus. It addresses common challenges in deploying machine learning models, such as inconsistent model versions, lack of observability, manual retraining, poor scalability, and unreliable predictions. ### Key Features: - **RESTful API**: Handles model predictions using FastAPI. - **Model Versioning**: Supports multiple model formats. - **Real-time Monitoring**: Integrates with Prometheus and Grafana for performance tracking. - **Data Drift Detection**: Tracks model performance and detects data drift. - **Automated Training Pipelines**: Includes a model registry for training and evaluation. - **Docker & Kubernetes**: Ready deployment using Docker and Kubernetes. - **CI/CD Integration**: Capable of integrating with GitHub Actions for automated training pipelines. ### Tech Stack: - **API Framework**: FastAPI (async/await) - **Machine Learning**: Scikit-learn, TensorFlow, PyTorch - **Monitoring**: Prometheus, Grafana - **Database**: PostgreSQL (async) - **Storage**: Local filesystem, S3, or cloud storage - **Containerization**: Docker, Docker Compose - **Orchestration**: Kubernetes manifests - **CI/CD**: GitHub Actions ### Why It Matters: This project offers a production-grade, end-to-end MLOps platform that ensures reliable, versioned model serving, real-time monitoring, automated drift detection, CI/CD-integrated training pipelines, scalable, cloud-native deployment, and

### Overview: This repository serves as a technical blog for developers, focusing on AI, MLOps, model serving, evaluation, and reproducible engineering practices. It aims to provide clear explanations, practical takeaways, and links to related projects, ensuring honesty and next steps. ### Key Features: - Vertex Generative AI Overview - Building an AI/MLOps Portfolio - Making model quality gates control the serving path ### Tech Stack: Makefile ### Why It Matters: By sharing my thought process and practical insights, this blog helps developers understand AI and MLOps better, making them more effective in their projects.

### Overview: This project provides a complete ML infrastructure on Kubernetes, including model serving, deployment with Helm, Argo CD, MLflow tracking, Prometheus metrics, and drift checks. It includes a deterministic linear rental scorer model and a lightweight scoring library for offline testing. ### Key features: - **Model Serving API**: FastAPI-based model serving API with health, prediction, and metrics endpoints. - **Scoring Library**: Lightweight scoring library and model metadata file for offline tests. - **Dockerfile**: Dockerfile for the model API. - **Helm Chart**: Helm chart for Kubernetes deployment, service, HPA, config, and service account. - **Argo CD**: Argo CD Application for GitOps delivery. - **Prometheus ServiceMonitor**: Prometheus ServiceMonitor and Grafana dashboard starter. - **MLflow Tracking Server**: MLflow tracking server local compose file. - **Input-Range Warnings**: Input-range warnings (not statistical drift detection) and documented model registry handoff. - **CI Tests**: CI tests that validate code, chart structure, and platform artifacts. ### Tech stack: Python ### Why it matters: This project demonstrates the complete setup of an ML infrastructure on Kubernetes, including model serving, deployment with Helm, Argo CD, MLflow tracking, Prometheus metrics, and drift checks. It provides a tested API packaging, explicit artifact identity, Helm deployment configuration, and Argo CD manifest, making it a valuable resource for developers looking to build and deploy ML models on Kubernetes.

### Overview: This project is a fully reproducible MLOps pipeline for rental price prediction, showcasing the entire lifecycle from data validation to model deployment. It includes Docker and Kubernetes manifests, artifact gating, CI/CD, and FastAPI serving for local and cloud deployment. ### Key Features: - Data validation and summary reporting - Train/test evaluation against a mean-price baseline - Quality gate with RMSE, MAE, R2, and baseline-lift thresholds - Generated model card and JSON quality report - Exportable model artifact with metadata and quality report - Local registry record with quality decision and image metadata - FastAPI serving layer with input-range monitoring warnings - Docker Compose API runtime - Helm chart and Argo CD Application for Kubernetes deployment - PrometheusRule, ServiceMonitor, and Grafana dashboard starters - CI checks for tests, compile safety, model quality, and pipeline compilation - Make targets for install, test, compile, evaluate, reports, artifact, registry, serve, and clean ### Tech Stack: - Python - Docker - Kubernetes - Helm - FastAPI - Argo CD - CI/CD - CI/CD ### Why It Matters: This project demonstrates a comprehensive MLOps pipeline that includes reproducibility, quality gates, CI/CD, and local and cloud deployment, making it a valuable resource for developers looking to build and deploy MLOps models.

### Overview: This project is an end-to-end AI/ML system designed for 16-day retail sales forecasting across 1,782 store-family series. The system evaluates 3 Ridge and 27 XGBoost configurations across 4 chronological folds, then packages the selected model for batch and API inference. ### Key features: - **Model Development**: Evaluates 3 Ridge and 27 XGBoost configurations across 4 chronological folds. - **Forecasting**: Produces 28,512 planning-ready forecasts per run. - **Error Analysis**: Consistent view of forecast errors across stores, categories, promotions, and holidays. - **Verification**: Historical batch/API/container verification matches every prediction. ### Tech stack: Python ### Why it matters: This project provides a comprehensive solution for retail planners, ensuring a consistent view of expected category sales before the next cycle begins. The system handles varying demand across different stores, product families, promotions, weekly patterns, and local events, producing detailed forecasts that are crucial for effective planning and decision-making.

Overview: This project showcases the complete lifecycle of an end-to-end fraud detection system, from raw data ingestion to deployment and automated retraining. It uses MLflow, FastAPI, drift detection, and automated retraining to demonstrate real-world engineering practices. Key features: - Data Ingestion - Data Cleaning - Feature Engineering - Model Training with MLflow - Model Registry - FastAPI Inference - Prediction Logging - Drift Detection with Evidently - Automated Retraining Tech stack: Python, scikit-learn, XGBoost, MLflow, FastAPI, Evidently AI, Git + GitHub. Why it matters: This project provides a comprehensive guide on building and deploying a production ML system, covering everything from data preparation to automated retraining, all in a way that mirrors industry practices. Repository: munzahmed07/mlops-fraud-detection Primary language: Python GitHub stars: 0 GitHub description: End-to-end fraud detection system with MLflow, FastAPI, drift detection, and automated retraining README excerpt:   ![FastAPI](https://img.shields.io/badge/FastAPI-Serving-green?style=for-the-badge&logo

### Overview: Axon is a Triton-compatible model server written in Rust, providing CPU and NVIDIA GPU inference capabilities. It supports ONNX Runtime, gRPC, and HTTP/REST for KServe v2, and includes features like BLS scripting, YAML configuration, and multi-GPU support. ### Key Features: - **KServe v2 API**: HTTP + gRPC, full model management. - **GPU Inference**: CUDA and TensorRT execution providers, per-model `KIND_GPU`/`KIND_CPU` placement, multi-GPU round-robin. - **Web Console**: Built-in UI at `/ui` (`--ui`), model list with device/platform, schema viewer, and inference playground. - **API Key Auth**: Optional Bearer token on HTTP + gRPC (health probes stay open). - **Ensemble Pipelines**: Declarative multi-model chaining via config. - **BLS (Rhai Scripting)**: Custom pre/post processing logic. - **23 Builtin Functions**: ML, NLP, CV, tabular preprocessing/postprocessing. - **HuggingFace Tokenizer**: Native tokenizer.json support. - **Structured Logging**: JSON file (daily rotation) + OTEL + filtered stdout. - **Inference Timeout**: Configurable per-server timeout. - **Circuit Breaker**: Auto-skip models that fail to load. - **Prometheus Metrics**: Latency, throughput, inflight, queue wait. - **Docker Multi-arch**: Linux/amd64 + Linux/arm64, plus a CUDA image

Overview: MLauncher is an enterprise-grade machine learning model deployment platform that simplifies the transition from experimental models to production services. It provides a unified interface for serving, monitoring, and managing models across various environments, ensuring reliability and performance. Key features: - Unified interface for model serving, monitoring, and management - Production-ready framework for seamless transitions between development, staging, and production environments - Adaptive serving engine for optimizing inference performance based on model characteristics and hardware capabilities - Multi-tenant, horizontally scalable architecture with clear separation of concerns and resource management Tech stack: Python Why it matters: MLauncher addresses the challenges of transitioning machine learning models from experimental to production services, providing a comprehensive and extensible framework that ensures enterprise-grade reliability and performance.

Overview: rs-llmctl is an enterprise-grade operations tool designed for running local and private LLM models, providing a default Rust binary, OpenAI-compatible serving, model lifecycle commands, resource budget planning, and observability. Key features: - A single Rust binary for operations - OpenAI-compatible serving endpoint - Model lifecycle commands - Resource budget planning and reporting - Quotas and audit trails - Usage reporting - OTel-ready observability Tech stack: Rust Why it matters: rs-llmctl simplifies the process of delivering and managing LLM models, making the operation of these models as straightforward as operating a real server infrastructure.

Overview: This project provides an inference stack for Nvidia DGX Spark, a high-performance AI supercomputer. It is designed for developers looking to deploy large language models on their desktops. Key features: - Inference stack for Nvidia DGX Spark - Mostly VLLM based - Single Spark deployment Tech stack: Shell Why it matters: This stack is useful for developers who want to experiment with large language models without the need for a dedicated AI supercomputer, making it accessible for the not-so-rich AI enthusiasts.

Overview: Ce dépôt contient une chaîne MLOps complète pour le prédiction de la variété d'un Iris, basée sur un dataset Iris et une régression logistique. L'objectif est de montrer la rigueur du pipeline, sans la difficulté de la donnée. Key features: - Entraînement (`backend/ml/train.py`) - Registre de modèle (`backend/scripts/register_model.py`) - Service (`backend/app/api.py`) - Interface (`frontend/`) - Conteneurisation (`backend/Dockerfile`, `frontend/Dockerfile`, `docker-compose.yml`) - Tests (`backend/tests/`) - Documentation (`docs/`) Tech stack: Python, MLflow, FastAPI, Docker, MkDocs, Azure ML, pytest, MkDocs, Docker, docker-compose, GitHub Actions Why it matters: Cette chaîne complète et reproductible montre comment un modèle peut être entraîné, documenté, et déployé en utilisant des outils standard comme MLflow, FastAPI, Docker, et MkDocs.

Overview: Building mission-critical machine learning systems requires addressing various engineering disciplines such as training-serving skew, covariate shift, inference latency spikes, and catastrophic deployments. Production ML Systems provides a comprehensive design and testing suite with dual-storage feature stores, high-throughput model serving, statistical drift observability, and resilient inference capabilities. Key features: - Dual-storage feature stores with point-in-time correctness - High-throughput model serving - Statistical drift observability (PSI / KS-test) - Resilient inference Tech stack: Python 3.10+ with support for MLOps and Observability features. Why it matters: This suite is essential for ensuring the reliability and performance of mission-critical machine learning systems, addressing common issues in production environments and providing a robust solution for enterprise-grade ML platforms.

Overview: A developer-friendly inference stack for Nvidia DGX Spark, designed for rich and not-so-rich AI enthusiasts alike. Key features: - VLLM-based inference engine - Single Spark deployment - Shell-based tech stack Tech stack: Shell Why it matters: Ideal for those looking to experiment with AI without the hefty cost of a full DGX system.