Projects

### Overview: End-to-end MLOps pipeline featuring a RandomForest classifier served via Flask, containerized, with CI to GHCR and a Render deployment. ### Key features: - **Machine Learning Model:** RandomForest classifier trained on the classic Iris dataset, achieving ~96%+ test accuracy. - **Interactive Frontend:** Stunning, responsive dark-mode UI with glassmorphic cards, sliders for feature inputs, and real-time auto-predictions as you drag the sliders. - **REST API:** Exposed `/health` and `/predict` endpoints served via production WSGI (Gunicorn). - **Automated CI/CD:** GitHub Actions workflow that automatically runs unit tests on every push, builds a Docker container, trains the model inside the container for reproducibility, and pushes the image to GitHub Container Registry (GHCR). - **Cloud Deployment:** Production configuration for a one-click deployment using Render Blueprints (`render.yaml`). ### Tech stack: Python (scikit-learn, Flask, pytest), Docker, GitHub Actions, Render. ### Why it matters: This project provides a complete, production-ready MLOps pipeline that demonstrates the entire lifecycle of a machine learning project, from model training to deployment and automated CI/CD.

### Overview: This project is a high-performance C++ AI/ML inference engine that supports both CPU and GPU inference using ONNX Runtime and optional LibTorch. It is cross-platform, available via CMake, Docker, and Codespaces, and includes asset scripts and ImageNet labels. ### Key Features: - **ONNX Runtime** (CPU default, GPU opt-in) for efficient AI/ML inference. - **LibTorch** (optional) for GPU inference, enhancing performance. - Cross-platform support via CMake, Docker, and Codespaces. - Auto-fetching of models/assets and asset scripts. - CLI and REST API for easy integration. - Cross-platform CI and PR merge gating. - Future support for multi-model and GPU training. ### Tech Stack: C++ ### Why it Matters: This project is essential for developers looking to integrate high-performance AI/ML inference into their projects, offering a robust and scalable solution that can be easily integrated across different platforms and environments.

### Overview: This project is an end-to-end MLOps pipeline implemented with FastAPI, MLflow, and scikit-learn. It features a champion/challenger promotion workflow, where new versions of a text classification model are trained, compared against the current champion, and only promoted if they perform better. The pipeline includes all necessary components for a full training, promotion, and prediction cycle, with Dockerized CI that smoke-tests the entire process. ### Key Features: - **Training**: New versions of the model are created and logged to MLflow. - **Promotion**: The latest version is compared against the current champion, and only the one that performs better is promoted. - **Prediction**: Predictions are served from the in-memory champion model. ### Tech Stack: - Python ### Why It Matters: This project is crucial for anyone involved in the development and deployment of machine learning models, particularly in the context of MLOps. It ensures that models are continuously improved and optimized, and that the best version is always serving predictions, reducing the risk of regressions and ensuring model reliability.

Overview: A developer-friendly inference stack designed for Nvidia DGX Spark, featuring a shell-based interface and primarily VLLM-based models. Key features: - Shell-based inference interface - VLLM-based models - Single Spark deployment Tech stack: Shell Why it matters: Ideal for developers seeking a straightforward and cost-effective solution for AI inference on a DGX Spark setup.

### Overview: This project is a historical machine learning regression project for predicting used-vehicle prices. It involved the use of XGBoost stacking and extended with serverless inference on AWS Lambda, API Gateway, ECR, Docker, and SAM. ### Key Features: - **Modeling**: Utilized XGBoost for regression. - **Serverless Inference**: Implemented using AWS Lambda, API Gateway, ECR, and SAM. - **Docker Container**: Containerized the model for deployment. - **AWS Resources**: Used AWS Lambda, API Gateway, ECR, Docker, and SAM for serverless inference. ### Tech Stack: - **Python**: 3.10 - **XGBoost**: Ensemble Regressor - **Docker**: Lambda Image - **AWS**: Lambda, API Gateway, ECR, Docker, SAM ### Why It Matters: This project demonstrates the practical application of machine learning in real-world scenarios, showcasing how to leverage serverless computing for efficient model deployment. It also highlights the importance of reproducibility and documentation in ensuring the reliability and transparency of the project.

**Overview:** This project is a production-style GCP MLOps predictive maintenance platform that demonstrates the entire workflow from model development to containerized serving, CI/CD readiness, and deployment evidence. It uses FastAPI for the prediction service, Docker for containerization, and Cloud Run for deployment, with Vertex AI lifecycle mapping and CI/CD integration. **Key features:** - Synthetic predictive maintenance data generation - Data ingestion, validation, and quality summaries - Feature engineering for sensor, rolling-window, lifecycle, and maintenance history features - Local model training, evaluation, feature importance, and approval gates - FastAPI prediction service with local model serving and future Vertex AI proxy mode - Docker, Artifact Registry, Cloud Run, Cloud Build, and GitHub-triggered deployment readiness - Cloud Run revisions, traffic splitting, and rollback evidence planning - Vertex AI custom training, Model Registry, endpoint, online prediction, and batch prediction mapping - Cloud Composer / Airflow and Vertex AI Pipelines / Kubeflow orchestration design - Explainability, monitoring, model versioning, and evidence organization docs **Tech stack:** Python **Why it matters:** This project showcases the entire GCP MLOps lifecycle, from data generation and validation to model training, deployment, and monitoring, providing a comprehensive guide for industrial predictive maintenance platforms. **Repository:** Ibraheemolasupogit/gcp-vertex-ai-mlops-predictive-platform **GitHub stars:** 0 **GitHub description:** Production-style GCP MLOps predictive maintenance platform with FastAPI serving,

Overview: SmarterRouter is an advanced LLM gateway designed for local AI labs, featuring semantic caching, model profiling, and automatic failover mechanisms. Key features: - Semantic caching - Model profiling - Automatic failover Tech stack: Python Why it matters: SmarterRouter enhances the performance and reliability of AI labs by optimizing model usage and ensuring smooth operation, making it a valuable tool for developers and researchers.

### Overview: This repository provides Kubernetes manifests for setting up an AI inference gateway using the Kubernetes Gateway API. It serves as a reference for deploying an AI inference service on a Kubernetes cluster. ### Key Features: - **Kubernetes Gateway API Manifests**: Contains essential resources for setting up a gateway, including HTTPS listeners, model services, and routing configurations. - **Tenant Namespaces**: Defines separate namespaces for different tenants, ensuring isolation and scalability. - **Routing Patterns**: Includes multiple routing patterns for different tenants, allowing for complex and configurable route management. - **Reliability**: Ensures high availability and low latency through Pod Disruption Budgets and rate limiting. - **Observability**: Monitors and logs the gateway's performance and access patterns. - **Security**: Implements network policies and quota management to secure the gateway and limit resource usage. ### Tech Stack: Kubernetes, Kubernetes Gateway API, Cert-Manager, Prometheus, Istio, and Helm. ### Why it Matters: This repository serves as a practical guide for developers looking to deploy AI inference services on Kubernetes, ensuring scalability, security, and observability.

Overview: MLPulse is a production-ready real-time feature store, statistical drift detector, and model shadowing platform designed for enterprise-grade machine learning and MLOps engineers. Key features: - Dual Online/Offline Feature Store - Point-in-Time (PIT) Correct Joins - Statistical Drift Detection Engine - Automated Model Shadowing - The 6 Security Pillars Tech stack: TypeScript Why it matters: MLPulse addresses the three catastrophic failure modes of machine learning systems in production, ensuring real-time online feature serving, zero training lookahead data leakage, silent data drift detection, and risk-free model deployments. The platform combines a dual online/offline feature store, PIT correct joins, statistical drift detection, automated model shadowing, and security pillars to provide a robust solution for enterprise-level machine learning and MLOps engineers.

Overview: This project is a production-grade FastAPI service designed to predict apartment prices based on 13 property features. It includes health checks, metadata endpoints, and API tests. Key features: - Predicts apartment prices using a trained regression model - Implements Pydantic validation for input data - Provides health and model-metadata endpoints for operational visibility - Runs on Python, using FastAPI, Pydantic, NumPy, and Uvicorn Tech stack: Python, FastAPI, Pydantic, NumPy, Joblib, Uvicorn Why it matters: This project demonstrates API design, ML model serving, and operational visibility in a production environment.

Overview: A simple, self-hosted vLLM solution with a Git-friendly YAML configuration for models and profiles, offering an intuitive CLI and TUI for a seamless development experience. Key features: - Self-hosted vLLM solution - Git-friendly YAML configuration - Intuitive CLI and TUI for easy management Tech stack: Python Why it matters: This project simplifies the self-hosting of vLLM models, providing a straightforward YAML configuration and a user-friendly CLI and TUI, making it ideal for developers looking to integrate vLLM into their projects with minimal setup.

### Overview: Type-Correcter-Ai is a Flask web application designed to correct typos in text by leveraging a character-level sequence-to-sequence neural network trained on the Ghost-Type-Corrector model. This project serves as part of a series of four projects exploring automatic typing correction, focusing on the integration of a model trained in a previous phase. ### Key Features: - **Flask Web App**: A simple web interface for correcting typos in text. - **Character-Level Seq2Seq Model**: Uses a character-level encoder-decoder LSTM to correct typos. - **NumPy Only**: The model is implemented entirely with NumPy, without the need for TensorFlow, making the server lightweight and fast. - **Dictionary Gate**: Ensures the model remains safe to run over real text by leaving known words, short words, and suggestions unchanged. - **Beam Search**: Uses beam search to pick the best real-word candidate, improving the correction rate. ### Tech Stack: - **Python**: The primary programming language used. - **NumPy**: Used for implementing the model and performing inference. ### Why It Matters: This project is part of a series exploring automatic typing correction, demonstrating the integration of a model trained in a previous phase. The use of NumPy only ensures the server is lightweight and starts instantly, making it suitable for deployment in a web application. ### Repository: - **Repository**: `JampaniKomal/Type-Correcter-Ai` - **GitHub Stars**: 0 - **GitHub Description**: Flask web app that fixes typos in text, serving a

Overview: This repository trains a simple gradient-boosted regressor model on California Housing and then builds a production-ready service using FastAPI. The service includes versioned artifacts, checksum verification, Prometheus metrics, drift monitoring, and a Docker container for deployment. Key features: - Versioned artifacts with SHA-256 checksums - Promote command for versioning - Request logging and structured logs - Prometheus metrics for monitoring - Drift monitoring based on request logs - Docker container for deployment Tech stack: Python Why it matters: This repository emphasizes the importance of production-level model serving practices, including versioning, checksum verification, and monitoring, which are crucial for ensuring the reliability and security of machine learning models in production environments.

Overview: Model Signature Guard is a compact, dependency-free validator designed to prevent incompatible feature records from entering model inference, ensuring safe and reliable model inputs. Key features: - Validates feature schema and ranges - Blocks incompatible records before inference - Does not detect feature drift or model accuracy Tech stack: Python Why it matters: Ensures model inputs are safe and compatible, preventing potential issues during inference and maintaining model integrity. Repository: Jdawgboo/model-signature-guard Primary language: Python GitHub stars: 0 GitHub description: Feature schema validator for safe model inference inputs and inference-time drift detection.

### Overview: Self-host any Hugging Face LLM on your own NVIDIA GPU behind an OpenAI-compatible API. This project allows you to run any Hugging Face model on your own NVIDIA GPU, serving it via the vLLM API. Models are cached locally, making switching between them quick and efficient. ### Key features: - **Any Hugging Face model** on your own NVIDIA GPU, served by vLLM. - **OpenAI-compatible API**, ensuring existing clients and SDKs work unchanged. - **One-command setup** that checks each model fits your GPU before downloading it. - **Fast switching**: models are cached locally, so changing models doesn't re-download them. ### Tech stack: Docker with NVIDIA GPU support. ### Why it matters: - **Efficiency**: Models are cached locally, reducing the need to re-download them. - **Ease of use**: The setup process is simple and requires minimal configuration. - **Flexibility**: Supports any Hugging Face model and can be easily integrated into existing workflows. - **Security**: The API key is optional, and the models are cached securely. ### Repository: Jimdrews/host-your-own-llm ### Primary language: Python ### GitHub stars: 0 ### GitHub description: Self-host any Hugging Face LLM on your own NVIDIA GPU behind an OpenAI-compatible API. One CLI, vLLM in Docker, models cached in a local S3 store.

Overview: This project is an end-to-end MLOps platform that self-hosts on a homelab using Terraform, K3s, ArgoCD, MLflow, BentoML, Evidently AI, Prometheus/Grafana, and more. It provides a production-ready pipeline for model experimentation, deployment, and monitoring. Key features: - Model experimentation and training - Automated model promotion via GitOps - Continuous integration and deployment using GitHub Actions - Model serving with BentoML and Prometheus/Grafana monitoring - Data drift detection with Evidently AI - Self-hosted infrastructure using Proxmox and K3s Tech stack: Terraform, K3s, ArgoCD, MLflow, BentoML, Evidently AI, Prometheus/Grafana, and more. Why it matters: This platform offers a zero-cloud-cost solution for MLOps, providing a complete end-to-end pipeline from model creation to production deployment and monitoring, all running on a self-hosted homelab.

Overview: A comprehensive inference stack designed for the Nvidia DGX Spark, providing a scalable and efficient solution for AI research and development. Key features: - Single Spark deployment - VLLM-based inference engine - Easy to set up and use - Primarily written in Shell Tech stack: Shell Why it matters: This stack is ideal for researchers and developers looking to perform AI inference on a desktop, making advanced AI research accessible to all.

### Overview: This project simulates the KV-cache fragmentation dynamics in continuous batching LLM servers, focusing on comparing contiguous vs paged allocation and compaction policies across sustained load intensities and request size distributions. ### Key Features: - **Fragmentation Growth Measurement:** Tracks fragmentation growth over time. - **Workload Intensity and Request Size Analysis:** Measures fragmentation under different workload intensities and request size distributions. - **Comparison of Allocation and Compaction Policies:** Compares contiguous vs paged allocation and compaction policies across various scenarios. ### Tech Stack: - Python - NumPy - Pandas ### Why It Matters: This simulator provides a comprehensive understanding of how KV-cache fragmentation accumulates over time in continuous batching LLM servers, offering insights into optimal allocation and compaction policies for improved server performance. ### Repository: - **GitHub Repository:** JohnScheuer/continuous-batching-fragmentation-sim - **GitHub Stars:** 0 - **Primary Language:** Python - **GitHub Description:** Discrete-event simulation of KV-cache fragmentation dynamics in continuous batching LLM servers, comparing contiguous vs paged allocation and compaction policies over sustained load. - **Status:** Complete - **License:** MIT License ### Output Files: - **timeseries.csv:** Records fragmentation over time, per tick. ### Quick Start: 1. Clone the repository: ```bash git clone https://github.com/JohnScheuer/continuous-batching-fragmentation-sim ``` 2. Navigate to the project directory: ```bash cd continuous-batching

Overview: This project simulates speculative KV-cache prefetching for multi-turn language models, focusing on reducing reload stalls, improving hit rates, and minimizing wasted GPU prefetch bandwidth. Key features: - Simulates KV-cache prefetching for LLM conversations - Measures stall reduction, hit rate, and waste of GPU prefetch bandwidth - Quantifies the impact of speculative prefetching on reload stalls and hit rate - Evaluates different prefetching policies Tech stack: Python Why it matters: This benchmark helps developers understand the effectiveness of speculative prefetching in reducing reload stalls and improving performance in multi-turn LLM conversations, providing insights into optimizing GPU usage and reducing waste in prefetching. Repository: JohnScheuer/kv-cache-prefetch-bench

### Overview This project simulates LLM serving workloads with discrete-event simulation, focusing on SLA-aware cluster sizing and Prefill/Decode resource optimization. It includes a Python-based planning and analysis layer, providing insights into GPU requirements, resource splits, and performance metrics. ### Key Features - **SLA-Aware Cluster Sizing:** Determines the minimum number of GPUs required to meet a P99 latency SLA. - **Prefill/Decode Resource Split Optimization:** Optimizes GPU usage between Prefill and Decode stages to minimize latency and maximize resource efficiency. - **Poisson Arrival Process:** Models realistic LLM serving workloads with prompt heterogeneity. ### Tech Stack - **C++20:** Implemented in the core simulator. - **Python:** Used for planning, analysis, and simulation. ### Why It Matters This project provides essential insights into LLM serving performance, helping developers optimize GPU usage and resource allocation to meet SLA requirements efficiently. ### GitHub Stars 0 ### Repository - **Repository:** JohnScheuer/llm-serving-trace-replay - **Primary Language:** Python - **GitHub Stars:** 0 - **GitHub Description:** Trace-driven discrete-event LLM serving simulator with SLA-aware GPU cluster sizing and prefill/decode resource optimization. ### Docs - **Results / Analysis:** <!--citation:7--> - **Engine (C++20):** `src/` - **Trace Generator (Python):** `traces/` - **Planner / Sweeps (Python):** `planner/