Projects

Overview: This repository contains a small demonstration of deploying machine learning models as an online API endpoint using Flask, a lightweight framework for building web applications. Key features: - Deploying ML models as an online API endpoint - Using Flask for inference server implementation - Accessing pretrained models through HTTP requests Tech stack: Python Why it matters: This repository provides a practical example of deploying machine learning models, showcasing how to use Flask to create an inference server that can be accessed via HTTP requests, making it easy to integrate ML models into web applications.

### Overview: This project is a production model serving system built with FastAPI, Docker, Prometheus/Grafana monitoring, A/B testing, shadow deployment, and CI/CD. It demonstrates the full production serving layer, including containerized inference, model version management, A/B testing, and automated evaluation gates. ### Key features: - **FastAPI** for the API layer - **Docker** for containerization - **Prometheus/Grafana** for monitoring - **A/B testing** for validation - **Shadow deployment** for safe validation - **CI/CD** with automated evaluation gates ### Tech stack: Python ### Why it matters: Training a model is just 10% of the work in MLOps. The other 90% is about serving, monitoring, versioning, rollbacks, and A/B testing. This project aims to demonstrate the full production serving layer, from containerized inference to automated evaluation gates, ensuring the model is always ready and performing well.

Overview: This project demonstrates a complete, real-world machine learning deployment workflow, from a trained model to a production-ready REST API served using FastAPI and Docker. Key features: - Dockerized application - Model packaged and deployed in a clean, simple, and professional way - REST API endpoint for predicting Titanic passenger survival - Preprocessing logic applied during both training and inference - Model trained offline using the Titanic dataset Tech stack: Python, FastAPI, scikit-learn, pandas, numpy, Pydantic, Docker, Uvicorn Why it matters: This project shows how a trained ML model can be packaged, served, and deployed in a clean, simple, and professional way, reflecting how ML systems are built in real-world production environments.

### Overview: This project serves as a production FastAPI service for an XGBoost model championed in Project 4. The model is trained on real UCI Bank Marketing data and deployed via Docker container, ensuring robust health checks and performance testing. The project is part of the MLOps portfolio series and is designed for real-world deployment. ### Key Features: - **Model Training**: The model is trained on the real UCI Bank Marketing dataset. - **FastAPI API**: A RESTful API is implemented using FastAPI for serving predictions. - **Docker Containerization**: The model is containerized using Docker for scalability and portability. - **pytest Tests**: All endpoints are tested using pytest for reliability and performance. - **CI/CD Pipeline**: A CI/CD workflow ensures continuous training, testing, and deployment of the model. ### Tech Stack: - **Python**: The primary programming language used for the project. - **FastAPI**: The FastAPI framework is utilized for the backend API. - **Docker**: The project is containerized using Docker for deployment and testing. - **XGBoost**: The XGBoost model is the champion model used in this project. - **pytest**: The testing framework used for all API endpoints. ### Why It Matters: This project showcases the practical application of a champion model in a production environment, demonstrating how to train, deploy, and test a model using the latest technologies in MLOps. It provides a clear example of how to integrate a model into a production system, ensuring reliability and performance. ### Repository: The project is

Overview: SmarterRouter is an intelligent LLM gateway and VRAM-aware router designed for Ollama, llama.cpp, and OpenAI. It features semantic caching, model profiling, and automatic failover for local AI labs. Key features: - Semantic caching - Model profiling - Automatic failover Tech stack: Python Why it matters: SmarterRouter enhances the performance and reliability of AI models by optimizing resource usage and ensuring consistent access, making it essential for AI labs and developers.

Overview: SmarterRouter is an advanced gateway and router designed for Ollama, llama.cpp, and OpenAI, featuring semantic caching, model profiling, and automatic failover for local AI labs. Key features: - Semantic caching - Model profiling - Automatic failover Tech stack: Python Why it matters: SmarterRouter simplifies the integration of Ollama, llama.cpp, and OpenAI by providing efficient routing and failover mechanisms, ensuring seamless communication and optimal performance.

Overview: This project is a Dockerized REST API serving a fine-tuned Turkish sentiment-analysis model. It is designed to be easily deployed and accessible via HTTP. Key features: - Fine-tuned Turkish sentiment-analysis model - API endpoint for sentiment analysis - Dockerized for easy deployment Tech stack: Python Why it matters: This project provides a reliable and scalable solution for sentiment analysis, making it ideal for applications requiring natural language processing in Turkish.

### Overview: This project demonstrates the end-to-end MLOps lifecycle, from training a scikit-learn model to deploying it as a production service using FastAPI, Docker, Prometheus metrics, and CI/CD. The model is trained on synthetic data to showcase reproducibility and generalizability. ### Key Features: - **Data Generation:** Synthetic data for training the model. - **Model Training:** Scikit-learn pipeline saved as a single artifact. - **FastAPI Service:** API endpoints for prediction, monitoring, and metrics. - **Swagger Documentation:** Interactive Swagger UI for developers. - **Prometheus Metrics:** Real-time monitoring and metrics. - **Docker Deployment:** Single Dockerfile for reproducibility. - **CI/CD Pipeline:** GitHub Actions for model retraining and tests. ### Tech Stack: Python ### Why It Matters: This project showcases the full lifecycle of a machine learning model from training to deployment, including reproducibility, observability, and CI/CD integration. It provides a clear demonstration of how to build, train, and deploy a model in a production environment, making it useful for developers and MLOps enthusiasts alike.

Overview: This project is a FastAPI-based ML inference service designed for predicting diabetes risk using the Pima Indians Diabetes dataset. It utilizes a scikit-learn model serialized with `pickle` and validated with Pydantic for input fields. Key features: - FastAPI-based inference service - Trained scikit-learn classifier - Pydantic-validated input fields - Synchronous prediction endpoint - Docker containerized for easy deployment Tech stack: Python Why it matters: This service provides a fast and reliable way to predict diabetes risk based on standard Pima Indians Diabetes features, making it useful for applications requiring quick predictions without the need for complex medical diagnostics. Repository: kelly26ici/diabetes-api Primary language: Python GitHub stars: 0 GitHub description: FastAPI ML inference service for Pima diabetes risk prediction — scikit-learn model, Pydantic-validated 8-feature input, containerised

Overview: SmarterRouter is an intelligent LLM gateway and VRAM-aware router designed for local AI labs, featuring semantic caching, model profiling, and automatic failover. Key features: - Semantic caching - Model profiling - Automatic failover Tech stack: Python Why it matters: SmarterRouter optimizes performance and reliability for AI research and development, ensuring smooth operation and efficient resource management.

### Overview: 이 프로젝트는 모델 교체를 안전하게 수행하기 위한 ML Platform를 설계하고 구현하였습니다. 이 플랫폼은 Accuracy와 Drift Gate 기반으로 Promotion / Shadow 분기를 구현하였으며, 모델 교체에 대한 리스크를 통제하였습니다. ### Key Features: - **Accuracy + Drift Gate 기반 Promotion / Shadow 분기**: 모델 교체에 대한 리스크를 통제하기 위해 Accuracy와 Drift Gate 기반으로 Promotion / Shadow 분기를 구현하였습니다. - **SSOT (Single Source of Truth)**: 모든 식별자/정책은 중앙 정의, 문자열 하드코딩 제거, TaskGroup prefix mismatch 방지 등으로 안전성을 확보하였습니다. - **Promotion / Shadow 전략**: Accuracy + Drift Gate 기반으로 분기를 구현하였으며, 모델 교체에 대한 리스크를 통제하였습니다. ### Tech Stack: - Python ### Why it Matters: 이 플랫폼은 모델 교체에 대한 리스크를 통제하고, 운영 환경에서 안전하게 모델 교체를 수행하는 데 큰 도움이 되었습니다.

Overview: This repository provides optimized, production-ready Dockerfiles for building Debian-based images for LLM inference on AMD GPUs using ROCm and vLLM. It offers two types of images: a wheel-based image for faster builds and minimal runtime, and source-build images for hardware-specific tuning and registry layer chunking. Key features: - A wheel-based image for vLLM 0.22 with ROCm 7.2.2, featuring a 2-stage Dockerfile, Python 3.12 via `uv venv`, and minimal runtime. - Source-build images for vLLM 0.14 and 0.8.5 with manual compilation of MoRi, FlashAttention, and aiter from source, targeting MI210 and MI300X GPUs. - Hardware-specific tuning and registry layer chunking for the source-build images. - Supported versions: vLLM 0.22, vLLM 0.14, and vLLM 0.8.5. Tech stack: Dockerfile. Why it matters: This repository provides a balanced approach to running vLLM on AMD GPUs, combining the best of both worlds: the latest upstream vLLM features and patches with the exact ROCm versions and AMD-specific optimizations needed for enterprise MLOps environments. Repository: kevinbazira/vllm-rocm-debian-images

### Overview: The **Wikipedia Companion** is a friendly AI tutor that uses local models to read Wikipedia facts aloud. It's designed for users in Uganda, where internet access is limited, and it offers a warm, engaging experience by running all AI components locally, fetching only the Wikipedia text from the internet. ### Key Features: - **Speech-to-Text:** Uses Whisper for fast transcription. - **Query Understanding:** Runs Qwen2.5-3B locally for understanding questions. - **Semantic Retrieval:** Utilizes all-MiniLM-L6-v2 for semantic ranking. - **Text Normalization:** Uses NVIDIA NeMo for text normalization. - **Speech Synthesis:** Uses Kokoro for lip-sync and reading. - **Follow-Along Reading & Lip-Sync:** Uses Wav2Vec2-CTC for forced alignment. ### Tech Stack: - Python - Docker - Whisper (local) - Qwen2.5-3B (local) - all-MiniLM-L6-v2 (local) - NVIDIA NeMo (local) - Kokoro (local) - Wav2Vec2-CTC (local) ### Why It Matters: This app is built for the community in Uganda, where internet access is limited, and it offers a warm, engaging experience by running all AI components locally, fetching only the Wikipedia text from the internet. It's a perfect solution for users who want to learn from Wikipedia facts without the need for expensive internet data.

### Overview: A Text-to-Speech (TTS) service prototype that fetches Wikipedia articles, cleans the text, splits it by section, and generates `.mp3` files asynchronously. ### Key features: - Fetches Wikipedia articles - Cleans the text by removing citation brackets and edit tags - Splits content by section - Generates `.mp3` files asynchronously ### Tech stack: Python ### Why it matters: This prototype serves as a foundational tool for developing and testing TTS functionalities, ensuring the service can efficiently handle large amounts of text data and generate high-quality audio outputs.

Overview: A production-style KServe platform designed for Kubernetes inference operations, featuring champion/challenger rollout, shadow scoring, request validation, canary gates, and rollback controls, along with observability and guided study artifacts. Key features: - Champion and challenger model aliases - Canary traffic routing - Shadow scoring for champion-routed requests - Request validation with a documented prediction contract - Idempotent prediction handling by `request_id` - Structured prediction logs - Latency, error rate, throughput, route mix, and score distribution monitoring - Canary promotion gates - Model rollback to previous champion - Minikube/KServe migration notes Tech stack: Python Why it matters: This platform provides a production-ready solution for model serving, ensuring high availability, scalability, and observability, making it ideal for production environments. Repository: kevinmeix1/kserve-model-serving-platform Primary language: Python GitHub stars: 0 GitHub description: Production-style KServe platform with Open Inference V2, canary and rollback controls, a tested ServeOps console, observability, and guided study artifacts.

### Overview: This project serves the game-retrieval-bench retrieval pipeline over HTTP, including a load characterization that answers the offline benchmark's inability to determine which retrieval strategy is actually servable under a latency SLO. The service is CPU-only, single-instance, and reproducible on commodity hardware. ### Key features: - **Async Dispatch**: Synchronous, CPU-bound pipeline runs on an owned thread pool. - **In-Process LRU Cache**: Keyed by `(query, strategy, k)`. - **Per-Client Token-Bucket Rate Limiting**: Rate-limited to 429 and `Retry-After`. - **Load-Shed Degradation**: Routes `hybrid_rerank` requests to `hybrid` under budget pressure. - **Per-Request Timeout Backstop**: `504` for genuine hangs. - **Per-Request Metrics**: `POST /search` runs a selectable strategy through the shared pipeline. - **Load-Characterization Metrics**: `GET /metrics` provides counts, latency percentiles, stage timing means, cache hit-rate, degraded/rate-limited counts, and pool depth. - **Structured Per-Request Logging**: Query *hash*, never the raw query. ### Tech stack: Python ### Why it matters: The project measures the performance of the retrieval pipeline under a latency SLO, providing insights into which strategies are actually servable and where the time goes.

### Overview: This project is an end-to-end e-commerce recommendation system that leverages implicit feedback to rank items based on user behavior. It features multiple collaborative filtering models and a hybrid approach to balance ranking accuracy, catalog coverage, and interpretability. The system is designed for both offline evaluation and online serving with FastAPI and Streamlit. ### Key Features: 1. **Popularity Baseline**: Ranks items by global interaction frequency. 2. **Item-Item Collaborative Filtering (Co-Visitation)**: Learns item similarity from user co-interaction patterns. 3. **Implicit Matrix Factorization (ALS)**: Captures deeper personalization patterns through latent embeddings. 4. **Hybrid Recommender**: Combines ALS for ranking accuracy and Item-Item for catalog coverage. ### Tech Stack: - **Jupyter Notebook**: Primary language used for development and experimentation. ### Why It Matters: This project explores the trade-offs between ranking accuracy, catalog coverage, and interpretability in e-commerce recommendation systems. It mirrors real-world scenarios where users provide implicit feedback rather than explicit ratings, making it valuable for understanding how recommendation systems are deployed in modern e-commerce platforms.

### Overview: kubectl-inferon is a kubectl plugin designed to diagnose machine learning (ML) model inference services running on Kubernetes. It provides insights into the status, startup timing, and logs of these services, including KServe InferenceService and LLMInferenceService. The plugin supports various types of models and runtime environments, ensuring comprehensive monitoring of inference services. ### Key Features: - **List Services**: Displays all inference services in a namespace, including their status and model details. - **Status with Startup Timing**: Provides detailed status information, including startup timing, model format, storage location, and runtime environment. - **Test Requests**: Checks whether the model is serving requests and tests the response with input files or prompts for LLMs. - **Logs**: Retrieves logs from containers to diagnose issues and understand the behavior of the inference service. ### Tech Stack: Go ### Why It Matters: kubectl-inferon offers a powerful tool for Kubernetes administrators and developers to quickly identify and diagnose issues with ML inference services. By providing real-time insights into the status and performance of these services, it helps in optimizing resource utilization and troubleshooting common problems such as model download failures, pod crashes, and incorrect model responses. With its ability to list, diagnose, and test inference services, kubectl-inferon streamlines the monitoring and management of ML models running on Kubernetes, ensuring that these services are operational and performing as expected.

### Overview: A decision service that answers one line at a time with a label and a confidence, abstains when the confidence is low, falls back to rules when the model fails, and is measured against 200 hand-written cases by a CI gate that fails on a drop. ### Key features: - **Closed Answer Space**: Three labels, parsed strictly; a label outside the set, prose instead of an object, or a missing confidence is refused rather than resolved. - **Abstention as a First-Class Outcome**: Below the threshold the service reports `abstained: true` and still reports its best guess; the evaluation counts abstentions and reports coverage rather than scoring them as wrong. - **Falling Back to Rules**: Providers are tried in order; a fallback sets `degraded: true` with a stable reason (`provider_timeout`, `provider_unparseable`, …) that is also a metric label. - **Explanation that Can Never Fail a Decision**: Explanations are a second call, only when asked, attributed with `explain_by_model`, and counted when they fail. - **A Gate That Has Been Seen to Fail**: The evaluation compares against a committed baseline and exits non-zero on a drop; a unit test and a CI step both doctor the baseline and assert the failure. - **Cost and Latency Next to Quality**: One forward pass, no output tokens on the default path, and a report that carries the price it used so a cost figure cannot be read without its assumption. ### Tech Stack: Go, standard library only, no database,

### Overview: This project is a FastAPI REST service designed for binary dermatoscopic skin lesion classification using LeNet-5 models trained on the HAM10000 dataset. It includes a Dockerized container with GitHub Actions CI for continuous integration, ensuring efficient model serving and comprehensive testing. ### Key Features: - FastAPI inference service - Multiple model selection via `model_id` - Image upload (multipart/form-data) - Deterministic preprocessing pipeline - Pytest test suite - Docker container support - GitHub Actions CI pipeline ### Tech Stack: - Python, FastAPI, PyTorch - Docker, GitHub Actions ### Why it Matters: This project demonstrates model serving, API design, testing, Dockerisation, and CI, making it a valuable resource for developers looking to implement similar solutions in dermatology or any field requiring binary classification using convolutional neural networks.