Projects

Overview: This project is an end-to-end MLOps platform that integrates MLflow for experiment tracking, model registry, automated retraining, drift detection, feature store, and model serving. Key features: - MLflow experiment tracking - Model registry - Automated retraining - Drift detection (feature and prediction) - Feature store - Model serving Tech stack: Python Why it matters: This platform streamlines the entire ML lifecycle, from experimentation to deployment and monitoring, ensuring efficient and reliable model management.

Overview: This project provides a self-hosted version of vLLM, making it easy to manage and use through a YAML file and command-line interface. It includes a Git-friendly YAML format for models and profiles, allowing for a seamless and intuitive workflow. Key features: - YAML-based model and profile management - CLI and TUI for easy access and configuration - Git-friendly version control for model updates Tech stack: Python Why it matters: This project simplifies the management of large language models, making it accessible and easy to use for developers and organizations looking to self-host their own version of vLLM.

### Overview: This is a production-grade ML platform designed for handling training, model registry, feature serving, inference, monitoring, drift detection, automated retraining, and Kubernetes deployment. It covers deterministic data generation, shared feature engineering, model comparison, MLflow tracking, guarded promotion, FastAPI inference, Prometheus/Grafana monitoring, drift detection, champion/challenger retraining, and Kubernetes deployment. ### Key features: - Deterministic synthetic behavioral data with meaningful label relationships, missingness, imbalance, and controlled shift. - Strict schema/range/category/uniqueness contracts and leakage-safe sklearn pipelines. - PostgreSQL offline and Redis online feature-store adapters with freshness semantics. - Three candidate models, imbalance-aware selection, tuning, test metrics, lineage, and explicit quality gates. - Fail-safe API with readiness distinct from liveness, request IDs, bounded validation, JSON logs, and low-cardinality metrics. - Drift-aware retraining that creates a challenger; production is never blindly overwritten. - Non-root containers, least-privilege Kubernetes manifests, HPA, NetworkPolicy, and automated supply-chain scans. ### Tech stack: Python ### Why it matters: This platform provides a comprehensive solution for end-to-end production-oriented customer churn ML systems, ensuring robust and reliable performance in real-world applications.

Overview: This repository serves as the companion code for the 11-part "MLOps for DevOps Engineers" series, covering the complete production journey of a model from data and model versioning to continuous integration, cloud orchestration, observability, and drift detection. Key features: - Continuous Integration and Packaging (CI/CD) - Cloud Orchestration and Serving (AWS EKS) - Data and Model Versioning (DVC, MLflow) - Drift Detection and Auto-Retraining (Prometheus, Drift Detector Daemon) - FastAPI Inference Service - Multi-Stage Production Dockerfile Tech stack: Python, FastAPI, Docker, Kubernetes, AWS EKS, MLflow, DVC, Prometheus, Drift Detector Daemon Why it matters: This repository provides a comprehensive guide for DevOps engineers on how to deploy and manage machine learning models in a production environment, covering the entire lifecycle from data preparation to model versioning and retraining.

Overview: Thrash is a macOS control panel and menu-bar app designed to switch between local LLM backends on a machine, ensuring efficient memory usage and minimal RAM thrashing. It supports multiple models like llama.cpp, uzu, MLX, and LM Studio, allowing for the selective use of different backends without overloading the system. Key features: - Control panel (`web/panel.html`) for live RAM monitoring and lane management. - Menu-bar app (`menubar/Thrash.swift`) for a quick switch to any lane. - Streaming chat (`web/uzu-chat.html`) for real-time chat interactions. - CLI (`./thrash on|off|toggle|status [--json]|logs|doctor`) for terminal access. - Idle auto-unload (`./thrash on|off|toggle|status [--json]|logs|doctor`) for automatic lane shutdown. - RAM guard (`./thrash on|off|toggle|status [--json]|logs|doctor`) for low-memory warnings. - Notifications (`./thrash on|off|toggle|status [--json]|logs|doctor`) for when lanes finish loading or when free memory is low. - Log viewer (`./thrash logs <id>`) for lane-specific log tails. - Chat lane-picker (`./thrash on|off|toggle|status [--json]|logs|doctor`) for choosing which lane the chat talks to. - One JSON API (`./thrash on|off|toggle|status [--json]|logs|doctor`) for shared configuration and status. Tech stack: Thrash uses Python for

Overview: CoInfer is an experimental machine learning inference engine built from first principles, focusing on understanding model execution, memory management, batching, and performance optimisation. Key features: Model execution, autoregressive inference, memory management, batching, and performance optimisation. Tech stack: Python and PyTorch. Why it matters: CoInfer explores the systems and algorithms responsible for executing machine learning models efficiently, providing insights into inference beyond high-level framework APIs. Repository: cotenthusiast/CoInfer Primary language: Python GitHub stars: 0 GitHub description: An experimental machine learning inference engine built from first principles, exploring model execution, memory management, batching, and optimisation.

### Overview: Cris has a video factory, ADN Generator, that makes videos. Before trusting a video, someone has to answer quick questions about it, like "does the start grab you?" or "are the comments good?". laya-server is a small judge that answers those short yes-or-no and pick-one questions for free, on Cris's own computer, called WAR. When a question is too long, or the judge is not sure, it asks a bigger judge in the cloud, called Jev. For now it works in **shadow mode**: its answers are written down so they can be compared later, but it never stops a video. ### Key features: - **Laya Server**: A free, local decision server that answers short yes-or-no and pick-one questions. - **Jev Judge**: A bigger, cloud-based judge that answers longer questions. - **Cascade Functionality**: Answers can be sent to Jev if they are uncertain or too long, and a cascade can be set up to ensure confidence in the answer. - **Shadow Mode**: Answers are written down for comparison, but do not affect the video. ### Tech stack: JavaScript ### Why it matters: Cris can trust videos more quickly and easily, while still ensuring the quality of the answers. The system is designed to be transparent and auditable, with all decisions being recorded and compared. This makes it easy to verify the accuracy of the system and ensures that the videos are trusted based on the best possible judgment.

Overview: This project provides shared infrastructure for MLflow tracking, Prefect workflows, and SeaweedFS artifact storage, enabling the production of batch operations and online predictions. Key features: - MLflow tracking and serving - Prefect workflows for batch execution - SeaweedFS artifact storage Tech stack: Python Why it matters: This shared infrastructure supports the scientific workflow and model package management of various applications, ensuring secure and efficient deployment without requiring platform-specific configurations.

### Overview: VAHAAN is a portable and validated release-engineering layer for the SETU model, designed to bridge the gap between the MLX-trained SETU model and the public demo running on Linux CPU hardware. It ensures the model's integrity and performance, providing a consistent and reliable service. ### Key Features: - **SHA-256 Verification**: Ensures the model's integrity before it is used. - **Model Loading Once**: The model is loaded only once and never reloaded per request. - **Length-Bounded Requests**: Limits the length of requests to prevent abuse. - **Concurrency Control**: Bounded to one model slot to prevent overloading. - **Valid JSON Output**: Ensures the output is valid JSON and adheres to the specified schema. - **Type Safety**: Invalid output returns a typed 502 response; fields are never fabricated. - **Liveness and Readiness Endpoints**: Separate liveness, readiness, version, and Prometheus endpoints. - **Non-Root User**: The container runs as a non-root user to ensure security. ### Tech Stack: - Python 3.12 - FastAPI - Docker ### Why It Matters: VAHAAN ensures the model's performance and integrity, making it a crucial component for the SETU project. It provides a consistent and reliable service, addressing the gap between the MLX-trained SETU model and the public demo running on Linux CPU hardware.

### Overview: This project is a clean, reproducible implementation of an image classifier using EfficientNetB0 and transfer learning. It works with any image dataset organized by class, including flowers. The model achieves perfect accuracy and macro-F1 score on a small test set, demonstrating robust performance. ### Key Features: - **Backbone:** EfficientNetB0 (ImageNet-pretrained, loaded manually for robustness). - **Head:** GlobalAveragePooling2D → Dropout(0.5) → Dense(*num_classes*, softmax). - **Preprocessing & Augmentation:** In-model Keras augmentation (`RandomFlip`, `RandomRotation(0.1)`, `RandomZoom(0.1)`, `RandomContrast(0.1)`) and EfficientNet `preprocess_input`. - **Training schedule:** 3 stages (head-only → partial unfreeze → full fine-tune) with `ReduceLROnPlateau` and `EarlyStopping`. - **Class imbalance:** per-class sample weights derived from file counts in `data/train`. - **Artifacts:** best checkpoint, final weights, JSON training histories, evaluation report, and confusion matrix, optional prediction visualizations. ### Tech Stack: Python ### Why It Matters: This implementation showcases the effectiveness of EfficientNetB0 in image classification tasks, especially when combined with transfer learning. The robust backbone and fine-tuning capabilities make it suitable for various image datasets, including those with class imbalance and out-of-distribution testing.

Overview: The **MERIDIAN** enterprise MLOps reference blueprint is a sanitized architecture designed for operating multiple machine-learning workloads on a shared data, feature, training, serving, and monitoring platform. Key features: - Eight model families across demand forecasting, constrained reinforcement-learning pricing, anomaly detection, predictive maintenance, asset clustering, feasibility analysis, and NLP ticket routing. - Includes representative interfaces, configuration shapes, model skeletons, deployment manifests, diagrams, and operational controls. - Excludes proprietary logic and production data. Tech stack: Python Why it matters: This blueprint provides a technical foundation for understanding workload behavior, feature contracts, model governance, GPU serving, reliability controls, and unit economics, enabling credible infrastructure discovery before commercial sizing begins. Repository: daetan999/mlops-hosp Primary language: Python GitHub stars: 0 GitHub description: Sanitized enterprise MLOps reference blueprint spanning governed features, model lifecycle, GPU serving, drift response, and serving economics.

Overview: This repository is a production-ready Z-Image-Turbo inference service running on a Kaggle T4x2 GPU cluster. It leverages BF16 precision, dual-GPU device mapping, and a bounded single-flight queue to ensure reproducibility and performance. Key features: - Dual-GPU device mapping for efficient parallel processing - BF16 precision for reduced memory footprint and faster inference - Persistent FastAPI REST service for secure, authenticated image generation - Bounded single-flight queue to prevent GPU overloading and ensure memory integrity - Reproducible qualification evidence for model and service validation Tech stack: Python Why it matters: This project provides a reliable, reproducible, and transparent way to run Z-Image-Turbo on a Kaggle T4x2 GPU cluster, ensuring that the model and service can be inspected and validated without compromising security or performance.

Overview: InferTrace is a high-performance observability collector designed specifically for LLM inference latency, providing ML-aware spans and regression detection for model deploys. Key features: - ML-aware span schema - Per-model latency windows - Regression detection Tech stack: Go Why it matters: InferTrace is purpose-built for LLM inference pipelines, addressing the limitations of generic tracing tools and providing a solution for detecting latency regressions after model deployments. Repository: danielgraviet/infertrace

### Overview: Fraud detection platform leveraging XGBoost, ONNX Runtime, and automated deployment strategies. ### Key features: - **XGBoost Training**: Trained model for fraud detection. - **Go ONNX Serving**: Inference engine for model predictions. - **Drift Detection**: Continuous monitoring for model drift. - **Shadow Mode**: Canary deployment with automated rollback. - **Canary Deployment**: Early traffic split for model evaluation. - **Late-Label Evaluation Loop**: Simulates real-world transaction delays. ### Tech stack: Python ### Why it matters: This platform provides a robust end-to-end solution for fraud detection, ensuring model performance and reliability through continuous monitoring and automated deployment strategies.

Overview: This project is a highly available model inference platform designed for Kubernetes, featuring canary releases with automated analysis and rollback, no dropped requests on failure, autoscaling on in-flight requests, and SLO burn-rate alerts and dashboards. Key features: - Canary releases with automated analysis and rollback - No dropped requests on failure - Autoscaling on in-flight requests - SLOs with burn-rate alerts and dashboards - Kubernetes manifests hardened for restricted Pod Security Standard Tech stack: Python Why it matters: This platform is essential for ensuring model inference services are highly available and performant, especially in environments where pods can fail or traffic spikes. Repository: darshansatpathy/ha-model-inference-platform Primary language: Python GitHub stars: 0 GitHub description: Highly available model serving on Kubernetes: canary releases with automated analysis and rollback, autoscaling on in-flight requests, SLO burn-rate alerts and Grafana dashboards.

### Overview: This project is a proof of concept for credit card fraud detection, utilizing LightGBM for model training and FastAPI for serving. It focuses on exposing the real failure modes of machine learning serving, including encoder-contract drift, time-based leakage, and threshold governance, to provide insights into the real-world implications of ML deployment. ### Key Features: - **Modeling**: Uses LightGBM for training a model. - **API Endpoint**: Exposes a FastAPI endpoint for real-time prediction. - **Metrics**: Tracks the performance of the model using a strict time-based hold-out split, achieving a PR-AUC of 0.5926. - **Anonymization**: Only the most relevant fields are named, while others remain vendor-anonymized, providing a clear distinction between real and fake labels. - **Encoder Use**: The serving code uses the encoder it loads, ensuring that the model's behavior is consistent with the data it was trained on. ### Tech Stack: - **Programming Languages**: Jupyter Notebook (for model training and API development) - **Frameworks**: FastAPI for the API server - **Modeling**: LightGBM for the model training ### Why It Matters: Most "fraud detection POC" repos stop at a notebook that prints a metric. This project takes the next step by treating the handoff from notebook to API as seriously as the modeling itself. It focuses on the following: - **PR-AUC 0.5926**: Achieves a high PR-AUC score on a strict time-based hold-out

Overview: SmarterRouter is an intelligent LLM gateway and VRAM-aware router designed for Ollama, llama.cpp, and OpenAI. It offers semantic caching, model profiling, and automatic failover for local AI labs. Key features: - Semantic caching - Model profiling - Automatic failover Tech stack: Python Why it matters: SmarterRouter enhances the performance and reliability of AI models by optimizing resource usage and ensuring smooth operation in local AI labs.

### Overview: A model serving platform designed for efficient and scalable deployment of machine learning models using FastAPI, Docker, Prometheus, Grafana, canary releases, A/B testing, drift checks, and dynamic batching. ### Key Features: - **FastAPI Service**: Loads and serves models via HTTP. - **Model Registry**: Tracks and manages model versions. - **Model Store**: Reads and writes model artifacts. - **Model Server**: Loads models into memory and handles predictions. - **Predictors**: Wraps models from scikit-learn, PyTorch, and ONNX Runtime. - **Router**: Decides which model version to use based on A/B tests and canaries. - **Monitoring**: Metrics and alerts via Prometheus and Grafana. - **Dynamic Batching**: Background processing of predictions. - **Shadow Mode**: Simulates a different model for canary testing. - **Preprocessing Pipelines**: Supports per-model pipelines. ### Tech Stack: Python ### Why It Matters: This platform simplifies the deployment and management of machine learning models, making it easier to scale and test new versions of models without the overhead of complex infrastructure.

Overview: This DeepSeek Harness plugin allows users to run open-weight language models on their own GPU virtual machines on Hugging Face Spaces, providing live progress and cost tracking. The plugin starts the machine when a message is sent and pauses it when idle, ensuring that only the GPU is billed while it is in use. Key features: - Model selector: Each configured Space appears as a provider, with its models, context size, output limit, and reasoning levels. - Start on demand: A message to a paused Space starts it, and the conversation resumes when the model is ready. - Stop on its own: After a configurable idle time or per-start time limit, the plugin pauses the machine, showing a warning a few minutes before and allowing a one-click extension. - Live progress and cost tracking: The plugin displays a header indicator in each conversation, a sidebar tab showing progress, and notices for various states. It also provides a setup page for checking credentials, Space, manifest, models, and effort levels. Tech stack: JavaScript Why it matters: This plugin enables users to run open-weight language models on their own GPU virtual machines on Hugging Face Spaces, providing live progress and cost tracking, making it easier to manage and optimize model usage. Repository: dev-bravelo/dsh-hf-spaces Primary language: JavaScript GitHub stars: 0 GitHub description: DeepSeek Harness plugin: run open-weight LLMs on your own GPU virtual machine on Hugging Face (a Space), started on demand and paused when idle, with live progress and cost in the Harness.

Overview: This repository contains custom inference containers for TabPFN-3, a tabular foundation model, optimized for Amazon SageMaker. It includes payload encoding helpers, a benchmark harness, and documented findings on row capacity, subsample-ensemble, and mixed-data support. Key features: - Custom inference containers - Payload encoding helpers - Benchmark harness - Row capacity - Subsample-ensemble - Mixed-data support Tech stack: Python Why it matters: This repository provides valuable insights and performance benchmarks for developers working with TabPFN-3 models on Amazon SageMaker.