Projects

### Overview: This repository is a production-style diabetes hospital readmission prediction pipeline, featuring leakage-aware preprocessing, XGBoost modeling, FastAPI serving, Streamlit demo, MLflow tracking, monitoring, and Docker integration. ### Key Features: - **Raw Data Validation and Dictionary Generation:** Ensures data quality and produces reviewable artifacts. - **Leakage-Aware Preprocessing and Grouped Splitting:** Reduces leakage by normalizing tokens and performing patient-level splits. - **Clinically Informed Feature Engineering:** Adds engineered features for improved model performance. - **Binary and Multiclass Modeling:** Evaluates multiple model families and sampling strategies, promoting the best artifacts. - **Experiment Tracking:** Logs local runs via MLflow. - **API Serving:** Exposes prediction, batch prediction, and explanation endpoints. - **Streamlit Frontend:** Provides a portfolio-facing interface with deterministic explanation mode. - **Monitoring and Drift Reporting:** Generates summary reports and drift checks. - **Demo Workflow:** Produces reproducible API responses and a concise demo summary. ### Tech Stack: Python, FastAPI, MLflow, Streamlit, Docker, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow, MLflow

Overview: ASC is a powerful tool designed to decompile Android APKs directly, bypassing preprocessing and memory bloat. It provides a fast, user-friendly interface for security researchers, developers, and anyone interested in app analysis. Key features: - Lightning-fast processing speed - Clean and simple interface - Powerful analysis tools for code and resources - Built for professionals, suitable for security research and AI integration Tech stack: Python Why it matters: ASC simplifies the process of decompiling Android apps, making it accessible and efficient for various technical tasks related to app analysis and security research. Repository: supervisorysoftwarefoxhound8113/ASC Primary language: Python GitHub stars: 0 GitHub description: Query Android APKs directly as a database for instant decompilation, bypassing heavy preprocessing and memory bloat.

Overview: SmarterRouter is a cutting-edge gateway and router designed for Ollama, llama.cpp, and OpenAI, featuring semantic caching, model profiling, and automatic failover for local AI labs. Key features: - Semantic Caching - Model Profiling - Automatic Failover Tech stack: Python Why it matters: SmarterRouter optimizes performance and reliability in AI research and development by leveraging advanced caching and profiling techniques, ensuring seamless operation across different models and environments.

Overview: This project demonstrates a complete end-to-end machine learning pipeline, from training a convolutional neural network (CNN) on the MNIST dataset to serving the trained model using MLflow and Flask. Key features: - Train a CNN model on MNIST using TensorFlow. - Autolog the training process with MLflow for tracking and model registry. - Serve the trained model via a Flask API. - Draw digits in a browser to test the model. Tech stack: Jupyter Notebook Why it matters: This project provides a practical example of how to build a production-ready machine learning model pipeline, from training to deployment, using TensorFlow, MLflow, and Flask. Repository: swaroop002/mnist-mlflow-serving Primary language: Jupyter Notebook GitHub stars: 0 GitHub description: MNIST CNN trained and autologged to MLflow, served from the model registry behind a Flask API, tested by drawing digits in the browser.

Overview: ModelOpsForge is a production-oriented ML/MLOps system designed for reproducible training, evaluation, model versioning, API serving, and operational monitoring. It focuses on core MLOps engineering without unnecessary distributed infrastructure or feature bloat. Key features: - Train → Evaluate → Version → Serve → Monitor - Demonstrates a practical machine-learning lifecycle from reproducible training and evaluation through model registration, API serving, and operational monitoring. - Uses Python, pandas, NumPy, scikit-learn, MLflow, FastAPI, Pydantic, Prometheus Client, pytest, Ruff, Docker, and GitHub Actions. Tech stack: Python, pandas, NumPy, scikit-learn, MLflow, FastAPI, Pydantic, Prometheus Client, pytest, Ruff, Docker, GitHub Actions. Why it matters: ModelOpsForge provides a practical, production-oriented approach to machine learning and MLOps, focusing on core engineering without unnecessary complexity or infrastructure.

Overview: This is a fine-tuned small language model designed to answer FastAPI questions. It is packaged as a production service that leverages free-tier infrastructure, including an OpenAI-compatible API, streaming, retrieval from the official FastAPI documentation, and per-user API keys. The model's quality is a release gate, with a 108-item evaluation, and it fails predictably under load with Cloudflare Worker failover. Key features: - Fine-tuned model of TinyLlama-1.1B - Runs on CPU at 13 tokens/s - Answers two APIs: `POST /generate` and `POST /v1/chat/completions` - Grounds answers in the FastAPI documentation - Bounded concurrency and queue handling - Model quality as a release gate - Prometheus metrics, Grafana dashboard, and Sentry logging Tech stack: Python Why it matters: This model provides a cost-effective solution for answering FastAPI questions, leveraging the strengths of both CPU and GPU, and ensuring model quality through rigorous evaluation and release gates. Repository: syed-waleed-ahmed/Niche-Finetuned-Model

Overview: SmarterRouter is an intelligent LLM gateway and VRAM-aware router designed for Ollama, llama.cpp, and OpenAI. It features semantic caching, model profiling, and automatic failover for local AI labs. Key features: - Semantic caching - Model profiling - Automatic failover Tech stack: Python Why it matters: SmarterRouter enhances the performance and reliability of AI models by optimizing resource usage and ensuring consistent access, making it essential for AI labs and developers working with large language models.

Overview: A simple, self-hosted vLLM (Vector Language Model) solution that allows for easy management and deployment of language models using YAML configuration files. Key features: - YAML-based configuration for models and profiles - Git-friendly workflow for version control - CLI and TUI for intuitive management Tech stack: Python Why it matters: This project simplifies the process of managing and deploying language models, making it accessible and easy to use for developers and enthusiasts alike.

Overview: This project helps developers plan their hardware requirements for local AI deployment, preventing compatibility issues and saving on unnecessary upgrades. Key features: - Calculates memory and processing power needed for local LLM deployment - Provides benchmarks for different model sizes - Warns about hardware limitations for large models Tech stack: Python Why it matters: Helps developers avoid guesswork and overspending on hardware, ensuring smooth local AI deployment without compatibility issues.

Overview: This toolkit provides an end-to-end machine learning pipeline for Python, automating the process from data loading to model deployment. It uses scikit-learn for training and evaluation, and saves the entire process in a YAML file for version control and reproducibility. Key features: - YAML configuration for pipeline setup - Data loading from CSV, TSV, Parquet, JSON, and JSON lines - Schema inference and enforced schema validation - Preprocessing steps including imputation, scaling, and one-hot encoding - 20 built-in models and the ability to register custom models - Model evaluation with held-out test metrics and a baseline - Model bundles containing the fitted pipeline, metadata, schema, and library versions - Batch prediction capabilities with class probabilities and other columns passed through Tech stack: Python Why it matters: This toolkit simplifies the process of building and deploying machine learning models, ensuring reproducibility and consistency across different environments. It's particularly useful for teams looking to streamline their ML pipeline and maintain data integrity. Repository: tec-smith/ml-pipeline-toolkit Primary language: Python GitHub stars: 0 GitHub description: End-to-end machine learning pipeline toolkit for Python: preprocessing, training, evaluation and deployment of scikit-learn models from one YAML file.

Overview: This end-to-end machine learning pipeline toolkit for Python allows you to describe a run in one YAML file, load data, train and tune a scikit-learn model, evaluate it against a baseline, and save a versioned bundle that can be served as an HTTP API or used for batch predictions. Key features: - One YAML file per pipeline - Data loading from CSV, TSV, Parquet, JSON, and JSON lines - Schema inference and enforced on predictions - Preprocessing with imputation, scaling, and one-hot/ordinal encoding - 20 built-in models or custom models via `register_model` - Grid or random hyperparameter search for training - Held-out test metrics, cross-validation on training set, baseline comparison - Confusion matrix and Markdown report for evaluation - Model bundles with fitted pipeline, metadata, schema, metrics, and library versions - Batch prediction with class probabilities and other columns passed through - FastAPI serving with input validation, batch and body size limits, OpenAPI docs, and health and metadata endpoints Tech stack: Python Why it matters: This toolkit provides a streamlined path from data to model, using scikit-learn for modeling and a YAML-based configuration file for reproducibility and version control. It's particularly useful for teams looking to train models in production without writing glue code, and for learning how a production ML pipeline works.

### Overview: kvfit is a Python CLI designed to help developers plan the memory capacity and GPU usage for large language models (LLMs) and their associated KV caches. It evaluates the fit of a specific model on a given GPU or DGX system, providing detailed insights into weight and state sizes, as well as recommended layouts for optimal performance. ### Key features: - Evaluates the memory and GPU usage requirements of a specific Hugging Face model. - Provides a static estimate of model size and state usage. - Recommends optimal tensor parallel (TP), full-replica distributed parallel (DP), and SGLang DPA/EP layouts. - Offers a detailed explanation of why a deployment fits or runs out of memory. - Can check an installed vLLM or SGLang build and calibrate the estimate against a real target-host serving sweep. ### Tech stack: Python ### Why it matters: kvfit helps developers quickly and accurately plan the memory and GPU usage for LLMs and their KV caches, ensuring optimal performance and preventing potential memory issues.

Overview: tinyserve-llm is an LLM inference engine built from scratch, optimized for prefix cache hits in agentic workloads. It combines large KV pages with token-level prefix matching, significantly improving performance compared to other models. Key features: - Token-granular prefix reuse with 256-token pages - Copy-on-write forking for parallel samples - Cache-aware scheduling - One explicit boundary separating concerns Tech stack: Python Why it matters: tinyserve-llm is designed to maximize prefix cache hits, making it highly efficient for agentic workloads, particularly in multi-turn chat scenarios. It outperforms other models in terms of token prefilled capacity, achieving up to 2.3× more tokens prefilled compared to nano-vllm on a similar KV memory setup. Repository: tejkas/tinyserve-llm

Overview: `weightlint` is an offline preflight check tool designed to catch model-packaging failures before a model is loaded, ensuring that the model bundle is internally complete and load-shaped. Key features: - Detects missing shard files - Checks for unsafe index paths - Validates malformed or truncated SafeTensors files - Ensures consistent tensor spans - Identifies suspicious pickle-based weights - Checks for missing bundle metadata Tech stack: Python Why it matters: `weightlint` provides a small, local-first CLI and Python library that performs cheap, deterministic checks before a model is loaded, ensuring that the model bundle is complete and load-shaped, thus avoiding expensive and late failures that can occur when a loader has already spent time opening a large checkpoint or starting a GPU worker. Repository: thisbejim/weightlint Primary language: Python GitHub stars: 0

Overview: This is a production-style model inference platform designed for ML portfolios, featuring a versioned model registry, dynamic micro-batching, canary and shadow deployments with instant rollback, load shedding, and Prometheus metrics. Key features: - Dynamic micro-batching: Requests queue; a worker greedily drains what's queued and runs one vectorized predict, amortizing the model's fixed per-call cost. - Versioned registry: Models registered as `(name, version)` with stable/candidate aliases. - Canary deployment: Configured fraction of live traffic serves from the candidate; per-version metering judges it. - Shadow deployment: 100% of users get stable, traffic mirrored to the candidate and answers compared/logged, never returned. - Load shedding: Bounded queue: past max queue depth, requests get an immediate 429 + Retry-After instead of unbounded queueing that collapses p99 for everyone. - Metrics: Prometheus text at `/metrics`: QPS, p50/p95/p99, batch-size histogram, shed count, per-version routing, shadow disagreements. - Ops surface: `POST /deploy/{model}` (`candidate` / `promote` / `rollback` / `clear_candidate`), `GET /status`, `GET /healthz`. Tech stack: Python. Why it matters: This platform provides a robust and scalable solution for ML inference, ensuring high throughput, reliability, and performance optimization. Repository: tkarim45/model-serving Primary language: Python GitHub stars: 0 GitHub description:

Overview: A fine-tuned text-classification service was developed and benchmarked against an LLM zero-shot baseline, achieving 100% accuracy, 0.22ms latency, and $0 cost on a narrow, high-volume task. Key features: - Fine-tuned small model using TF-IDF and Logistic Regression - Benchmarked against LLM zero-shot baseline on accuracy, latency, and cost - Specialized model outperforms LLM on all three axes Tech stack: Python Why it matters: This project demonstrates the trade-off between accuracy, latency, and cost when using an LLM for text classification, showing that a small, specialized model can achieve better results on a narrow, high-volume task.

### Overview: This project is a model-serving gateway designed to deliver low latency and high throughput. It features dynamic micro-batching and load shedding, achieving a 54% throughput increase and a 915ms p99 latency cut from 2,289ms at 64-way concurrency compared to naive serving. ### Key features: - Dynamic micro-batching: Batches of up to 32 requests or up to 4ms deadline. - Load shedding: Automatically sheds excess load with an immediate 503 error and retry hint, keeping the tail latency bounded. - Benchmarked against the same model served naively, showing significant improvements in throughput and latency. ### Tech stack: - **asyncio + futures**: The batcher is 80 lines of reviewable stdlib code, no framework magic. - **NumPy MLP**: The model behind the gateway, with real matmuls size to dominate the inference cost. - **FastAPI + uvicorn**: The HTTP edge, with Pydantic validation and a single worker for clear bottleneck measurement. - **pytest + pytest-asyncio**: 14 tests and 98% coverage, pinning batching equivalence, shedding, and worker resilience. - **Docker**: Deployment, with all knobs as environment variables. - **GitHub Actions + ruff**: CI, running the batcher behavior suite on every push. ### Why it matters: The project solves the gap between a model's benchmark speed and its production latency, particularly in the serving layer where GPUs and BLAS libraries are throughput

Overview: This project is a MLOps platform that serves an Isolation Forest model using FastAPI, Docker, and Kubernetes. It includes CI/CD pipelines for continuous integration and deployment. Key features: - Dockerized FastAPI application - Isolation Forest model trained with scikit-learn - Kubernetes deployment on EKS - CI/CD setup with GitHub Actions and AWS ECR Tech stack: Python, Docker, FastAPI, Kubernetes, Docker Compose, GitHub Actions, AWS ECR, Prometheus, Docker multi-stage Why it matters: This platform automates the entire MLOps lifecycle, from model training to deployment, ensuring efficient and reliable ML operations.

Overview: LinML is an open-source project aimed at providing a Linux counterpart to Windows ML (WinML). It includes a Linux-first ONNX Runtime inference layer, a CLI, and a hardware provider catalog. Key features: - LinML provides a Linux-first inference integration layer. - It includes a CLI for easy execution and diagnostics. - LinML.Catalog automates the process of vendor registration, SHA-256 verification, and session attachment for external plugin execution providers. Tech stack: C# Why it matters: LinML aims to democratize machine learning on Linux by offering a unified, hardware-agnostic inference layer, similar to the Windows ML (WinML) ecosystem, but designed specifically for Linux environments.

Overview: SmarterRouter is an advanced gateway and router designed for AI labs, featuring semantic caching, model profiling, and automatic failover for seamless performance. Key features: - Semantic caching - Model profiling - Automatic failover Tech stack: Python Why it matters: SmarterRouter optimizes AI performance by intelligently managing local AI labs, ensuring smooth operation and efficient resource utilization.