Projects

Overview: AHEADDB Live is a local relational prediction system that integrates an existing Python statistical engine with a PostgreSQL prediction layer. It maintains explicit temporal tasks and correctness-first views, ensuring tasks are executed from identified snapshots and published with strict freshness checks. Key features: - Explicit source boundary prediction features - Integration with an existing Python statistical engine - PostgreSQL prediction layer for local execution - Retained generations and stricter freshness bounds Tech stack: Python Why it matters: AHEADDB Live provides a robust, explicit temporal prediction system that ensures tasks are executed from identified snapshots and published with strict freshness checks, making it suitable for applications requiring high reliability and correctness. Repository: nazeeh111/AHEADDB Primary language: Python GitHub stars: 0 GitHub description: Local relational predictions with explicit temporal tasks and correctness-first maintained views

### Overview: LLM FineTune Factory is a production-ready platform designed for fine-tuning and serving large language models (LLMs) across various domains. It offers seamless integration with Open WebUI, multiple fine-tuning techniques, and is 70% cheaper than proprietary APIs across five domain models. ### Key Features: - **5 Domain-Specific Models**: Human Resources, Financial Services, Customer Relations, Medical Services, Brand & Growth. - **Multiple Fine-tuning Techniques**: Full, DPO, PEFT, LoRA, QLoRA. - **OpenAI-Compatible API**: Seamless integration with Open WebUI. - **Production-Ready**: Docker, systemd, monitoring support. ### Tech Stack: Python ### Why it Matters: This platform simplifies the fine-tuning and serving process, making it accessible and cost-effective for businesses across different industries.

Overview: This self-hosted ML platform allows teams to train, ship, and run machine learning models on Kubernetes. Key features include project management, training pipelines, scheduling, data management, batch inference, model monitoring, project secrets, models, serving, and functions. Key features: - Project management with namespaces and quotas - Training jobs and pipelines on Argo Workflows - Scheduling and parameters management - Data management with immutable versions and lineage - Batch inference and model monitoring - Project secrets management - Models from MLflow registry or Hugging Face Hub - Serving with immutable revisions and canary rollouts - Language models with KServe's Hugging Face runtime Tech stack: Python, Kubernetes, Argo Workflows, Knative, MLflow, KServe, Kubernetes Secrets, OpenID Connect, Kubernetes Jobs, Kubernetes Pipelines, Kubernetes Deployments, Kubernetes Secrets, Kubernetes CronJobs, Kubernetes Jobs, Kubernetes Scheduling, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs, Kubernetes Jobs,

Overview: This project is an end-to-end solution for predicting medium-voltage (MV) grid fault risk, featuring scalable ML pipelines, CI/CD for ML, feature engineering, artefact versioning, model serving, and monitoring-ready outputs. Key features: - Builds a risk score for MV assets/segments from structured operational data. - Produces monitoring-ready outputs with a predictable schema and stable interfaces. - Ships a serving layer (API) and a demo UI for interactive scoring. - Tracks experiments and artefacts for reproducibility and rollback. Tech stack: Python Why it matters: This system is designed for production use, ensuring scalable ML pipelines, CI/CD for ML, feature engineering, artefact versioning, model serving, and monitoring-ready outputs, making it ideal for predictive maintenance in MV grids. Repository: nepryoon/mv-grid-fault-risk Primary language: Python GitHub stars: 0 GitHub description: End-to-end MV grid fault risk scoring: feature engineering, tracked training, artefact versioning, API serving, monitoring-ready outputs.

Overview: This project is an inference gateway that optimizes batching, queue management, and performance evaluation for AI model requests using bounded micro-batching, backpressure, and per-request routing. Key features: - Bounded micro-batching - Queue backpressure - Per-request result routing - Reproducible performance evaluation Tech stack: TypeScript Why it matters: This project simplifies the implementation of inference gateways by eliminating the need for GPUs, Kubernetes, or external services, focusing solely on optimizing batch handling and performance metrics. Repository: nolanefe/grpc-inference-gateway Primary language: TypeScript GitHub stars: 0 GitHub description: Inference gateway for batching AI model requests through bounded micro-batching, queue backpressure, per-request result routing, and reproducible performance evaluation.

Overview: This MLOps project includes an automated machine learning pipeline with a focus on data ingestion, model training, inference API, monitoring, and deployment using FastAPI, AWS, Terraform, Prometheus/Grafana, and Slack integration. Key features: - Data Ingestion: Fetches records from a data lake endpoint with error handling. - ML Model Training: Uses RandomForest or GradientBoosting classifier with auto-retraining. - Auto-Retraining: Trains the model until the accuracy reaches 0.8. - AWS Deployment: Deployed on EC2 with public endpoints. - Prometheus Metrics: All required metrics exposed at `/metrics`. - Prom + Grafana + Alertmanager: Full monitoring stack with Docker Compose. - Alerts: Slack notifications for critical events (503 errors, drift, accuracy drops). - Docker Compose: Complete orchestration of all services. Tech stack: Python, FastAPI, AWS, Terraform, Prometheus, Grafana, Alertmanager, Slack, Docker Compose. Why it matters: This project provides a comprehensive solution for machine learning models in a production environment, ensuring robust monitoring and alerting, and efficient deployment using modern tools and practices. Repository: noumanh11/mlops-inference-pipeline Primary language: Python GitHub stars: 0 GitHub description: MLOps inference pipeline with FastAPI, AWS, Terraform, Prometheus/Grafana, drift detection, automated retraining, and real-time alerting.

Overview: This project is a Python-based tool designed to deploy ML models from Metaflow flows, ensuring full lineage tracking and easy management of model versions. Key features: - Deploy ML models from Metaflow flows - Full lineage tracking - Automatic environment handling for `@conda` and `@pypi` steps - Explicit package specification for `@initialize` Tech stack: Python Why it matters: This tool simplifies the process of deploying ML models from Metaflow flows, making it easier to manage model versions and ensuring traceability, which is crucial for maintaining and troubleshooting production environments.

Overview: LiliuxFlow is a local AI model serving and operation application designed for Apple Silicon, providing a streamlined deployment, monitoring, and management of local AI models. Key features: - Local AI model serving - OpenAI-compatible interface for apps - Model switching with llama-swap - Request handling with LiteLLM - Lily-powered local operations - Self-hosted setup for privacy-focused users - Tools for managing models, monitoring activity, and organizing the setup Tech stack: Python Why it matters: LiliuxFlow simplifies the process of running local AI models, making it accessible to non-technical users. It offers a user-friendly interface, allowing easy switching between models and managing local operations without requiring deep server knowledge.

Overview: Okik is a powerful command-line interface (CLI) tool designed to simplify the process of deploying various services using different frameworks on any cloud platform. With Okik, users can effortlessly deploy and manage their services directly on any cloud infrastructure without the complexity of handling their own infrastructure. Key features: 1. Managed Kubernetes clusters: Deploy services on Kubernetes clusters without the need to manage the underlying infrastructure. Okik handles all aspects of cluster management, including provisioning, scaling, and monitoring, allowing users to focus on building their applications. 2. Multi-cloud support: Deploy services across various cloud providers seamlessly. This feature allows for greater flexibility and prevents vendor lock-in, enabling users to choose the best cloud solution for their specific needs. 3. Framework agnostic: Compatible with a wide range of popular frameworks and technologies. Use Hugging Face, FastAPI, TensorFlow, PyTorch, and more to build your services, all managed by Okik. 4. LLM and GenAI integration: Easily incorporate Large Language Models and Generative AI capabilities into your applications. Okik provides seamless integration with popular LLM frameworks and GenAI tools, enabling advanced natural language processing and content generation within your services. 5. Automated predictive scaling: Easily scale your services up or down based on demand. Okik automatically adjusts resources to meet your application's needs, ensuring optimal performance and cost-efficiency, even for resource-intensive LLM and GenAI workloads. 6. Built-in monitoring: Keep track of your services' performance and health with integrated monitoring tools. This feature provides real-time insights into your application's behavior,

### Overview: This project presents three LLM serving backends for GPT-2-124M, built from scratch. The backends are benchmarked against each other under identical concurrent load, showcasing continuous batching and paged KV-cache for improved performance and reliability. ### Key features: - Continuous batching with paged KV-cache - Static batching - Naive serial ### Tech stack: Python ### Why it matters: The implementation provides a detailed reference implementation of continuous batching and paged KV-cache, showcasing their effectiveness in a production environment. It benchmarks against existing methods and highlights areas where it behaves differently, providing valuable insights for developers and researchers.

### Overview: Streaminfer is a fast, real-time machine learning inference tool designed for quick responses and smooth operation. It supports adaptive batching, backpressure control, and seamless model hot-swapping, ensuring zero downtime during updates. ### Key features: - Real-time processing - Adaptive batching for efficient data handling - Model hot-swapping for zero-downtime updates - Zero-downtime model hot-swapping - Supports popular tools like Python and PyTorch ### Tech stack: Python ### Why it matters: Streaminfer is crucial for applications requiring fast, real-time analysis, such as live data processing systems. Its features ensure smooth operation and minimal disruption, making it ideal for environments where performance and uptime are critical.

Overview: The dgx-spark-inference-stack is a Docker-based inference stack designed specifically for the Nvidia DGX Spark, offering a simple and efficient way to serve AI models on your personal computer. It utilizes vLLM technology and Docker for easy application management and is tailored for generative AI tasks. Key features: - Simple setup - Local model serving - Docker support - ML Ops ready - Ideal for generative AI Tech stack: JavaScript Why it matters: This stack simplifies the process of deploying AI models on a personal computer, making it easier to get started with generative AI tasks without the need for complex infrastructure.

Overview: HiveMesh is a decentralized platform enabling peer-to-peer leasing of models, agents, and machine intelligence. Key features: - Secure and transparent leasing of AI resources - Facilitates collaboration and sharing among developers and researchers - Supports various types of AI models and agents Tech stack: Node.js, React, MongoDB Why it matters: HiveMesh simplifies the process of accessing and using AI resources, promoting a more collaborative and efficient development ecosystem.

Overview: This repository is a concurrent request scheduler designed to test and improve the performance of serving systems. It features a pluggable model backend and is built using C++ with a bounded queue, Python via pybind11, and FastAPI integration. Key features: - Health endpoint - C++ bounded, thread-safe FIFO with immediate full-queue rejection - FIFO batch pop and multi-producer/multi-consumer native stress test - pybind11 boundary that releases the GIL around queue operations - Model runner abstraction with explicit batch assembly/splitting and a padding-leak invariance test - Python `RequestRegistry` with exactly-once lifecycle operations - Transactional, non-blocking `AdmissionController` - Fixed-size batch Tech stack: Python Why it matters: This project is essential for developers looking to understand and optimize the concurrency core of serving systems, particularly in terms of request queue management, batch formation, and deadline handling.

### Overview: LLM ServeVerdict is an automated senior-review checklist designed to evaluate and decide between two different setups of an LLM inference server. It compares the evidence and outcomes of both setups, ensuring that only improvements that meet specific criteria are promoted, rejecting any changes that could compromise correctness or stability. ### Key features: - **Evidence Comparison**: Runs the same evidence for both setups to ensure consistency. - **Speed and Correctness Check**: Evaluates speed and correctness together, ensuring that only improvements that meet the criteria are promoted. - **Stability and Trust**: Promotes only when there is sufficient and trusted evidence, ensuring that the decision is based on robust metrics. ### Tech stack: Python ### Why it matters: LLM ServeVerdict serves as an automated senior-review checklist for LLM inference-server changes, ensuring that only improvements that meet specific criteria are promoted. This tool helps maintain the correctness and stability of the LLM inference server without relying on the opinion of the LLM itself.

### Overview: This project is a FastAPI-based churn prediction API designed for a D2C personal-care brand. It serves a trained Random Forest model to provide real-time churn probability, risk level, and recommended actions to the CRM team. The API is optimized for integration with products and deployment-ready, ensuring seamless integration and deployment. ### Key features: - **Real-time Scoring**: The API can predict churn probability for individual or batches of customers in real-time. - **REST Endpoints**: It provides RESTful APIs for easy integration and access to the model. - **Model Serving**: The model is trained and served via FastAPI endpoints, ensuring high performance and scalability. ### Tech stack: - **Python**: The primary language used for development. - **FastAPI**: The framework used to build the API, providing a robust and efficient solution for microservices. ### Why it matters: This project is crucial for the D2C brand as it helps in identifying at-risk customers, enabling targeted interventions to prevent churn. The API's real-time scoring capabilities ensure quick responses to customer inquiries, enhancing customer satisfaction and retention. ### Repository: The project is stored in the GitHub repository `patelmegh1234/d2c-churn-part4-fastapi-churn-api`, with a total of 0 GitHub stars. The repository contains the following files: - `main.py`: The FastAPI application. - `model.pkl`: The trained Random Forest pipeline. - `test_main.py`: A pytest test suite for 12 test cases. - `monitoring_plan

Overview: A developer-friendly inference stack for Nvidia DGX Spark, designed for home use with a focus on vLLM technology. Key features: - Single-Node Spark Inference - vLLM-Based Inference - Easy-to-Use Interface - Compatibility with DGX Spark Tech stack: Shell Why it matters: This stack provides a straightforward solution for developers looking to integrate AI inference capabilities on their DGX Spark systems, making advanced AI processing accessible to those with limited resources.

Overview: An ML serving platform that detects and automatically rolls back traffic to the last stable version when models' input distribution drifts. Key features: - Detects input distribution drifts in deployed models. - Automatically rolls back traffic to the last stable version. - Runs three different model types (fraud detection, satellite imagery, AI-text detection) to prove the reliability layer generalizes. Tech stack: Python Why it matters: This project builds the end-to-end layer of reliability engineering for ML models, testing it with injected drift on three different data types. The models themselves are intentionally simple, using well-known public datasets, to focus on the evaluation of the serving and reliability engineering aspects. Repository: pearlin-in/adaptive-ml-platform Primary language: Python GitHub stars: 0 GitHub description: ML serving platform that detects deployed model's input distribution drifts and automatically rolls back traffic to the last stable version — built across three unrelated model types (fraud detection, satellite imagery, AI-text detection) to prove the reliability layer generalizes, not just the model.

### Overview: This project is a production-oriented NLP/ML engineering laboratory designed to demonstrate reproducible data workflows, classical machine learning, PyTorch, Transformers, fine-tuning, dense vector search, retrieval, reranking, evaluation, serving, and MLOps. The repository includes Python data engineering, scikit-learn classical ML baselines, PyTorch dataset/training infrastructure, Hugging Face Transformers, sentence embeddings, dense vector search with FAISS, BM25 lexical retrieval, hybrid retrieval with Reciprocal Rank Fusion, bi-encoder retrieval, cross-encoder reranking, classification and retrieval evaluation, error analysis, experiment provenance, model artifact metadata and registry lifecycle, FastAPI model serving, health/readiness endpoints, inference telemetry, Docker deployment, and GitHub Actions quality gates. ### Key features: - Python data engineering and reproducibility - scikit-learn classical ML baselines - PyTorch dataset/training infrastructure - Hugging Face Transformers and fine-tuning - sentence embeddings - dense vector search with FAISS - BM25 lexical retrieval - hybrid retrieval with Reciprocal Rank Fusion - bi-encoder retrieval - cross-encoder reranking - classification and retrieval evaluation - error analysis - experiment provenance - model artifact metadata and registry lifecycle - FastAPI model serving - health/readiness endpoints - inference telemetry - Docker deployment - GitHub Actions quality gates ### Tech stack: Python ### Why it matters: This project demonstrates the importance of starting with the simplest defensible baseline and adding

### Overview: This is a minimal project designed to explore the architecture of an online inference service, focusing on API, features, versioning, batching, fallback, and observability. It aims to provide a learning track, a measurement bench, and an interview trainer, all while adhering to the principles of a lean engineering approach. ### Key features: - **API**: A stable interface for requests. - **Features**: Fetching online features, selecting model versions, executing inference, applying post-processing rules, and returning a decision. - **Versioning**: Version routing, batching, and fallback paths. - **Observability**: Monitoring and logging to track performance and service time. ### Tech stack: Python ### Why it matters: This project serves as a practical exercise in lean engineering, focusing on the design and implementation of an online inference service. It provides a learning track, a measurement bench, and an interview trainer, all while adhering to the principles of a lean engineering approach.