Projects

### Overview: This self-study course provides a comprehensive guide for professionals looking to design and implement production-level machine learning systems. The course covers the entire lifecycle of a machine learning project, including data quality, feature pipelines, experimentation, evaluation, serving, monitoring, and governance. It emphasizes architecture, requirements, and trade-offs, ensuring that the system around the model is treated as the primary engineering artifact. ### Key features: - **Design-first, not algorithm-first.** The emphasis is on architecture, requirements, and trade-offs. - **End-to-end.** Covers the full production ML lifecycle, including MLOps, deployment, observability, drift, and governance. - **Failure-driven.** Every module exists to prevent specific production failure modes. - **Vendor-neutral Python.** Uses scikit-learn, PyTorch, Pandera/Great Expectations, MLflow, FastAPI, Docker, Prometheus/Grafana, and no single-cloud assumption. ### Tech stack: Python ### Why it matters: This course is crucial for professionals who can train models in a notebook but need to design and implement production-level ML systems. It covers everything from data quality and feature pipelines to serving, monitoring, and governance, ensuring that the system around the model is treated as the primary engineering artifact. Repository: akshayDev17/ML-Engineer Primary language: Python GitHub stars: 0 GitHub description: Self-study course on designing production ML systems: 18 modules from data quality and feature pipelines to serving, monitoring and drift,

Overview: `fraud-ml-engine` is a comprehensive machine learning fraud detection engine designed for real-time risk scoring, anomaly detection, and decision services. It supports feature engineering, model benchmarking, explainability, and API-based scoring. Key features: - Feature pipelines - Model benchmarking across multiple families - Explainability with SHAP/LIME outputs - Production API delivery Tech stack: Python, FastAPI, XGBoost, LightGBM, PyTorch, SHAP, LIME, Plotly, Streamlit. Why it matters: This engine centralizes fraud model experimentation and serves, making it suitable for iterative model development and efficient production integration.

### Overview: Mojo-Bento is a Mojo library that serves as the core of BentoML, providing an epoll HTTP/1.1 server, typed JSON endpoints, health checks, metrics, streaming capabilities, and more. It is designed to be a drop-in replacement for BentoML 1.4.39, with features comparable to the latest version. ### Key Features: - **Server**: Supports epoll with `EPOLLONESHOT`, keep-alive, pipelining, request limits, and `Expect: 100-continue`. - **Endpoints**: Provides typed JSON endpoints with BentoML's 400 validation body and 404/405 responses. - **Operations**: `/healthz`, `/livez`, `/readyz`, `/metrics` with BentoML's metric names. - **Streaming**: Supports BentoML's chunked generator and real SSE. - **Batcher**: Implements BentoML's `CorkDispatcher` and `Optimizer` for batcher decisions. - **Model Store**: Utilizes BentoML's directory layout for model storage. ### Tech Stack: - **Mojo**: The primary language used for development and deployment. ### Why It Matters: Mojo-Bento offers a robust and feature-rich solution for developers looking to integrate BentoML's capabilities into their Mojo applications, ensuring compatibility and performance with the latest BentoML features.

### Overview: This repository is a comprehensive knowledge system designed for senior developers, solution architects, staff engineers, and technical leads. It focuses on the full system of AI solutions, including agent orchestration, model serving, training and adaptation, RAG data architecture, observability, evaluation, tool governance, security, and production operations. The repository is built from 17 real AI repositories and is organized into a course-style path for learning AI solution architecture. ### Key Features: - **Bilingual by Design:** English and Vietnamese learning paths for teams working across local and global engineering contexts. - **Repository-Grounded:** Concepts are tied to real AI repositories instead of abstract slides. - **Architect-Ready:** Includes templates, capstone, assessment rubrics, production checklists, and governance reviews. - **Production-First:** Every major topic connects to failure modes, observability, security, and release gates. ### Tech Stack: HTML ### Why It Matters: - **Bilingual Learning:** Provides a bilingual learning experience for teams working in both English and Vietnamese. - **Real-World Examples:** Concepts are tied to real AI repositories, making the learning more practical and relevant. - **Production-Ready:** Includes templates, capstone, assessment rubrics, and production checklists, ensuring that the knowledge is practical and ready for production use. - **Security and Observability:** Every major topic connects to security and observability, ensuring that the learning is comprehensive and up-to-date. ### Repository: anhtnt90dev/ai-solution-architecture - **Primary Language:** HTML

### Overview: Multi-LoRA-Serve is a production-informed inference gateway that allows you to use a single base model alongside multiple LoRA adapters for inference requests. It supports OpenAI-compatible APIs, leverages Prometheus for monitoring, and uses React for frontend integration. ### Key features: - **Dynamic Adapter Resolution**: Handles different locations of weights (local filesystem, Hugging Face Hub, S3) and selects the first hit for resolution. - **OpenAI-Compatible Surface**: Supports `POST /v1/chat/completions`, `POST /v1/completions`, and `GET /v1/models` with SSE streaming. - **Tenant and Header Routing**: Matches `X-Tenant-Id` or arbitrary headers, and allows explicit adapter selection from `model` or `X-Adapter-Name`. - **Cache and Load State**: Tracks adapter metadata in a thread-safe LRU cache and uses Redis for load state tracking. - **Observability**: Provides Prometheus metrics for observability, including histograms and counters, and integrates with Grafana for easy monitoring. ### Tech Stack: - Python 3.12+ - React 18 - Prometheus - Docker Compose - Redis ### Why it Matters: - **Efficiency**: Reduces the need for multiple model replicas, saving GPU resources. - **Flexibility**: Allows for easy addition or removal of adapters without reloading the base model. - **Scalability**: Tracks and manages adapter load state, ensuring the most up-to-date information is available. - **Ease of Use**: Supports OpenAI-compatible APIs

### Overview: BentoML is an open-source unified inference platform for building, packaging, and deploying machine learning models as scalable REST API services. It is maintained by API Evangelist and provides an independent profile of a company's publicly available API surface. ### Key features: - **Model Packaging**: Automatically exposes model inference logic as HTTP endpoints. - **RESTful API**: Provides a RESTful interface for model inference. - **Scalability**: Designed to be highly scalable and performant. - **Ease of Use**: Simple Python class decorators for defining services. ### Tech stack: Python ### Why it matters: BentoML simplifies the process of deploying machine learning models as RESTful APIs, making it easier for developers to integrate ML capabilities into their applications without the complexity of traditional machine learning frameworks.

### Overview: Cumulus Labs is an independent, third-party profile of a public API surface, maintained by API Evangelist. This repository contains an aggregated view of a company's publicly available APIs, including specifications, documentation, and specifications for public use. The information is sourced from publicly accessible materials such as the company's website, developer portal, and documentation. ### Key Features: - **Public API Surface**: Aggregate view of a company's publicly available APIs. - **Specifications**: OpenAPI, AsyncAPI, JSON Schema, and `apis.json`, `llms.txt` and similar. - **Public Repositories**: Source code and documentation hosted on public repositories. ### Tech Stack: API Evangelist ### Why it Matters: Cumulus Labs provides an independent, third-party profile of a company's publicly available APIs, sourced from publicly accessible materials. This repository is maintained by API Evangelist, an organization that does not operate, host, resell, or support the APIs. The information is assembled from various sources, including the company's website, developer portal, and documentation, and is not obtained by breaching a system, defeating an access control, or using credentials of any kind. The Kin Score and Agent Readiness rating are independently calculated scores of a company's public API artifacts, produced by API Evangelist against a published rubric. The profile is not a certification, endorsement, security assessment, or audit, and it scores published artifacts, not the quality, safety, or security of the software. Corrections, re-scores, and removal requests are free and can be made within one business

### Overview: Inferact is an independent, third-party profile of a public API surface, maintained by API Evangelist. This repository provides an AI inference infrastructure startup's public API artifacts, assembled from publicly available materials. ### Key Features: - Publicly available API surface - Profiled by API Evangelist - Independent assessment - No affiliation with the company ### Tech Stack: Unknown ### Why It Matters: Inferact is an independent assessment of a company's public API artifacts, providing a neutral and factual representation of the API surface. This can be valuable for developers seeking to understand and integrate with a company's APIs.

### Overview: KServe is an independent, third-party profile of a public API surface, maintained by API Evangelist. It provides a standard model inference platform on Kubernetes, built for highly scalable use cases. ### Key features: - KServe is a standard model inference platform. - It supports multiple ML frameworks including TensorFlow, PyTorch, scikit-learn, XGBoost, and more. - KServe is built for highly scalable use cases. - It provides performant, standardized inference protocol across various ML frameworks. ### Tech stack: Kubernetes ### Why it matters: KServe offers a standardized and performant way to deploy and run machine learning models, making it easier to scale and integrate with existing infrastructure.

### Overview: Kubeflow is an open-source machine learning platform designed for Kubernetes, offering tools for training, serving, tuning, and managing ML models across the full lifecycle. ### Key features: - Training and serving ML models - Tuning and managing ML models - Deploying ML workflows on Kubernetes - Simplifying deployments of ML workflows ### Tech stack: Kubernetes, Python, and various machine learning frameworks. ### Why it matters: Kubeflow simplifies the deployment of machine learning workflows on Kubernetes, making it accessible and scalable.

Overview: This profile is an independent, third-party overview of a company's publicly available API surface, maintained by API Evangelist. It does not operate, host, resell, or support the company's APIs. Key features: - API surface overview - Public API artifacts - Kin Score and Agent Readiness rating - Independent assessment Tech stack: HTML Why it matters: This profile provides a comprehensive overview of a company's publicly available API surface, including their API artifacts and ratings, without breaching any system or using credentials.

### Overview: This is an independent, third-party profile of a company's publicly available API surface, maintained by API Evangelist. It is not affiliated with or endorsed by the company unless stated on the profile. ### Key features: - Public API surface profile - Kin Score and Agent Readiness ratings based on publicly available artifacts - No affiliation with the company, no warranty or guarantee - No download or installation required ### Tech stack: None ### Why it matters: This profile provides a snapshot of the publicly available API surface, helping developers understand and interact with the company's services.

### Overview: Ray is an independent, third-party profile of a public API surface, maintained by API Evangelist. It consists of Ray Core (a distributed runtime) and a set of AI libraries for Python and AI workloads. ### Key features: - Ray Core: A distributed runtime for Python and AI workloads. - Ray Train: AI libraries for training models. - Ray Data: Data processing libraries. - Ray Tune: Optimization libraries. - Ray Serve: Deployment libraries. - RLlib: Reinforcement learning libraries. ### Tech stack: Ray ### Why it matters: Ray is an open-source unified compute framework that scales Python and AI workloads from a laptop to a cluster. It is maintained by Anyscale and is useful for developers looking to leverage AI and machine learning in their Python projects.

### Overview: This repository is an independent, third-party profile of a company's publicly available API surface, maintained by API Evangelist. It includes a collection of APIs, frameworks, and platforms for scalable machine learning model inference serving, deployment, and management. ### Key Features: - Scalable Inference Serving - KServe Open Inference Protocol - BentoML ### Tech Stack: Python, Kubernetes, Docker, Flask, OpenAPI ### Why It Matters: This profile provides a comprehensive overview of the company's public API surface, including its offerings and technologies used. It is useful for developers looking to understand and integrate the company's services into their projects.

### Overview: Seldon is an independent, third-party profile of a company's publicly available API surface, maintained by API Evangelist. It provides REST and gRPC inference APIs following the Open Inference Protocol, enabling standardized model deployment at scale. ### Key Features: - **REST and gRPC APIs**: Seldon offers both REST and gRPC interfaces for accessing the company's machine learning models. - **Open Inference Protocol**: The APIs adhere to the Open Inference Protocol, ensuring standardized model deployment and communication. - **Kubernetes Native**: Seldon is designed for Kubernetes environments, making it easy to integrate with existing infrastructure. ### Tech Stack: Seldon uses Kubernetes as its primary technology stack, enabling efficient deployment and management of machine learning models. ### Why it Matters: Seldon provides a comprehensive solution for enterprises looking to deploy, monitor, manage, and explain machine learning models at scale. Its REST and gRPC APIs, along with the Open Inference Protocol, offer a standardized approach to model deployment, making it easier for organizations to integrate with existing systems and adhere to industry standards.

Overview: Inferact is an independent profile of a public API surface, maintained by API Evangelist. This repository contains an assembly of information gathered from publicly accessible sources, including the company's website, developer portal, and documentation, to assess the public API artifacts. Key features: - API surface profile - Kin Score and Agent Readiness rating - Independent assessment - No affiliation with the company - Public and private repositories - No software download Tech stack: API Evangelist Why it matters: This repository provides an independent assessment of a company's public API surface, helping developers understand the capabilities and potential of the API without relying on the company's own information or services.

### Overview: A machine learning-powered system for predicting property prices and recommending similar properties in Pakistan's Real Estate and Housing Data. ### Key features: - Price prediction for Rent/Sale Properties - Similar Property Recommendations - Location-Based Pricing Tiers ### Tech stack: - Flask REST API - Scikit-Learn Pipelines - XGBoost Prediction - Scikit-Learn Nearest Neighbours (NN) Recommendations with KMeans Clustering ### Why it matters: Streamlines property valuation with automated price estimation and intelligent recommendations for Pakistan's Real Estate and Housing Data.

Overview: LiveBatch is a lightweight, framework-agnostic middleware designed to maximize GPU/TPU utilization by dynamically batching inference requests in real time. Key features: - Plug-and-play: Drop-in proxy over any HTTP ML server - Dynamic batching: Configurable by max latency or batch size - Written in Go: Fast, concurrent, production-ready - Transparent: Accepts and returns single inference calls - Efficient: Great for reducing GPU/TPU underutilization Tech stack: Go Why it matters: LiveBatch simplifies the process of optimizing GPU/TPU utilization by providing a straightforward and efficient solution for ML teams and microservices. Repository: arbiseeos/livebatch Primary language: Go GitHub stars: 0 GitHub description: A lightweight, framework-agnostic middleware that dynamically batches inference requests in real time to maximize GPU/TPU utilization.

Overview: This project is a Rust-based CLI tool designed to manage local MLX model servers on Apple Silicon. It allows users to list, start, stop, and check the status of models served by `mlx-vlm` or `mlx-lm`. Key features: - List, start, stop, and check the status of models served by `mlx-vlm` or `mlx-lm`. - Discover and manage models on disk without hardcoding. - Interactive and direct start/stop options. - Pin a model's port (persists). - Download models from HuggingFace and delete installed models. - Automatically detect and serve models based on their configuration. Tech stack: Rust 1.70+. Why it matters: This tool simplifies the management of MLX model servers, allowing users to easily discover, start, stop, and manage models on their Apple Silicon systems without the need for additional Python installations or external dependencies. Repository: arcasilesgroup/mlx-cli Primary language: Rust GitHub stars: 0 GitHub description: Manage local MLX model servers on Apple Silicon

Overview: A comprehensive, production-ready Machine Learning serving platform designed for high performance, reliability, scalability, and robust observability. Key features: - Microservices architecture with FastAPI-based API Gateway - Containerized model services for toxicity classification and comment ranking - Centralized MLflow model registry and tracking - Observability stack including Prometheus, OpenTelemetry Collector, and Grafana Tech stack: Python Why it matters: This platform ensures high performance, reliability, scalability, and robust observability, making it ideal for production environments. Repository: arec1b0/ml-platform Primary language: Python GitHub stars: 0 GitHub description: A comprehensive, production-ready Machine Learning serving platform designed for high performance, reliability, scalability, and robust observability.