Projects

Overview: SVRaaS is a containerized deployment platform for generating, storing, and serving Sentiment Volatility Ratio (SVR) model forecasts using R, Plumber2, Docker, and Docker Swarm. Key features: - SVRaaS deploys and manages the Sentiment-Volume Ratio (SVR) model forecasts. - It uses R, Plumber2, Docker, and Docker Swarm for deployment and orchestration. - It includes a persistent model runner, shared NFS artifact storage, and an internal Plumber2 API service. - The model runner monitors FRED for new model data, runs the AR, ARIMA, MLP, and LSTM workflows sequentially, and exports dashboard-ready JSON artifacts. - The shared NFS storage is treated as a cache and is rebuilt when needed. - The API service provides read-only access to the shared artifact store and assembles coherent four-model result sets. Tech stack: R, Plumber2, Docker, Docker Swarm. Why it matters: SVRaaS simplifies the deployment and management of SVR model forecasts, making them accessible and reliable for public use.

### Overview: This project is a course project for the "Machine Learning Engineer" course at Yandex Practicum, focusing on serving a price model online. The implementation includes FastAPI, Docker, Prometheus, and Grafana for monitoring and visualization of the model's performance. ### Key Features: - **FastAPI**: A microservice for predicting property prices using CatBoost. - **Docker**: Containerization of the FastAPI microservice. - **Docker Compose**: Single command to start the microservice, Prometheus, and Grafana. - **Prometheus Metrics**: Basic HTTP metrics and custom Prometheus metrics. - **Grafana Dashboard**: Four-level monitoring dashboard with infra, real-time, and application panels, and load scripts. ### Tech Stack: - Python - FastAPI - CatBoost - Docker - Prometheus - Grafana ### Why It Matters: This project demonstrates the practical application of machine learning in a real-world scenario, specifically focusing on property price prediction. It showcases how to deploy a model online, monitor its performance, and visualize the results using Docker, Prometheus, and Grafana. The project is designed to help engineers understand and improve the reliability and performance of their machine learning models in production environments. ### Repository: The repository is named `liliya-ily/mle-project-sprint-3-v001-main` and is in the primary language Python. It currently has 0 GitHub stars. The README provides detailed instructions on how to set up and run the project, including how to deploy the microservice, monitor metrics, and visualize the

### Overview: This project is a course project for the Yandex Practicum, focusing on serving a price model online. It utilizes FastAPI and CatBoost with Docker for containerization, Prometheus for monitoring, and Grafana for dashboards. The project includes a Swagger API documentation, Docker Compose setup, Prometheus metrics, and Grafana dashboards for real-time monitoring. ### Key Features: - **FastAPI**: A microservice for the price prediction model. - **CatBoost**: A model for predicting apartment prices. - **Docker**: Containerized environment for microservices. - **Prometheus**: Metrics collection and monitoring. - **Grafana**: Real-time monitoring dashboard. ### Tech Stack: - Python: 3.10 - FastAPI: 0.111 - CatBoost: 1.2.5 - Docker: Compose - Prometheus: 0.111 - Grafana: 0.111 ### Why It Matters: This project is designed to meet the needs of the Yandex Nедвижимости team, allowing them to receive real-time predictions from the model. The microservice is deployed on Docker containers, and the metrics are monitored using Prometheus and Grafana, ensuring that the system can be quickly identified and addressed if any issues arise. ### Repository: - Repository: liliya-ily/practicum-s3-apartment-price-service - Primary Language: Python - GitHub Stars: 0 - GitHub Description: Course project (Yandex Practicum, MLE): serving

### Overview: This project demonstrates the use of BentoML for model serving, showcasing features such as adaptive micro-batching, multi-framework support, model packaging, containerization, and benchmarking. ### Key Features: - **Adaptive Micro-Batching**: Automatically batches individual requests for GPU/CPU throughput optimization. - **Multi-Framework Support**: Unified serving for scikit-learn, XGBoost, PyTorch, and Transformers models. - **Model Packaging**: Build portable Bento archives with all dependencies. - **Containerization**: One-command Docker image generation from Bento. - **Custom Runners**: Specialized runners for XGBoost, PyTorch, and ensemble models. - **Benchmarking**: Latency, throughput, and batching effectiveness measurement. ### Tech Stack: Python ### Why It Matters: This project provides a unified and opinionated framework for serving machine learning models, offering built-in features such as adaptive batching, model versioning, and containerization. It simplifies the deployment process, making it easier to implement advanced features like A/B testing and custom request pipelines.

### Overview: A FastAPI-based model-serving microservice designed for deploying scikit-learn models. It includes features like request validation, health checks, Prometheus metrics, rate limiting, async processing, and structured logging. ### Key Features: - **Request Validation**: Pydantic v2 models with strict input validation, type coercion, and descriptive error messages. - **Health & Readiness Probes**: Kubernetes-compatible `/health` and `/ready` endpoints with dependency checks. - **Prometheus Metrics**: Request latency histograms, prediction counters, error rates, and model version tracking. - **Rate Limiting**: Per-endpoint rate limiting with Redis-backed slowapi. - **Async Batch Processing**: Batch prediction endpoint with chunked processing for large payloads. - **Structured Logging**: JSON-formatted request/response logging with correlation IDs. - **Model Hot-Reload**: Update model versions without restarting the service. - **Request Tracing**: Unique request IDs propagated through middleware and logs. ### Tech Stack: - FastAPI - Docker - Kubernetes - Prometheus - Pydantic v2 - Redis - Slowapi ### Why It Matters: This microservice template is designed for observability and horizontal scalability, making it ideal for deploying and managing scikit-learn models in a production environment. The combination of FastAPI, Docker, Kubernetes, and Prometheus ensures that the service is both efficient and easy to monitor and scale.

### Overview: This project is a Spring Boot (Java) REST service designed for fraud model inference. It includes features such as health checks, circuit breakers (Resilience4j), metrics (Micrometer/Prometheus), and containerized deployment. The architecture is based on a microservice design, with specific emphasis on the preprocessing and inference stages. ### Key Features: - **Health Checks**: Ensures the service is operational. - **Circuit Breakers**: Handles failures and retries. - **Metrics**: Provides real-time monitoring and logging. - **Preprocessing**: Normalizes and validates input data. - **Model Service**: Uses ONNX Runtime for inference and circuit breaker protection. - **ONNX Runtime Engine**: Optimized for multi-threaded inference. - **Actuator Endpoints**: Provides health and metrics endpoints for monitoring. ### Tech Stack: - **Java 17+** - **Spring Boot** - **ONNX Runtime** - **Resilience4j** - **Micrometer/Prometheus** ### Why It Matters: - **Spring Boot Over FastAPI**: This project explores the use of Spring Boot as an alternative to FastAPI for ML model serving, highlighting the benefits of Spring Boot's enterprise ecosystem, type safety, and lower startup time. It also emphasizes the differences in inference overhead and monitoring compared to FastAPI. - **Decision**: FastAPI was chosen for the primary serving project due to its tight integration with the Python ML ecosystem, lower serialization overhead with NumPy arrays, and faster model development iteration. ### Project Structure: - **src/main/java/com

Overview: SmarterRouter is an intelligent LLM gateway and VRAM-aware router designed for Ollama, llama.cpp, and OpenAI. It features semantic caching, model profiling, and automatic failover for local AI labs. Key features: - Semantic caching - Model profiling - Automatic failover Tech stack: Python Why it matters: SmarterRouter enhances the performance and reliability of AI models by intelligently managing VRAM usage and providing failover capabilities, ensuring optimal performance in local AI labs.

### Overview: This project is an AI drawing-review microservice that separates the workflow of models, inputs, and checks into a FastAPI service with pluggable inference backends, typed results, a reproducible evaluation harness, and container/Kubernetes packaging. It uses synthetic drawings and keeps every result pending review. ### Key features: - **API Boundary:** Multipart input validation, stable Pydantic response/error schemas, JSON and CSV results, liveness/readiness, metrics, and no document content in logs. - **Backend Boundary:** Detector, OCR, and interpreter protocols; deterministic stubs plus YOLO, Tesseract, and LLaVA-through-vLLM clients. - **Evaluation:** Synthetic fixtures with known labels, extraction metrics, business-defect gates, and code/backend/config lineage. - **Packaging:** A non-root container, kustomize overlays, CI tests and manifest checks, and a recorded local Kind deployment with an evaluation job and rollback. ### Tech stack: Python ### Why it matters: This project is crucial for ensuring the reliability and accuracy of AI-driven drawing analysis workflows, making it easier to hand over models, inputs, and checks to other environments without compromising quality.

Overview: This project is an LLM inference optimization platform designed to reduce costs and improve latency by routing requests to the most efficient model, caching repeated prompts, and utilizing dynamic batching and TensorRT optimization for self-hosted models. Key features: - Intelligent model routing - Semantic and exact caching in Redis - Dynamic batching and continuous batching for self-hosted models - Quantization and TensorRT engines for model optimization - GPU scheduling across Kubernetes - Observability with Prometheus and Grafana dashboards Tech stack: PyTorch, CUDA, TensorRT, ONNX, Kubernetes, NVIDIA GPUs, GPU scheduling, Docker, Terraform. Why it matters: This platform helps reduce GPU costs, improve latency, and optimize resource usage, making it valuable for developers and businesses looking to improve the performance and cost-effectiveness of their generative AI applications.

Overview: This project provides an inference stack for Nvidia DGX Spark, a powerful AI supercomputer, tailored for developers on a budget. Primarily based on vLLM technology, it offers a streamlined solution for AI inference tasks. Key features: - vLLM-based inference stack - Single Spark deployment - Designed for developers on a budget Tech stack: Shell Why it matters: This project simplifies the process of deploying AI inference on a budget-friendly Nvidia DGX Spark setup, making advanced AI capabilities accessible to developers without the need for extensive hardware or software expertise.

Overview: ModelCourier is an outbound AI task relay system designed for Raspberry Pi, ESP32, and other small devices, enabling them to send tasks to a public control plane and receive results from local GPU workers without needing public IP, port forwarding, or inbound tunnels. Key features: - Supports Raspberry Pi, ESP32, and other small devices - Uses FastAPI for public API, SQLite for database, and local artifact files for lightweight model dependencies - Enables devices to submit versioned tasks, pull matching capabilities, and retrieve normalized results from local GPU workers - Optional support for Faster-Whisper, FunASR, and Ultralytics adapters in the `packages/provider-*` directory - Provides a small device-side client for Python and Raspberry Pi in the `packages/sdk-python` directory - Offers a documentation on REST contract and lifecycle in `docs/protocol.md` and a guide for adding another local model backend in `docs/provider-development.md` - Supports deployment in a single-process profile with reverse-proxy examples in the `deploy/` directory Tech stack: Python Why it matters: ModelCourier simplifies the task of sending and receiving AI tasks between edge devices and local GPU workers, making it useful when an edge device can reach the public internet but a GPU workstation cannot accept inbound connections.

Overview: 이 미션은 PyTorch 모델을 양자화 및 ONNX 형식으로 변환하고 추론 결과를 검증하는 실습입니다. Key features: - PyTorch 기본 형식 변환 - 양자화 모델 변환 - ONNX 형식 변환 - 추론 결과 검증 Tech stack: Python, PyTorch, ONNX, ONNX Runtime, Transformers, Jupyter Notebook Why it matters: 이 미션은 PyTorch 모델의 다양한 변환 방법과 추론 결과를 검증하는 실습을 제공하며, 이를 통해 모델의 성능과 용량에 대한 이해를 높일 수 있습니다.

### Overview: The "Cold-Start Tax" project explores the transient latency and energy consumption issues faced by IoT devices that duty-cycle, specifically those running on a Raspberry Pi 5. This work measures the cold-start tax, decomposes it into its components, and proposes a residency policy to mitigate these issues. ### Key Features: - **Cold-Start Tax Measurement**: Measures the transient latency and energy consumption of IoT devices duty-cycling. - **Cold-Wake Latency Decomposition**: Decomposes the cold-start tax into weight-loading and page-cache pressure. - **Energy Measurement**: Measures the energy consumption using on-board PMIC telemetry. - **Residency Policy**: Proposes a residency policy to reduce the cold-start tax, specifically targeting the eviction cliff effect. ### Tech Stack: - Python ### Why It Matters: This project provides a comprehensive understanding of the cold-start tax and proposes a practical solution to mitigate its impact on IoT device performance, particularly on duty-cycled edge devices like Raspberry Pi 5.

Overview: This project serves as a comprehensive inference stack for Nvidia DGX Spark, a powerful AI supercomputer. It is designed for developers looking to leverage advanced AI capabilities without the need for a dedicated server. Key features: - VLLM-based inference engine - Single Spark deployment - Primarily used for vLLM tasks Tech stack: Shell Why it matters: This stack provides a cost-effective solution for developers to experiment with advanced AI models, making it accessible to those without the resources for a full-scale AI supercomputer.

Overview: ModelOps Control Room is a runnable, deterministic release demo featuring gated canaries, rollback, fallback routing, and audit evidence without a GPU. It demonstrates model comparison, statistical admissibility policy, traffic-step state machine, and fallback decisions. Key features: - Gated canaries - Rollback - Fallback routing - Audit evidence Tech stack: Python Why it matters: It provides a production-shaped API and Kubernetes contracts for model comparison and release management without requiring a GPU, demonstrating quality/latency/cost gates, canary steps, rollback, fallback routing, and audit evidence.

### Overview: Bring your own open-weight Hugging Face checkpoints into Vertex AI Model Garden, customize them with non-default hyperparameters via serverless managed open model tuning, and serve both baseline and fine-tuned weights on fully managed Vertex AI dedicated endpoints. The pipeline is automated from Google Cloud Shell via `./deploy/run_cloud_shell.sh`, or follows a complete manual Console UI and CLI/SDK walkthrough. ### Key features: - Customizable Hugging Face checkpoints - Serverless managed open model tuning - Fully managed Vertex AI dedicated endpoints - Zero GKE clusters required ### Tech stack: Python ### Why it matters: This project allows developers to leverage their existing open-weight models without the need for GKE clusters, simplifying the process of integrating and serving their models in Vertex AI.

Overview: LinML is the Linux counterpart to Windows ML (WinML), providing a unified, hardware-abstracted inference layer, CLI, and catalog library for ONNX Runtime. Key features: - LinML CLI for easy model inference - LinML.Catalog for automated plugin execution provider discovery and registration - LinML.Catalog library for pure contracts and data models Tech stack: C# Why it matters: LinML simplifies the integration of machine learning models on Linux, making it easier to deploy and run models across various hardware platforms.

### Overview: Glacier Engine is an experimental full AI Runtime project written in Zig, designed to treat artifact identity, resource admission, placement, scheduling, execution, continuation, token and media publication, provider usage, and cost as explicit state transitions that can be rejected, replayed, and verified. It offers tested building blocks, credential-free demos, portable evidence formats, and independent verifiers. ### Key features: - **Atomic token publication.** KV rows, RNG state, sampler counters, and output words are committed together or remain invisible. - **Exact resource admission.** `ResourceBank` and `LeaseTree` make logical ownership and release part of the execution contract. - **Deterministic multi-request scheduling.** `LaneWeave` provides bounded, weighted service with replayable decisions and fail-closed permits. - **Portable workload-pressure evidence.** A versioned mixed image/audio/video scenario drives real admission, weighted service, overload rejection, timeout, cancellation, and final release; Zig exact replay and an independent Python verifier agree on every outcome, trace, summary, and root. - **Generated workload corpus.** Four retained seeds expand into 32 bounded deterministic open-loop scenarios through coordinate-addressed SHA-256 decisions. Zig and an independent Python implementation reproduce every unchanged W0/W1 case, while an exact-signature synthetic fixture proves deterministic local-minimum shrinking. - **Deterministic closed-loop conformance.** A separately versioned, finite-source controller maintains a declared logical in-flight target. Terminal outcomes create FIFO successor credits for the next logical step; canonical lineage and phase

Overview: This project aims to predict the number of transactions each visitor makes at a Google Merchandise Store using a logistic regression model in BigQuery. Key features: - Predicts the number of transactions each visitor makes - Sorts results and selects the top 10 visitors by transactions - Utilizes BigQuery ML for machine learning models Tech stack: Python, BigQuery, Logistic Regression Why it matters: This project demonstrates how to create, evaluate, and use machine learning models in BigQuery, specifically focusing on logistic regression for predicting customer behavior at a Google store.

### Overview: A self-hosted, air-gapped AI control plane designed for GPU monitoring, model registry and deployment, OpenAI-compatible gateway, agents with MCP and tools, RAG, speech and OCR, and a fully offline install bundle. ### Key features: - GPU monitoring - Model registry and deployment - OpenAI-compatible gateway - Agent platform (LangGraph + MCP + skills + tools) - RAG - Speech and OCR - Fully offline installation ### Tech stack: Python ### Why it matters: Built to a 28-module specification across 10 phases, every phase has a passing acceptance gate that runs on a laptop with no GPU. The platform is designed to be self-hosted and air-gapped, ensuring secure and isolated operation.