AI Trending

Project directory

Projects

Open-source AI projects tracked by the leaderboard, ordered by GitHub stars.

Python 1 stars

RudraTyagi1135/phising-detection-mlops-system

### Overview: PhishGuard AI is a production-oriented phishing detection MLOps system that classifies URL/network-security feature records as phishing or legitimate. The system combines FastAPI inference services, modular ML training pipeline, schema validation and drift detection, Scikit-learn and XGBoost model selection, MLflow experiment tracking, DVC and DagsHub artifact versioning, MongoDB Atlas data ingestion and prediction audit logging, Docker packaging, GitHub Actions CI/CD, and AWS ECR and ECS Fargate deployment assets. ### Key features: - **FastAPI Inference Services**: Real-time phishing detection using FastAPI. - **Modular ML Training Pipeline**: Scalable and reproducible training pipeline. - **Schema Validation and Drift Detection**: Ensures data integrity and model drift. - **Scikit-learn and XGBoost Model Selection**: Combines multiple models for improved accuracy. - **MLflow Experiment Tracking**: Tracks ML experiments and model performance. - **DVC and DagsHub Artifact Versioning**: Versioned datasets and models for reproducibility. - **Docker Packaging**: Ensures consistent deployment across environments. - **GitHub Actions CI/CD**: Automates the build, test, and deployment process. - **AWS ECR and ECS Fargate**: Cloud-native deployment with AWS services. ### Tech stack: - **Python**: The primary programming language used. - **FastAPI**: Backend framework for real-time predictions. - **Scikit-learn**: For model training and evaluation. - **XGBoost**: For model training and prediction. - **DVC
Unknown 1 stars

TGPSKI/redline

Find max vLLM context config on a single GPU with a directed-workflow agent skill

Overview: Redline is a directed-workflow agent skill designed to find the maximum vLLM context configuration on a single GPU, leveraging a safe context ceiling. Key features: - Finds the safe context ceiling for a vLLM-served model on a fixed GPU. - Climbs `--max-model-len` and `--max-num-batched-tokens` one variable at a time. - Records real KV-cache and concurrency signal from vLLM boot logs. - Maintains state in a ladder file for persistent rung records. - Reads the last rung and dispatches to the next phase. Tech stack: Directed-workflow agent skill. Why it matters: Redline is crucial for optimizing vLLM model context configurations on GPUs, ensuring efficient and safe model performance.
Python 1 stars

TR-3N/ai-junior-data_scientist

Built an AI Junior Data Scientist agent using LangChain AgentExecutor, Ollama, and FastAPI on a 10k‑row bank‑churn dataset; automated EDA and baseline modeling (logistic regression) and served results via a REST API with latency and usage metrics, achieving ~0.82 accuracy on held‑out customers.

### Overview: This project showcases an AI-powered Junior Data Scientist Agent that automates Exploratory Data Analysis (EDA) and baseline machine learning modeling on a real-world fintech bank churn dataset. The agent uses a fully local LLM via Ollama, orchestrated with LangChain's AgentExecutor, and exposes results via a FastAPI backend. ### Key Features: - Loads and inspects the dataset - Performs EDA (dataset overview, numeric summaries, value counts) - Trains a baseline churn prediction model - Explains results in natural language - Serves all functionality via a REST API ### Tech Stack: - Python 3.x - pandas, numpy - scikit-learn - Ollama - LangChain - FastAPI - Uvicorn - LangChain-ollama - LangChain-classic - AgentType.ZERO_SHOT_REACT_DESCRIPTION ### Why it Matters: This project demonstrates how agentic LLMs can autonomously decide when to run Python data tools, train models, and explain results in natural language without using any paid APIs, achieving ~0.816 accuracy on the churn prediction task using a baseline Logistic Regression model on the bank churn dataset (10k rows).
Rust 1 stars

VeyrForge/GreenEngine

Run LLMs on everyday hardware: memory-smart MoE scheduling, KV paging, and the `ge` CLI for local GGUF inference. Pairs with Green Compress.

### Overview: Green Engine is a local runtime for large language models, designed to run on consumer GPUs and CPUs. It supports memory-smart multi-branch expert (MoE) scheduling, KV paging, and the `ge` command-line interface for offline inference and model management. ### Key Features: - **`ge run` / `ge chat serve` on `.gguf`**: Launches `llama-cli`, `llama-server`, or `llama_cpp` with ordinary GGUF models. - **Phase 1 Compressed GGUF**: `greencompress export-gguf` output runs with `ge chat serve --model file.gguf`. - **Model Search / Pull / Bench / MCP Stack**: Active development. - **Native `.green` Dense Generate**: Experimental, `ge run model.green` and `ge chat serve --model model.green` run through `engine-core`. - **Tiny MoE `.green` Generate**: Experimental, `tiny-moe.green` opens and generates through the native router. - **Long-Context / Paged KV**: Experimental, bounded-RAM KV is live on native decode with full-window default, optional tighter hot cap, and `K=Q8` keys for RAM savings. - **Green-Compressed Inference in `ge run`**: Experimental, native `.green` path exists for dense and tiny-MoE engineering, while production chat still prefers exported GGUF via `llama.cpp`. ### Tech Stack: Rust ### Why It Matters: Green Engine simplifies the process of running large language models on hardware
Python 1 stars

WaffleBits/secure-gpu-inference-gateway

Inference gateway with JWT auth, role policy, token budgets and audit logging, benchmarked against direct vLLM

Overview: This Python service acts as an inference gateway for an LLM server, enforcing JWT authentication, role-based policies, rate and token limits, and recording audit logs and traces. It benchmarks against direct vLLM and a mock backend. Key features: - JWT authentication - Role-based access control - Rate and token limits enforcement - Audit logging and tracing - Redis-based limiter for Redis mode - Real vLLM throughput tracking Tech stack: Python Why it matters: This service provides a secure and efficient way to control access to an LLM, ensuring that only authorized users can interact with the models, and it benchmarks against real vLLM performance. Repository: WaffleBits/secure-gpu-inference-gateway
Python 1 stars

WaffleBits/triton-inference-benchmark

Authenticated Triton/OpenAI serving benchmark with clock-aware agents, agent/coordinator restart reconciliation, privacy-safe aggregates, and reliability gates.

Overview: This project is a load generator designed for Triton/OpenAI inference serving, focusing on clock-aware agents, agent/restart reconciliation, privacy-safe aggregates, and reliability gates. Key features: - Mock backend for testing - Separate harness from client interface - Conservative behavior across machines - Never mistake a chunk for a token - Limited to single-host CI for now Tech stack: Python Why it matters: This benchmark ensures that the harness itself holds up, demonstrating that two authenticated TLS agents on one host recover completed work after response loss and restarts, and the target receives its eight requests with zero duplicates.
Python 1 stars

ahmadalsharef2/llm-mcp-fastapi

FastAPI + MCP server for LLM inference — chat API with tool-calling, streaming responses, and model-context-protocol support.

### Overview: This project is a FastAPI-based backend server that integrates with the Model Context Protocol (MCP) to expose tools to local LLMs, such as Ollama, for use with a chat API. It supports streaming responses and model context protocol support, making it easy to extend with new tools. ### Key features: - **FastAPI Backend**: Implements MCP-compatible endpoints. - **Tool Listing**: `GET /mcp/tools/list`. - **Tool Invocation**: `POST /mcp/tools/invoke`. - **Example Tool**: `get_weather(city)`. - **Works with Ollama and any LLM supporting tool calling**. - **Easy to Extend**: Add new tools in minutes. ### Tech stack: - **Python**: FastAPI backend. - **MCP**: Model Context Protocol. - **Ollama**: Local LLM for tool calling. ### Why it matters: This project provides a clean and efficient way to integrate local LLMs with tools, allowing for seamless tool invocation and streaming responses, making it ideal for chat APIs and other applications requiring model context protocol support. ### Repository: - **GitHub Stars**: 1 - **GitHub Description**: FastAPI + MCP server for LLM inference — chat API with tool-calling, streaming responses, and model-context-protocol support. - **Primary Language**: Python - **GitHub URL**: [https://github.com/ahmadalsharef2/llm-mcp-fastapi](https://github.com/ahmadalsharef2/llm-mcp-fastapi)
Python 1 stars

ahmadalsharef994/llm-mcp-fastapi

FastAPI + MCP server for LLM inference — chat API with tool-calling, streaming responses, and model-context-protocol support.

### Overview: This project is a FastAPI backend that exposes a Model Context Protocol (MCP) compatible endpoint, allowing for the integration of local LLMs with a chat API. It supports tool calling and streaming responses, making it easy to extend with new tools. ### Key features: - **FastAPI Backend**: Provides MCP-compatible endpoints. - **Tool Listing**: `GET /mcp/tools/list` endpoint. - **Tool Invocation**: `POST /mcp/tools/invoke` endpoint. - **Example Tool**: `get_weather(city)` tool. - **Works with Ollama and any LLM supporting tool calling**. - **Easy Extension**: Adds new tools in minutes. ### Tech stack: - **Python**: FastAPI backend. - **MCP**: Model Context Protocol. - **Ollama**: Local LLM for tool calling. - **Tool Calling**: Enables tools to be called from the API. ### Why it matters: This project is useful for developers looking to integrate local LLMs with a chat API, providing a clean REST API for tool calling and model-context-protocol support. It simplifies the process of extending the system with new tools and tools that can be easily integrated with any LLM that supports tool calling. ### Repository: ahmadalsharef994/llm-mcp-fastapi ### GitHub stars: 1 ### GitHub description: FastAPI + MCP server for LLM inference — chat API with tool-calling, streaming responses, and model-context-protocol support.
Python 1 stars

ai-art-dev99/mlops-capstone

End-to-end MLOps: training → MLflow → containerized FastAPI → Helm+HPA → Prom/Grafana → drift checks → CI/CD with GitHub Actions

Overview: This project demonstrates an end-to-end MLOps pipeline, covering model training with MLflow, deployment using Docker and Helm, observability with Prometheus and Grafana, drift checks, and CI/CD using GitHub Actions and Terraform. Key features: - Model training with MLflow metrics and artifacts - Serving with a Dockerized FastAPI application - Kubernetes deployment via Helm with Horizontal Pod Autoscaler (HPA) - Observability with Prometheus metrics and Grafana dashboard - Data drift check via a CronJob - CI/CD with GitHub Actions and GitHub Container Registry (GHCR) - Infrastructure as Code (IaC) with Terraform for GKE Autopilot Tech stack: Python, MLflow, Docker, Helm, Prometheus, Grafana, Kubernetes, Terraform, GitHub Actions Why it matters: This project showcases best practices for model deployment and operationalization, making it a valuable resource for developers looking to implement production-ready MLOps pipelines.
Python 1 stars

ai-lab-tech/mlflow-plugin

MLflow deployment plugin and Triton model flavor for NVIDIA Triton Inference Server via Triton Control.

Overview: This project is a plugin for MLflow that allows developers to log trained models and generate Triton serving files, making it easier to deploy models using NVIDIA Triton Inference Server. Key features: - Logs trained models to MLflow - Generates Triton configuration, serving code, model assets, and example requests Tech stack: Python Why it matters: This plugin simplifies the process of deploying machine learning models using Triton, making it easier for developers to integrate MLflow with Triton without needing to write custom deployment code.
Jupyter Notebook 1 stars

algorithmiaio/githubactions-modeldeployment-demo-algorithmiaalgo

Demo ML repository, using Algorithmia Model Deployment Github Action, to auto deploy on an algorithm hosted on Algorithmia

Overview: This repository demonstrates a streamlined workflow for automating the deployment of an XGBoost model and its inference script to Algorithmia, utilizing the Algorithmia Model Deployment Github Action. Key features: - Automated deployment of an XGBoost model and inference script - Continuous integration and deployment using Algorithmia Github Action - Scalable inference endpoint for ML models hosted on Algorithmia - Integration with Jupyter notebooks for model training and evaluation - Deployment triggered by Git push to the main branch - Configuration options for either notebook or model file path - Model manifest file containing metadata and upload details Tech stack: Jupyter Notebook Why it matters: This example showcases a practical approach to automating the deployment of machine learning models, making it easier for developers to integrate ML workflows into their repositories and streamline the deployment process.
Jupyter Notebook 1 stars

algorithmiaio/githubactions-modeldeployment-demo-githubalgo

Demo ML repository, using Algorithmia Model Deployment Github Action, to auto deploy on an Algorithmia algorithm backed by Github

Overview: This repository demonstrates the automated deployment of an XGBoost model and its inference script to Algorithmia using the Algorithmia Model Deployment Github Action. Key features: - Automated deployment of an XGBoost model - Continuous integration and deployment for model updates - Integration with Algorithmia's scalable inference endpoint - Support for both Jupyter notebook and saved model file configurations Tech stack: Jupyter Notebook Why it matters: This example showcases the seamless integration of machine learning models and inference scripts into Algorithmia, automating the deployment process and ensuring model updates are reflected in the inference API.
Jupyter Notebook 1 stars

algorithmiaio/githubactions-modeldeployment-template

Template ML repository to get started with Algorithmia Model Deployment Github Action integration

Overview: This is a template ML repository designed to integrate Algorithmia Model Deployment Github Action for continuous deployment, suitable for Jupyter notebook training and evaluation of ML models. Key features: - GitHub Actions integration - Algorithmia Model Deployment Github Action integration - Continuous deployment of ML models to Algorithmia Tech stack: Jupyter Notebook Why it matters: This template helps streamline the deployment process for ML models, making it easier to integrate Algorithmia's continuous deployment workflow into existing Jupyter notebook-based ML workflows. Repository: Algorithmiaio/githubactions-modeldeployment-template Primary language: Jupyter Notebook GitHub stars: 1 GitHub description: Template ML repository to get started with Algorithmia Model Deployment Github Action integration README excerpt: This repository provides a template for integrating Algorithmia's Model Deployment Github Action with Jupyter notebooks, suitable for ML model training and continuous deployment. It includes necessary files and can be used as a starting point for creating a new repository or as a reference for integrating the Github Action into an existing ML repository.
Jupyter Notebook 1 stars

amine-akrout/Smoker_detection

Smoker Detection deep learning model served via a Web App using TensorFlow, tensorflow-serving, flask and Docker compose

### Overview: This project features a deep learning model for detecting smokers using a Convolutional Neural Network (CNN) trained on TensorFlow. The model is deployed via a Flask web app and Docker containers, allowing it to be accessed via a web interface. ### Key Features: - Deployed model using TensorFlow InceptionV3 - Web app served via Flask - Dockerized environment for efficient deployment - Data split into training and testing sets ### Tech Stack: - Python - Flask - TensorFlow - Docker - Docker Compose ### Why it Matters: This project showcases the integration of deep learning models with modern web technologies, making it accessible and deployable in a containerized environment.
Unknown 1 stars

api-evangelist/databricks

Databricks — independent third-party profile of a public API surface, by API Evangelist. Collection of Databricks REST APIs for managing workspaces, clusters, jobs, and data operations.

### Overview: This repository is an independent, third-party profile of a company's publicly available API surface, maintained by API Evangelist. It provides a collection of Databricks REST APIs for managing workspaces, clusters, jobs, and data operations. ### Key features: - Databricks REST APIs - Workspace management - Cluster management - Job management - Data operations ### Tech stack: Unknown ### Why it matters: This profile offers a comprehensive overview of Databricks' public API surface, allowing developers to quickly access and utilize the company's APIs for their projects.
Unknown 1 stars

api-evangelist/google-tensorflow

Google TensorFlow — independent third-party profile of a public API surface, by API Evangelist. Google TensorFlow is an open-source machine learning framework providing APIs and tools for building, training, and deploying ML models, including TensorFlow Serving for model inference and TensorFlow Hub for reusable model components.

### Overview: This repository is an independent, third-party profile of Google TensorFlow, an open-source machine learning framework. It provides APIs and tools for building, training, and deploying ML models, including TensorFlow Serving for model inference and TensorFlow Hub for reusable model components. ### Key Features: - TensorFlow Serving: API for model inference. - TensorFlow Hub: Platform for reusable model components. ### Tech Stack: Google TensorFlow ### Why it Matters: This profile offers a comprehensive overview of Google TensorFlow, including its APIs and tools, making it a valuable resource for developers looking to integrate machine learning into their projects.
Unknown 1 stars

api-evangelist/tensorflow

TensorFlow — independent third-party profile of a public API surface, by API Evangelist. TensorFlow is an end-to-end open source machine learning platform developed by Google. It provides a comprehensive ecosystem of tools, libraries, and community resources for building and deploying ML-powered applications, including model training, serving, mobi

Overview: This repository is an independent, third-party profile of a company's publicly available API surface, maintained by API Evangelist. It provides a comprehensive ecosystem of tools, libraries, and community resources for building and deploying ML-powered applications, including model training, serving, and mobile applications. Key features: - Model training - Model serving - Mobile applications Tech stack: Python Why it matters: This profile offers a detailed understanding of the company's API surface, including its public repositories and specifications, making it useful for developers looking to integrate the company's services into their projects.
Unknown 1 stars

api-evangelist/triton

Triton Inference Server — independent third-party profile of a public API surface, by API Evangelist. NVIDIA Triton Inference Server provides a cloud and edge inferencing solution optimized for both CPUs and GPUs. Triton supports an HTTP/REST and gRPC protocol that allows remote clients to request inferencing for any model being managed by the serv

### Overview: Triton Inference Server is an independent, third-party profile of a public API surface maintained by API Evangelist. It provides a cloud and edge inferencing solution optimized for CPUs and GPUs, supporting HTTP/REST and gRPC protocols. ### Key features: - Triton Inference Server - Supports HTTP/REST and gRPC protocols - Optimized for both CPUs and GPUs - Cloud and edge inferencing solution ### Tech stack: Python, C++, and Rust ### Why it matters: Triton Inference Server offers a cloud and edge inferencing solution that is optimized for both CPUs and GPUs, making it suitable for a wide range of applications. The platform supports HTTP/REST and gRPC protocols, allowing remote clients to request inferencing for any model being managed by the service.
Unknown 1 stars

atryx/mlops-engineering-roadmap-2026

MLOps engineering roadmap for 2026 — experiment tracking, model serving, feature stores, pipelines, monitoring, GPU infra, and platform engineering for ML

Overview: This repository provides a structured roadmap for becoming a production MLOps engineer in 2026, covering foundational tools, experiment tracking, data pipelines, model training infrastructure, model serving, CI/CD, monitoring, GPU infrastructure, platform engineering, and more. Key features: - Experiment tracking with MLflow - Data pipelines and feature stores - Model training infrastructure - Model serving and inference - CI/CD for ML - Monitoring and observability - GPU infrastructure and cost - Platform engineering for ML Tech stack: Python, Docker, Git, Linux, Cloud basics (AWS/Azure/GCP), SQL, ML basics Why it matters: This comprehensive guide helps bridge the gap between data science experimentation and production systems, ensuring reliable and scalable ML deployments.
Python 1 stars

bdllhdrss3/EmbedServe

A high-performance, local embedding service for running models like Qwen, providing semantic text embeddings via FastAPI, with caching and concurrency control.

Overview: This project is a high-performance, local embedding service powered by Qwen models, designed to provide semantic text embeddings. It includes caching, concurrency control, and flexible configuration options. Key features: - 🚀 High Performance: Optimized for throughput with concurrency control - 💾 Intelligent Caching: Disk-based caching with configurable size limits - 📊 Monitoring: Cache hit/miss statistics for performance tuning - 🔧 Configurable: Environment variables for all settings - 🐳 Containerized: Ready-to-use Docker support - 🔄 Format Options: Return embeddings as JSON lists or binary numpy arrays - 🔌 Model Flexibility: Easy to swap embedding models Tech stack: Python Why it matters: This service addresses the challenges of running embedding models efficiently at scale, providing a self-hosted solution that reduces costs, ensures privacy, minimizes latency, and offers customization options. Repository: bdllhdrss3/EmbedServe Primary language: Python GitHub stars: 1 GitHub description: A high-performance, local embedding service for running models like Qwen, providing semantic text embeddings via FastAPI, with caching and concurrency control.