Projects

Overview: This project is a PyTorch-based toy project that generates images from captions using a Generative Adversarial Network (GAN) and Conditional Self-Attention (CLS) model. Key features: - Uses Flickr-30K dataset for image generation - Implements sentence encoding using SkipThought model - Utilizes PyTorch framework for model training - Runs on Google Colab for easy experimentation Tech stack: PyTorch Why it matters: This project showcases the potential of combining GANs with CLS models to generate realistic images from captions, demonstrating the capabilities of modern deep learning techniques.

Overview: This repository demonstrates a minimal example of deploying a fine-tuned BERT model using FastAPI. Key features: - Fine-tuning and deploying BERT model using FastAPI - Experiment with 3 BERT and BERT-variant models on the GLUE CoLA dataset - Use HuggingFace Transformers Trainer API for training and validation - Performance metric: Matthew's correlation - Quantization of BERT model layers for inference optimization Tech stack: Python, FastAPI, HuggingFace Transformers, PyTorch Why it matters: This project serves as a minimal example to showcase how to deploy a BERT model using FastAPI, demonstrating the capabilities of the framework in serving machine learning models.

Overview: This is a real-time monitoring dashboard designed for local LLM inference servers running on Apple Silicon, including llama.cpp, vllm-mlx, oMLX, and Splash backends. It tracks various metrics such as global stats, per-server cards, system cards, and GPU usage, providing a comprehensive overview of server performance. Key features: - Global stats - Per-server cards with live metrics - System card - Sparkline charts per server - Throughput chart - GPU chart - Thermals & GPU throttling - Auto-refresh via SWR polling Tech stack: Next.js 16, React 19, TypeScript, Tailwind CSS 4, Recharts, SWR Why it matters: This dashboard offers a single interface to monitor multiple LLM inference servers, providing real-time insights into performance and resource usage, making it invaluable for developers and administrators managing these systems. Repository: janhilgard/vllm-mlx-dashboard Primary language: TypeScript GitHub stars: 8 GitHub description: Real-time monitoring dashboard for local LLM inference servers (vllm-mlx + llama.cpp) — built with Next.js

### Overview: Vexpresso is a simple and scalable multi-modal vector database built with Daft, designed for easy integration with various data sources and efficient querying of images, audio, and text. ### Key features: - **Simple**: Vexpresso is lightweight and easy to get started! - **Flexible**: Supports arbitrary datatypes, including images, audio, and text. - **Scalable**: Can be scaled using Ray to multi-gpu / cpu clusters. - **Persistent**: Easy Saving and Loading functionality: Vexpresso has easily accessible functions for saving / loading to Hugging Face datasets. ### Tech stack: Vexpresso is built with Python and uses Daft as its backend. ### Why it matters: Vexpresso is designed to be simple, flexible, scalable, and persistent, making it an ideal choice for developers looking to integrate multi-modal data efficiently and easily. Repository: jesterlabs/vexpresso GitHub stars: 8 GitHub description: Vexpresso is a simple and scalable multi-modal vector database built with Daft.

Overview: This plugin adds a simplified Vector Database type to Unreal Engine 5, designed for creating, managing, and querying vector databases with support for various data types and distance metrics. Key features: - Create vector databases using the 'Create Vector Database' node - Add entries of different types: strings, objects, and structs (with support for most variable types) - Retrieve entries using various matching methods: Get Top N Matches, Get Top N Struct Matches, and Get Detailed Top N Matches - Remove entries based on vector matches with optional range and multiple occurrence removal - Database statistics and management functions Tech stack: Unreal Engine 5 C++ Why it matters: This plugin leverages the power of embeddings to create semantic search functionalities, making it useful for applications that require efficient and accurate data retrieval based on vector representations.

Overview: This project is a community-maintained vLLM hardware plugin for Ascend NPUs, designed to run inference workloads on Ascend hardware without altering the entire codebase. Key features: - Implement vLLM's hardware backend interface - Map vLLM's memory management, kernel execution, and device synchronization primitives to Ascend's CANN runtime and kernel libraries - Support for Ascend 910B and 910A, with experimental support for LoRA adapters and speculative decoding Tech stack: Python Why it matters: This plugin provides a cost-effective way to run vLLM workloads on Ascend NPUs, leveraging the hardware acceleration while maintaining compatibility with the upstream vLLM codebase. Repository: jmerelnyc/vllm-ascend Primary language: Python GitHub stars: 8 GitHub description: Community-maintained vLLM hardware plugin for Ascend NPUs

Overview: This project provides a reproducible build for serving Blackfrost's uncensored GLM-5.3-Flash model on four DGX Spark nodes using four NVIDIA DGX-4 and NVFP4 attention configurations with DFlash2 speculative decoding. The build is based on vLLM v0.30.0 and includes the vLLM main nightly build with the Humming MoE backend and Z.ai's official chat template. Key features: - Reproducible build - Four DGX Spark nodes with NVFP4 attention and DFlash2 speculative decoding - vLLM main nightly build with Humming MoE backend - Z.ai's official chat template - Stock vLLM images with patched files Tech stack: Python Why it matters: This build is designed to be highly reproducible and efficient, ensuring consistent performance and quality across multiple runs. The use of DGX Spark nodes and NVFP4 attention configuration provides a powerful computing environment for large language models, while DFlash2 speculative decoding enhances model performance. The choice of vLLM main nightly build with Humming MoE backend ensures compatibility with the latest developments in the vLLM project.

Overview: This project implements a document query system using LangChain, ChromaDB, and OpenAI's language models, enabling users to create a searchable database from markdown documents and efficiently retrieve information through natural language queries. Key features: - Vector database creation from markdown documents - Embedding and query cost estimation - Similarity searches on the database - AI-powered response generation for user queries Tech stack: Python Why it matters: This project provides a powerful solution for efficient and effective information retrieval, leveraging the strengths of LangChain, ChromaDB, and OpenAI's language models to create a robust and user-friendly query engine for developers and researchers.

Overview: NeuralForge is an open-source platform designed for training, evaluating, and deploying machine learning models, focusing on reproducibility, scalability, and ease of use. Key features: - Experiment management - Data pipeline - Model registry - Hyperparameter optimization - Distributed training - Model serving - Web dashboard - Command-line interface Tech stack: Python, PyTorch Why it matters: NeuralForge simplifies the entire machine learning workflow, making it accessible for researchers and engineers who want to iterate quickly on model architectures without sacrificing reproducibility or code quality. Repository: kamalesh404/NeuralForge Primary language: Python GitHub stars: 8 GitHub description: AI model training platform — train, fine-tune, deploy ML models in browser with WebGPU acceleration

### Overview: VELOCITI is a benchmarking project designed to evaluate and visualize compositional reasoning in video-language tasks. It includes strict entailment benchmarking and provides tools for evaluating models using CLIP and Video-LLMs. ### Key features: - **Benchmarking with Strict Entailment**: VELOCITI evaluates compositional reasoning tasks with a focus on strict entailment, ensuring that models can reason in a logical and consistent manner. - **Data Collection**: The dataset is available on HuggingFace and can be downloaded using the provided script. - **Environment Setup**: Users can set up the environment using the provided script, which includes installation of dependencies. - **Evaluation Scripts**: Scripts are provided for evaluating models using CLIP and Video-LLMs, including comprehensive metrics and examples of how to run the scripts. - **Citation**: The project is cited in a relevant conference paper, emphasizing its utility and importance in the field of video-language reasoning. ### Tech stack: Python ### Why it matters: VELOCITI is crucial for researchers and developers looking to evaluate and improve compositional reasoning models in video-language tasks. It provides a standardized benchmark that can help in the development and validation of models, ensuring that they can handle complex reasoning tasks effectively.

### Overview: Qwenvert simplifies the integration of Claude Code with local Qwen models, ensuring privacy and security. ### Key Features: - **Local Model Integration:** Configures Claude Code to work with local Qwen models, keeping your code private. - **Zero Inference Costs:** No internet required, eliminating cost and security risks. - **End-to-End Privacy:** Ensures that your code and data remain secure and private. ### Tech Stack: Python ### Why It Matters: Privacy, Security, and Compliance. Zero Inference Costs. No Internet Required.

## Overview: A curated selection of AI tools and models, backed by AI Product Registry facts, for developers. ## Key features: - **Models**: - Language & Search Models - Image Models - Video Models - Speech Models - Music Models - 3D & Spatial Models - **Agent Development Infrastructure**: - Agent Frameworks & Infrastructure - Agent Payments & Identity - Agent & Model Evaluation - Agent Development Platforms - LLM & Agent Frameworks - **Coding & App Development**: - Coding Tools - App Builders - Developer Platforms - **Image & Design**: - Image Creation & Editing - Design & User Research - Multimodal Creation - **Video & Filmmaking**: - Video Generation & Editing - Filmmaking & Localization - **Voice, Speech & Audio**: - Speech & Transcription Applications - Music, Audio & Podcasts - **Search & Web Data**: - AI Search Applications - Standalone Web Search APIs - SERP APIs - Model-Hosted Web Search Tools - URL Fetch APIs - Web Scraping & Extraction - Web Browsers - Browser Automation & Infrastructure - **Data & AI Infrastructure**: - Cloud & Model Infrastructure - Data Platforms & Analytics - Model Access & Routing - Model Hubs & Communities - Model Runtime Tools

### Overview: This repository contains 8 tested recipes for FLUX 2 Pro prompts, including example renders, aspect ratios, and the model ID (`flux-2-pro`) along with the cost per image. It provides a concise and effective way to generate high-quality images for marketing and design purposes. ### Key Features: - **8 Recipes**: Detailed prompts for various subjects, including products, food photography, and more. - **Aspect Ratios**: 1:1, 4:3, 8:5, etc., to ensure the best visual representation. - **Model ID**: `flux-2-pro` for consistent results. - **Cost Per Image**: $0.024 per image, with a minimum of $1 for the entire set. ### Tech Stack: Python ### Why It Matters: Teams using FLUX 2 Pro (`flux-2-pro`) benefit from a single, OpenAI-compatible endpoint that supports 300+ models, eliminating the need for monthly subscriptions or prepaid plans. The API is priced based on the number of calls, ensuring cost-effective usage. The prompt structure is designed to guide the model towards the most effective rendering, avoiding unnecessary guesswork. The asynchronous nature of the API calls allows for efficient management of tasks and retries, making the process seamless and reliable.

Overview: This project is a minimal demo showcasing the capabilities of a RAG (Retrieval-Augmented Generation) system using LangChain and Milvus. It includes all the necessary components to build a basic RAG system. Key features: - Milvus as the vector database - Ollama for document tokenization and interaction - Preparing documents and loading them into the vector database - Running the RAG system with a simple command Tech stack: Python Why it matters: This project serves as a practical demonstration of how to integrate and utilize LangChain and Milvus for building a RAG system, making it valuable for developers looking to implement similar functionalities in their projects. Repository: laolarou726/RAG_Demo Primary language: Python GitHub stars: 8 GitHub description: This is a minimal demo project to show the capabilities of a RAG system using `LangChain` and `Milvus`, it contains all the things you required to build a basic RAG system.

### Overview: mac_linuxgpu is a project that runs the unmodified upstream Linux `amdgpu` driver on macOS, specifically for AMD Radeon GPUs over Thunderbolt on Apple Silicon Macs. It leverages the `amdgpu` + `amdkfd` code used on Linux, using a DriverKit extension to host the compiled sources. ### Key Features: - **Driver Bring-Up:** The driver runs the unmodified `amdgpu_pci_probe` on the R9700 GPU, ensuring firmware images are loaded on demand. - **KFD Compute Sessions:** Each client has its own KFD process with its own GPU address space, and HSA queues are MES-scheduled user queues. - **HSA Runtime:** `libhsa-runtime64.dylib` is installed, and HRX/Loom-based software such as LemonSeed Engine finds it without configuration. - **Clean Lifecycle:** Clients can come and go, and sessions close through upstream removal, interrupt drain, and endpoint isolation, keeping the driver reusable. - **Observability:** A read-only observer reads the driver's retained log and session state at any time, providing the same data as `amdgpu_top` on Linux. ### Tech Stack: - **Language:** C ### Why It Matters: mac_linuxgpu provides a way to run the Linux kernel driver on macOS, specifically for AMD GPUs, without the need for reimplementing the driver. This allows GPU software written for Linux/ROCm to run on Apple Silicon Macs, leveraging the clean and reusable lifecycle of the driver.

Overview: A Telegram bot designed to answer questions about the character Brock, fine-tuned with the help of Gemini 1.5 pro. Key features: - Answers questions about Brock - Fine-tuned with Gemini 1.5 pro data - Responds to prompts starting with "/brock" Tech stack: Python, Vertex AI, Aiogram, Telegram Bot SDK Why it matters: This bot provides a convenient way to interact with Brock, leveraging advanced fine-tuning for improved accuracy and relevance. Repository: lgsurith/Brock_v1-Bot Primary language: Python GitHub stars: 8 GitHub description: A Telegram Bot about Brock fine-tuned with the help of Gemini 1.5 pro

### Overview: MOSS YXBOX 是一个将 ComfyUI 官方的 "Create Bounding Boxes" 和 "MOSS-Image Layout Prompt" 节点融合成一个中文节点的工具。它支持一块画布容纳所有框，16 像素点阵 + 吸附，可滚轮缩放、右键平移。每个框一行，支持多选、拖拽排序、参数锁、背景图最多 64 张，以及四种画布尺寸来源。参数名与悬停说明全中文，支持前置提示词、布局 JSON、后置提示词、草稿坐标显示。 ### Key features: - **一块画布装下所有框**：物体框、文字框、角色框画在同一块画布上，16 像素点阵 + 吸附，可滚轮缩放、右键平移。 - **逐行列表**：每个框一行，直接改类型、文字、描述、宽高、主色、x/y；每个参数带小锁，支持多选、拖拽排序、撤销重做。 - **背景图最多 64 张**：接图自动当画布背景并压暗，逐张可拖、缩、调层、填满画布；「导入原图」还会把 PNG 里记录的工作流布局读回来。 - **四种画布尺寸来源**：自定义 / 按缩放规则 / 按导入原图的原始尺寸

Overview: This project implements model serving in pipelines, providing examples for wine quality analysis, product recommendations, and air traffic delay prediction. It includes reusable libraries for writing Pipelines streamlets and managing models. Key features: - Wine quality analysis using models served within a streamlet process. - Product recommendations using models served as a service. - Predicting air traffic delays using an H2O embedded "MOJO" model. Tech stack: Scala Why it matters: This project demonstrates the integration of model serving capabilities within the Pipelines framework, showcasing how models can be efficiently managed and deployed in a streamlet-based architecture, making it useful for various machine learning use cases.

### Overview: HSML (Hopsworks Machine Learning API) is a library designed to interact with the Hopsworks Model Registry and Model Serving. It simplifies the process of exporting, managing, and deploying models, making it easier to integrate with Hopsworks from external Python environments. ### Key Features: - **Model Registry and Serving Integration**: Connects to the Hopsworks Model Registry and Model Serving APIs. - **Model Management**: Supports model creation, deployment, and deletion. - **Model Export**: Allows exporting models to local directories or files. - **Model Deployment**: Enables starting and managing model deployments. - **Prediction**: Supports making predictions with deployed models. ### Tech Stack: Python ### Why It Matters: HSML provides a straightforward and efficient way to integrate machine learning models into the Hopsworks ecosystem, making it easier for developers to manage and deploy models within the Hopsworks environment.

### Overview: OpenJev is an open-source decision engine that leverages type-safe answers and raw softmax probabilities to provide precise and reliable decisions without the need for external APIs. It allows you to run any local language model, including Qwen, on your machine, eliminating the cost and privacy concerns associated with cloud-based solutions. ### Key Features: - **Type-Safe Answers**: Returns answers with calibrated probabilities, ensuring no free-text generation or hallucination. - **Raw Softmax Probabilities**: Provides raw softmax values for masked logits, offering accurate and model-native confidence. - **Local Execution**: Runs on any local language model, eliminating the need for external APIs and maintaining privacy. ### Tech Stack: Python ### Why It Matters: - **Cost-Free**: Eliminates the cost of external APIs and data leakage. - **Privacy**: All decisions are executed locally, ensuring no data leaves your machine. - **Accuracy**: Provides accurate and model-native confidence through raw softmax values. - **Versatility**: Works with various local language models, including Qwen, Llama, and others. ### Repository: - **lookski/openjev** - **Primary Language**: Python - **GitHub Stars**: 7 - **GitHub Description**: Open decision engine: type-safe answers with raw softmax probabilities from a local LLM or any OpenAI-compatible API. Crush radar for WeChat/QQ/Telegram chats - verdict + next-move advice. ### Quick Start: ```bash $ python demo.py ``` ### Why It Works: - **Decoding Eliminated**: Decoding