Projects

### Overview: llmBench is a high-depth benchmarking tool designed to measure the raw performance of local LLM runtimes (Ollama, llama.cpp) while providing deep hardware intelligence. Unlike standard benchmarks, llmBench maps your local performance against the **LMSYS Chatbot Arena** global leaderboard using parameter-aware estimation. ### Key Features: - **Deep Hardware Intelligence**: Performs a technical deep dive into CPU architecture, memory subsystem, GPU analytics, and storage/display details. - **Mode 1: Hardware Forensic Analysis**: Identifies bottlenecks and provides optimized deployment recommendations. - **Mode 2: LLM Performance & Global Index**: Stresses the inference runner and maps results to the global ecosystem. ### Tech Stack: Python ### Why it Matters: Beyond simple CPU/GPU names, llmBench provides a technical deep dive into the hardware stack, offering insights into cache hierarchies, memory subsystems, GPUs, and more. This tool helps identify bottlenecks and optimize deployment recommendations, making it a valuable asset for developers looking to improve their local LLM performance.

### Overview: Conditional diffusion models with spatial attention and latent embedding (cDAL) have been developed for medical image segmentation. This model enhances accuracy by incorporating spatial attention and a latent embedding mechanism, making it suitable for high-quality segmentation tasks. ### Key features: - **Spatial Attention Map**: Computes a spatial attention map based on features learned by the discriminator to improve segmentation accuracy. - **Latent Embedding**: Incorporates a random latent embedding to reduce the number of training and sampling steps, making the model faster. - **Conditional Diffusion Process**: Utilizes a convolutional neural network (CNN) based discriminator at each time-step to distinguish between generated and real labels. ### Tech stack: Python ### Why it matters: cDAL offers improved segmentation accuracy by leveraging spatial attention and latent embedding, making it particularly effective for medical image segmentation tasks. Its conditional diffusion process ensures better quality and efficiency in image generation, making it a valuable tool in medical imaging applications.

### Overview: JoakimCarlsson/ai is a Go toolkit designed for building AI agents and applications across multiple providers. It includes a unified LLM client, agent framework with handoffs, tool calling, streaming, structured output, MCP integration, and cost tracking. The library is organized into 50 independent Go modules, allowing you to import only the modules you need, reducing transitive bloat. ### Key Features: - **Per-vendor modules**: Pull only the SDKs you need, no transitive bloat. - **LLM**: Chat, streaming, tool calling, structured output, reasoning. - **Agent framework**: Sub-agents, handoffs, fan-out, sessions, persistent memory, context strategies. - **Voice agent**: Low-latency streaming STT → LLM → TTS pipeline with barge-in, filler audio, tool-call sounds, sessions, hooks, handoffs, toolsets, and memory. - **Embeddings**: Text, multimodal, and contextualized. - **Image generation**: OpenAI, Gemini, xAI. - **Audio**: TTS (ElevenLabs, OpenAI, Google Cloud, Azure Speech) and STT (OpenAI Whisper, ElevenLabs Scribe, Deepgram, AssemblyAI, Google Cloud). - **Rerankers**: Voyage AI, Cohere. - **Decision models**: TypeSafe Jev (System One): typed choice, score and yes/no answers without text generation. - **Fill-in-the-middle**: Mistral, DeepSeek. - **Batch processing**: Native batch APIs

Overview: OLOL is a Python package that provides a unified API endpoint for transparently distributing inference requests across multiple Ollama instances, maintaining compatibility with the Ollama API and adding clustering capabilities. Key features: - Transparent API Compatibility - Automatic Load Balancing - Model Awareness - Session Affinity - Redundancy - Monitoring - Distributed Inference - Layer Partitioning - Auto-Detection - Hardware Optimization - Transparent API Tech stack: Python Why it matters: OLOL simplifies the process of building a powerful multi-host cluster for Ollama AI models, making it easier to scale and maintain a reliable inference system.

Overview: This repository provides a PyTorch implementation of DeepSpeech and DeepSpeech2, serving as a baseline for other speech recognition models. Key features: - PyTorch-based DeepSpeech and DeepSpeech2 models - Pre-trained weights for both networks and performance statistics - Mixed-precision training roadmap Tech stack: PyTorch Why it matters: This implementation offers a stable and performant baseline for speech recognition tasks, facilitating easy comparison with other models and aiding in the development of advanced speech recognition systems.

### Overview: NeuroBrix is an open-source inference engine designed for developers who need to run multiple AI models on their own GPUs or CPUs without the need for a separate pipeline for each model family. It supports models for language, vision, speech, image, and video generation, and runs them from a single container format. NeuroBrix is compatible with NVIDIA GPUs and is in progress for Apple Silicon. ### Key features: - **One Engine and One Container Format:** Models are packaged as `.nbx` containers that contain the computation graph, weights, and runtime defaults. The engine reads all of this from the container, eliminating the need for hand-written pipelines. - **Two Independent Compute Branches:** The default branch runs the model through PyTorch, while the Triton branch runs through NeuroBrix's own Triton kernels. Both branches share the same container and placement plan, allowing for easy verification of the engine's performance. - **Placement is Computed, Not Configured:** NeuroBrix automatically detects the machine's devices and memory, and plans where each component runs. `neurobrix run --explain-plan` provides the chosen plan and alternatives that were rejected, without loading the model. - **Requests that Do Not Fit Are Refused:** When a request cannot fit on the hardware, NeuroBrix stops with an error message that names what was needed and what was available, instead of silently degrading the output. ### Tech Stack: - Python 3.10 or newer - PyTorch 2.1 or newer ### Why it Matters:

Overview: Seedream 5.0 Pro API is an OpenAI-compatible endpoint offering image generation with layered pricing, precise text rendering, and multi-reference editing. It supports 1K and 2K image resolutions and includes a model id, pricing details, and request parameters. Key features: - Layered pricing for 1K and 2K images - Precise text rendering and multi-reference editing - OpenAI-compatible endpoint with a base URL and `Authorization` header - Supports 300+ models, including image, video, and language - $1 minimum charge per task, no pre-paid subscription - Real-time pricing and cost reporting Tech stack: Python Why it matters: Seedream 5.0 Pro API simplifies access to high-quality image generation and text rendering, making it ideal for developers looking to integrate advanced AI capabilities into their projects without the complexities of managing multiple APIs or pre-paid subscriptions. Repository: a2hnks5fo-t/seedream-5-0-pro-api-cn Primary language: Python GitHub stars: 46

### Overview: GPT Image 2.5 Sunburst API is a powerful tool for generating and editing images using the GPT-2.5 model, optimized for OpenAI compatibility. This API allows developers to create stunning visuals without the need for expensive subscriptions, making it an excellent choice for anyone looking to enhance their projects with high-quality images. ### Key Features: - **Model Compatibility:** Compatible with GPT-2.5, offering a wide range of image generation options. - **Pricing:** Starts at $0.010625 per image, with a flexible pricing model that scales with usage. - **Ease of Use:** Assynchronous API, allowing developers to quickly start using the API within minutes. - **Performance:** Supports up to 16 references and PNG, JPEG, or WebP formats, providing excellent performance. ### Tech Stack: - Python ### Why It Matters: - **Cost Efficiency:** Offers a cost-effective alternative to traditional image generation tools, reducing the need for expensive subscriptions. - **Ease of Use:** Simplifies the process of generating high-quality images, making it accessible to developers of all skill levels. - **Scalability:** Supports flexible pricing models, allowing users to adjust the cost based on their usage needs. ### Repository: - **Repository:** a85c7zq/gpt-image-2.5-sunburst-api-pt - **GitHub Stars:** 30 - **GitHub Description:** ChatGPT Imagens — GPT Image 2.5 Sunburst API (`gpt-image-2.

### Overview: Veo 3.1 Fast Prompts: 8 Tested Recipes with Outputs <!-- conv-kit:v1 --> <p align="center"> <img src="assets/badges/price.svg" alt="observed unit price"> <img src="assets/badges/billing.svg" alt="billing model"> <img src="assets/badges/compat.svg" alt="OpenAI-compatible endpoint"> </p> <p align="center"> <img src="assets/01-product-ad-ceramic-mug-thumb.jpg" width="820" alt="Sample video frame generated through APIMart"> </p> > **from $0.064 per second** at 720P (1080P $0.08 · 4K $0.2) — one OpenAI-compatible endpoint at `https://api.apimart.ai/v1`, no monthly plan required. *(observed 2026-10-12)* **[Get an API key](https://go.apimart.ai/k-0e282c)** · **[Live pricing](https://go.apimart.ai/k-81ebb8)** · **[Model page](https://go.apimart.ai/k-656ff3)** · [⚡ 60-second quickstart](#quickstart) **Why teams call Veo 3.1 Fast (`veo3.1-fast`) through APIMart** - **One key, entire catalog

### Overview: SkyReels V4 Std Prompts: 8 Tested Recipes With Outputs SkyReels V4 Std Prompts offers tested image prompt recipes with example renders, aspect ratios, the model ID (`skyreels-v4-std`), and the cost per image. This repository provides a concise guide to using the APIMart API for generating high-quality image prompts at a competitive price. ### Key Features: - **Model ID:** `skyreels-v4-std` - **Prompt Structure:** - **Subject:** Concrete Object or Person - **Light:** Direction + Quality - **Framing:** Ratio + Lens Language - **Setting:** Surface, Backdrop, Environment - **Finish:** Render or Film Character - **Example Prompts:** - **Product Ad:** "Frosted glass perfume bottle" rendered in 5 seconds, estimated cost: $0.088 per second. - **Cinematic:** "Rain soaked street at night" with a lone figure, estimated cost: $0.088 per second. - **Food Ad:** "Steamed food on a wooden plate" rendered in 5 seconds, estimated cost: $0.088 per second. ### Tech Stack: - Python ### Why It Matters: - **$0.088 per second** on the APIMart route, unifying OpenAI-compatible endpoint. - **No subscription or pre-paid plans**—recycles $1 and uses it on calls. -

Overview: Wan 2.7 R2V API (`wan2.7-r2v`) provides access to a high-performance AI video model through the APIMart endpoint, enabling quick and efficient control over style and characters with a single API key. Key features: - Supports multiple models including text-to-video and image-to-video - Offers a flexible pricing model with per-second billing based on resolution - Allows up to five image and video references for enhanced control - Provides detailed model information on the model page - Supports non-blocking polling for task status updates Tech stack: Python Why it matters: This API is crucial for developers looking to integrate advanced AI video capabilities into their projects, offering superior visual quality and performance at a competitive cost. Repository: ap6rg/wan-2-7-r2v-api-ja Primary language: Python GitHub stars: 32 GitHub description: The latest generation AI video model, offering superior visual quality, high motion coherence, and professional-grade performance. Use wan2.7-r2v with a prompt and reference images or videos to control style and characters; up to five image and video references can be combined. README excerpt: # Wan 2.7 R2V API (`wan2.7-r2v`) <!-- conv-kit:v1 --> <p align="center"> <img src="assets/badges/price.svg" alt="observed unit price"> <img src="assets/badges/billing.svg" alt="billing model"> <img src="assets/badges/compat.svg

Overview: FLUX 2 Max API is an OpenAI-compatible endpoint that provides high-resolution image generation at a cost of $0.02 per megapixel, making it an excellent choice for those looking to optimize their image processing workflows without the need for a monthly subscription. Key features: - Supports multiple output resolutions, including 1K, 2K, and 3K. - Charges per megapixel, with a minimum cost of $0.02 per image. - Offers a single key for access to over 300+ models. - Supports both prompt-based and image reference inputs. - Provides detailed pricing information and a live pricing page. Tech stack: Python. Why it matters: FLUX 2 Max API is designed to be highly efficient and cost-effective, making it ideal for developers and businesses looking to streamline their image processing tasks without the complexities of managing a subscription-based system.

### Overview: Switch Diffusion Transformer: Synergizing Denoising Tasks with Sparse Mixture-of-Experts This repository contains the official pytorch implementation of the paper "Switch Diffusion Transformer: Synergizing Denoising Tasks with Sparse Mixture-of-Experts". It includes codes for the improved version of DiT and DTR with the sparse mixture-of-experts. ### Key Features: - Improved version of DiT and DTR with sparse mixture-of-experts - Pytorch implementation for efficient model training and inference - Supports training on ImageNet dataset - Provides example training and sampling scripts for easy reproduction of results ### Tech Stack: - Python - PyTorch ### Why it Matters: - Enhances performance on ImageNet dataset by improving the synergy between denoising tasks - Demonstrates effective use of sparse mixture-of-experts in diffusion models - Provides a practical tool for researchers and practitioners in the field of diffusion models ### Repository: - GitHub: [byeongjun-park/Switch-DiT](https://github.com/byeongjun-park/Switch-DiT) - Stars: 49 - Description: Official pytorch implementation of the paper "Switch Diffusion Transformer: Synergizing Denoising Tasks with Sparse Mixture-of-Experts" ### Updates: - Initial Release: 2024.03.14 ### Setup: - Requires 8 80GB A100 GPUs - Use `pip install -r requirements.txt` to set up environment - Example training script

Overview: FunASR.NET 是一个基于 .NET 10 和纯 C# 实现的普通话 ASR/TTS 和电话音频处理工具，支持 CPU SIMD 和 Vulkan GPU 混合后端。它基于固定 F16 编码器、Q8_0 解码器和两种中文 VITS 模型，提供浏览器和 SIP 音频示例，以及详细的中文文档和真实原版测速对照。 Key features: - ASR (Automatic Speech Recognition) 实现 - 实时转写功能 - 中文 TTS (Text-to-Speech) 功能 - 浏览器和电话音频处理示例 - 强大的 CPU SIMD 和 Vulkan GPU 混合后端支持 - 详细的中文文档和真实原版测速对照 Tech stack: C# Why it matters: FunASR.NET 提供了一个高性能、轻量级的 ASR/TTS 工具，适用于需要在浏览器和 SIP 系统中处理语音的开发者。它通过纯 C# 实现，减少了对 Python、ONNX Runtime 等依赖，提高了开发效率和稳定性。 Repository: hnlyf168/FunAsrNet GitHub stars: 49

### Overview: ComfyUI自定义节点：ANIMA3提示词随机生成器。它是一款专注于生成成人向NSFW内容的插件，包含五级NSFW档位闸门、三层权重及角色库2383位/场景搭配261条，纯本地零依赖，仅生成提示词，不生成图片。 ### Key features: - **五级NSFW档位闸门**：控制提示词的档次，低档绝不越界。 - **候选池先按档位收敛，抽完再体检、超档回滚**：确保低档内容不漏。 - **三层权重**：段、字段、选项权重，0=关闭，3.0=必现。 - **纯本地零依赖**：无需第三方依赖，仅需Python和原生前端JS。 ### Tech stack: 纯Python + 原生前端JS。 ### Why it matters: - **合规声明**：详细说明插件包含成人向内容，用户需达到法定年龄并知悉内容。 - **用途负责**：生成内容的用途由使用者自行负责，不得用于制作涉及未成年人的性化内容。 - **全面覆盖**：包含角色库2383位、场景搭配261条、词池、档位表、发型、角色固定特征等丰富资源。 ### GitHub metadata: - **Repository**: lingdongbudong/Comfyui-Anima-Wardrobe-V4 - **Primary language**: Python - **GitHub stars**:

Overview: This repository implements the Denoising Diffusion Probabilistic Model (DDPM) using the MindSpore framework. The implementation is based on the work by lucidrains and is designed for training and inference using MindSpore. Key features: - Training and inference using MindSpore - Support for both L1 and L2 loss types - Support for various model architectures and loss functions Tech stack: MindSpore Why it matters: This implementation is useful for researchers and developers who are working on image processing and generative models using MindSpore, a popular framework for deep learning in China. Repository: lvyufeng/denoising-diffusion-mindspore GitHub stars: 49

Overview: Voice DevTools is a developer toolset designed for debugging and building real-time voice agents, supporting multiple models. Key features: - Real-time debugging capabilities - Model support for various models - Easy integration with API keys for authentication Tech stack: TypeScript Why it matters: This toolset simplifies the development process for voice agents, enhancing efficiency and accuracy. Repository: outspeed-ai/voice-devtools Primary language: TypeScript GitHub stars: 49

### Overview: ExtremeRouter is a self-hosted AI gateway that routes traffic from your AI coding tools to over 300 providers, including Claude Code, Cursor, Antigravity, Copilot, Codex, Gemini, OpenCode, Cline, OpenClaw, and many more. It features 6 combo strategies, smart routing, format translation, and real-time quota tracking, offering 20-40% token savings and 304+ providers in one unified endpoint. ### Key Features: - **304+ Providers, One Endpoint**: API-key, OAuth, free-tier, and 39 web-cookie providers behind a single `/v1` surface. - **Format Translation**: OpenAI ↔ Claude ↔ Gemini ↔ Responses ↔ Antigravity ↔ Kiro ↔ Cursor. - **6 Combo Strategies**: fallback, round-robin, fusion, swarm, cascade, smart-routing. - **Hierarchical Swarm Engine**: Gatekeeper → Manager → Workers → Audit → Synthesis with live SSE telemetry. - **Smart Routing**: Task-aware routing: tool-calling tasks go to tool-capable models, research goes to cookie/free providers. - **Cascade Strategy**: Cheap → capable escalation with self-reported confidence gating. - **RTK Token Saver**: Compresses `tool_result` content before sending, saving 20-40% input tokens on every request. - **Ponytail & Caveman**: YAGNI-first and terse-response prompt injectors, up to 65% fewer output tokens. - **Smart 3-tier Fallback

Overview: SmartApp Framework is a Python framework that allows developers to create smart apps for the Salut virtual assistants. Key features: - SmartApp Framework - Integration with Salut virtual assistants - Support for various features like calling for a taxi, checking the weather, controlling smart home devices, and booking appointments Tech stack: Python Why it matters: This framework simplifies the process of creating smart apps for Salut virtual assistants, making it easier for developers to create engaging and useful applications for users.

### Overview: VisionGPT-2 is a multimodal image captioning model combining ViT and GPT-2, trained on the MS-COCO dataset. The project aims to create a simple yet effective solution for image captioning, demonstrating the feasibility of using pre-trained models for image understanding. ### Key features: - Combines ViT and GPT-2 for image captioning. - Trained on MS-COCO dataset. - Implements the model from scratch, including ViT layers and GPT-2. - Uses mixed-precision training with PyTorch AMP. - Supports both random and deterministic generation. - Provides detailed dataset preparation and training metrics. ### Tech stack: - PyTorch - HuggingFace ### Why it matters: VisionGPT-2 offers a practical approach to multimodal image captioning, showcasing the potential of combining pre-trained models for efficient and effective image understanding.