Projects

### Overview: EmbedCatalog is a reproducible catalog designed to help developers choose the optimal embedding layout for vector retrieval systems under quality, memory, and latency constraints. It evaluates candidate layouts, measures retrieval quality, storage, and latency, and provides a statistically annotated Pareto catalog for operators to scan against their deployment budget. ### Key Features: - **Quality Evaluation**: Measures retrieval quality, storage, and latency under one protocol. - **Empirical Decision Making**: Answers the physical layout decision by empirically evaluating post-hoc vector layouts. - **Pareto Catalog**: Returns a statistically annotated Pareto catalog that operators can use to scan against their deployment budget. - **Small Advisor**: Answers the deployment question directly by providing a layout or an infeasibility certificate. ### Tech Stack: Python ### Why It Matters: Modern retrieval systems often store one dense vector per object. At production scale, the embedding column can become one of the largest materialized structures in the system. EmbedCatalog answers this decision empirically, providing a conservative first storage reduction for any vector collection, a strong default when memory dominates and exact float scoring is not required, and the best aggressive layout depending on the backbone, dataset, and budget.

### Overview: Seedance 1.5 Pro Prompts: 8 Tested Recipes with Outputs ### Key Features: - **Model ID:** `seedance-1.5-pro` - **Cost per Image:** $0.0104 per second - **Effective Price:** $0.0416 for the entire set - **Aspect Ratios:** 85mm, 24mm, 1:1, 3:4, etc. - **Duration:** 5 seconds each - **Prompt Structure:** 4 recipes per row, including subject, light, framing, setting, and finish ### Tech Stack: Python ### Why It Matters: Seedance 1.5 Pro offers a cost-effective way to generate high-quality images with detailed prompts, making it ideal for designers, marketers, and anyone looking to enhance their visual content without breaking the bank.

### Overview: Seedream 5.0 Lite prompts offers tested image prompt recipes with example renders, aspect ratios, the model id (`seedream-5-0-lite`), and the cost per image. This repository provides a concise overview of eight Seedream 5.0 Lite prompts, each accompanied by an example render, ratio, and estimated cost. ### Key Features: - **Model ID:** `seedream-5-0-lite` - **Aspect Ratios:** 1:1, 3:4, 4:3, 5:7, 8:5, 16:9, 2:3, 3:2 - **Cost per Image:** $0.028 ### Tech Stack: - Python ### Why It Matters: Seedream 5.0 Lite prompts offer a cost-effective, OpenAI-compatible endpoint for image generation, making it ideal for developers looking to integrate Seedream 5.0 Lite into their projects without the need for a monthly subscription. The concise and practical nature of these prompts ensures efficient and effective use of the model, reducing the need for complex integration and minimizing the risk of wasted calls.

Overview: GPT Image 1.5 API is an OpenAI-compatible endpoint providing access to the GPT Image 1.5 model, including pricing details, parameters, and examples for API integration. Key features: - Model id: `gpt-image-1.5-official` - Pricing: Token-based, with rates for `text_input`, `cached_text_input`, `image_input`, and `cached_image_input` - Parameters: `model`, `prompt`, `size`, `resolution`, `n`, `image_urls`, `nsfw_check` - Example: `curl` command for API integration Tech stack: Python Why it matters: This API offers a cost-effective way to integrate GPT Image 1.5 into applications, leveraging OpenAI's model with minimal setup required. Repository: jw6nt/gpt-image-1-5-api-ru GitHub stars: 15

Overview: This repository contains 8 tested recipes for Nano Banana 2 Lite prompts, including example renders, aspect ratios, and the cost per image. Key features: - 8 Nano Banana 2 Lite prompts with example renders, aspect ratios, and cost per image. - Model: `gemini-3.1-flash-lite-image-ext`. - Effective price: $0.0125 per image, estimated cost of the whole set: $0.1. Tech stack: Python. Why it matters: This set of Nano Banana 2 Lite prompts provides tested image prompt recipes, helping developers create effective and cost-efficient image generation tasks.

Overview: LLM Labs is a community-driven collection of open demos, tutorials, and deployment examples for building and running LLM-powered applications. Key features: - Simple agents using open-source models - Streamlit apps for quick prototyping - Docker and Kubernetes deployment examples - Guides and how-tos for local and cloud inferencing Tech stack: Python Why it matters: This repository serves as a valuable resource for developers looking to build and deploy LLM-powered applications, providing both tutorials and practical examples.

Overview: GDB is a real-world benchmark designed to evaluate AI models on professional graphic design tasks, covering layout reasoning, typography, SVG editing, and more. It includes 40 benchmark pipelines and 46 evaluation tasks, based on the Lica dataset. Key features: - 40 benchmark pipelines covering 46 evaluation tasks - 49 evaluation tasks defined in the paper - 40 benchmarks and 46 paper tasks not 1:1, with multiple tasks per benchmark - 2 categories, 8 layout tasks, 5 Lottie tasks, 8 SVG tasks, 5 template tasks, 9 typography tasks, and 6 temporal tasks - 4 layout-understanding tasks not included in the repo - Benchmarks and paper tasks not 1:1, with two benchmarks scoring multiple tasks per model call Tech stack: Python Why it matters: GDB provides a comprehensive and realistic benchmark for evaluating AI models on graphic design tasks, offering a valuable resource for developers and researchers in the field. Repository: lica-world/GDB GitHub stars: 15

### Overview: ComfyUI 自定义节点：摄影人物提示词随机生成器 ### Key Features: - **一键出题**：生成完整中文提示词，每次 Queue 自动换新。 - **模板装配**：63个随机池/949条中文条目按固定10段模板装配。 - **角色联动**：内置2165位角色，按角色特征组织服装、场景、气质、体型。 - **内容强度**：五档控制，可锁定、编辑、重摇、复用参数。 - **三类权重**：轴级、NSFW、外观细节权重，可调整。 - **角色外观锁定**：同一角色的发色、发型、瞳色保持一致。 - **历史与收藏**：每条提示词可复制、放大查看、复用参数。 - **自检报告**：每次出题都给出等级、联动、权重、冲突纠错、字数报告。 ### Tech Stack: Python + ComfyUI ### Why it Matters: - **纯本地、零依赖**：仅用 Python 标准库 + ComfyUI 本体。 - **不使用大模型**：无需任何模型、GPU、网络。 - **模板填空式**：把63个随机池/949条中文条目按固定10段模板装配成提示词。 - **角色联动**：内置2165位角色，按角色特征组织服装、场景、气质、体型。

Overview: This tool converts text into vector embeddings and stores them in the Qdrant search engine, leveraging Rust for efficient handling of large datasets and fast search results. Key features: - Converts text to vector embeddings using popular AI libraries. - Stores vector embeddings in Qdrant for fast search. - Provides a REST API for querying the search engine. Tech stack: Rust Why it matters: This tool is designed to efficiently handle large datasets and deliver lightning-fast search results, making it suitable for applications requiring high performance and scalability.

### Overview: This project is a practical GenAI-Eshop application utilizing Semantic Kernel, multi-agent orchestrations, Mcp tools, A2A Agents, Semantic Search, and more. It showcases the integration of various AI and developer tools to enhance the functionality and user experience of an e-commerce platform. ### Key Features: - **Semantic Kernel Integration:** Utilizes the Semantic Kernel for multi-agent orchestrations and AI-related services. - **Qdrant Semantic Kernel Connector:** Stores vector data for semantic meaning search and hybrid search using vector data and Semantic Kernel. - **Semantic Kernel EmbeddingGenerator:** Generates vector data for semantic search using providers like Ollama, AzureOpenAI, and OpenAI. - **Semantic Kernel ChatCompletion:** Communicates with different models for generating responses using providers like Ollama, AzureOpenAI, and OpenAI. - **Mcp Tools:** Utilizes Mcp tools based on http and json-rpc for calling endpoints in our Mcp server and third-party tools by LLMs for fine-grained functions. - **Multi-Agent Orchestrations:** Enables communication between local and external agents using parent-child agent relationships and different Semantic Kernel’s Agent Orchestration Patterns. - **Agent2Agent Protocol (A2A):** Calls and uses external agents using the A2A protocol based on http and json-rpc. - **Vertical Slice Architecture:** Provides a high-level architecture. - **Minimal APIs:** Simplifies handling requests. - **OpenTelemetry:** Collects logs, metrics, and distributed traces. - **.NET Aspire:** Orchestrates

Overview: QLoRA-LLM is a simple custom QLoRA implementation designed for fine-tuning a language model (LLM) with basic tools like PyTorch and Bitsandbytes, decoupled from Hugging Face. Key features: - Custom QLoRA fine-tuning - Use of basic tools (PyTorch, Bitsandbytes) - Decoupling from Hugging Face transformers and PEFT - Implementation of custom dropout layers instead of standard LoRA layer - Adaptability for other LLM models Tech stack: Python Why it matters: This project provides an opportunity to gain a deeper understanding of QLoRA fine-tuning without relying on Hugging Face's tools, offering greater control and customization options.

Overview: This project showcases the capabilities of NVIDIA NIM, a suite of optimized cloud-native microservices, by demonstrating how to set up and run a Retrieval-Augmented Generation (RAG) pipeline. Key features: - NVIDIA NIM (NVIDIA Inference Model) - Retrieval-Augmented Generation (RAG) pipeline Tech stack: Python Why it matters: This project highlights the ease and efficiency of deploying advanced AI technologies, making them accessible to a wider audience, including developers, through streamlined APIs and optimized microservices.

Overview: This project is an implementation of the Quantization Meets Projection algorithm for approximate k-Nearest Neighbor search, achieving up to a 3x efficiency speed-up with only 1/3 the length of quantized code while maintaining the same accuracy. Key features: - High-Ratio Vector Quantization (HNSW) - Efficient Index Time and Space - Significant Performance Improvement over State-of-the-Art Methods Tech stack: C++ Why it matters: This algorithm leverages data distribution to achieve better distance correction and higher vector compression ratio, significantly outperforming existing AKNN search methods in terms of speed and efficiency.

Overview: The Renyi Kernel Entropy (RKE) metric is introduced to evaluate the diversity of generated samples in generative models, providing a novel approach to assessing the quality and diversity of models' output. Key features: - R\'enyi Kernel Entropy (RKE) metric - Evaluation of generative models on standard image datasets - Provides a ranking of widely-used generative models based on the RKE score of their generated samples Tech stack: Python Why it matters: The RKE metric offers a distinct and powerful method for evaluating the diversity of generated samples in generative models, offering insights that are not easily obtainable with traditional metrics.

Overview: WiSP is a routing-aware expert paging system designed for memory-constrained Mixed-Experts (MoE) models, specifically for serving large models like Qwen3-30B-A3B on GPUs with limited VRAM. It optimizes resource usage by treating resident experts as a cache, allowing the model to run efficiently on a GPU with a fixed VRAM budget. Key features: - WiSP pages experts like vLLM pages KV — and then decides, on one fixed GPU budget, how many bytes each side gets. - It serves models that do not otherwise fit on the GPU at up to 2.0x the decode throughput of vanilla vLLM's static `--cpu-offload-gb` at the same VRAM budget. - The system is drop-in compatible with vLLM, allowing seamless integration into existing pipelines. Tech stack: WiSP is built on Python, leveraging the vLLM framework and integrating with the MoE model. Why it matters: WiSP optimizes resource usage and improves performance by efficiently managing memory and CPU resources, making it particularly useful in scenarios where GPU memory is limited. This system allows for the effective use of GPU resources even when the model size exceeds the available VRAM, thereby enhancing the overall performance and efficiency of the MoE model on constrained hardware.

### Overview: LlamaToaster is a tool designed to help developers and enthusiasts run and benchmark models on their own hardware, without the need for expensive data centers. It provides a simple and intuitive interface to test the performance of various models and configurations, ensuring optimal results for the target platform. ### Key Features: - **Self-Hosted Benchmarking:** Run the benchmark on your own machine or connect a machine to the hosted instance at llamatoaster.com. - **Model Compatibility:** Checks whether a model will run on your machine based on available memory. - **Performance Metrics:** Measures the speed of prefill and generation, providing detailed insights into the performance. - **Configuration Tuning:** Offers a map of context against layers, flash attention on and off, and various K/V cache types to optimize performance. - **Comparison and Tuning:** Compares your chosen configurations with the defaults and shows what moved the number, helping you fine-tune for your specific use case. - **Hosted Instance:** Provides a hosted instance at llamatoaster.com for easy access and testing. ### Tech Stack: TypeScript ### Why It Matters: LlamaToaster simplifies the process of running and benchmarking models, making it accessible to developers without requiring specialized knowledge or expensive infrastructure. It ensures that the best performance is achieved on the target hardware, providing a reliable and straightforward solution for developers to test and optimize their models.

Overview: llamaMan is a browser-based UI and API front-end designed for managing multiple llama.cpp server instances, including an Ollama-compatible proxy, making it easy to run and monitor multiple instances without modifying the llama.cpp code itself. Key features: - Universal GPU image: Supports NVIDIA, AMD (ROCm), Intel Arc, and CPU. - Model library + downloader: HuggingFace pulls models with speed limits, resume, and auto-retry. - One-click launch and presets: Every llama.cpp knob is a first-class UI field, with live preset updates. - Ollama + OpenAI proxy on `:42069`: Open WebUI drops in; models auto-start on demand. - Image & PDF input: Vision models with `--mmproj`; PDFs rasterized page-by-page. - Instance monitoring: Per-GPU VRAM, container CPU/RAM, and throughput/latency. - Auth: User accounts, API keys (bearer tokens), and per-endpoint auth toggle. - Multi-node clustering (opt-in): Heterogeneous nodes as one logical cluster; aggregated dashboard. - Strata engine (opt-in, NVIDIA): Launch Strata instances alongside llama.cpp ones. - Knowledge base + MCP server (opt-in, MariaDB backend): Store documents as topics, search them with custom embedding model, and expose the whole thing to any MCP client. Tech stack: Python. Why it matters: llamaMan simplifies the management of llama.cpp server instances, making it easier to run and monitor multiple instances without modifying the underlying code. The Ollama

### Overview: CC-Vision is a project that adds the ability for non-multimodal Claude Code main models to "see" pasted screenshots. This is achieved by implementing a `UserPromptSubmit` hook that automatically calls a multimodal model to parse the image into text, which is then injected into the main model's context. This way, the main model can still be cheap and fast, as it only borrows eyes, while the user sees a textual description of the image. ### Key features: - **Non-invasive**: The main model's configuration remains untouched, just adding a hook. - **Any OpenAI-compatible multimodal API**: Supports models like OpenAI, DashScope, Zhipu, SiliconFlow, and Moonshot. - **Provider presets**: `VISION_PROVIDER` fills in base_url + default model, with `VISION_MODEL` optional and can be overridden. - **Idempotent**: The same image is never parsed twice, tracked in `.vision-processed`. - **Never blocks**: Any crash returns `{}`, allowing the conversation to continue. - **Standalone test mode**: `--test-image` verifies API config without going through Claude Code. ### Tech stack: - Python ### Why it matters: This project provides a solution for non-multimodal Claude Code models to "see" pasted screenshots, making them more versatile and useful for various applications.

Overview: ToriiDB is a Go embedded key-value database that supports Redis-style commands, a shared socket daemon, and vector search. Key features: - One daemon per directory - Crash-safe snapshot persistence - JSON field mutation - Built-in vector search - Redis-style command router Tech stack: Go Why it matters: ToriiDB simplifies key-value storage with a shared socket daemon and vector search capabilities, making it suitable for applications requiring efficient and scalable data management.

Overview: Pinecone BYOC is a project that allows developers to deploy Pinecone in their own cloud account, providing full control over the infrastructure. Key features: - Deploy Pinecone in your own cloud account (AWS, GCP, or Azure) with full control over your infrastructure. - Quick start with interactive setup and Pulumi project generation. - Provisioning takes approximately 25-30 minutes. Tech stack: Python Why it matters: Pinecone BYOC simplifies the process of deploying Pinecone in your own cloud account, offering full control and ease of setup.