Projects

### Overview: Recent advancements in image and video generation have significantly improved visual quality and controllability. However, multi-stage creative tasks still require planning, coordination, and revision. Agentic visual creation plays a crucial role in integrating language or multimodal agents to coordinate generative models, editing tools, and creative software, thereby facilitating real-world creative workflows. This survey provides a structured overview to help researchers quickly understand the field, its representative approaches, and their connections. It categorizes existing work into four main categories based on the agent's role in production: visual generation, visual editing, visual composition, and visual programming. The survey also compares how agents plan, use tools, maintain memory, incorporate feedback, and collaborate. Additionally, it reviews benchmark design, training data, and evaluation methods, focusing on feedback that supports revision. Finally, it discusses current limitations and future directions for more reliable and interactive visual creation systems. ### Key Features: - **Target-Centered Visual Generation**: Unified Workflow Routing for Agentic Image Generation - **Coordinated Visual Generation**: Agentic Ontology-guided Image Generation and Evaluation for Rare-event Data Augmentation in Safety-critical Perception - **Visual Editing**: Tool-Using Unified Multimodal Models for Agentic Image Generation - **Visual Composition**: Cognitive-structured Multimodal Agent for Multimodal Understanding, Generation, and Editing - **Visual Programming**: Workflow-Based Visual Programming and Code-Driven Visual Programming ### Tech Stack: Unknown ### Why It Matters: This survey provides a structured overview to help researchers quickly understand the field, its representative approaches, and

Overview: This project guides developers in fine-tuning the newly released Llama-3.2 lightweight models, providing a detailed guide on how to harness the power of these models using Python. Key features: - Fine-tuning Llama-3.2 models - Using a CSV dataset for training - Detailed instructions for modifying the training script - Generating a masterpiece model Tech stack: Python Why it matters: This guide is essential for developers looking to leverage the capabilities of the Llama-3.2 models, making it easier to fine-tune these models on their own data, leading to improved performance and applications.

Overview: This repository is a heavily annotated, from-scratch CUDA inference runtime developed specifically for Qwen2.5-Coder-7B on NVIDIA H100 (sm_90). The project includes custom INT4 packer, fused GEMV, paged KV, split-KV attention, and CUDA graph decode, all commented for clarity. It aims to provide a learning experience rather than a direct replacement for llama.cpp. Key features: - Custom INT4 weight format - Fused GEMV - Paged KV - Split-KV attention - CUDA graph decode Tech stack: Cuda Why it matters: This repository offers a detailed, annotated approach to decoding, focusing on understanding the decode process rather than just implementing a pre-existing solution. It provides insights into the reasoning behind certain design choices, making it useful for developers looking to understand and improve their own decoding stacks.

### Overview: This project is a comprehensive dataset generation skill designed for agent IDEs and AI coding assistants, offering 14 specialized sub-skills, 19 pipeline entry scripts, and 15 shared utility modules. It enables the creation of SFT and DPO training datasets for Claude Code, Codex CLI, Antigravity IDE, Claude Cowork, Cursor, Windsurf, and GitHub Copilot. ### Key Features: - **14 Specialized Sub-skills**: Tailored for specific use cases in AI development. - **19 Pipeline Entry Scripts**: Including `build_loop.py`, `generate.py`, `collect.py`, `research.py`, `verify.py`, `dedup.py`, `coverage.py`, `audit.py`, `export.py`, `grounding.py`, and more. - **15 Shared Utility Modules**: Enhancing the overall utility and flexibility of the skill. ### Tech Stack: Python ### Why it Matters: This skill is crucial for developers and AI coders, providing a robust and versatile toolset for generating high-quality datasets that are essential for training AI models and improving the efficiency of AI-driven development processes.

Overview: This project is an experiment-driven LLM engineering lab that explores token economics, API costs, caching, context growth, latency, model routing, rate limits, concurrency, and batching using Groq. It includes 10 reproducible Python experiments and is based on real requests to the Groq API. Key features: - Token Economics - API Cost Calculation - Context Growth & Context Bloat - Prompt Caching - Output Control & Invocation Optimization - Model Routing - Streaming & Perceived Latency - Rate Limiting & Retry Strategies - Async Processing & Concurrency - Batch Processing - Conversation Summarization & Context Economics Tech stack: Python Why it matters: This project provides a comprehensive understanding of LLM token economics, API performance, and optimization techniques, making it valuable for developers looking to improve their LLM engineering practices. Repository: Burakkylmz/llm-token-economics-lab GitHub stars: 21 GitHub description: An experiment-driven LLM engineering lab with 9 reproducible Python experiments exploring token economics, API costs, caching, context growth, latency, model routing, rate limits, concurrency, and batching using Groq.

### Overview: In this series, we are demonstrating how to integrate and utilize the leading artificial intelligence (AI) tools available in the market with Delphi, focusing on practical applications in Delphi-based ERP systems. ### Key Features: - **LLMs (Language Learning Models)**: Implementing language learning models for text-based interactions. - **RAG (Retrieval-Augmented Generation)**: Using AI to generate text based on retrieved information. - **MCP (Model Context Protocol)**: Interacting with AI models using context protocols. - **Agents**: Automating tasks using AI agents. - **Function Calling/Tool Calling**: Integrating AI tools into Delphi applications. - **Multimodal**: Combining text, images, and voice for more comprehensive interactions. - **TTS (Text-to-Speech)**: Generating speech from text. - **STT (Speech-to-Text)**: Converting speech to text. - **Toolsets**: Pre-configured tool sets for common tasks. ### Tech Stack: Delphi, AI libraries, and APIs. ### Why it Matters: This series provides a practical guide for developers to integrate AI tools into their Delphi applications, making them more efficient and user-friendly.

Overview: Ensemble Integration: a customizable pipeline for generating multi-modal, heterogeneous ensembles. Key features: - Multi-modal data integration - Late fusion approach - Sklearn-like models compatibility - Built-in nested cross-validation - Feature selection for unstructured data Tech stack: Python Why it matters: This package offers a flexible and powerful method for integrating multi-modal data, making it valuable for predictive modeling tasks that require heterogeneous ensemble approaches.

Overview: inferd is a native Apple Silicon inference engine designed to handle multiple AI agents simultaneously. It leverages C++, Objective-C++, and Metal, and runs on macOS 27 and 26 with Apple Silicon, focusing on long context agents and efficient memory management. Key features: - Native Apple Silicon inference engine - Supports many concurrent AI agents - Handles long contexts with continuous batching and speculative decoding - Uses an 8-bit paged KV cache and a prefix cache for efficient context management - SSD tier for contexts that do not fit in memory - API for the agent framework to decide what to keep in the cache Tech stack: C++, Objective-C++, Metal, Python 3.12 Why it matters: inferd is a cutting-edge solution for AI agents on Apple Silicon, designed to handle large numbers of concurrent agents efficiently. It's particularly useful in scenarios where multiple AI agents need to be processed simultaneously, such as in large-scale AI applications or in environments with high memory constraints. Repository: Incept5-AI/inferd Primary language: Python GitHub stars: 21 GitHub description: Native Apple Silicon inference engine for many AI agents at once: C++/Metal, agent-contexts API, 8-bit paged KV, SSD tier (Qwen3.8)

### Overview: Unraid CA template and Docker build for Mem0 (OpenMemory). This mega-container utilizes s6-overlay to seamlessly bundle the pre-compiled Qdrant Vector Database, the Python FastAPI/MCP server, and the Next.js Dashboard UI into a single, click-and-play local AI memory layer. Built for homelabs, privacy, and true agentic AI memory retention. ### Key features: - OpenMemory web UI on port `3000` - OpenMemory API / MCP server on internal localhost port `8765` - Embedded Qdrant vector store - Persistent appdata storage for SQLite and Qdrant state - Upstream backup/export helper scripts bundled into the image - Same-origin UI routing to the API so a standard Unraid install does not need separate browser-facing API networking - Unraid CA template at [mem0-aio.xml](mem0-aio.xml) ### Tech stack: Docker ### Why it matters: This image provides a user-friendly, self-hosted installation of OpenMemory with Qdrant vector database and a customizable Next.js dashboard, all within a single container. It simplifies the process of setting up AI memory retention for homelabs, ensuring privacy and security. The bundled nature of the Qdrant database and the OpenMemory configuration makes it easy to deploy without requiring manual setup of external dependencies.

Overview: 甲作 Atelier是一款手机端的AI美甲推荐与虚拟试戴产品，旨在降低网图与实物不符的风险，通过真实手照提供值得尝试的款式方向，再用AI帮助用户预览比较。 Key features: - AI推荐美甲款式方向 - 虚拟试戴功能 - 上传或拍摄手照 - 选择路径推荐 Tech stack: TypeScript Why it matters: 通过提供真实手照的AI推荐，甲作Atelier帮助用户在购买前进行预览和比较，减少因网图与实物不符带来的决策风险，提高购买体验和满意度。

### Overview: ModelForge 是一款面向小团队的 NLP 模型一站式服务平台，旨在解决数据集管理、训练、评估、在线部署及 Prompt 评测等痛点问题。 ### Key features: - **数据集管理与版本控制**：统一管理训练集和评估集，提供快照、版本管理、数据模板下载等功能。 - **训练流程**：支持多种任务类型，如分类、命名实体识别、多标签分类等，提供实时进度条和MLflow记录实验与产物。 - **评估流程**：支持评估集版本发起评估，提供横向对比多个模型版本的功能。 - **模型版本管理**：支持模型版本提升生命周期阶段，提供模型注册与查询功能。 - **在线部署**：一键部署模型到模型服务器，提供API接口和部署管理功能。 - **坏例修复**：提供坏例回流、回流训练集、自动评分和修复功能。 - **Prompt 评测**：支持大模型 Prompt 评测，提供盲测人工评估和自动评估功能。 ### Tech stack: ModelForge 使用Python进行开发和部署。 ### Why it matters: ModelForge旨在解决小团队在NLP模型训练与应用过程中遇到的多个痛点，提供一站式解决方案，帮助团队更高效地进行模型训练、评估和部署，同时通过Prompt评测确保模型效果的准确性和可复现性。

Overview: ViewCraft is an evolving framework designed to enhance multimodal agents' ability to perceive, transform, and interpret evidence. It combines observation selection, visual transformations, layout, and reading instructions to create reusable skills that can be applied across various tasks. Key features: - Observation selection - Visual transformations - Layout and reading instructions Tech stack: Python Why it matters: ViewCraft leverages Python 3.11+ and an MIT License, offering a framework that can significantly improve the performance and efficiency of multimodal agents in handling evidence and tasks. Its ability to evolve what agents see and how they read it makes it a valuable tool in various applications, including image analysis, document understanding, and more. Repository: Little-Fridge/ViewCraft GitHub stars: 21 GitHub description: Evolving what multimodal agents see and how they read it. Framework, ten reusable task skills, and paper resources.

Overview: This project implements the research paper "Fine-Tuning is Fine, if Calibrated" presented at NeurIPS 2024. Key features: - Fine-tuning of pre-trained models on various datasets. - Evaluation of checkpoint models using OfficeHome and ImageNet datasets. - Reproduction of evaluation results for academic purposes. Tech stack: Python Why it matters: This implementation provides a practical way to fine-tune pre-trained models on different datasets, ensuring that the calibrated models are effective and reliable. Repository: OSU-MLB/Fine-Tuning-Is-Fine-If-Calibrated Primary language: Python GitHub stars: 21 GitHub description: Official Implementation of "Fine-Tuning is Fine, if Calibrated", NeurIPS 2024

Overview: Solace AI is an empathetic mental health companion designed to provide personalized support through natural conversations. It uses advanced multi-agent architecture to create a judgment-free space where users can freely express themselves and receive compassionate guidance tailored to their unique needs. Key features: - Core Capabilities: Solace AI understands and responds to emotions and personality through natural conversations. - Text-Based Interaction: Users can communicate with Solace AI through text-based messages. - Voice Interaction: Solace AI can also be interacted with via voice, offering a more immersive experience. - Technology Stack: The project is built using Python and FastAPI, leveraging libraries such as Transformers for language understanding and TensorFlow for machine learning. - Why it Matters: Solace AI is a significant advancement in mental health technology, providing a judgment-free space for users to express themselves and receive tailored support. Repository: Rayyan9477/Solace-AI Primary language: Python GitHub stars: 21 GitHub description: Solace AI is an empathetic mental health companion that understands your emotions and personality to provide personalized support through natural conversations. It creates a judgment-free space where you can express yourself freely and receive compassionate guidance tailored just for you. README excerpt: ⚠️ OUTDATED README (flagged 2026-07-21). This file describes a legacy Streamlit/Gemini monolith (app.py, chat.py, launch_api.py, config.yaml) that is NOT the current codebase. The repo is now 10 FastAPI microservices under services/ + shared libs

Overview: Uncertainty-o is a model-agnostic framework designed to uncover uncertainty in large multimodal models. It leverages multimodal prompt perturbation to generate diverse responses, which are then clustered and quantified using semantic entropy to measure the uncertainty. Key features: - Multimodal prompt perturbation - Semantic clustering and entropy calculation - Quantification of epistemic uncertainty Tech stack: Python Why it matters: Uncertainty-o provides a unified approach to evaluating and mitigating uncertainty in various tasks, making it valuable for researchers and practitioners working with large multimodal models. Repository: Ruiyang-061X/Uncertainty-o Primary language: Python GitHub stars: 21 GitHub description: [EMNLP'26] Uncertainty-o: One Model-agnostic Framework for Unveiling Uncertainty in Large Multimodal Models

Overview: QChatAi is a project that enables users to interact with PDF files by converting them into vector embeddings and using them to generate relevant answers through a large language model. Key features: - User can upload a PDF file. - The file is processed to extract and split content into chunks. - Each chunk is converted into a vector embedding. - The embeddings and chunks are stored in a PostgreSQL database. - Users can ask questions, and the system retrieves the most relevant context from the stored PDFs. - The system uses a large language model to generate answers based on the context provided. Tech stack: TypeScript, Node.js, PostgreSQL, BullMQ, Docker, Next.js, NestJS, OpenAI Embeddings, PyEvctor, Pinecone, Faiss, Chromadb. Why it matters: This project offers a novel way to interact with PDFs by leveraging vector embeddings and large language models, potentially revolutionizing how users can access information from documents.

Overview: This project has been significantly improved, offering faster startup times, better search functionality, and zero dependencies. It includes a new search algorithm that supports code understanding and incremental updates. Key features: - Faster startup times - Improved search algorithm - Zero dependencies Tech stack: TypeScript Why it matters: This project offers a 10x improvement in performance and functionality compared to the previous version, making it a better choice for developers looking for a more efficient and accurate code search solution.

### Overview: R-Sparse is a novel training-free activation sparsity approach designed for efficient inference of large language models (LLMs), particularly on edge devices. It achieves high sparsity levels in advanced LLMs by leveraging the sparsity of input channels and singular value components, eliminating the need for active channel prediction. ### Key Features: - R-Sparse achieves comparable performance at 50% model-level sparsity. - It reduces computation and memory movement, making it suitable for small-batch on-device applications. - The approach is applicable to Llama-2/3 and Mistral models across ten diverse tasks. ### Tech Stack: - Python ### Why It Matters: R-Sparse offers a significant improvement in efficiency by reducing the model size and computational load, making it highly effective for inference on edge devices.

Overview: This project provides a Matlab implementation of Fast Multi-Label Canonical Correlation Analysis (ml-CCA), based on the ICCV-2015 work "Multi-Label Cross-Modal Retrieval". Key features: - Fast ml-CCA implementation - Supports multi-label data - Implements advanced correlation techniques Tech stack: Matlab Why it matters: This implementation is useful for applications requiring efficient multi-label cross-modal retrieval, such as image and text-based content recommendation systems.

### Overview: This project focuses on the classification of large-scale multi-modal (text and image) products, utilizing the 2020 SIGIR E-Commerce Workshop challenge dataset. The goal is to predict the product type codes for unseen test data, with the primary objective of true multi-modal learning. The organizers have released approximately 99,000 product listings in tsv format, including 84,916 training samples, 937 phase 1 testing samples, and 8435 phase 2 testing samples. The training dataset consists of product titles, descriptions, images, and their corresponding product type codes, with 27 categories and varying sample counts per category. ### Key Features: - Decision-level fusion scheme for leveraging multi-modal product information. - Modal-specific classifiers built separately from textual and image modal data. - Late fusion strategy learned from class probabilities predicted by each modal classifier. - Utilization of textual titles and images for true multi-modal learning. ### Tech Stack: Jupyter Notebook ### Why it Matters: This project demonstrates effective multi-modal learning and decision-level fusion in large-scale product classification, showcasing the potential of combining textual and image data for improved classification accuracy.