Projects

Overview: This project provides a visual representation of how an agent loop operates, offering a more intuitive and visual explanation compared to traditional reading. It showcases the step-by-step process of an agent's loop, from proposing actions to executing them until the loop stops. Key features: - Visual representation of an agent loop - Detailed token-level visualization - Support for multiple models (Gemma 4 E2B, Qwen3.5 0.8B) - Option to run the loop using different harnesses (built-in, pi, Claude/GPT) - MTP speculative decoding for Gemma 4 E2B - Support for both English and Traditional Chinese language Tech stack: Python Why it matters: This project simplifies the understanding of agent loop mechanics, making it easier to grasp through a visual interface rather than reading about it. It aligns with the learning objectives of the agent atlas course, providing practical examples that can be explored and experimented with. Repository: Agentic-System-Lab/agent-loop-visualizer Primary language: Python GitHub stars: 51 GitHub description: This project is for learning: it shows how an agent loop works underneath, in a way more intuitive and visual than reading about it. Behind an agent's chat box runs a loop: step after step, the model proposes an action and the harness carries it out, until it stops. README excerpt: <img src="app/assets/logo.svg" alt="" width="72"> # Agent Loop Visualizer English | [繁體中文](README.zh-TW.md

### Overview: Nano Banana 2.1 API is a developer guide that provides runnable examples for generating and editing images using Google's Nano Banana 2.1 model through MuAPI. This repository offers Python, JavaScript, and curl examples, as well as detailed documentation on the endpoints, task lifecycle, and result polling. ### Key features: - **Text to Image**: Generates an image from a prompt. - **Image Edit**: Combines reference images with a plain-language instruction. - **Image Edit (Advanced)**: A more advanced image editing endpoint. - **API Key Management**: Instructions on how to create and manage API keys. ### Tech stack: Python, JavaScript, and curl examples. ### Why it matters: Nano Banana 2.1 is an improved version of Google's Nano Banana 2 and Nano Banana Pro, offering better visual design, mask-based editing, and subject consistency. This API guide helps developers integrate this model into their projects, ensuring they can leverage the latest advancements in image generation and editing. Repository: Anil-matcha/Nano-Banana-2.1-API GitHub stars: 51 GitHub description: Nano Banana 2.1 API: generate and edit images with Google’s Nano Banana 2.1 through MuAPI. Python and JavaScript examples, curl, request schema.

Overview: E-Library-Agent is a virtual agent designed to help you manage and search through your virtual library of books and papers. It uses the ingest-anything library for data ingestion and is powered by LlamaIndex, Qdrant, and Linkup for advanced search capabilities. Key features: - Ingests all the files you feed to it - Retrieves information from the library - Searches for new books or papers on the internet Tech stack: Python Why it matters: This project showcases the capabilities of ingest-anything, LlamaIndex, Qdrant, and Linkup, providing a comprehensive solution for virtual library management and search. Repository: AstraBert/e-library-agent Primary language: Python GitHub stars: 51

Overview: mediagen is an image and video generation skill designed for coding agents like Gemini, OpenAI, and Kie AI. It allows you to specify the model, aspect ratio, size, output name, and directory, as well as quality preset, video duration, and whether to mark the result. The skill is configured with a single key, and it handles the prompt writing, model selection, and error recovery. Key features: - Generates images and videos from descriptions or edited media - Automatically chooses the appropriate provider based on the task - Uses the correct model for providers like Gemini and OpenAI - Writes prompts properly and suggests marking AI-generated output - Recovers from configuration errors on its own Tech stack: TypeScript Why it matters: mediagen simplifies the process of generating images and videos for coding agents, making it easier to create professional-looking outputs without needing to learn complex APIs.

Overview: This project aims to experiment with the Web Speech API by creating a voice note app that records speech, converts it to text, and then converts the text back into speech. Key features: - Speech recognition - Speech synthesis (text-to-speech) Tech stack: JavaScript Why it matters: This app showcases the capabilities of the Web Speech API, enabling real-time text-to-speech and speech-to-text conversions, which can be useful for creating interactive voice interfaces. Repository: DKMitt/speech-to-text-js GitHub stars: 51 GitHub description: The Voice Note App's purpose is to experiment with the Web Speech API by using the browser interface to record human speech and convert it into text and then convert the text back into speech.

### Overview: This project introduces a novel pipeline for adverse weather restoration, named **T3-DiffWeather**. The method addresses the challenges posed by unpredictable and varied weather degradations in the real world by combining sub-prompts, leveraging a prompt pool, and incorporating Depth-Anything constraints. The approach is designed to adaptively tackle unforeseen weather inputs and provides scene-specific conditions for the diffusion process, ensuring distinctive representations for both types of prompts. Experimental results demonstrate superior performance across synthetic and real-world datasets, outperforming existing diffusion techniques in terms of computational efficiency. ### Key Features: - **Adverse Weather Restoration**: Restores degraded images and videos due to adverse weather conditions. - **Prompt Pool**: Allows the network to autonomously combine sub-prompts to construct weather-prompts. - **Depth-Anything Constraint**: Provides scene-specific conditions for the diffusion process. - **Contrastive Prompt Loss**: Ensures distinctive representations for both types of prompts. ### Tech Stack: - Python ### Why It Matters: Our method addresses the significant challenges in adverse weather restoration by combining sub-prompts, leveraging a prompt pool, and incorporating Depth-Anything constraints. This approach ensures adaptability to unforeseen weather inputs and provides scene-specific conditions, leading to superior performance across various datasets compared to existing techniques.

### Overview: Binary Latent Diffusion is an implementation of the Binary Latent Diffusion model, a type of generative model that uses binary latent variables for image generation. This project aims to provide a practical and accessible way to experiment with and understand the Binary Latent Diffusion model. ### Key Features: - **Model Implementation**: The project includes a complete implementation of the Binary Latent Diffusion model. - **Visualization**: The README provides an example of the architecture and results using a sample input image. - **Training Details**: The project includes training details and results, showing the effectiveness of the model in generating high-quality images. ### Tech Stack: Python ### Why It Matters: Binary Latent Diffusion is a promising model for generating images from binary latent variables, offering a unique approach to image generation. This project provides a practical way to explore and understand the model, making it valuable for researchers and developers interested in generative models and image synthesis.

Overview: This project provides a C# interface for language models, utilizing TorchSharp for most functionality and specialized CUDA ops for certain tasks. The aim is to offer a native C# experience for production-ready applications. Key features: - Python resources interop - Support for converting models and tokenizers - Native support for serving language models - Model parallel capabilities - Specialized CUDA ops for inference Tech stack: C# Why it matters: This project simplifies the integration of language models into C# applications, making it easier to develop and deploy language models in a native C# environment.

### Overview: LayoutLLM-T2I is a text-to-image generation model that addresses the challenges of layout planning and image generation by utilizing Large Language Models (LLMs) to elicit layout guidance from text prompts. This approach aims to generate high-fidelity images that are semantically aligned with the textual input without the need for manual guidance. ### Key features: - **Coarse-to-Fine Layout Planning**: Generates a coarse-grained layout based on a textual prompt using in-context learning from Large Language Models. - **Fine-grained Object-Interaction Diffusion**: Synthesizes high-fidelity images by conditioning on both the prompt and the automatically generated layout. - **Benchmarking**: Evaluates the Numerical, Spatial, and Semantic reasoning capabilities of text-to-image generative models using a new benchmark dataset. ### Tech stack: Python ### Why it matters: Our model outperforms state-of-the-art methods in terms of layout and image generation, making it a significant advancement in the field of text-to-image generation.

Overview: Qinti is an open-source frontend application built with Godot for image generation, designed to be both user-friendly and suitable for professional use. Key features: - Infinite canvas - Save and load progress - Controlnet integration - Inpaint, outpaint, and drawing to AI image - Model resources management - Textural inversion, LoRAs, Lycoris, etc - Undo and Redo - Automatic1111 Web UI and SD.Next backend support - Modular image metadata generation - Inbuilt tutorials - Automatic1111 Web UI installation (beta feature) - Themes Tech stack: GDScript Why it matters: Qinti aims to provide a user-friendly interface for image generation, focusing on versatility and ease of use, making it suitable for both personal and professional applications. Repository: Samachiy/Qinti GitHub stars: 51 GitHub description: An AI image generation frontend focused on ease of use, versatility, and capability for professional uses, packaged as a click-and-run executable. README excerpt:

Overview: This project introduces a novel approach to mitigate mode collapse in score distillation for text-to-3D generation, addressing a significant challenge in the field of 3D mode collapse. The method is based on the paper "Taming Mode Collapse in Score Distillation for Text-to-3D Generation" by Peihao Wang et al., which was presented at CVPR 2024. Key features: The project provides a solution to the mode collapse problem in score distillation, offering a way to alleviate the Janus problem. It includes a script for visualizing the training trajectory of matching two Gaussian distributions using various score distillation schemes, demonstrating the effectiveness of the method. Tech stack: The project utilizes Python, with a focus on the threestudio framework for text-to-3D generation experiments. Why it matters: The method presented in this project is significant because it addresses a critical issue in the field of 3D mode collapse, providing a practical solution to improve the quality and diversity of 3D generation models.

### Overview: This repository contains the official PyTorch implementation for the paper "Diffusion Posterior Sampling for Linear Inverse Problem Solving: A Filtering Perspective". The project introduces an efficient diffusion sampling algorithm that guarantees asymptotic accuracy, providing a specific instance of Bayesian filtering. This method seamlessly integrates with existing Markovian diffusion samplers and requires no re-training of the model, ensuring accurate samples from the Bayesian posterior. ### Key Features: - Efficient diffusion sampling algorithm for linear inverse problems. - Guarantees asymptotic accuracy. - Integrates with existing Markovian diffusion samplers. - No model re-training required. - Seamless integration with all Markovian diffusion samplers. - Guarantees accurate samples from the Bayesian posterior as particle counts increase. ### Tech Stack: - Python 3.8 - PyTorch >= 1.7.0 - CUDA >= 10.2 ### Why It Matters: This method provides a practical and accurate solution for zero-shot diffusion posterior sampling, offering significant improvements over existing zero-shot diffusion posterior samplers. It is particularly useful in applications like image inpainting, super-resolution, and motion deblurring, where accurate posterior sampling is crucial for achieving high-quality results.

### Overview: Seedream 5.0 Pro API (`seedream-5-0-pro`) is a comprehensive OpenAI-compatible API endpoint offering precise image generation and editing at a competitive price. ### Key Features: - **Model ID and Pricing**: Seedream 5.0 Pro (`seedream-5-0-pro`) supports 1K and 2K image models with layered pricing, providing a detailed catalog of models. - **Pricing Model**: The API charges based on the number of images generated, with tiered pricing for different resolutions. - **Model Compatibility**: The endpoint supports multiple image and video models, making it versatile for various tasks. - **Billing Model**: The pricing is based on the number of images generated, with a flat rate of $0.014625 per image, making it cost-effective. - **Model Page**: Detailed information on the model can be found on the model page, including the exact tiers and pricing. - **Model Parameters**: The API supports various parameters such as `model`, `prompt`, `size`, `resolution`, `n`, and `image_urls`. ### Tech Stack: Python ### Why It Matters: Seedream 5.0 Pro API is an essential tool for developers looking to integrate high-quality image generation and editing into their projects without the need for a monthly subscription. The API offers a competitive pricing model and a comprehensive catalog of models, making it a valuable addition to any developer's toolkit.

Overview: DGCCA-pytorch is an implementation of Deep Generalized Canonical Correlation Analysis (DGCCA) using PyTorch, as described in the paper by Adrian Benton et al. at the 4th Workshop on Representation Learning for NLP in 2019. Key features: - DGCCA implementation using PyTorch - Supports multiple view data - Non-linear feature extraction using neural networks - Extends Deep Canonical Correlation Analysis (DeepCCA) for more than two views Tech stack: PyTorch Why it matters: DGCCA provides a powerful tool for analyzing and comparing data with multiple views, making it useful in various applications such as image and text analysis, recommendation systems, and more. Repository: arminarj/DeepGCCA-pytorch Primary language: Jupyter Notebook GitHub stars: 51 GitHub description: An implementation of Deep Generalized Canonical Correlation Analysis (DGCCA or Deep GCCA) with pytorch. README excerpt:

Overview: A live subtitle app that uses VOSK for speech recognition and Google Translate for translation, providing real-time subtitles for any live audio or video stream. Key features: - Real-time subtitles - Supports multiple languages - Uses VOSK for speech recognition - Uses Google Translate for translation Tech stack: Java Why it matters: This app offers a more accurate and extensive translation method compared to the MLKIT Translate API, providing better overall quality for live subtitle functionality. GitHub stars: 51 Repository: botbahlul/VOSK-Powered-Live-Subtitle-V3

Overview: DriftKit Framework is a comprehensive Java AI framework designed for the entire prompt lifecycle management, including versioning, testing, and tracing, all integrated within the framework itself rather than relying on external platforms. Key features: - Versioning and management of prompts - Testing and tracing capabilities - Multi-agent orchestration - Built-in audio processing Tech stack: Java 21, Maven, Spring Boot 3.3.x, MongoDB, Node.js Why it matters: This framework simplifies the development process by integrating prompt lifecycle management directly into the application, ensuring that all necessary steps are handled internally, reducing the need for external tools and improving efficiency. Repository: driftkit-ai/driftkit-framework GitHub stars: 50 GitHub description: The only Java AI framework with a complete Dev → Test → Prod prompt lifecycle, featuring multi-agent orchestration and built-in audio processing.

Overview: Ignis is a high-performance inference engine for Qwen3.8 and Qwen-Image-2.1, designed for local coding agents. It offers a peak throughput of 170 tok/s on a single RTX 5090 GPU, making it ideal for quick responses and low-resource environments. Key features: - Supports Qwen3.8 (27B) and Qwen-Image-2.1 (125B MoE) inference. - Built with Rust and CUDA, optimized for Windows and Linux. - Measured at 170 tok/s on a single RTX 5090 GPU. Tech stack: Rust + CUDA. Why it matters: Ignis is specifically designed for local coding agents, providing fast and efficient inference capabilities on a single GPU, making it ideal for applications requiring quick responses and low resource usage.

### Overview: HARTOS is an AI-native operating system that runs on your own hardware, providing models with direct access to your machine's resources. It federates peer-to-peer without a central broker, making it open-source and compatible with OpenAI APIs. Booting with 8GB of RAM, it supports a variety of hardware tiers, from a modest version to unlock speculative decoding on CUDA cards. ### Key features: - Direct access to hardware resources - Federates peer-to-peer without a central broker - Compatible with OpenAI APIs - Runs on 8GB of RAM - Supports multiple hardware tiers ### Tech stack: Python ### Why it matters: HARTOS is designed to be an AI-native operating system that runs on your own hardware, providing models with direct access to your machine's resources. It federates peer-to-peer without a central broker, making it open-source and compatible with OpenAI APIs. Booting with 8GB of RAM, it supports a variety of hardware tiers, from a modest version to unlock speculative decoding on CUDA cards.

Overview: This project is a web interface for Stable Diffusion, ControlNet, and Inpaint, providing a user-friendly interface for generating images with advanced features. Key features: - Stable Diffusion image generation - ControlNet integration for advanced image manipulation - Inpainting functionality to repair damaged images Tech stack: Python Why it matters: This project offers a practical tool for artists and designers to create high-quality images with enhanced capabilities, making it valuable for creative professionals and enthusiasts alike. Repository: kadirnar/Stable-Diffusion-ControlNet-WebUI Primary language: Python GitHub stars: 52

Overview: LocalCloud is a developer-friendly solution that allows you to run PostgreSQL, MongoDB, vector databases, AI models, Redis cache, job queues, and S3-like storage locally, eliminating the need for cloud infrastructure and DevOps. Key features: - Instantly accessible PostgreSQL, MongoDB, vector databases, AI models, Redis cache, job queues, and S3-like storage. - Works seamlessly with Python, Node.js, Go, Java, Rust, PHP, .NET, or any language. - Provides standard service APIs that integrate with your existing code regardless of technology stack. - Supports multiple platforms: macOS, Linux, Windows. Tech stack: Go Why it matters: LocalCloud offers a developer-friendly solution that allows startups, enterprises, corporate developers, demo heroes, remote teams, students, and learners to run their applications and databases locally, bootstrapped, and without the need for cloud infrastructure or DevOps.