Projects

Overview: MulmoChat is a research prototype exploring a new paradigm for multimodal AI chat experiences, enabling users to interact with AI through rich visual and interactive content directly on canvas. Key features: - Integrates visual and interactive elements (images, maps, games) into the chat interface. - Enables seamless communication between visual and textual inputs. - Demonstrates the architecture, design patterns, and user experience principles necessary for building truly multimodal chat interfaces. Tech stack: TypeScript Why it matters: This prototype aims to redefine the interaction between AI and humans, offering a more engaging and intuitive way of communicating with AI through a combination of text and visual elements. Repository: receptron/MulmoChat Primary language: TypeScript GitHub stars: 177 GitHub description: Research prototype for chat that is not a text stream: keep talking while images, maps and playable games appear on the canvas. Each kind of answer is a plugin on the GUI Chat Protocol.

Overview: This demo showcases the vector search similarity capability within Redis Stack and Redis Enterprise, turning Redis into a highly performant vector database for various applications. Key features: - Vector Similarity Search - by image - by text - Multiple Vector Indexing Types - HNSW - Flat (brute-force) - Hybrid Queries - Apply tags as pre-filter for vector search Tech stack: TypeScript Why it matters: This demo provides a practical example of how Redis can be used as a vector database, offering high performance and scalability for applications requiring vector similarity search. Repository: redis-developer/redis-product-search Primary language: TypeScript GitHub stars: 177 GitHub description: Visual and semantic vector similarity with Redis Stack, FastAPI, PyTorch and Huggingface.

Overview: This project focuses on 3D object detection from heterogeneous robot platforms such as vehicles, drones, and quadrupeds. It introduces a new dataset and a cross-platform 3D domain adaptation framework to enhance the generalization ability of 3D object detectors. Key features: - A new dataset for multi-platform 3D object detection. - A cross-platform 3D domain adaptation framework. - A comprehensive benchmark study of state-of-the-art 3D object detectors on cross-platform scenarios. Tech stack: Python Why it matters: This work addresses the practical yet challenging task of 3D object detection from heterogeneous robot platforms, providing a valuable dataset and framework that can be applied across different platforms, enhancing the generalization ability of 3D object detectors.

### Overview: AVID is a versatile video inpainting method that addresses the challenges of text-guided video inpainting across various video durations and tasks. It introduces Any-Length Video Inpainting with Diffusion Model (AVID), a novel approach that addresses temporal consistency, structural fidelity, and variable video length. ### Key Features: - **Temporal Consistency**: Ensures the edited video maintains temporal coherence. - **Structural Fidelity**: Supports different inpainting types at varying structural fidelity levels. - **Variable Video Length**: Generates videos of any desired duration. ### Tech Stack: - **Diffusers**: Built on top of [diffusers](https://github.com/huggingface/diffusers). - **AnimateDiff**: Utilized for generating videos. - **Tune-A-Video**: Integrated for comprehensive experiments. ### Why It Matters: AVID is a significant advancement in the field of video inpainting, offering a flexible and effective solution for a wide range of video editing tasks, including any-length videos with varying structural fidelity.

### Overview: This repository contains the official code for the paper "What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks" presented at the NeurIPS 2023 Datasets and Benchmarks Track. The benchmark includes eight tasks and evaluates the performance of various large language models on these tasks. ### Key Features: - **Tasks**: The repository includes benchmarks for eight different tasks in chemistry, such as name prediction, reaction yield prediction, and more. - **Models**: It provides results and performance metrics for multiple large language models, including GPT-4, GPT-3.5, Davinci-003, LLaMA2, and Galactica. - **Data**: The datasets used in the benchmarks are available for download and are categorized by task. ### Tech Stack: - **Primary Language**: Jupyter Notebook ### Why It Matters: This repository serves as a comprehensive benchmark for evaluating the capabilities of large language models in chemistry, providing valuable insights into their performance across various tasks.

### Overview: This tutorial provides a comprehensive guide on deploying and running `llama.cpp` on an Android device, utilizing the Adreno GPU. It includes a detailed setup for building and testing the llama.cpp application, along with a solid foundation for developing your own Android LLM applications. ### Key Features: - Deploying `llama.cpp` on an Android device and running it using the Adreno GPU. - Utilizing `llama-cpp-python` with a custom-built `llama.cpp` version that supports Adreno GPU with OpenCL. - Enables large-scale inference evaluation directly on Android. - Provides a solid foundation for developing your own Android LLM applications. ### Tech Stack: - `llama.cpp` and `llama-cpp-python` are utilized for the application development. ### Why it Matters: This tutorial offers a practical approach to deploying AI models on Android devices, leveraging the powerful Adreno GPU for large-scale inference tasks. It serves as a valuable resource for developers looking to integrate AI capabilities into Android applications, particularly for applications requiring high-performance inference capabilities.

Overview: DAEDAL is a training-free denoising strategy that unlocks dynamic and variable-length generation for DLLMs, achieving performance comparable, and sometimes superior, to meticulously tuned fixed-length baselines. Key features: - Dynamic length adjustment - Variable-length denoising - Training-free inference Tech stack: Python Why it matters: By resolving the static length constraint, DAEDAL enables more efficient and capable generation, bridging a critical gap with Autoregressive models and paving the way for more efficient and capable generation.

Overview: VEFX-Bench is a comprehensive benchmark designed to evaluate text-driven video editing and visual effects, focusing on 5,049 annotated examples across 9 categories and 32 subcategories. Key features: - 5,049 annotated examples - 9 categories and 32 subcategories - 32B reward model for scoring edits Tech stack: Python Why it matters: VEFX-Bench provides a robust framework for evaluating text-driven video editing and visual effects, ensuring that models can be objectively assessed across multiple dimensions of quality.

Overview: This repository provides sample applications for building generative AI solutions using AWS Generative AI CDK Constructs. Key features: - SageMaker JumpStart model - SageMaker Hugging Face model - SageMaker Hugging Face model on AWS Inferentia2 - SageMaker custom endpoint - SageMaker multimodal custom endpoint - SageMaker image to video endpoint - LLM on SageMaker in GovCloud PDT - Amazon Bedrock Agents - Python Samples - .NET Samples - Contract Compliance Analysis Tech stack: AWS Generative AI CDK Constructs Why it matters: These samples demonstrate the capabilities of AWS Generative AI CDK Constructs by providing practical examples of how to build generative AI solutions using various models and hardware accelerators.

### Overview: GGNN is a state-of-the-art GPU-based nearest neighbor search algorithm designed for handling billion-scale, high-dimensional datasets. It leverages CUDA for efficient computation and supports sharding for parallel processing on multiple GPUs. ### Key features: - **GPU Acceleration**: Utilizes CUDA for fast computation. - **Sharding Support**: Can execute on multiple GPUs through sharding. - **Direct Data Exchange**: Data can be exchanged directly with other code (e.g., torch tensors) without copying through CPU memory. - **C++ and CUDA Implementation**: Written in C++ and CUDA. - **Python Bindings**: Available via nanobind for Python integration. - **Citing This Project**: The algorithm is based on the method proposed in the paper "GGNN: Graph-based GPU Nearest Neighbor Search" by Fabian Groh, Lukas Ruppert, Patrick Wieschollek, and Hendrik P.A. Lensch. ### Tech stack: - **Programming Languages**: C++ and CUDA. - **Dependencies**: - CUDA Toolkit 12 or newer. - GCC/Clang C++ compilers version 10 or newer. - glog and gflags development libraries. - nanobind bindings for Python integration. ### Why it matters: GGNN is designed to efficiently handle large-scale graph-based nearest neighbor searches, making it ideal for applications requiring high performance and scalability, such as recommendation systems, bioinformatics, and more. ### Repository: - **GitHub Repository**: cgtuebingen/ggnn

### Overview: MindCube is a comprehensive framework designed to evaluate spatial reasoning abilities of vision-language models. It includes a dataset of 21,154 questions across 3,268 images, testing cognitive mapping, perspective-taking, and mental simulation. The project aims to assess whether VLMs can form spatial mental models from limited views, similar to human behavior. The framework is open-source and includes code and data for training and evaluation. ### Key Features: - **Dataset**: 21,154 questions across 3,268 images. - **Modules**: Scaffold data curation, prompt generation, model inference, training, and comprehensive evaluation. - **Evaluation Metrics**: Positions, orientations, and dynamics. - **Model Training**: Uses "map-then-reason" approach to improve accuracy from 37% to 77%. ### Tech Stack: Python ### Why it Matters: MindCube provides a robust framework to evaluate the spatial reasoning abilities of vision-language models, offering insights into how AI models can simulate and understand spatial information from limited views, which is crucial for advancing the field of multimodal AI.

### Overview: Forge is an open-source, self-hosted platform designed for visually building, testing, and shipping AI agents and workflows without relinquishing control over infrastructure. It allows developers to wire together agents, tools, knowledge, and logic on a canvas, ground them in data, connect them to systems, and deploy to various endpoints like email, APIs, MCP servers, or web widgets. Forge is built on the MIT-licensed LangChain + LangGraph framework and offers a production-ready, zero-infra local development environment. ### Key features: - **Fully Open Source (MIT).** No proprietary core, no usage caps, no vendor lock-in. - **Zero-infra Local Dev.** Boots on SQLite + embedded Chroma + an in-process scheduler - no Docker, Postgres, or Redis required to start. - **Production-Ready.** Swap to Postgres + pgvector, Redis, and a real secret store with config only - a hardening guard refuses to boot with insecure defaults. - **Observable by Default.** Every run is a span waterfall with tokens, latency, and cost down to fractions of a cent. - **Observability.** Real-time monitoring of tool executions, agent performance, and system metrics. - **Tool Builder with Response Projection.** Register REST, GraphQL, Code, SQL, MCP, or built-in tools and test them live against real inputs. A JMESPath response projection trims payloads before reaching the model, and an SSRF guard screens outbound calls with optional retries, rate limits, and caching. - **Built-ins (time,

Overview: The repository contains a production-grade vLLM cluster setup for DeepSeek-V4-Flash-0731, featuring a self-healing 2-node vLLM cluster, tuned DSpark speculative decoding, and full benchmarks. The cluster is equipped with 2x NVIDIA DGX Spark, linked by a single 400G QSFP DAC cable, showcasing high bandwidth and low latency. Key features: - Self-healing 2-node vLLM cluster - Reboot-verified setup - Tuned DSpark speculative decoding (~75 tok/s) - Full benchmarks - OpenAI Codex CLI integration Tech stack: Shell Why it matters: This repository provides a comprehensive guide for developers looking to set up a production-grade vLLM cluster, ensuring high performance and reliability with minimal downtime. Repository: raullenchai/twinspark Primary language: Shell GitHub stars: 176

### Overview: Faster Chat is a blazingly fast, privacy-first chat interface for AI that works with any LLM provider—cloud or completely offline. It connects to OpenAI, Anthropic, Google, Groq, Mistral, xAI, DeepSeek, and more, or runs completely offline with Ollama, LM Studio, or llama.cpp. Your conversations stay on your machine, ensuring no vendor lock-in, no tracking, and full control. ### Key Features: - 📄 Server-side SQLite storage — Conversations persist across devices and browser tabs. - 🤖 17 built-in providers (plus 50+ more auto-discoverable via models.dev): OpenAI, Anthropic, Google, Google Vertex, Amazon Bedrock, Azure, Groq, Mistral, xAI, DeepSeek, Cohere, Fireworks, Cerebras, OpenRouter, Replicate, Ollama, LM Studio. - 🔍 Cross-conversation memory — AI remembers your preferences, projects, and context across chats. - 🖼️ Web search — AI can search the web and cite sources inline (Brave Search). - 📥 File attachments with preview and download. - 📝 Markdown rendering with syntax highlighting (Shiki) and LaTeX support. - 🎨 Themable UI — 15+ color themes, dark/light mode, custom fonts, syntax highlighting themes. - 🎤 Voice input/output — Speech-to-text and text-to-speech capabilities. - ⌨️ Keyboard shortcuts for power users (Ctrl+B sidebar, Ctrl

Overview: This GitHub repository provides comprehensive code for building Business-Ready Generative AI Systems (GenAISys) from scratch. It guides developers through the process of architecting and implementing advanced AI controllers, intelligent agents, and dynamic Retrieval Augmented Generation (RAG) frameworks. The projects demonstrate practical applications across various domains. Key features: - Implement an AI controller with a conversation AI agent and orchestrator at its core - Build contextual awareness with short-term, long-term, and cross-session memory - Cross-domain automation with multimodal reasoning, image generation, and voice features - Expand a CoT agent by integrating cutting-edge models like GPT-4 and DeepSeek-R1 - Ensure stability, scalability, and production readiness with Pinecone integration Tech stack: Jupyter Notebook Why it matters: This repository offers a practical guide for developers looking to create robust and adaptable generative AI systems, ensuring they can tackle complex business challenges with confidence.

**Overview:** NexusDesk is an open-source, self-hostable, and extensible general-purpose agent platform built around ReAct and Harness engineering, designed to streamline AI integration into business workflows. **Key features:** - AI customer service as the first complete open-source reference application - Extensible and customizable through extension contracts and business capabilities - Secure and scalable architecture with PostgreSQL, Redis, Kubernetes, and Docker support - Self-hosted and Docker-based deployment with a default requirement for PostgreSQL **Tech stack:** Python **Why it matters:** NexusDesk simplifies the integration of AI into business processes, providing a robust, self-hosted platform that validates AI solutions through real-world applications. **Repository:** GitHub - [GroundedCoreAI/nexusdesk](https://github.com/GroundedCoreAI/nexusdesk) **GitHub stars:** 136 **GitHub description:** Open-source, self-hosted, extensible Agent platform built around ReAct and Harness engineering, with AI customer support as its first complete reference application.

Overview: This project introduces **Invertible Diffusion Models (IDM)**, a novel method for compressed sensing (CS) that leverages diffusion models for efficient image reconstruction. IDM reimagines a large-scale diffusion sampling process as a reconstruction model and fine-tunes it end-to-end to directly recover original images from CS measurements, overcoming the limitations of current CS neural networks. Key features: - Invertible Diffusion Models (IDM) - Efficient end-to-end fine-tuning - Reduces GPU memory usage by up to 93.8% Tech stack: Python Why it matters: By addressing the challenges of training and inference efficiency, IDM enables rapid deployment of CS models, making them more practical for real-world applications. The method's ability to directly reconstruct images from CS measurements without the need for one-step noise estimation learning significantly outperforms existing state-of-the-art CS networks in terms of PSNR, achieving up to 10.09dB gain and 14.54 times faster inference speed compared to the recent diffusion-based approach DDNM. Repository: Guaishou74851/IDM GitHub stars: 175 GitHub description: (TPAMI 2025) Invertible Diffusion Models for Compressed Sensing [PyTorch]

Overview: HyperDreamBooth is an implementation of HyperNetworks for fast personalization of text-to-image models, focusing on accelerating the generation of personalized images from textual prompts. Key features: - Real-time image generation - Textual prompt-based image generation - HyperNetworks for efficient model optimization Tech stack: Python Why it matters: This project aims to significantly reduce the time required for generating personalized images, making it particularly useful for applications requiring rapid response times. Repository: JiauZhang/hyperdreambooth Primary language: Python GitHub stars: 175

### Overview RapidFire AI is a powerful experiment execution framework designed to accelerate AI customization. It transforms slow, sequential processes into rapid, intelligent workflows with hyperparallelized execution, interactive control, and automatic optimization. ### Key Features - **Hyperparallelized Execution**: Higher throughput, concurrent execution of data shards to show side-by-side differences. - **Interactive Control (IC Ops)**: Stop, Resume, Clone-Modify, and optionally warm start runs in real-time. - **Automatic Optimization**: Intelligent orchestration to optimize GPU utilization with minimal overhead. ### Tech Stack JavaScript ### Why It Matters RapidFire AI delivers 16-24x higher throughput without additional resources, making AI customization faster and more impactful. It simplifies the complex process of AI experimentation, enabling teams to explore multiple configurations simultaneously and compare results in real-time.

Overview: ChatIndex is a context management system designed to efficiently navigate and utilize long conversation histories through hierarchical tree-based indexing and intelligent reasoning-based retrieval. Key features: - Context Tree Specification - Dynamic Retrieval Resolution - Lossless Fallback - Efficient Reasoning Tech stack: Python Why it matters: ChatIndex provides a scalable and robust solution for managing long conversational contexts, addressing the context management challenge in AI chat assistants by combining the completeness of an index system with the flexibility of a hierarchical memory structure. Repository: VectifyAI/ChatIndex GitHub stars: 175 GitHub description: ChatIndex: Tree indexing and retrieval for long conversational memory