Projects

### Overview: FlexiDreamer is an official implementation of FlexiDreamer, a framework that enables the generation of high-quality 3D meshes from single images using FlexiCubes, an advanced mesh optimization technique. The project aims to address the challenges of sparse-view reconstruction and post-processing extraction by directly reconstructing meshes from multi-view images. ### Key features: - Direct reconstruction of high-quality meshes from multi-view images. - Utilizes FlexiCubes for advanced mesh optimization. - Incorporates hybrid positional encoding and orientation-aware texture mapping. - Reduces geometric holes and surface noise through eikonal and smooth regularization. - Generates 3D meshes in approximately 1 minute, significantly outperforming previous methods. ### Tech stack: Unknown ### Why it matters: FlexiDreamer offers a novel approach to 3D content generation by directly reconstructing meshes from single images, addressing the challenges of sparse-view reconstruction and post-processing extraction. The project aims to enhance the efficiency and quality of 3D mesh generation, making it more accessible and practical for various applications.

Overview: TensaLang is a Tensor-first programming language that allows developers to write the Model's inference engine in a high-level language, then compile it through MLIR to multiple targets such as CPU, CUDA, and ROCm. Key features: - Source code for attention, sampling, tiling, and memory placement - MLIR as the core IR - Language features for hinting tile sizes, parallel indices, and memory placement - Overridable builtins - Target-dependent lowering pipelines Tech stack: C++ Why it matters: TensaLang flips the traditional monolithic approach to LLM inference, allowing developers to write the Model's inference logic in a high-level language and then compile it through MLIR to multiple targets, enabling flexibility and customization. GitHub stars: 77

### Overview: InverseSR is an unsupervised method for MRI super-resolution that leverages a pre-trained Brain LDM model to generate powerful image priors for T1w brain MRIs. This approach aims to adapt to various MRI SR problems at test time and find the optimal latent representation for super-resolution. ### Key features: - **Unsupervised MRI Super-Resolution**: No ground truth data required for training. - **Latent Diffusion Model (Brain LDM) Utilization**: Leverages a pre-trained Brain LDM for generating image priors. - **Adaptability**: Capable of being applied to different MRI SR settings. - **Optimal Latent Representation**: Finds the optimal latent representation $z^∗$ in the Brain LDM latent space, mapping it to the SR MRI $G(z^∗)$. ### Tech stack: Python ### Why it matters: This innovative technique offers a new approach to MRI super-resolution, potentially improving the quality of medical imaging without the need for ground truth data, making it a significant advancement in the field of medical imaging.

Overview: ArtFusion is a learning-based arbitrary style transfer diffusion model that offers enhanced controllability and faithful representation of artistic details. Key features: - Controllable arbitrary style transfer - Captures core style characteristics overlooked in SOTA methods - Flexible balance between source content and reference style Tech stack: Jupyter Notebook Why it matters: ArtFusion provides a powerful tool for users to achieve diverse stylization preferences with high control, making it valuable for various artistic applications. Repository: ChenDarYen/ArtFusion GitHub stars: 77 GitHub description: ArtFusion: Controllable Arbitrary Style Transfer using Dual Conditional Latent Diffusion Models License: MIT License

### Overview: JavisGPT is a unified multi-modal language model designed to understand and generate synchronized sound videos, making it a powerful tool for audio-visual comprehension and generation. ### Key Features: - **Unified Model**: JavisGPT combines audio and visual understanding to create a single model capable of comprehending and generating sound videos. - **Pretrained Weights**: Utilizes pre-trained weights from JavisDiT-v1.0-jav to enhance audio-video generation. - **Datasets**: Cares for the **JavisInst-Omni** dataset for instruction tuning, **MM-PreTrain** and **AV-FineTune** datasets for multimodal alignment, and the **JavisUnd-Eval** dataset for evaluating audio-video understanding. ### Tech Stack: Python ### Why It Matters: JavisGPT offers a comprehensive solution for audio-visual comprehension and generation, making it a valuable tool in the field of multimodal AI. Repository: JavisVerse/JavisGPT

Overview: EvoWorld is an official implementation of a generative world model that explicitly reconstructs and utilizes a 3D memory for egocentric video generation. This model synthesizes future frames with strong 3D consistency by projecting an evolving 3D point cloud onto future views as conditioning for a video diffusion model. Key features: - Single-clip generation - 3-clip iterative generation Tech stack: Python Why it matters: EvoWorld offers a novel approach to world generation that enhances the realism and coherence of video sequences by leveraging explicit 3D memory, making it particularly useful for creating immersive and consistent virtual environments. Repository: JiahaoPlus/EvoWorld Primary language: Python GitHub stars: 77 GitHub description: EvoWorld: Evolving Panoramic World Generation with Explicit 3D Memory

Overview: Genima fine-tunes Stable Diffusion to generate joint actions on RGB observations, aiming to improve the realism and action capability of the generated images. Key features: - Fine-tuning of Stable Diffusion to generate joint actions - Support for various diffusion models (e.g., SD1, SD2, SD3) - Pre-trained checkpoints available for different tasks and demonstrations Tech stack: Python Why it matters: Genima aims to enhance the realism and action capability of generated images by fine-tuning Stable Diffusion, making it suitable for applications requiring realistic joint actions, such as robotics and simulation environments. Repository: GitHub - https://github.com/MohitShridhar/genima Primary language: Python GitHub stars: 77 GitHub description: Official Code Repo for GENIMA README excerpt: # GENIMA ### [Generative Image as Action Models](https://genima-robot.github.io) [Mohit Shridhar*](https://mohitshridhar.com/), [Yat Long (Richie) Lo*](https://richielo.github.io/), [Stephen James](https://stepjam.github.io/) [CoRL 2024](https://www.corl.org/) Genima fine-tunes Stable Diffusion to draw joint-actions on RGB observations.  This repo is for reproducing the RLBench results from the paper. For the latest updates, see: [genima-robot.github.io](https://genima-robot

### Overview: OmniMCP is a perceive-plan-act computer-use experiment repository designed for AI model interaction and deep UI understanding. It focuses on enabling rich UI context and precise interaction execution through Model Context Protocol (MCP) and Microsoft's OmniParser. ### Key Features: - **Visual Perception:** Utilizes OmniParser to understand UI elements. - **LLM Planning:** Plans next actions based on goal, history, and visual state. - **Agent Executor:** Orchestrates the perceive-plan-act loop. - **Action Execution:** Controls mouse/keyboard using `pynput`. - **CLI Interface:** Simple entry point for running tasks. - **Auto-Deployment:** Optional AWS EC2 deployment for OmniParser server. - **Debugging:** Generates timestamped visual logs per step. ### Tech Stack: Python ### Why it Matters: OmniMCP is essential for researchers and developers looking to enhance the interaction capabilities of AI models through visual analysis and precise execution. Its modular design and rich UI context make it a valuable tool in the field of AI and computer use experiments. Repository: OpenAdaptAI/OmniMCP GitHub Stars: 77 GitHub Description: Labs: a perceive-plan-act computer-use experiment. Not the OpenAdapt product. The product is openadapt-flow: a compiled program that halts unless an independent check passes.

### Overview: Voice-to-Text ChatGPT: Transcribe with Whisper API 🚀 ### Key features: - Record voice and transcribe using OpenAI's Whisper API - Configurable shortcut button for quick microphone access - Compatible with main inputs on chat.openai.com and edit-inputs - Customize prompt for better API voice recognition - Clean interface with mic button - Support for multiple Whisper prompts for versatile contexts - Download transcriptions as sound files - Snippets feature (beta) for pasting text ### Tech stack: JavaScript ### Why it matters: Elevate your ChatGPT experience with effortless voice-to-text transcription using the Whisper API, all within your Chrome browser.

### Overview: CoPaint is an implementation of the paper "Towards Coherent Image Inpainting Using Denoising Diffusion Implicit Models". It addresses the coherence issue in image inpainting by using a Bayesian framework to modify both the revealed and unrevealed regions of the image. ### Key features: - **Coherent Inpainting**: CoPaint ensures coherence between the inpainted regions and the surrounding context. - **Bayesian Framework**: Utilizes a Bayesian approach to modify both the revealed and unrevealed regions of the image. - **Time-Travel Trick**: Implements a time-travel trick to further improve coherence. ### Tech stack: Python ### Why it matters: CoPaint effectively resolves the coherence issue in image inpainting, leading to more natural and coherent results compared to existing methods.

Overview: PicPilot is a powerful AI-driven platform designed to enhance, generate, and transform images and videos for branding purposes. Key features: - Flux Inpainting for detailed image editing - SDXL (Stable Diffusion XL) with LoRA for high-quality image generation - SDXL Outpainting for seamless image extension - Image to Video Generation using CogvideoX - Batch processing support with configurable batch sizes and timeouts Tech stack: Python Why it matters: PicPilot simplifies the process of creating professional product photography and visual narratives, making it an essential tool for businesses aiming to enhance their brand presence through stunning visuals. Repository: VikramxD/PicPilot Primary language: Python GitHub stars: 77 GitHub description: PicPilot is a scalable solution that leverages state-of-the-art Text to Image Models to extend and enhance images and create product photography in seconds for your brand.

### Overview: This repository provides a comprehensive survey on evaluating reasoning capabilities in multimodal large language models (MLLMs). It includes benchmarks for various reasoning tasks such as mathematical reasoning, chart reasoning, scientific reasoning, code generation, multi-image-based inductive reasoning, social and cultural knowledge reasoning, algorithmic problem solving, action prediction, and spatial reasoning. ### Key Features: - **Mathematical Reasoning**: Benchmarks for evaluating mathematical reasoning in large multimodal models. - **Chart Reasoning**: Evaluating reasoning in visual contexts using datasets like Mathvista, MathVerse, MathScape, and ChartBench. - **Scientific Reasoning**: Benchmarking models on scientific reasoning tasks. - **Code Generation**: Evaluating models for generating code. - **Multi-Image Based Inductive Reasoning**: Assessing models' ability to reason from multiple images. - **Social and Cultural Knowledge Reasoning**: Benchmarking models on social and cultural knowledge tasks. - **Algorithmic Problem**: Evaluating models for solving algorithmic problems. - **Action Prediction**: Benchmarking models for predicting actions. - **Spatial Reasoning**: Assessing models' ability to reason about spatial relationships. ### Tech Stack: The repository uses Python, Jupyter notebooks, and Hugging Face datasets for data processing and evaluation. ### Why it Matters: This repository is crucial for researchers and developers in the field of AI and ML, as it provides a standardized way to evaluate and compare the reasoning capabilities of different large language models. It helps in identifying the strengths and weaknesses of models and in guiding the development of more accurate and

Overview: Wan 2.7 Video Edit API (`wan2.7-videoedit`) is a model id, pricing, and parameters for the OpenAI-compatible APIMart endpoint, providing high-quality video editing services at a cost-effective rate. Key features: - Supports 300+ models including Wan 2.7 Video Edit API (`wan2.7-videoedit`). - Offers a single API key for accessing all models and resources. - Supports both text-to-video and image-to-video routes. - Pricing is per generated second, with a default tier of 0.083 per second. - Supports optional instructions and up to four reference images. - Non-blocking design for efficient processing. Tech stack: Python Why it matters: This API simplifies the process of video editing by providing a single API key for multiple models, making it easier and more cost-effective to use AI for video creation.

### Overview: Bud500 is a comprehensive Vietnamese ASR dataset consisting of approximately 500 hours of audio, covering various topics and accents. It is designed to enhance speech recognition research and development. ### Key Features: - **Audio Content**: Includes podcast, travel, book, food, and other topics, with Vietnamese accents from different regions. - **Public Access**: Available for free public use, enhancing accessibility and research opportunities. - **Structured Data**: Each entry contains an audio file and its transcription, facilitating easy access and analysis. - **Split Splits**: The dataset is divided into train, validation, and test sets, ensuring balanced and effective evaluation. ### Tech Stack: - **Languages**: Vietnamese - **Dataset Type**: Parquet files ### Why It Matters: - **Enhanced Research**: Provides a rich dataset for ASR researchers to improve their models. - **Accessibility**: Facilitates research and development by making the dataset publicly accessible. - **Community Impact**: Supports the growth of AI talent in Vietnam, contributing to the development of a skilled AI community. ### Repository: - **GitHub Repository**: [apluka34/Bud500](https://github.com/apluka34/Bud500) - **GitHub Stars**: 77 - **Repository Description**: Bud500: A Comprehensive Vietnamese ASR Dataset ### Languages: - **Languages**: Vietnamese ### Dataset Structure: - **Data Fields**: - **Audio**: Contains the path to the audio file, the decoded audio array, and

### Overview: ID-Patch is an official implementation of a paper presented at CVPR 2025 titled "ID-Patch: Robust ID Association for Group Photo Personalization." This project proposes a method called ID-Patch, which is designed to link identity features to 2D positions in group photo personalization. It addresses the issue of identity leakage and the increasing generation time with an increasing number of identities by generating ID patches and ID embeddings. ### Key Features: - **ID-Patch Method**: A fast and robust method for linking identity features to 2D positions. - **ID Patches**: Placed on the conditional image for precise spatial control. - **ID Embeddings**: Fused with text embeddings to enhance identity resemblance. ### Tech Stack: - Python ### Why It Matters: ID-Patch is significant because it provides a solution to the problem of identity leakage and the increasing generation time in group photo personalization. By linking identity features to specific positions, it ensures that the identities are accurately represented in the generated images, making the personalization process more robust and efficient. ### Repository: - **Repository**: bytedance/ID-Patch - **Stars**: 77 - **Description**: Official implementation of CVPR 2025 paper "ID-Patch: Robust ID Association for Group Photo Personalization." This work proposes ID-Patch, a method that links identity features to 2D positions via visual patches and embeddings. ### Tech Stack: - **Python** ### Disclaimer: The released HuggingFace model differs from the paper's

### Overview: This repository is dedicated to collecting and sharing research papers on diffusion guidance methods, focusing on cutting-edge papers in the field. ### Key features: - Papers by Topic: The repository categorizes papers by topic, including foundational and theoretical studies, correction methods, adaptive guidance, self-attention guidance, external objectives, alternative conditioning, and guidance-free methods. - How to Contribute: The repository provides a table of contents and a guide on how to contribute to the project. - To Do: The repository includes a list of things that need to be done. ### Tech stack: Unknown ### Why it matters: This collection of papers provides a comprehensive overview of diffusion guidance methods, helping researchers and practitioners stay updated on the latest advancements in the field.

### Overview: Gigastt is a Rust-based speech-to-text (STT) server that leverages GigaAM v3 for high-quality Russian and multilingual recognition, powered by ONNX Runtime. It offers offline inference capabilities and supports a variety of audio file formats. ### Key features: - **Model Support:** Russian, English, Kazakh, Kyrgyz, and Uzbek. - **Offline Inference:** Models are downloaded once and can be used offline. - **CLI and HTTP/WS:** Supports CLI, HTTP, and WebSocket interfaces. - **Batch Processing:** Facilitates batch processing and directory watching. - **File Transcription:** Transcribes WAV, AAC/M4A, MP3, FLAC, OGG/Vorbis, OGG/Opus, and WebM/Opus files. - **Punctuation and Text Normalization:** Built-in punctuation and text normalization for the `e2e_rnnt` model. - **Model Setup and Sizes:** Models are downloaded once and can be provisioned offline. ### Tech stack: Rust ### Why it matters: Gigastt provides a robust, offline STT solution for developers, offering high-quality recognition and offline inference capabilities, making it ideal for applications requiring speech processing without internet connectivity. Repository: ekhodzitsky/gigastt

Overview: This repository contains the official implementation of the paper "Anyattack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Models" presented at CVPR 2025. The method aims to generate adversarial examples on a wide range of commercial Vision-Language Models (VLMs) using self-supervised learning techniques. Key features: - Large-scale: The approach is designed to work effectively on large-scale datasets like LAION-400M. - Self-supervised: AnyAttack utilizes self-supervised learning for generating adversarial examples. Tech stack: Python Why it matters: The repository provides a practical tool for researchers and practitioners to evaluate the effectiveness of adversarial attacks on VLMs, contributing to the advancement of defense mechanisms in AI systems. Repository: jiamingzhang94/AnyAttack GitHub stars: 77 GitHub description: CVPR 2025 - Anyattack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Models

Overview: Super STT allows you to dictate directly into any app on Linux using a simple shortcut, making speech-to-text effortless for everyone. Key features: - A model-agnostic engine that supports a growing library of speech models. - An open protocol that allows any app, in any language, to request transcriptions, stream live audio visualizations, or drive recording with per-app consent. - A desktop app, CLI, and COSMIC applet, all of which are built-in to the installation process. Tech stack: Rust Why it matters: Super STT simplifies speech-to-text integration, making it accessible and efficient for users on Linux platforms. Repository: jorge-menjivar/super-stt Primary language: Rust GitHub stars: 77 GitHub description: Super STT let's you speak, and your words are typed straight into whatever app is focused README excerpt: <div align="center"> <img src=".github/assets/super-stt-icon.svg" width="128" height="128" alt="Super STT"> # Super STT **Speak into any app on Linux. Your words appear as text.** *One shortcut to dictate anywhere • Any model from a growing library • An open protocol any app can build on • Built in Rust* [](https://jorge-menjivar.github.io/super-stt/coverage/) </div> https://github

### Overview: This repository provides a Dockerized version of VQGAN+CLIP for zero-shot text-to-image generation. It simplifies the process of training and deploying the model, making it accessible for both local and production environments. ### Key features: - **Model Training**: A pre-trained VQGAN+CLIP model is available in the repository. - **Zero-Shot Generation**: The model can generate images from text inputs, without the need for additional training data. - **Docker Support**: The model is packaged in a Docker container, making it easy to deploy on any machine with Docker installed. - **Configuration Files**: Two configuration files are provided (`local.json` and `docker.json`) to customize the model's behavior. - **GPU and CPU Support**: The model can be run on both CPU and GPU, with specific requirements for GPU performance. ### Tech stack: Python ### Why it matters: This repository simplifies the process of zero-shot text-to-image generation by providing a pre-trained model and a Dockerized solution, making it accessible for both local and production environments.