Projects

### Overview: SGS, or Single-cell and Spatial Multiomics Browser, is a user-friendly, collaborative, and versatile tool designed for the visualization of single-cell and spatial multiomics data. It integrates advanced features for comparative visualization, multi-panel coordinate view, and abundant visualization functions, enabling researchers to unlock novel insights from scMulti-omics data. ### Key Features: - **Advanced Visualization Functions**: Enhances the clarity and interactivity of visualizations. - **Comparative Visualization**: Facilitates the comparison of different datasets and experimental conditions. - **Multi-Panel Coordinate View**: Provides a comprehensive view of data across multiple panels. - **Collaborative Exploration**: Enables multiple users to work on the same dataset simultaneously. - **Seamless Data Format Conversion**: Supports various data formats and seamlessly converts between them using the SgsAnnData R package. ### Tech Stack: - **Docker**: Primary technology for graphical one-click installation. - **Flutter**: Used for implementing the graphical user interface. ### Why It Matters: SGS empowers researchers to unlock novel insights from scMulti-omics data by providing a user-friendly, collaborative, and versatile browser for visualization and analysis. Its advanced features and seamless data format conversion capabilities make it an indispensable tool in the field of single-cell and spatial multiomics research. ### Repository: Repository: `fanglu0411/sgs` GitHub Stars: 101 GitHub Description: SGS, a user-friendly, collaborative, and versatile browser for visualizing single-cell and spatial multiomics data. ### Primary Language: JavaScript ### Installation

Overview: Way Back Home is an immersive AI workshop platform designed to teach Google Cloud AI technologies through an engaging narrative. Participants learn to build intelligent agents while rescuing a stranded space explorer. Key features: - Multi-turn image generation with Gemini (Nano Banana) - Multi-agent systems for crash site analysis - Event-driven agents for processing incoming SOS signals - Agent orchestration for coordinating group rescue efforts Tech stack: The project uses Next.js 14, Three.js, React Three Fiber, Tailwind CSS for frontend, FastAPI, Firestore, Firebase Storage, and Google Cloud Run for backend and infrastructure. Why it matters: Way Back Home provides a hands-on learning experience that combines Google Cloud AI technologies with an engaging narrative, making it an excellent resource for developers looking to understand and apply AI concepts in a practical setting.

Overview: This JAX research toolkit, `gemma_penzai`, extends the original Penzai package to support visualizing, manipulating, and understanding Gemma models with multi-modal support, including Gemma 3, for more interpretability research. Key features: - Multi-modal support with vision and multimodal components. - Support for Vision Transformers (ViTs) and basic components. - Implementation of a new attention mask for MLLMs. - Decoding algorithms for MLLMs. Tech stack: JAX research toolkit for building, editing, and visualizing neural networks. Why it matters: This toolkit provides a comprehensive solution for visualizing and understanding Gemma models, enhancing interpretability and research capabilities in the field of large language models.

### Overview: MegaFusion is an innovative project that extends diffusion models to generate higher-resolution images without needing further tuning. This work was presented at WACV 2025 and is available on the official repository. ### Key features: - **SDM-MegaFusion**: Extends StableDiffusion to generate higher-resolution images. - **SDXL-MegaFusion**: Extends StableDiffusion-XL to generate higher-resolution images. - **SD3-MegaFusion**: Extends StableDiffusion-3 to generate higher-resolution images. - **ControlNet-MegaFusion**: Integrates ControlNet into diffusion models for more detailed image generation. - **IP-Adapter-MegaFusion**: Combines IP-Adapter with diffusion models for improved image quality. ### Tech stack: Python ### Why it matters: MegaFusion offers a significant advancement in the field of image generation by extending existing diffusion models to higher resolutions without the need for further tuning, making it a valuable tool for creators and researchers alike.

### Overview: LLMChat is a Discord chatbot that leverages cutting-edge language models like GPT-4, 3.5, 3, and LLaMA for text generation and high-quality voice chat with ElevenLabs, Azure TTS, Silero, and Bark models. It supports local LLaMA models and OpenAI Whisper for speech recognition, making it a versatile tool for developers looking to integrate advanced AI into their projects. ### Key Features: - **Realistic Voice Chat**: Compatible with ElevenLabs, Azure TTS, Silero, and Bark models for immersive voice chat experiences. - **Image Recognition**: Utilizes BLIP for recognizing images in messages. - **Long-Term Message Recall**: Detects similar topics from past conversations using OpenAI's embeddings. - **Custom Bot Identity**: Allows users to personalize their bot's name and identity. - **Support for OpenAI Models**: Compatible with all OpenAI text completion and chat completion models. - **Local LLaMA Models**: Supports local LLaMA models for improved performance. - **Local OpenAI Whisper**: Enables speech recognition using OpenAI Whisper or Google/Azure TTS. - **Chat-Optimized Commands**: Designed for efficient interaction with the bot. ### Tech Stack: Python ### Why It Matters: LLMChat is a powerful tool for developers looking to integrate advanced AI capabilities into their Discord projects. Its ability to generate realistic text and provide high-quality voice chat experiences makes it a valuable addition to any developer's toolkit. The bot's features, including image recognition and local model support,

### Overview: The ChatGPT/DeepSeek Voice Assistant is a Python-based project that enables spoken conversation with OpenAI or DeepSeek large language models using a Raspberry Pi or desktop. It listens to speech, processes the conversation through the OpenAI/DeepSeek service, and responds back, similar to Apple Siri, Amazon Alexa, Google Nest Home, and Mi XiaoAi. ### Key features: - Real-time voice dialogue - Continuous dialogue history - Local wake word support ### Tech stack: Python ### Why it matters: This project is a practical implementation of a voice assistant that allows for seamless spoken communication with AI models, making it accessible and useful for a wide range of applications and users.

Overview: libwaifu is a local image generator that runs on your machine, using your existing GPU to create high-quality images from prompts. It supports various models like SDXL, Anima, and Qwen-Image, and allows for voice conversion and layer decomposition. Key features: - Local AI image generation - Supports multiple models - GPU-accelerated inference - Voice conversion - Layer decomposition Tech stack: Rust + C++, CUDA / Metal / Vulkan Why it matters: libwaifu provides a powerful and efficient way to create high-quality images locally, without the need for cloud services or external APIs. This makes it ideal for users who want to generate images quickly and locally, without compromising on quality.

### Overview: The Great GPT Firewall is a curated list of websites that restrict access to AI Agents, AI crawlers, and GPTs using the `robots.txt` file. It aims to provide a controlled environment for AI-related content. ### Key features: - Contains a curated list of websites that employ `robots.txt` to restrict access to AI Agents, AI crawlers, and GPTs. - Updated monthly to ensure the most current information. - Includes user agents for various AI-related projects and bots. ### Tech stack: Python ### Why it matters: This repository serves as a valuable resource for developers and content creators who need to manage the accessibility of their websites, ensuring that AI-related content is protected and not accessed by unauthorized bots.

Overview: SoundCTM is an advanced Pytorch implementation of a unified score-based and consistency model for full-band text-to-sound generation, presented at ICLR 2025. This project includes the official codebase, checkpoints, and a demo page for the SoundCTM-DiT backbone. Key features: - SoundCTM-DiT: A unified model combining score-based and consistency techniques for full-band text-to-sound generation. - Pytorch Implementation: The project is written in Python using Pytorch. - GitHub Stars: 101: The project has gained significant traction on GitHub. - NeurIPS'25 Workshop: The paper was presented at the NeurIPS'25 audio image generation workshop. Tech stack: The project uses Pytorch for the implementation. Why it matters: SoundCTM offers a comprehensive solution for text-to-sound generation, combining the strengths of score-based and consistency models to achieve high-quality results.

Overview: Artifex is a Python library designed for creating and fine-tuning task-specific Small Language Models without the need for large labeled datasets. It generates synthetic data on-the-fly to fine-tune models for specific tasks, making it suitable for offline NLP and text classification. Key features: - Using pre-trained task-specific Small Language Models on CPU - Fine-tuning them on CPU without any training data - Tracking model performance locally with built-in evaluation and monitoring tools Tech stack: Python Why it matters: Artifex simplifies the process of creating and fine-tuning Small Language Models, making it accessible for tasks that don't require large labeled datasets, such as text classification and moderation.

Overview: This is the official TypeScript client for interacting with a Weaviate instance, providing easy access to the powerful indexing and querying capabilities of Weaviate. Key features: - Easy integration with TypeScript - Support for various Weaviate operations - Documentation available in multiple formats Tech stack: TypeScript Why it matters: This client simplifies the process of working with Weaviate, making it easier for developers to integrate with the powerful indexing and querying features of the platform. Repository: weaviate/typescript-client Primary language: TypeScript GitHub stars: 101

### Overview: DiffAbXL is a benchmarking tool designed to evaluate generative models for antibody sequence design. The repository includes the official implementation of the DiffAbXL benchmark, which was originally titled "Benchmarking Generative Models for Antibody Design" but was later updated to focus on ranking antibody sequence designs. ### Key features: - **Benchmarking Models**: Provides a standardized way to evaluate and compare generative models for antibody sequence design. - **Benchmarking Datasets**: Includes datasets and licenses for benchmarking models. - **Training**: Includes scripts and configurations for training generative models. - **Interface for Benchmarking Models**: Guides on how to build an interface for benchmarking models. ### Tech stack: Python ### Why it matters: DiffAbXL is crucial for researchers and developers in the field of antibody design, as it provides a standardized way to evaluate and compare generative models. This tool helps in identifying the most effective models for antibody sequence design, thereby accelerating the development process and improving the accuracy of antibody design.

### Overview: This repository is a curated list of research on diffusion and flow matching models for tabular data, serving as the official repository for a survey paper titled "Diffusion and Flow Matching Models for Tabular Data: A Survey." The project aims to explore advancements in these models and highlight key challenges, current progress, and future directions in the field. ### Key Features: - **Research Papers**: Curated list of research papers on diffusion and flow matching models for tabular data. - **Survey Paper**: Official repository for the survey paper "Diffusion and Flow Matching Models for Tabular Data: A Survey." - **Repository**: Contains links to the survey paper and curated research papers. - **Starred Repository**: Has 100 GitHub stars, indicating its popularity among the developer community. ### Tech Stack: - **Research Papers**: The repository includes links to the survey paper and curated research papers. - **Survey Paper**: The official repository for the survey paper "Diffusion and Flow Matching Models for Tabular Data: A Survey." ### Why It Matters: This project is crucial for researchers and developers working in the field of tabular data, as it provides a comprehensive overview of the current state-of-the-art in diffusion and flow matching models. It serves as a valuable resource for understanding the challenges and future directions in this area, making it an essential tool for anyone interested in advancing the field of tabular data analysis. ### Repository: Diffusion-Model-Leiden/awesome-diffusion-models-for-tabular-data - **Primary Language**: Unknown - **GitHub Stars**:

### Overview: RGBD2: Generative Scene Synthesis via Incremental View Inpainting using RGBD Diffusion Models ### Key features: - Sequentially generates novel RGBD views along a camera trajectory. - The scene geometry is simply the fusion result of these views. ### Tech stack: Python ### Why it matters: This project presents a new solution for generating novel RGBD views along a camera trajectory, which is useful for various applications such as scene synthesis and augmented reality. The use of RGBD diffusion models allows for efficient and effective scene synthesis, making it a significant contribution to the field of computer vision and graphics. ### GitHub stars: 100 ### Repository: Karbo123/RGBD-Diffusion ### Primary language: Python ### Download files: 1. Preprocessed ScanNetV2 dataset: `https://drive.google.com/file/d/12MUFPsLxJakr5bnLO5XsyGQ4lEN9q2Wb/view?usp=share_link` 2. Model checkpoint: `https://drive.google.com/file/d/1R2fvrnVx4ORh3d9Z5n_NHf97X93S78vo/view?usp=share_link` ### Preparation: 1. Clone the repository: ```bash git clone git@github.com:Karbo123/RGBD-Diffusion.git --depth=1 cd RGBD-Diffusion git submodule update --init --recursive ``` 2. Set up the

### Overview: Sequential Diffusion Language Models (SDLM) enhance pre-trained autoregressive language models by adaptively determining generation length and maintaining KV-cache compatibility, achieving high efficiency and throughput. ### Key Features: - **Efficient Generation**: SDLM generates D tokens per step, keeping the longest consecutive n confident tokens (1 ≤ n ≤ D) to save computation. - **High Efficiency**: Achieves nearly 5x faster decoding speed compared to HuggingFace Transformers’ vanilla autoregressive decoding. - **Improved Accuracy**: Matches the accuracy of comparable autoregressive models while operating approximately 2x faster. ### Tech Stack: Python ### Why it Matters: SDLM significantly improves the efficiency and throughput of pre-trained autoregressive language models, making them more suitable for real-time applications and large-scale deployment.

Overview: This project introduces Interleaving Reasoning to the Text-to-Image Generation field, aiming to achieve state-of-the-art performance on benchmark metrics. The method significantly enhances image quality, detail, and aesthetic aspects. Key features: - Interleaving Reasoning for Better Text-to-Image Generation - Improves image quality, detail, and aesthetic aspects - Achieves superior performance across multiple T2I benchmarks Tech stack: Python Why it matters: This approach offers a novel method for enhancing image quality and detail in text-to-image generation, significantly outperforming existing models on various benchmarks. Repository: Osilly/Interleaving-Reasoning-Generation GitHub stars: 100 GitHub description: An early exploration of Interleaving Reasoning for Text-to-Image Generation, aiming to achieve state-of-the-art performance. It also significantly improves image quality, detail, and aesthetic aspects.

### Overview: Seedance 2 Generator is an open-source Next.js SaaS template designed for Seedance 2.0, Seedance 2.5, and Seedance 2 Mini video generation. It comes with Stripe billing, credits, NextAuth, and Prisma integration out of the box, making it a complete, self-contained SaaS boilerplate for generating, editing, and managing high-quality AI videos. ### Key Features: - **Seedance 2.0**: Generates high-quality AI videos using Seedance 2.0 engine. - **Seedance 2.5**: Utilizes Seedance 2.5 engine for more advanced features. - **Seedance 2 Mini**: Offers a fast, affordable model for Seedance 2 Mini engine. - **Stripe Integration**: Enables seamless payment processing. - **NextAuth**: Facilitates user authentication. - **Prisma**: Supports database management. - **Prismarine**: Provides a playground for experimenting with different models. ### Tech Stack: - **Frontend**: Next.js - **Backend**: Node.js (Express.js) - **Database**: Prisma ORM - **Payment Gateway**: Stripe ### Why It Matters: Seedance 2 Generator simplifies the process of creating and managing AI videos, making it easier for developers and content creators to integrate Seedance technology into their projects. The inclusion of Stripe and NextAuth ensures a smooth and secure payment experience, while Prisma and Prismarine offer robust database and model management capabilities, respectively. ### Repository: The repository is hosted on GitHub, where

### Overview: This project is an image tagging and searching application that leverages the power of Llama 3.2 Vision and ChromaDB to provide a seamless image management experience. ### Key features: - Folder Selection and Image Discovery: On first launch, the application prompts users to select a folder by inputting the full path of a folder. It then recursively scans this folder and its subfolders to discover images (supports `png`, `jpg`, `jpeg`, and `webp` formats). - Image Indexing: Initializes a JSON-based index to track images within the selected folder. This index is updated dynamically to reflect new or deleted images. - Intelligent Tagging: Utilizes Llama 3.2 Vision with Ollama to generate descriptive tags for each image. This includes identifying elements/styles, creating a short description, and extracting any text present within the images. - Vector Database Storage: Stores image metadata (path, tags, description, text content) in a ChromaDB vector database for efficient vector search. - Natural Language Search: Enables users to search images using natural language queries. The application performs a hybrid full-text search and vector search on the stored metadata to find relevant images. - User Interface: Provides a user-friendly web interface built with Tailwind CSS and Vue3 for browsing and interacting with images. - Image Grid: Displays images in a responsive grid layout. - Image Modal: On clicking a thumbnail, a modal opens, displaying the image along with its tags, description, and extracted text. - Progress Tracking: Shows real-time progress during batch

Overview: SRUM is a post-training reward fine-tuning method designed for Unified Multimodal Models, leveraging UMMs' inherent understanding capabilities to enhance generative abilities and bridge performance gaps caused by conflicts during the training phase. Key features: - SRUM: A post-training reward fine-tuning method for Unified Multimodal Models. - Fine-grained self-rewarding: Enhances generative abilities. - Unified multimodal models: Bridges performance gaps caused by conflicts during training. Tech stack: Python Why it matters: SRUM demonstrates exceptional generalization across both common positions and world knowledge, showcasing its potential in improving the performance of Unified Multimodal Models.

### Overview: Self-hostable RAG platform designed for document ingestion, embedding, and vector search. Ideal for developers and content creators who need a robust and scalable self-hosted solution for semantic, keyword, and hybrid search. ### Key Features: - **Document Ingestion**: Supports PDF, DOCX, PPTX, HTML, Markdown, and images via Docling. - **Embedding Providers**: Compatible with OpenAI, OpenAI-compatible gateways, Cohere, and Voyage. - **Embedding Presets**: Save named provider/model configs once, reuse across collections. - **Turbopuffer Search**: Retrieves vectors, chunk text, metadata filters, and keyword search via Turbopuffer. - **Namespace Isolation**: Each collection maps to a Turbopuffer namespace for scoped writes, exports, truncation, and deletion. - **Reranking**: Cohere reranking for improved result relevance. - **Multi-Collection Queries**: Search across collections in a single request. - **Generated Chat**: Stateless backend-grounded playground chat with streaming and citations. - **Batch Operations**: Bulk upload, delete, status checks, and queries. - **S3/R2 Connector**: Mirror bucket prefixes with manual or scheduled sync. - **Status Polling**: REST endpoints for document and batch processing status. - **Auth, Audit, Scopes**: Admin accounts, session cookies, scoped `bigrag_sk_…` API keys, and full audit/access logs. - **Metadata Controls**: Per-collection metadata schemas, file validation, and content-hash deduplication at ingest