Projects

Overview: GraphEdit is a project that utilizes large language models to learn graph structure, focusing on the application of graph neural networks (GNNs) and language models in graph structure learning. Key features: - Implementing various graph neural networks (GNNs) including GCN, MLP, and RevGAT. - Utilizing datasets like Citeseer, Cora, and Pubmed for training and evaluation. - Providing a main script for training edge predictors and applying graph structure learning. - Offering a set of utilities for data preprocessing, evaluation, and model adaptation. Tech stack: Python Why it matters: GraphEdit demonstrates the integration of large language models with graph neural networks, showcasing the potential for advanced graph structure learning through the application of language models.

Overview: A simple ChatGPT AI tutorial that guides developers on how to generate images, text, and code using ChatGPT, including its limitations. Key features: - Generating images based on input prompts - Generating text descriptions for GitHub profiles and other applications - Generating code in seconds for simple tasks - Limitations on data usage and potential for false information Tech stack: Python Why it matters: This tutorial provides a practical guide for developers to leverage ChatGPT for various tasks, including image generation, text descriptions, and code creation, while highlighting the importance of understanding the AI's capabilities and limitations.

Overview: RAGU is a modular framework that turns raw text into a knowledge graph and answers queries over it, designed to be easily integrated into existing projects without rewrites. Key features: - **Retrieval that fits the question**: Local graph neighborhoods, global community summaries, naive chunk-vector RAG, and a mix engine with query planning and optional token-by-token streaming. - **Reason over structure, not just chunks**: Extraction yields `Entity` and `Relation` typed by the NEREL ontology, then Leiden community detection clusters the graph. - **Async by design**: Batched embeddings, batched vector queries, and rate-limited LLM clients throughout. Tech stack: Python Why it matters: RAGU simplifies the process of turning text into a knowledge graph and answering queries over it, making it easier to integrate into existing projects without rewrites and providing a more comprehensive way to retrieve information compared to traditional nearest-neighbor lookups.

### Overview: Flovart is an open-source agent-native visual production workspace that allows humans and coding agents to edit the same live canvas in real-time. It combines the DaVinci Resolve Studio 21.1 host layer with an experimental Agent creative tool, enabling seamless collaboration and dynamic content creation. ### Key features: - **Agent Native Integration**: Flovart seamlessly integrates with coding agents, allowing them to work alongside humans on the same live canvas. - **Local-First Workflow**: The workflow is designed to be local-first, ensuring that all edits and processes are performed on the user's local machine. - **BYOK Image/Video Models**: Users can use their own image and video models without any external dependencies, making the workflow flexible and accessible. ### Tech Stack: - TypeScript ### Why it Matters: Flovart is designed to streamline the creative process by combining the strengths of human creativity and machine intelligence. By allowing agents to understand the context and collaborate with humans, Flovart aims to enhance the quality and efficiency of visual production workflows, making it a valuable tool for both artists and developers. Repository: avabbbb/Flovart Primary language: TypeScript GitHub stars: 144 GitHub description: Open-source agent-native visual production workspace where humans and coding agents edit the same live canvas — local-first, BYOK image/video models.

Overview: Dafthunk is a workflow execution platform built on Cloudflare's infrastructure, enabling visual workflow automation with AI-powered nodes, serverless execution, live monitoring, persistent storage, and triggers/integrations. Key features: - Visual Workflow Editor - AI-Powered Nodes - Serverless Execution - Live Monitoring - Persistent Storage - Triggers & Integrations Tech stack: TypeScript, React Flow, Hono, Cloudflare Workers, Cloudflare D1, R2, AI, Drizzle ORM, Zod, Tailwind CSS, Shadcn/UI. Why it matters: Dafthunk provides a powerful, visual workflow automation platform with AI capabilities, serverless execution, and persistent storage, making it ideal for developers looking to streamline their workflows and integrate AI into their applications.

### Overview: Con*ffusion* is an official implementation of the "Con*ffusion*: Confidence Intervals for Diffusion Models" paper. This project provides confidence intervals around each pixel in the generated image, ensuring that the true value falls within these bounds with a specified probability. The implementation uses a pre-trained diffusion model to predict these intervals in a single forward pass, making it significantly faster than traditional methods. ### Key Features: - **Confidence Intervals**: Generates lower and upper bounds around each pixel in the generated image. - **Context Visualization**: The true pixel value is guaranteed to fall within these bounds with a probability set by the user. - **Inpainting and Super-Resolution**: Supports both inpainting and super-resolution tasks. - **Fast Sampling**: Uses a pre-trained diffusion model to predict interval bounds in a single forward pass, significantly reducing the sampling time. ### Tech Stack: - **Python**: The primary programming language used for this implementation. ### Why It Matters: This project fills a gap in the current state of diffusion models by providing statistical guarantees regarding the generated results, making it suitable for applications where accuracy and reliability are paramount.

### Overview: Whisper To Input is an Android keyboard that utilizes OpenAI's Whisper model for speech-to-text (STT/ASR) functionality. It supports multiple languages including English, Chinese, Japanese, and even mixed languages, making it versatile for users around the world. ### Key Features: - **Speech-to-Text (STT/ASR) with Whisper Model**: Recognizes spoken text and converts it into written language. - **Supports Multiple Languages**: Supports English, Chinese, Japanese, and mixed languages. - **Flexible Backend Services**: Available with OpenAI API, Whisper ASR Webservice, or NVIDIA NIM. - **Configuration**: Allows users to choose between OpenAI API, Whisper ASR Webservice, or NVIDIA NIM, depending on the backend service. ### Tech Stack: - **Kotlin**: The primary programming language used for development. ### Why It Matters: Whisper To Input provides a convenient and efficient way to input text using speech, making it particularly useful for those who prefer speaking over typing. The versatility of the app makes it accessible to a wide range of users, enhancing their productivity and communication skills.

Overview: DiffusionRet is a novel generative text-video retrieval framework presented at ICCV 2023, aiming to address the limitations of discriminative solutions by modeling the correlation between text and video as their joint probability. Key features: - Diffusion-based text-video retrieval framework - Models the retrieval task as a process of gradually generating joint distribution from noise - Addresses the limitations of current discriminative solutions by focusing on maximizing the joint probability p(candidates, query) Tech stack: Python Why it matters: DiffusionRet provides a generative perspective to text-video retrieval, offering a more holistic and data-driven approach to identifying relevant content, especially in the context of out-of-distribution data.

Overview: This project is a local RAG (Relevance-Augmented Generation) research agent built using Langgraph, DeepSeek R1, and Ollama. It acts as a deep researcher, gathering, analyzing, and summarizing information based on user instructions. Key features: - Dynamic search through local documents - Advanced insight extraction using DeepSeek R1 - Real-time web search expansion - Structured report generation Tech stack: Python, Langgraph, DeepSeek R1, Ollama, ChromaDB, Streamlit, Tavily API. Why it matters: This system provides an efficient and adaptive research process, ensuring high-quality and relevant outputs while minimizing unnecessary or low-value data.

### Overview: Stellar Chat is a versatile multi-modal chat application designed for developers. It allows users to create custom agents, generate images, utilize visual recognition, and engage in voice interactions. The application seamlessly integrates with local LLMs and commercial models, including OpenAI, Gemini, Perplexity, and Claude, and supports the conversation with uploaded documents and websites. ### Key Features: - **Support for Local Open Source Models**: Integrate and utilize local open source models through the OLLAMA platform. - **Support for Commercial Models**: Easily use commercial models like OpenAI, Gemini, Perplexity, and Claude. - **Visual Recognition**: Utilize the powerful visual recognition capabilities of the GPT-4-Vision model and Gemini Vision. - **Support for TTS & STT**: Enable text-to-speech (TTS) and speech-to-text (STT) functionalities within the application. - **Text to Image Generation**: Generate images from text inputs using advanced models such as Stable Diffusion and DALL-E 3. - **Multimodal Chat**: Analyze text, images, and voice inputs to provide a comprehensive conversational experience. - **Prompt Store**: Store and manage prompts for future use. - **Custom Agent Creation (GPTs)**: Create custom agents using GPT models. - **Message and Conversation Search**: Search for messages and conversations efficiently. - **Custom Action Creation for App Integration**: Create custom actions for app integration. - **Multi-Agent Chat Capability**: Enable multi-agent chat functionality. ### Tech Stack: C# ### Why It Matters: Stellar

### Overview: This educational implementation of the Discrete Flow Matching paper provides an educational exploration of discrete image generation, focusing on the case of discrete image generation. The notebook (`notebook/discretefm.ipynb`) is designed to complement the original paper and includes educational notes with formulas from the paper. ### Key features: - Discrete image generation - Educational implementation of Discrete Flow Matching - Exploring the theoretical framework of the paper ### Tech stack: Jupyter Notebook ### Why it matters: This notebook serves as an educational resource for understanding the Discrete Flow Matching paper, providing a practical way to explore and learn from the theoretical concepts presented in the research.

### Overview: InfiniCube is a cutting-edge project that introduces a dynamic 3D driving scene generation pipeline. This innovative system leverages world-guided video models to create unbounded and controllable driving scenes, aiming to enhance the realism and variety of virtual driving environments. ### Key Features: - **Dynamic Scene Generation**: InfiniCube generates realistic driving scenes with a focus on dynamic elements such as weather conditions and traffic patterns. - **World-Guided Video Models**: Utilizes models that are conditioned on real-world maps to ensure the generated scenes are contextually relevant and realistic. - **Unbounded Scene Creation**: The system can create scenes that are not limited by traditional constraints, providing a wide range of driving scenarios. - **Controllable Scenarios**: Users can easily adjust parameters to create scenes that match specific requirements, such as day or night conditions, weather types, and traffic densities. ### Tech Stack: - **Python**: The primary programming language used for developing and training the models. - **Waymo Open Dataset**: The dataset used for generating the voxel world and video generation. ### Why It Matters: InfiniCube offers a significant advancement in the field of virtual driving simulation by providing highly realistic and controllable driving scenes. This technology can be crucial for research, training, and development in the automotive and gaming industries, where the ability to create diverse and contextually relevant driving scenarios is essential. ### Repository: - **Repository**: `nv-tlabs/InfiniCube` - **GitHub Stars**: 143 - **GitHub Description**: [ICCV

Overview: This repository contains the PyTorch implementation of the paper "WDM: 3D Wavelet Diffusion Models for High-Resolution Medical Image Synthesis," authored by Paul Friedrich, Julia Wolleb, Florentin Bieder, Alicia Durrer, and Philippe C. Cattin. Key features: - Wavelet-based medical image synthesis framework - Scalable diffusion models for high-resolution image synthesis - Capable of generating images at resolutions up to 256 x 256 x 256 - State-of-the-art image fidelity and sample diversity scores compared to GANs, Diffusion Models, and Latent Diffusion Models Tech stack: Python Why it matters: This implementation is essential for researchers and developers looking to enhance medical image synthesis and generation, offering a practical solution for high-resolution image synthesis tasks in the medical field. Repository: pfriedri/wdm-3d GitHub stars: 143

Overview: Zoltar is a comprehensive library designed to serve TensorFlow, XGBoost, and scikit-learn models in production environments. Key features: - TensorFlow model serving - XGBoost model serving - Scikit-learn model serving Tech stack: Java Why it matters: Zoltar simplifies the deployment and management of machine learning models, making it easier for developers to integrate and deploy models in production environments without the need for specialized knowledge.

### Overview: BELM: Bidirectional Explicit Linear Multi-step Sampler for Exact Inversion in Diffusion Models ### Key Features: - **Exact Inversion**: Implements a method to find the initial noise corresponding to a sample in diffusion models. - **Bidirectional Explicit Constraint**: Derives from a bidirectional explicit constraint to ensure mathematically exact inversion. - **Optimal Sampler**: Proposes the Optimal BELM (O-BELM) sampler through minimizing Local Truncation Error (LTE). ### Tech Stack: - Python ### Why It Matters: BELM provides a theoretical foundation for exact inversion in diffusion models, ensuring high-quality sampling without heuristic methods.

### Overview: The project aims to evaluate and compare the performance of different Large Language Models (LLMs) across various use cases, focusing on their ability to handle specific tasks within an integrated pipeline. This approach is crucial for ensuring that the chosen LLMs are suitable for their intended roles in complex systems. ### Key Features: - **Agentic RAG for Deep Document Analysis**: Evaluates LLMs based on their performance in deep document analysis tasks. - **Multi-Agent System for Scientific Research**: Measures the effectiveness of LLMs in scientific research through various agents and scenarios. - **Vision with Reasoning of Poorly Scanned Forms**: Tests LLMs on vision tasks involving poorly scanned forms. ### Tech Stack: Jupyter Notebook ### Why It Matters: By evaluating LLMs in real-world scenarios and using a comprehensive evaluation framework, this project ensures that the chosen LLMs are optimal for their specific roles in complex systems, thereby improving overall system performance and reliability.

### Overview: Legend of Elya is a Zelda-style dungeon game featuring a real-time 6.36M-parameter ternary transformer powered by a nano-GPT on the VR4300 MIPS III CPU. The game runs at 1.23 tok/s scalar and 2.19 tok/s on the RSP overlay, measured under ares, but has not been verified on silicon. ### Key Features: - **Sophia Elya AI NPC**: AI dialog triggered by pressing A near the character. - **8-layer Transformer**: Full 8-layer transformer running on the MIPS R4300i CPU. - **Real Matrix Multiply, Real Softmax, Real Attention**: All operations are performed on the 93.75 MHz CPU from 1996. - **Live Token Counter**: Tokens appear character-by-character with a live tok/s counter. - **32 Prompts**: Covers identity, Elya lore, RustChain, hardware trivia. - **On-Axis Inference**: Inference math is performed on the MIPS R4300i FPU. - **Ternary Quantization**: Parameters are quantized to 2-bit weights + float16 block scales. - **Ternary Weight File**: 1,984 KB on cartridge ROM, 2,031,628 bytes. - **1.23 tok/s Scalar, 2.19 tok/s RSP Overlay**: Measured under ares, never on silicon. - **32 KB KV Cache**:

Overview: SimGRAG is an innovative approach that leverages similar subgraphs to enhance knowledge graph-driven retrieval-augmented generation, making it a significant contribution to the field of natural language processing. Key features: - Supports plug-and-play usability with LLMs, embeddings, and vector databases. - Addresses various tasks such as question answering and fact verification. - Built on open-source solutions for Ollama, Nomic Embedding Model, and Milvus. Tech stack: Python Why it matters: SimGRAG offers a versatile solution that can be easily integrated with existing large language models, embeddings, and vector databases, making it a valuable tool for researchers and developers looking to improve their knowledge graph-driven retrieval-augmented generation capabilities. Repository: YZ-Cai/SimGRAG GitHub stars: 142 GitHub description: Official code of the ACL 2025 paper "SimGRAG: Leveraging Similar Subgraphs for Knowledge Graphs Driven Retrieval-Augmented Generation"

Overview: This repository aggregates articles and code from the Decoding ML team, providing hands-on projects for developers to test and learn from. Key features: - Code snippets for various machine learning and artificial intelligence domains - Articles tagged by main topics like large language models, computer vision, and more - Open-source projects under the MIT license Tech stack: Jupyter Notebook Why it matters: This repository serves as a comprehensive resource for developers looking to apply machine learning concepts and technologies in practical scenarios.

### Overview: This project is a comprehensive primer on probabilistic machine learning (PML) tailored specifically for finance and investing applications. The book is designed to provide a solid foundation in the essential concepts of PML and how to apply them effectively to financial and investment problems, using Python code. ### Key Features: - **PML Primer**: A detailed introduction to probabilistic machine learning concepts. - **Finance and Investing Applications**: Practical examples and case studies demonstrating how PML can be used in financial modeling and investment strategies. - **Python Code Implementation**: Hands-on Python code to illustrate the concepts discussed in the book. - **Educational Resources**: Access to the entire book for free, along with additional resources from O'Reilly Media's learning platform. ### Tech Stack: Python with Jupyter Notebook. ### Why It Matters: This book serves as a valuable resource for graduate students and professionals in finance and economics, as well as MBA students in India. It addresses the critical need for a deeper understanding of probabilistic machine learning in financial applications, providing a bridge between theory and practical application. The book's content is supported by citations from prestigious journals and academic publications, making it a reliable resource for those seeking to advance their knowledge in this field.