Projects

Overview: JDiffusion is a diffusion model library for generating images and videos using Jittor, Jtorch, and Diffusers. It includes examples and usage instructions for various models like Stable Diffusion, Latent Consistency Model (LCM), Control-Net, SDXL, and Instruct-Pix2Pix. Key features: - Stable Diffusion - Latent Consistency Model (LCM) - Control-Net - SDXL - Instruct-Pix2Pix Tech stack: Python Why it matters: JDiffusion simplifies the process of creating high-quality images and videos using advanced diffusion models and cutting-edge deep learning frameworks, making it accessible for developers and researchers alike.

### Overview: MooshieUI is a user-friendly front-end interface designed for beginners to generate images, videos, and music using ComfyUI. It offers a guided workflow that simplifies the process of generating content without requiring extensive knowledge of ComfyUI's node-graphs. ### Key Features: - **ComfyUI Integration**: Seamless integration with ComfyUI for image, video, and music generation. - **NovelAI Backend**: Optional integration with the NovelAI API for additional image generation. - **Desktop App**: Runs as a native app on Windows/Linux and Apple Silicon candidates. - **Browser/Server Mode**: Built-in web server for LAN/Docker-friendly use. - **Svelte/Rust**: Developed using Svelte 5 and Rust for a clean, guided workflow. - **Image Modes**: Supports image generation through ComfyUI and optional image generation via NovelAI. - **Video Generation**: Offers MiniMax H3, model stacks, and timeline-based video generation. - **Music Generation**: Allows for YuE2 song creation and playlist management. - **Prompting Guide**: Provides detailed instructions on how to create prompts effectively. - **Models & Hub**: Supports a variety of ComfyUI models and auto-detection of supported architectures. ### Tech Stack: - **Frontend**: Svelte 5 + Rust - **Backends**: ComfyUI, NovelAI API ### Why It Matters: MooshieUI simplifies the process of generating content using ComfyUI, making it accessible to beginners. Its guided workflow and clean interface make it easier to create

### Overview Sparse inference for transformer-based language models is optimized through the implementation of sparse multiplication and fusing up/down projections in the MLP layers. This project leverages low-rank weight activations based on Deja Vu and Apple's LLM in a Flash. It achieves significant improvements in terms of throughput, memory usage, and inference speed, with a 1.6-1.8x gain in total token throughput and a 26.4% reduction in memory usage. ### Key Features - **Sparse Multiplication**: Efficient computation of sparse tensors. - **Fused Projections**: Combining up/down projections for improved performance. - **Differential Weight Caching**: Optimizes memory usage and inference speed by selectively caching and updating weights. ### Tech Stack - **Python**: The primary programming language used for the implementation. - **Torch**: For CPU-based sparse inference. - **CUDA**: For GPU-based sparse inference. ### Why It Matters - **Performance Boosts**: Achieves a 1.6-1.8x gain in total token throughput and a 26.4% reduction in memory usage. - **Efficiency Improvements**: 6.7× faster cache updates and better cache locality, leading to faster inference times. - **Memory Efficiency**: Reduces memory usage by 26.4%, making it suitable for resource-constrained environments. ### Implementation Details - **Sparse MLP Layer**: The project includes a Sparse MLP layer implementation that fuses up/down projections and uses low-rank weight activations. - **Weight Cache**: Manages

### Overview: IT3D is a text-to-3D generation method that addresses the limitations of existing approaches by explicitly synthesizing multi-view images. This study leverages large diffusion models to generate high-quality images based on coarse 3D models, aiming to improve the realism and detail of 3D models. ### Key Features: - **Explicit View Synthesis**: IT3D generates multi-view images to synthesize views from coarse 3D models. - **Realistic 3D Models**: The method ensures that the generated images are based on realistic 3D models, improving the quality and realism of the final output. - **Discriminator Integration**: A discriminator is integrated to guide the training of 3D models, addressing inconsistencies and variance issues in large diffusion models. ### Tech Stack: Python ### Why It Matters: IT3D offers a significant improvement over existing text-to-3D techniques by providing more realistic and detailed 3D models. The explicit view synthesis and discriminator integration help in overcoming the inherent challenges of large diffusion models, making the generated 3D models more effective and useful in applications such as virtual reality, augmented reality, and more. ### Repository: - **Primary Language**: Python - **GitHub Stars**: 222 - **GitHub Description**: [AAAI'2024] IT3D: Improved Text-to-3D Generation with Explicit View Synthesis - **Repository URL**: [https://github.com/buaacyw/IT3D-text-to-3D](https://github.com/buaacyw/

### Overview: ClovaCall is a large-scale Korean goal-oriented dialog speech corpus for automatic speech recognition of contact centers, introduced at the Interspeech 2020 conference. This dataset includes 112,000 pairs of short sentences and their corresponding spoken utterances in a restaurant reservation domain, providing valuable resources for ASR models in diverse call-based reservation services. ### Key features: - **Raw Dataset**: Contains approximately 112,000 pairs of short sentences and spoken utterances. - **Clean Dataset**: Silence-free version, with 67% of the data being clean. - **Structured JSON Format**: Provides a structured JSON file for easy access and integration. ### Tech stack: Python ### Why it matters: ClovaCall offers a comprehensive dataset for ASR models, particularly useful for contact center applications, enhancing the accuracy and relevance of speech recognition in diverse call-based scenarios. ### Repository: clovaai/ClovaCall ### GitHub stars: 222 ### GitHub description: ClovaCall dataset and Pytorch LAS baseline code (Interspeech 2020)

Overview: RAG Arena is an open-source Next.js project designed to evaluate and showcase various Retrieval-Augmented Generation (RAG) chatbot methods. Users can interact with a real-time leaderboard that tracks their performance, with responses unblurred to reveal the RAG techniques used. Key features: - Real-time leaderboard for users to track performance and see how their choices impact the chatbot's responses. - Ability to choose and interact with different RAG chatbot methods through a dynamic retriever, allowing for varied responses based on user input. - Supabase integration for database operations and real-time data management. - OpenAI integration for enhanced chat completions using SupabaseVectorStore for document matching. Tech stack: TypeScript, Next.js, LangChain, Supabase, OpenAI, Neo4j, Poetry, Flask. Why it matters: This project provides a comprehensive platform for developers and researchers to evaluate and compare different RAG chatbot methods, offering insights into their effectiveness and learning capabilities through user feedback and performance tracking. Repository: firecrawl/rag-arena GitHub stars: 222 GitHub description: Open-source RAG evaluation through users' feedback.

### Overview: This repository contains the Automatic Speech Recognition project, focusing on the DeepSpeech and Seq2Seq models using TensorFlow. The project aims to provide a ready-to-use pipeline for automatic speech recognition, with TensorFlow 2.0 support and easy model monitoring. ### Key Features: - 🎧 Automatic Speech Recognition: DeepSpeech & Seq2Seq (TensorFlow) - 📊 Ready-to-use pipeline with pre-trained model - 🚀 Eager `TensorFlow 2.0` model - 🚀 Monitor model weights, activations, and gradients ### Tech Stack: - Python ### Why it Matters: The project provides a simple and effective solution for automatic speech recognition, leveraging the power of TensorFlow and pre-trained models to achieve high accuracy. It is designed to be easily adjustable, allowing users to explore various enhancements such as feature extraction, data augmentation, and language model integration. The provided code examples demonstrate how to fine-tune the pipeline for better performance and evaluate its effectiveness on the LibriSpeech benchmark. ### Repository: - **Repository**: rolczynski/Automatic-Speech-Recognition - **GitHub Stars**: 222 - **Primary Language**: Python - **GitHub Description**: 🎧 Automatic Speech Recognition: DeepSpeech & Seq2Seq (TensorFlow) - **Installation**: Use `pip install automatic-speech-recognition` or clone the repository and create a new environment with `conda` for GPU support. ### References: - **Fundamental Repositories**: - Baidu: [DeepSpeech2 - A P

Overview: This project profile is for developers who want to learn robotics and ROS 2 by doing. It focuses on creating a 3D printed robot arm using ROS 2, Arduino, MoveIt!, and Amazon Alexa. Key features: - 3D printed robot arm - ROS 2 and Arduino powered - Controlled via MoveIt! 2 and Amazon Alexa - Developed in the Udemy course "Robotics and ROS 2 - Learn by Doing! Manipulators" Tech stack: Python Why it matters: This project provides hands-on experience in robotics, robotics software development, and the integration of voice control systems, making it valuable for developers looking to enhance their skills in these areas. Repository: AntoBrandi/Robotics-and-ROS-2-Learn-by-Doing-Manipulators GitHub stars: 221 GitHub description: The course material for the Udemy course "Robotics and ROS 2 - Learn by Doing! Manipulators" by the same author.

### Overview: This repository is a curated collection of papers related to artificial intelligence (AI) and large language models (LLM) for education. It serves as a valuable resource for researchers, educators, and students interested in the latest advancements in AI and LLM applications within the educational domain. ### Key Features: - **Surveys, Analyses & Perspectives**: Comprehensive surveys, analyses, and perspectives on AI and LLM in education. - **Teaching & Learning Lifecycle**: Papers categorized by specific tasks such as tutoring, material preparation, learner modeling, and assessment. - **Application Domains**: Papers focused on specific application domains within education, such as personalized learning, intelligent tutoring systems, and goal-oriented learning. - **Datasets, Benchmarks & Toolkits**: Resources for datasets, benchmarks, and toolkits related to AI and LLM in education. ### Tech Stack: Jupyter Notebook ### Why it Matters: This curated catalog provides a comprehensive overview of the latest research and practical applications of AI and LLM in education, making it an essential resource for anyone interested in advancing the field.

Overview: This is a high-performance llama.cpp optimized for AMD Strix Halo (gfx1151), featuring Flash-Attention and MoE-prefill fixes, bundled with a current Mesa driver. It includes a performance-enhancing payload and is portable across various environments. Key features: - Flash-Attention and MoE-prefill fixes - Current Mesa driver bundled - Vulkan and HIP support - Portable directory, Docker, and distrobox Tech stack: Python Why it matters: This optimized llama.cpp significantly improves performance for long-context, quantized-KV workloads on AMD Strix Halo, offering a faster quantized KV cache alternative to the penalty of traditional quantization. Repository: Nathanw1014/strix-halo-llamacpp Primary language: Python GitHub stars: 220

Overview: This project is a series of episodes that explore the most valuable insights from groundbreaking books in AI, Machine Learning, and Data Science. Each episode simplifies complex theories into practical knowledge, making advanced topics more accessible and actionable. Key features: - Discoveries from groundbreaking books in AI, Machine Learning, and Data Science - Simplifies complex theories into practical knowledge - Helps accelerate your learning journey - Transforms advanced topics into actionable insights Tech stack: Python, Jupyter Notebook, Markdown Why it matters: This project is essential for developers looking to enhance their understanding of AI, Machine Learning, and Data Science. By providing practical insights from top-tier books, it helps make these complex topics more accessible and actionable, ultimately improving their learning journey.

Overview: This project collects and compares GPU benchmarks from various cloud providers to determine fixed per token costs for language model inference, aiding users in making informed decisions on cost-effective LLM options and efficient GPU selections. Key features: - GPU benchmark collection and comparison - Fixed per token cost calculation for LLMs - Comprehensive LLM price comparison tool - GPU benchmark table Tech stack: TypeScript Why it matters: This project helps users efficiently select cost-effective LLM options and affordable language models by providing insights into AI model pricing, facilitating GPU inference, and benchmarking.

Overview: FLASK is a task-agnostic evaluation protocol for language models, focusing on fine-grained instance-wise skill set evaluation using alignment skill sets. Key features: - Fine-grained language model evaluation - Alignment skill sets as metrics - Supports various LLMs and OpenAI API keys Tech stack: Python Why it matters: FLASK provides a standardized way to evaluate language models, ensuring consistent and accurate performance metrics across different models and domains, making it valuable for researchers and developers in the field of natural language processing.

### Overview Aesthetic Code Generation with Decoupled Cross-Modal Rewards (AesCode) is a project that transforms prompts into editable, verifiable HTML/CSS infographics. It leverages image generators as aesthetic guidance, ensuring that the generated code remains structured, editable, and verifiable. The project introduces a setting where an image generated from the same prompt serves as aesthetic guidance for the code model, separating semantic requirements from visual cues through graph-structured supervision and decoupled cross-modal rewards. ### Key Features - **Image as Aesthetic Guidance**: The project uses image generators as a source of aesthetic guidance, ensuring the generated code remains structured, editable, and verifiable. - **Graph-Structured Supervision**: AesCode trains the model to separate semantic requirements from visual cues through graph-structured supervision and decoupled cross-modal rewards. - **Training Pipeline**: The project learns from the reference through three steps: representing the target as a design graph, routing each property to the right reward, and normalizing rewards before combining them. ### Tech Stack - **Primary Language**: Python ### Why It Matters AesCode addresses the limitations of off-the-shelf vision-language models by explicitly training them to separate semantic requirements from visual cues. This approach ensures that the generated code remains structured, editable, and verifiable, making it an attractive option for creating information-rich visual artifacts.

Overview: This project provides a Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) implementation using Theano, focusing on Connectionist Temporal Classification (CTC). It includes a toy training example. Key features: - Recurrent Neural Network (RNN) - Long Short-Term Memory (LSTM) - Connectionist Temporal Classification (CTC) - Toy training examples Tech stack: Python Why it matters: This project is useful for developers looking to implement and experiment with RNNs and LSTMs for sequence learning tasks, such as text classification, speech recognition, and more. Repository: rakeshvar/rnn_ctc Primary language: Python GitHub stars: 221 GitHub description: Recurrent Neural Network and Long Short Term Memory (LSTM) with Connectionist Temporal Classification implemented in Theano. Includes a Toy training example.

Overview: This repository contains a sample application and TypeScript SDK for speech recognition using websockets in JavaScript. It supports the Microsoft Cognitive Services Speech SDK, including the Unified Speech Service, and is deprecated due to the new Speech Service. Key features: - Speech Recognition using WebSockets - TypeScript SDK - Supports Microsoft Cognitive Services Speech SDK, including Unified Speech Service - Deprecated due to the new Speech Service Tech stack: TypeScript Why it matters: This repository provides a practical example of using the Microsoft Speech SDK with websockets, which is useful for developers looking to integrate speech recognition into their web applications.

### Overview: CineScale is a novel tuning-free inference framework designed to enable pretrained video diffusion models to generate high-fidelity 4K videos at resolutions far beyond those encountered during training. This project is significant as it addresses the gap between training and inference by unlocking the ability to generate high-resolution videos without the need for fine-tuning. ### Key Features: - **Tuning-Free Inference**: CineScale allows pretrained models to generate high-resolution videos without the need for fine-tuning, making it suitable for applications requiring high-quality video generation. - **4K Video Generation**: It enables the generation of 4K videos from low-resolution inputs, achieving high perceptual quality and maintaining temporal behavior and semantic structure. - **Open-Source**: The project is open-sourced, allowing researchers and developers to contribute to and build upon the framework. ### Tech Stack: Python ### Why It Matters: CineScale is the first tuning-free inference framework that enables pretrained video diffusion models to generate high-fidelity videos at resolutions far beyond those encountered during training. This breakthrough opens up new possibilities for applications that require high-resolution video generation, such as immersive 4K video experiences in virtual reality and augmented reality applications.

Overview: REST is a retrieval-based speculative decoding method designed to accelerate the generation speed of large language models (LLMs). It utilizes a datastore to retrieve and employ draft tokens, offering a faster alternative to speculative decoding and other methods like blockwise parallel decoding and Medusa. Key features: - Accelerates any pre-existing language model - Uses a datastore for efficient token retrieval - Does not require extra training steps - Functions as a plug-and-play solution Tech stack: C Why it matters: REST is a significant advancement in generating text quickly, making it a valuable tool for developers and researchers looking to improve the performance of their LLMs without the need for additional training. Repository: FasterDecoding/REST GitHub stars: 220 GitHub description: REST: Retrieval-Based Speculative Decoding, NAACL 2024

### Overview: Alpamayo is an open platform designed to accelerate the development of safe, transparent, and reasoning-based autonomous vehicles. It consists of vision-language-action (VLA) models, simulation frameworks, reinforcement learning infrastructure, and physical AI datasets and tools. The VLA models scale from the 10B-parameter Alpamayo 1 Nano and Alpamayo 1.5 Nano to the 34B-parameter Alpamayo 2 Super, each generating driving trajectories alongside CoC reasoning traces that make every decision transparent and auditable. AlpaSim provides closed-loop policy simulation, and AlpaGym enables reinforcement learning on top to learn from edge-case failures across massively parallel GPU simulations. The Physical AI AV Dataset spans 2500+ cities across 25 countries and provides real-world long-tail training data. The CoC Auto-Labeling Pipeline annotates data with meta-actions and reasoning labels automatically. ### Key features: - **Alpamayo 2 Super**: An open 34B reasoning Vision Language Action model for end-to-end autonomous driving, built on the NVIDIA Cosmos 3 backbone. - **AlpaGym**: A closed-loop reinforcement learning framework for training end-to-end autonomous driving policies in simulation. - **Alpamayo-Coc-AutoLabeler**: An open-source pipeline that generates Chain-of-Causation (CoC) reasoning labels for driving clips automatically. ### Tech stack: Python ### Why it matters: - **Open platform**: Model weights, inference code, closed-loop

### Overview: ToolRet is an official code repository for the ACL 2025 conference paper "Retrieval Models Aren’t Tool-Savvy: Benchmarking Tool Retrieval for Large Language Models". This project aims to provide a comprehensive tool retrieval benchmark and a large-scale training dataset to evaluate and optimize information retrieval models specifically designed for tool retrieval tasks. ### Key Features: - A first comprehensive tool retrieval benchmark to systematically evaluate existing information retrieval models on tool retrieval tasks. - A large-scale training dataset to optimize the expertise of information retrieval models on tool retrieval tasks. - The project includes main code for evaluation and training scripts, as well as a dataset and paper for the accepted paper at ACL 2025. ### Tech Stack: JavaScript ### Why It Matters: This project is significant because it provides a standardized benchmark for evaluating and optimizing information retrieval models specifically designed for tool retrieval tasks, which is crucial for the development of large language models in the field of tool learning.