Projects

### Overview: This project is an official PyTorch implementation of the AAAI 2024 paper "Latent Space Editing in Transformer-based Flow Matching". It focuses on latent space editing techniques using Transformer-based flow matching, aiming to enhance the flexibility and control of flow manipulation. ### Key features: - **Adding Sequentially**: Allows adding elements in a sequential manner. - **Replacing Elements**: Enables the replacement of specific elements in a flow. - **Rescaling**: Facilitates the scaling of flow elements to maintain consistency. ### Tech stack: Python ### Why it matters: This project provides a powerful toolset for flow manipulation and editing, leveraging the capabilities of Transformer-based flow matching to achieve high-level control over visual flows. ### Repository: dongzhuoyao/uspace ### Primary language: Python ### GitHub stars: 53 ### GitHub description: An official PyTorch implementation of AAAI 2024 paper "Latent Space Editing in Transformer-based Flow Matching"

Overview: AI Image Generator is a desktop application designed for creators to quickly produce high-quality artwork, photorealistic renders, and creative concepts using simple text prompts. Key features: - Text-to-Image (txt2img) - Image-to-Image (img2img) - Image Upscaling - Inpainting & Outpainting - Style Blending - Smart Prompt Engine - Model & LoRA Support - Full Creative Control - ControlNet Integration - Hardware Accelerated Rendering - 100% Private & Offline - Gallery Manager - Installation Tech stack: Python, PyTorch, Stable Diffusion, ControlNet, NVIDIA GPU Why it matters: AI Image Generator simplifies the process of creating stunning visuals, making it accessible for a wide range of creators and professionals without the need for complex coding or command-line setups.

### Overview: vLLM Qwen 3.6-27B (AWQ-INT4) with DFlash speculative decoding on AMD Strix Halo, featuring a 256K context, OpenAI compatibility, and Docker support. This model achieves 24.8 t/s single-stream performance, making it highly cost-effective compared to DGX Spark FP8+DFlash+MTP. ### Key features: - **27B Parameter Model**: High-performance model with 27 billion parameters. - **DFlash Speculative Decoding**: Enhances decoding speed and accuracy. - **256K Context**: Efficient memory usage for improved performance. - **OpenAI Compatibility**: Matches OpenAI's specifications for seamless integration. - **Docker Support**: Easy deployment using Docker containers. ### Tech stack: Python ### Why it matters: This model is designed for developers and researchers looking for high-performance, low-cost alternatives to expensive GPU setups. Its compatibility with Docker makes it ideal for cloud-based research and development environments.

### Overview: InferCrane is an open-source infrastructure designed for the full inference lifecycle, providing deploy, observe, scale, optimize, and safe release of self-hosted models behind a single endpoint. It supports various cloud providers and models, ensuring compatibility with OpenAI-compatible APIs. ### Key Features: - **Model Deployment**: Deploy open weights into cloud or Kubernetes environments. - **Optimization**: Profile traffic, search for optimal candidates, and keep only qualified wins. - **Agent Sandboxing**: Persistent workspace for isolated agent tasks. - **Open-Weight Support**: Compatible with vLLM, SGLang, and custom OCI models across AWS, GCP, Kubernetes, and RunPod. - **Model APIs**: Show only currently qualified offers, ensuring no false availability promises. - **BYOC Deployments**: Adopt open weights into existing infrastructure without transferring ownership. ### Tech Stack: Go ### Why It Matters: InferCrane simplifies the process of deploying and optimizing models, ensuring that only the best models are used and that the deployment is secure and efficient. This approach helps in maintaining a stable and reliable inference endpoint, making it easier to manage and scale models effectively.

### Overview: ComfyUI-StableAudioX is a powerful audio generation extension for ComfyUI, integrating AudioX models for high-quality audio synthesis from text and video inputs. It offers features like text-to-audio, text-to-music, video-to-audio, enhanced conditioning, and professional audio processing. ### Key Features: - **Text to Audio**: Generate high-quality audio from text descriptions. - **Text to Music**: Create musical compositions with style, tempo, and mood controls. - **Video to Audio**: Extract and generate audio from video content. - **Enhanced Conditioning**: Separate CFG scales, conditioning weights, negative prompting, and prompt enhancement. - **Professional Audio Processing**: Volume control with LUFS normalization, limiting, and precise gain staging. - **Video Processing**: Mute videos and combine with generated audio. ### Tech Stack: Python ### Why It Matters: This extension enhances the audio generation capabilities of ComfyUI, making it suitable for creating high-quality audio from various inputs, including text and video, with advanced features for better control and processing.

### Overview: WeChat iLink Multimodal Bot This project is a practical and extensible foundation for building AI-powered WeChat assistants. It integrates WeChat with Qwen models, supports multimodal interaction, function calling, and local RAG, making it suitable for various applications like personal assistants, knowledge bots, customer-service workflows, and automation agents. ### Key Features: - **WeChat Integration**: Supports QR-code login and text, image, voice, file, and image replies. - **Multimodal AI**: Enables chat, intent recognition, image understanding, image generation, ASR, and TTS. - **Function Calling**: Provides validated tool schemas, multi-round execution, and structured results. - **Parallel Tools**: Runs independent calls concurrently on Java 21 virtual threads. - **Reusable Skills**: Combines multiple tools into deterministic workflows. - **Local RAG**: Uses a lightweight JSON knowledge base for keyword retrieval. - **Live Information**: Offers weather, web-connected news, translation, and calculations. - **Layered Routing**: Predictable routing through Skill → RAG → LLM/Tool. ### Tech Stack: Java This project is built using Java 21, Spring Boot 4.1.0, Node.js 18+, and WeChat iLink. ### Why It Matters: This project is more than a chat demo; it's a foundational tool for building AI-powered WeChat assistants. It provides a practical and extensible framework for integrating WeChat with Qwen models, handling multimodal interaction, function calling, and local RAG

Overview: This project provides official PyTorch code for out-of-distribution detection using denoising diffusion models (DDPMs) and latent diffusion models (LDMs). The code supports both 2D and 3D data and is based on recent research in computer vision. Key features: - DDPMs for 2D/3D data - LDMs for higher dimensional data - Support for FashionMNIST, SVHN, CIFAR10, CelebA datasets - Data processing and model training using Python - Tensorboard logging for experiment tracking Tech stack: Python Why it matters: This project is essential for researchers and developers looking to improve unsupervised out-of-distribution detection in computer vision applications, leveraging cutting-edge diffusion models for more accurate and efficient detection. Repository: marksgraham/ddpm-ood GitHub stars: 53

Overview: Tech Interview GPT is a speech-to-text mock tech interview powered by ChatGPT, designed to provide in-depth feedback and customizable questions based on the user's development field, work experience, and spoken language. Key features: - Interview Feedback powered by ChatGPT - Customizable based on development field, work experience, and spoken language - Record and edit answer using speech recognition - Automatically save interview history for user to see previous interviews and retry on request - Responsive design for various devices Tech stack: Vite, React, Tailwind CSS Why it matters: Tech Interview GPT offers a comprehensive and personalized tech interview experience, making it an invaluable tool for developers looking to improve their skills and showcase their abilities. Repository: moonkorea00/tech-interview-GPT Primary language: TypeScript GitHub stars: 53 GitHub description: Speech to text mock tech interview powered by ChatGPT README excerpt: <br> <p align="center"> <a href="https://tech-interview-gpt.vercel.app/" target="_blank" rel="noopener"> <img width="539" alt="tech-interview-gpt" src="https://github.com/moonkorea00/moonkorea/assets/78708082/1a8284ef-79c3-4879-854c-00da859dc6d6"> </a> </p> <p align="center"> Speech to text <b>mock tech interview

Overview: This repository provides a beginner's guide to ChromaDB's Python client, covering topics such as adding data, querying collections, updating and deleting data, and using different embedding functions. Each topic is accompanied by detailed READMEs and Python scripts for practical understanding. Key features: - Adding data - Querying collections - Updating data - Deleting data - Using embedding functions Tech stack: Python Why it matters: This repository offers a comprehensive guide for beginners to ChromaDB, making it easier to understand and use the library effectively.

Overview: NeuronDB is a PostgreSQL extension designed for vector similarity search, k-Nearest Neighbors (kNN), embeddings, machine learning, and hybrid full-text + vector retrieval. It allows these functionalities to be performed directly within the PostgreSQL database, eliminating the need for separate, synchronized databases. Key features: - Vector types: `vector`, `vectorp`, `vecmap`, `vgraph`, `rtext`, `halfvec`, `binaryvec`, `sparsevec` (8 types) - Index access methods: HNSW and IVF only, PQ and OPQ for quantization, not separate index types - Distance metrics: L2, cosine, inner product, L1, Hamming, Jaccard, and others - ML: 25+ algorithm families (train/predict/evaluate): linear regression, XGBoost, LightGBM, CatBoost, K-Means, etc. - SQL: ~650+ functions and operators (vector, ML, embeddings, RAG, indexing) - GPU: CUDA, ROCm, Metal (distance and search; index build is CPU only) - Background workers: neuranq, neuranmon, neurandefrag, neuranllm Tech stack: C Why it matters: NeuronDB integrates vector similarity search, kNN, embeddings, machine learning, and hybrid full-text + vector retrieval directly into PostgreSQL, eliminating the need for separate, synchronized databases and providing significant performance improvements over standard PostgreSQL extensions. Repository: neurondb/neurondb GitHub stars:

### Overview: TaskNet Studio is a developer-friendly environment for fine-tuning ModernBERT and orchestrating multi-task learning pipelines. It offers a coherent, opinionated, and extensible canvas for researchers and teams to work on NLP tasks without losing track of the project. ### Key Features: - **Composable Task Definitions**: Declare tasks in plain configuration files, mix and match them across experiments without touching Python source. - **First-class ModernBERT Support**: Native handling of rotary embeddings, unpadded sequences, and graceful length generalization. - **Multi-Task Learning Strategies**: Joint training, gradient surgery, uncertainty weighting, and task-sampling schedules, all selectable with a single keyword. ### Tech Stack: HTML ### Why It Matters: TaskNet Studio treats multi-task learning as a garden, not a pipeline. Tasks are planted alongside one another, sharing the same soil (the ModernBERT backbone), drawing on the same water (the tokenizer and optimizer), and yet each flowering with its own head, its own loss, and its own metrics. The result: better parameter efficiency, improved generalization on low-resource tasks, and a codebase that reads more like a recipe book than a labyrinth of scripts. Repository: petkyletampu28-a11y/tasknet-multitask GitHub Stars: 53 GitHub Description: Modern Multi-Task NLP Fine-Tuning Framework with ModernBERT 2026

### Overview: Point Cloud Diffusion Models for Automatic Implant Generation is a PyTorch implementation of a novel approach for generating patient-specific implants using 3D point cloud diffusion models and voxelization networks. The method aims to automate the implant generation process by proposing an ensemble of different implants per defect, improving upon existing approaches that often result in a loss of detail or contextual information. ### Key features: - **Point Cloud Diffusion Models**: A combination of 3D point cloud diffusion models and voxelization networks for generating high-quality implants. - **Voxelization Networks**: A network used to generate implants from 3D point cloud data. - **Ensemble of Implants**: The method proposes an ensemble of different implants per defect, allowing physicians to choose the most suitable one. ### Tech stack: - Python ### Why it matters: This method addresses the tedious and manual process of designing implants by leveraging the recent success of Diffusion Probabilistic Models. By proposing an ensemble of different implants, it aims to improve the quality and accuracy of implant generation, making it a valuable tool in the field of medical imaging and 3D printing.

Overview: This project is an Artificial Intelligence/Machine Learning course at the University of Central Florida, covering topics such as supervised learning, unsupervised learning, reinforcement learning, linear regression, loss functions, gradient descent, and neural network anatomy. Key features: - Slides for lectures on artificial intelligence, machine learning, and deep learning. - Slides for lectures on supervised, unsupervised, and reinforcement learning. - Slides for a deep dive into fundamental machine learning concepts. - Slides for a detailed explanation of the anatomy of a neural network. - Slides for an experiment on the effect of learning rate on gradient descent. - Slides for visualizing bivariate functions. - Slides for creating density and contour plots. - Slides for creating three-dimensional plots. - Slides for a mathematical derivation of gradient for linear regression. - Slides for a notebook for solving linear regression using stochastic gradient descent. Tech stack: Jupyter Notebook Why it matters: This project provides a comprehensive overview of machine learning concepts, including supervised and unsupervised learning, reinforcement learning, linear regression, loss functions, gradient descent, and neural network anatomy. It also includes practical examples and visualizations to help understand these concepts.

### Overview: Banana Level Up is an interactive web demo powered by Gemini that allows users to create and inspect AI-generated avatar characters from text prompts. The app features a responsive design, multiple viewing angles, and a 360° viewer for characters. ### Key Features: - **Interactive AI Assistant**: Guides users through the Banana Level Up world. - **Natural Language Avatar Generation**: Generates new characters from text descriptions. - **Character Selection**: Choose from 8 unique base characters. - **360° Viewer**: Inspect characters from all angles. - **Multiple Views**: Switch between 4-direction and 8-direction views. - **Bring Your Own Key**: Use Google Gemini or OpenRouter for avatar generation. - **Responsive Design**: Adapts to mobile and desktop screens. ### Tech Stack: JavaScript ### Why it Matters: Banana Level Up provides an engaging and educational experience for users interested in AI and 3D avatar creation. The app's consistent character rendering, style fusion, and automated creative workflows make it a valuable tool for both learning and generating new characters.

### Overview: RatGAN is a Conditional GAN designed for text-to-image synthesis, specifically tailored for the CUB dataset. It achieves an IS score of around 5.0 on the CUB dataset with a 2080 TI. ### Key Features: - Conditional GAN architecture - Text-to-image synthesis - Fine-tuned for CUB dataset - Achieves IS score of 5.0 on CUB ### Tech Stack: Python, PyTorch, easydict, nltk, scikit-image ### Why it Matters: RatGAN provides a powerful tool for generating high-quality images from textual descriptions, making it particularly useful in applications like image captioning and text-to-image synthesis. Its performance on the CUB dataset is notable, achieving a score of 5.0, which is highly competitive in the field of text-to-image synthesis. ### Repository: GitHub - [senmaoy/RAT-GAN](https://github.com/senmaoy/RAT-GAN) ### Primary Language: Python ### GitHub Stars: 53 ### Installation: ```bash git clone https://github.com/senmaoy/RAT-GAN.git cd RAT-GAN/code/ ``` ### Requirements: - Python 3.8 - Pytorch 1.11.0+cu113 - easydict - nltk - scikit-image - A 2080 TI (set nf=32 in *.yaml) or a 3090 32GB (set nf=64

Overview: This project provides a Python library, web application, and command-line tool for transcribing Pinyin to International Phonetic Alphabet (IPA). It includes tone markers attached to the vowel of each syllable. Key features: - Command-line interface - Web application - Python library Tech stack: Python Why it matters: This tool simplifies the process of converting Pinyin, a phonetic representation of Chinese, into IPA, making it easier for developers and learners to work with Chinese phonetics.

Overview: This repository showcases the fine-tuning process of Meta's Llama 2 & 3 models on the SQuAD v2 task, aiming to explore the possibility of combining the reasoning abilities of large foundation models with the specialization capabilities of encoder models. Key features: - Fine-tuning of Meta's Llama 2 & 3 models on the SQuAD v2 task. - Use of TRL (Transformer Reinforcement Library) for fine-tuning. - Exploration of how to make foundation models more honest about what they cannot answer and to always give answers in a predictable format. Tech stack: Python. Why it matters: This project demonstrates the potential of combining the reasoning abilities of large foundation models with the specialization capabilities of encoder models, showcasing the possibility of creating specialized models on a wide range of tasks without the need for custom training.

### Overview: X-MDPT, or Cross-view Masked Diffusion Prediction Transformers, is a novel approach designed for pose-guided human image generation. This framework demonstrates exceptional scalability and performance, significantly improving metrics like FID, SSIM, and LPIPS as model size increases. It surpasses state-of-the-art methods on the DeepFashion dataset, achieving an FID of 7.42 with a 33MB model, which is $11\times$ smaller than the prior most efficient Unet latent diffusion approach. The framework is efficient, with a compact model achieving $5.43\times$ faster inference speed compared to SOTA pixel-based diffusion PIDM. ### Key Features: - **Cross-view Masked Diffusion Transformers**: A novel framework for pose-guided human image generation. - **Exceptional Scalability and Performance**: Demonstrates improved metrics like FID, SSIM, and LPIPS as model size increases. - **Compact Model**: Achieves an FID of 7.42 with a 33MB model, surpassing state-of-the-art approaches with $11\times$ fewer parameters. - **Efficient Training and Inference**: Achieves $5.43\times$ faster inference speed compared to SOTA pixel-based diffusion PIDM. ### Tech Stack: - Python ### Why it Matters: X-MDPT stands out due to its compact model size and efficient training and inference, making it a significant advancement in the field of image synthesis and pose-guided image generation.

Overview: MiniASR is a lightweight, simple, and fast end-to-end automatic speech recognition toolkit designed for easy customization and efficient deployment. It utilizes self-supervised pre-trained models and minimal fine-tuning for minimal code and minimal modification. Key features: - Minimal Code 📄 - Minimal Training ⏱ - Simple and Flexible ⚙️ - Colab Compatible 🧪 Tech stack: Jupyter Notebook Why it matters: - Fast fine-tuning for real-world applications.

### Overview: SPEC is an official implementation of the paper "Synthesize, Diagnose, and Optimize: Towards Fine-Grained Vision-Language Understanding." It is designed to evaluate the fine-grained vision-language understanding capabilities of visual language models. The repository includes a comprehensive dataset and code for benchmarking and fine-tuning models. ### Key features: - **Fine-Grained Vision-Language Understanding**: SPEC focuses on evaluating models' ability to understand fine-grained concepts in both images and text. - **Six Subsets**: The dataset consists of six distinct subsets that cover different dimensions of understanding, including Size, Position, Existence, and Count. - **Official Implementation**: The repository provides a complete implementation of the paper, including code, checkpoints, and datasets. ### Tech stack: - **Primary Language**: Jupyter Notebook - **GitHub Stars**: 53 ### Why it matters: SPEC is a crucial benchmark for evaluating the fine-grained vision-language understanding capabilities of models. It provides a standardized way to measure and compare the performance of different models, making it an essential resource for researchers and developers in the field of computer vision and natural language processing.