Projects

### Overview: Generative models are a fundamental part of machine learning, used to generate realistic data samples. This project aims to implement generative models from scratch, starting with NumPy, then moving to C, and finally to CUDA, to uncover the true nature of generative models and how they are typically hidden by popular frameworks. ### Key Features: - **NumPy Implementation**: Focuses on the algorithm itself, ensuring the model's core functionality is correctly implemented. - **C Implementation**: Hand-written tensors and memory management, providing a deeper understanding of the underlying data structures. - **CUDA Implementation**: Hand-written GPU kernels, showcasing how generative models can be accelerated on modern GPUs. ### Tech Stack: - **Python**: The primary language used for implementing the generative models. - **NumPy**: For basic tensor operations and data manipulation. - **C**: For the implementation of generative models, providing a deeper understanding of the data structures and algorithms. - **CUDA**: For the implementation of generative models on GPUs, showcasing the potential for acceleration. ### Why it Matters: Understanding generative models from scratch is crucial for developers and researchers. It helps in creating more efficient and optimized models, and it provides insights into how these models are typically hidden by popular frameworks, allowing for better design and implementation choices.

### Overview: This project is an official PyTorch implementation of the paper "Progressive Deblurring of Diffusion Models for Coarse-to-Fine Image Synthesis". The code aims to enhance image synthesis by gradually removing noise and amplifying signals, addressing the limitations of traditional diffusion models in considering the relative importance of each frequency component. ### Key Features: - **Progressive Deblurring**: The model synthesizes images in a coarse-to-fine manner, enabling the blurring and deblurring processes to be applied sequentially. - **Rotated Coordinate System**: The diffusion process is generalized to enable diffusion in a rotated coordinate system with different velocities for each component of the vector. - **Blur Diffusion**: A special case of the proposed model, where each frequency component of an image is diffused at different speeds, leading to a more realistic and detailed image synthesis. ### Tech Stack: The project uses Python, leveraging the PyTorch framework for implementation. ### Why It Matters: This approach offers a more realistic and detailed image synthesis process by considering the relative importance of each frequency component, thereby improving the quality of generated images.

Overview: WorFBench is a unified workflow generation benchmark designed for large language models (LLMs). It features multi-faceted scenarios and intricate graph workflow structures, aiming to evaluate the capabilities of LLM agents in generating workflows. Key features: - Multi-faceted scenarios - Intricate graph workflow structures - Comprehensive evaluation using subsequence and subgraph matching algorithms Tech stack: Python Why it matters: WorFBench provides a standardized benchmark for evaluating LLMs in workflow generation, addressing the limitations of existing frameworks and enabling more accurate and detailed performance assessments.

### Overview: Vibe Creating is an open-source AI video-prompt skill designed to simplify the process of turning rough ideas into model-ready text-to-video prompts. This skill is compatible with various text-to-video models and can be integrated into Claude Code, Codex, OpenClaw, and Hermes, as well as Seedance 2.0, Kling, Veo, Hailuo, Wan, Vidu, Gemini Omni, and Grok Imagine. ### Key Features: - **Judgment-first Approach**: Determines if the input is suitable for Vibe Creating before rewriting, ensuring the model doesn’t flatten the intended shots. - **Focus on Story**: Emphasizes the narrative over technical specifications, allowing the model to interpret and find the best shots. - **Trust the Model**: Drops low-value technical parameters, focusing on the intent of the shot. - **Four-Part Output**: The skill provides a fixed format for responses: **Judgment**, **Action**, **Result**, and **Notes**. ### Tech Stack: JavaScript ### Why It Matters: Vibe Creating simplifies the process of translating rough ideas into model-ready text-to-video prompts, making it easier and more intuitive for creators to work with AI-generated videos. It ensures that the model’s output is tailored to the intended style and narrative, enhancing the creative process without unnecessary technical specifications.

### Overview: X-Prompt is an official implementation of a model that aims to enable universal in-context image generation in auto-regressive vision language foundation models. ### Key features: - Universal In-Context Image Generation - Auto-Regressive Vision Language Foundation Models - High Performance and Versatility ### Tech stack: Python, PyTorch, and other relevant libraries. ### Why it matters: X-Prompt is a significant advancement in the field of image generation, offering a more versatile and efficient approach to in-context image generation compared to existing methods.

Overview: Agent For Dummies 是一个零基础入门学习笔记和编程平台，旨在帮助开发者快速掌握 PyTorch 和 Agent 的基本概念和应用。 Key features: - Agent 基础入门学习笔记 - PyTorch 与 Agent 从 0 到 1 快速入门编程平台 - 阅读友好功能 Tech stack: Python Why it matters: 该平台为初学者提供了一个直观的学习路径，帮助他们快速上手 PyTorch 和 Agent，适合那些对整个项目或长代码感到困惑的朋友们。

Overview: Swarms-Torch is a PyTorch library that implements novel swarm intelligence architectures, including Particle Swarm Optimization with Transformers, Ant Colony Optimization, and more, for enterprise-grade AI systems. Key features: - Novel architectures for bio-inspired algorithms - Enterprise components like Mixture of Mambas and Switch Mixture of Experts - Advanced model merging techniques like HyperSlice, Random subspace merging, and Weighted evolutionary crossover - Production-ready implementations for large-scale deployment Tech stack: Python Why it matters: Swarms-Torch provides cutting-edge solutions for next-generation AI systems, offering superior performance and scalability compared to traditional architectures. Repository: kyegomez/swarms-pytorch GitHub stars: 157 GitHub description: Swarming algorithms like PSO, Ant Colony, Sakana, and more in PyTorch 😊

### Overview: IMAGPose is a unified conditional framework for pose-guided person generation, enabling versatile image generation with high detail fidelity, pose alignment, and cross-view consistency. This innovative approach addresses limitations of existing methods by incorporating feature-level conditioning, image-level conditioning, and cross-view attention, resulting in superior performance across various user scenarios. ### Key Features: - **Feature-Level Conditioning (FLC)**: Combines texture features from the VAE encoder with semantic features from the image encoder, addressing missing detail information. - **Image-Level Conditioning (ILC)**: Injects variable number of source image conditions and introduces a masking strategy for image and pose alignment. - **Cross-View Attention (CVA)**: Introduces decomposing global and local cross-attention to ensure local fidelity and global consistency. ### Tech Stack: - Python 3.8 (Recommended: Anaconda or Miniconda) - PyTorch 2.0.0 - CUDA 11.8 ### Why It Matters: IMAGPose is a significant advancement in the field of image generation, particularly in pose-guided person image generation. By overcoming limitations of existing methods, it offers a unified framework that can generate multiple target images with different poses simultaneously and from multi-view source images, making it highly versatile and practical for various applications. ### Repository: muzishen/IMAGPose ### GitHub Stars: 157 ### GitHub Description: [NeurIPS 2024] 🕺IMAGPose🕺: A Unified Conditional Framework for Pose-Guided

### Overview: Lilly is an offline Bosnian-English translator that translates typed Bosnian text into English, translates spoken English into Bosnian, and reads text from photographs. It is a public, offline artifact with reproducible paths and evidence of its performance. ### Key Features: - **Translate**: 43.25 BLEU / 68.10 chrF2, 0 leaked tags - **Reply**: 32.22 BLEU / 61.55 chrF2, 99.2% Bosnian forms (244/246) - **Listen**: 11.9% word error - **Read**: 67.0% found / 65 invented, 57.8% / 450 on test-v2 ### Tech Stack: Python 3.12+ ### Why It Matters: Lilly is a tool for individuals who need to communicate in Bosnian, providing a practical solution for those who do not have the time or resources to learn the language.

Overview: PipeANN is a low-latency, billion-scale, and updatable graph-based vector store optimized for SSDs, designed for both large-scale and memory-constrained scenarios. Key features: - Ultra-Low Latency: 1ms for 1 billion vectors (top-10, 90% recall), with only 1.14x-2.02x of in-memory index. - High Throughput: 20K QPS for 1 billion vectors, outperforming DiskANN and SPANN. - Efficient Updates: Insert/delete with minimal search interference (1.07x fluctuation). - Speculative Filtering: 3K QPS & 6ms latency for attribute-filtered ANNS on 100 million vectors. - Memory Efficient: >10x less memory than in-memory indexes (~40GB for 1B vectors). - Easy-to-Use: Both Python (`faiss`-like) and C++ interfaces supported. - Seamless Integration: Milvus-compatible API (drop-in MilvusClient + gRPC server). - Multi-SSD Scaling: Scales to 70K QPS & 2ms tail latency on 1B vectors (4 SSDs, SPDK backend). Tech stack: C++ Why it matters: PipeANN is designed for high-performance vector storage, making it ideal for applications requiring low latency, large-scale data processing, and efficient updates. Its low memory footprint and ability to scale to multiple SSDs make it a powerful tool for modern data management.

### Overview: Paddock is a Rust-based inference server designed for NVIDIA GPUs and Apple Silicon, offering an OpenAI and Anthropic-compatible API, GGUF + safetensors + MLX support, and built-in Studio for model exploration and comparison. It aims to be the fastest and most modern inference platform for models fitting on a single GPU. ### Key Features: - **OpenAI and Anthropic API Compatibility**: Supports OpenAI and Anthropic APIs. - **Native Engine**: Runs directly on the GPU, eliminating the need for wrappers. - **Built-in Studio**: Simplifies model exploration and comparison. - **Metal Backend**: Utilizes Metal for efficient GPU processing. - **NVIDIA Driver**: Required for NVIDIA systems. - **Swift Native App**: Available for macOS, combining manager and Studio in one window. - **Swift App**: Includes model management, chat, and Metal backend. - **Notarized Downloads**: Developer ID signed DMG and PKG, notarized for macOS. ### Tech Stack: - Rust ### Why It Matters: Paddock is designed to make it easier for organizations and companies to run open models in production, particularly on Blackwell GPUs such as the RTX 5090, RTX PRO 6000, and B200. It aims to provide the fastest and most modern inference platform, focusing on performance and ease of use. ### GitHub Stars: 135

### Overview: MFS provides a unified workspace for AI agents, integrating scattered context sources such as code, memory, docs, databases, SaaS, and more into a searchable and browsable interface. ### Key features: - **Unified Workspace**: MFS aggregates context sources into a single, searchable interface. - **Code & Repositories**: Tracks and indexes code and repositories. - **Memory & Skills**: Manages past-session transcripts and reusable skills. - **Docs & Knowledge**: Crawlable PDFs, design specs, and web pages. - **Chat & Mail**: Slack, Discord, and email integration. - **Issues & CRM**: Jira, Linear, HubSpot, and Zendesk integration. - **Databases & Object Stores**: Postgres, Mongo, BigQuery, Snowflake, S3, and Drive integration. ### Tech stack: Python ### Why it matters: MFS simplifies the management and retrieval of scattered context sources, making it easier for AI agents to access and utilize the vast amount of information they need.

Overview: PapersChat is an AI-powered application that allows users to chat with their papers and gather information from ArXiv and PubMed. Key features: - Chat with papers - Access information from ArXiv and PubMed - Use of LlamaIndex, Qdrant, and Mistral AI technologies Tech stack: Python Why it matters: PapersChat simplifies the process of interacting with academic papers, providing quick access to relevant information without the need for extensive research.

### Overview: CoIR is a comprehensive benchmark designed to evaluate the performance of code information retrieval systems. It includes various datasets and metrics to assess the effectiveness of different approaches. ### Key features: - **Benchmark Datasets**: Includes datasets such as COCO, COCO-ST, COCO-TC, and COCO-TC2. - **Metrics**: Evaluates systems based on precision, recall, F1-score, and others. - **Datasets**: Contains real-world code snippets and queries for practical testing. ### Tech stack: Python ### Why it matters: CoIR is crucial for developers and researchers in the field of code information retrieval, providing a standardized way to measure and compare the performance of different systems.

Overview: KT-Speech-Crawler is an automated tool designed to construct a speech recognition dataset from YouTube videos, enhancing the accuracy of automatic speech recognition systems. Key features: - Automatically extracts speech segments from YouTube videos - Generates a corpus for training speech recognition models - Supports multiple languages and video formats Tech stack: Python Why it matters: This project is crucial for researchers and developers looking to improve the performance of speech recognition systems, providing a reliable and scalable dataset for training models. Repository: EgorLakomkin/KTSpeechCrawler Primary language: Python GitHub stars: 157

Overview: AReA is an efficient image generation model that adapts attention mechanisms to generate images beyond their training resolution. It uses two phases: cross-scale coarse attention and adaptive windowed fine attention, achieving significant speedup and model agnosticism. Key features: - Cross-scale coarse attention with full resolution queries and tokens in smaller grids. - Adaptive windowed fine attention with shrinking windows and a global-KV bridge. - Model agnostic and compatible with various diffusion transformers. - No need for RoPE extrapolation, extra parameters, or fine-tuning. - Speedup grows with resolution, up to 3.2x in experiments. Tech stack: Python Why it matters: AReA simplifies the process of generating high-resolution images by adapting attention mechanisms, making it more efficient and easier to use compared to traditional diffusion transformers.

### Overview: LM-Proxy is a lightweight, OpenAI-compatible HTTP proxy server/gateway that unifies access to multiple Large Language Model providers and local inference using a single, standardized API endpoint. Designed for seamless integration with OpenAI's API, it allows developers to switch between cloud providers and local models without modifying existing client code. ### Key Features: - **Lightweight and Extensible**: Built with Python and FastAPI, LM-Proxy is highly efficient and can be easily extended with additional features. - **OpenAI API Compatibility**: Fully compatible with OpenAI's API format, ensuring seamless integration with existing clients. - **Real-time Token Streaming**: Supports real-time token streaming for better performance and efficiency. - **Secure Virtual API Key Management**: Provides secure management of virtual API keys for improved security. - **Standalone and Library Usage**: Can be used as a standalone HTTP service or as an importable Python library, making it versatile for various development needs. ### Tech Stack: Python/FastAPI ### Why it Matters: LM-Proxy simplifies the integration of multiple LLM providers and local inference, making it easier to switch between providers and local models without changes to existing client code. Its real-time token streaming and secure API key management features enhance performance and security, making it a valuable tool for developers working with large language models.

Overview: Media Agent is a project that scrapes data from Twitter and Reddit, summarizes it, and engages in interactive conversations using Langchain and ChromaDB. Key features: - Scrapes tweets/submissions on behalf of user accounts or keywords. - Embeds tweets/submissions using OpenAI. - Indexes embeddings in ChromaDB and enriches with metadata. - Provides a summary and potential questions for interaction. - Opens a chat session with the tweets. - Saves the conversation with metadata. - Offers a rich terminal UI and logging features. Tech stack: Python Why it matters: This project showcases the integration of Langchain, ChromaDB, and other tools to create a conversational AI system, making it useful for developers looking to build interactive data-driven applications.

### Overview: Explore sample applications and tutorials demonstrating the prowess of Amazon Bedrock with Python. Learn to integrate Bedrock with databases, use RAG techniques, and showcase experiments with langchain and streamlit. ### Key features: - Seamless integration with Amazon Bedrock and Aurora PostgreSQL. - Integration with LangChain for advanced functionalities. - Integration with Amazon RDS for database access. - Integration with OpenSearch for vector database storage. - Integration with Amazon Aurora for efficient backend processing. - Integration with Amazon RDS for database access. - Integration with Amazon OpenSearch for vector database storage. - Integration with Amazon Aurora for efficient backend processing. - Integration with Amazon RDS for database access. - Integration with Amazon OpenSearch for vector database storage. - Integration with Amazon Aurora for efficient backend processing. - Integration with Amazon RDS for database access. - Integration with Amazon OpenSearch for vector database storage. - Integration with Amazon Aurora for efficient backend processing. - Integration with Amazon RDS for database access. - Integration with Amazon OpenSearch for vector database storage. - Integration with Amazon Aurora for efficient backend processing. - Integration with Amazon RDS for database access. - Integration with Amazon OpenSearch for vector database storage. - Integration with Amazon Aurora for efficient backend processing. - Integration with Amazon RDS for database access. - Integration with Amazon OpenSearch for vector database storage. - Integration with Amazon Aurora for efficient backend processing. - Integration with Amazon RDS for database access. - Integration with Amazon OpenSearch for vector database storage. - Integration with Amazon Aurora for efficient backend processing. - Integration with Amazon RDS for database

### Overview: This project, "Continuous-Time Distribution Matching for Few-Step Diffusion Distillation" (CDM), addresses the challenge of few-step diffusion distillation by employing a dynamic continuous-time schedule during backward simulation. The approach involves sampling intermediate anchors uniformly from (0, 1] to align text-image conditions and data distributions at on-trajectory anchors. To address inter-anchor inconsistency, the proposed CDM objective explicitly extrapolates off-trajectory latents using the predicted velocity. ### Key Features: - **Dynamic Continuous-Time Schedule**: The project employs a dynamic continuous-time schedule during backward simulation, sampling intermediate anchors uniformly from (0, 1]. - **CFG Augmentation (CA) and Distribution Matching (DM)**: CA and DM operate on this dynamic schedule to align text-image conditions and data distributions at on-trajectory anchors. - **Extrapolation of Off-Trajectory Latents**: To address inter-anchor inconsistency, the proposed CDM objective explicitly extrapolates off-trajectory latents using the predicted velocity. ### Tech Stack: - **Python**: The primary programming language used for the project. ### Why It Matters: This project is significant because it provides a novel solution to the few-step diffusion distillation problem, offering improved alignment and distribution matching between text and image conditions. The dynamic continuous-time schedule and the explicit extrapolation of off-trajectory latents contribute to more accurate and consistent results, making it a valuable contribution to the field of few-step diffusion distillation.