Projects

### Overview: Transformer Transformer is a unified model designed to generate complete robot designs based on manipulation demonstrations. It optimizes robot performance by generating every link, joint, motor, and inertial property, achieving significant improvements in tracking error and joint speed compared to the original ALOHA platform. ### Key Features: - **Unified Model**: Generates complete robot designs from manipulation demonstrations. - **Motion-Conditioned Co-design**: Optimizes robot performance for specific tasks. - **RoboTokens**: A unified tokenization of robot embodiments, states, and actions. - **Procedural Robot Generation**: Generates robots over parameterized design spaces. - **Data Generation**: Uses Mink and RL controllers for data generation. - **Training & Inference**: Trains and inferences on cross-embodiment control and motion-conditioned robot co-design. - **CMA-ES Baselines**: Compares with CMA-ES for optimization. - **Blender Pipelines**: Behind all robot visualizations. ### Tech Stack: - **Python**: The primary programming language used for development and training. ### Why It Matters: Transformer Transformer provides a unified approach to robot design, optimizing performance for specific tasks, and significantly improving the accuracy and efficiency of robot motion.

### Overview: Build production AI agents that generate images, video, and audio on serverless GPUs. This course offers a free, open-source project-based course for developers, covering everything from setting up serverless GPUs to scaling and monitoring production systems. ### Key Features: - Build and monitor AI agents that run image, video, and audio models on serverless GPUs. - Learn how to handle expensive, slow, and failure-prone GPU workloads. - Understand how to give agents GPU models as tools and build generate → evaluate → regenerate loops. - Master the skills to fan out batch workloads on spot instances, handle failures, and report cost per asset. - Add evaluations, tracing, and SLOs to ensure media agents cost-effectively and meet quality standards. ### Tech Stack: Python, PyTorch, TensorFlow, Docker, Kubernetes, AWS, GCP, and OpenCV. ### Why It Matters: This course is essential for developers looking to build and deploy production AI systems. It covers the entire process from setting up serverless GPUs to scaling and monitoring production systems, ensuring that AI agents can handle large media artifacts efficiently and cost-effectively.

Overview: MGM-Omni is an open-source omni-chatbot designed for processing text, image, video, and speech inputs, and generating both text and speech responses. It leverages MiniGemini and MiniGemini v2 to handle long-form speech understanding and generation, making it a versatile multi-modal AI assistant. Key features: - Supports audio, video, image, and text inputs. - Handles long-form speech inputs up to 10 minutes. - Enables voice cloning with zero-shot training. - Supports streaming generation for live conversations. - Provides zero-shot voice cloning with short clip training. Tech stack: Python. Why it matters: MGM-Omni is a significant advancement in the field of multi-modal AI, offering superior performance in understanding and generating long-form speech, making it highly valuable for applications requiring detailed and detailed speech processing and generation.

Overview: A fast version of llama.cpp optimized for NVIDIA RTX 30-series GPUs, particularly the RTX 3090 and 3090 Ti, designed for Qwen3.8 models. It supports a 262K context and improves performance by 110+ tok/s over a 100K-token generation at temperature 1. Key features: - Significantly improved performance on RTX 3090 and 3090 Ti - Supports Qwen3.8 models, including Qwen3.8-27B with MTP draft head - Supports low memory configurations for 200k+ context - Faster Flash-Next writing and prompt reading - Supports MB-MTP with MTP draft head for faster writing Tech stack: C++ Why it matters: This fork of llama.cpp significantly boosts performance on RTX 30-series GPUs, making it ideal for high-performance AI applications and developers looking to optimize their models for better performance on their hardware.

### Overview: SodaMem is a self-evolving, agentic memory layer designed for AI agents. It ensures facts are superseded instead of overwritten, profiles entity profiles on demand, and answers with zero LLM calls, providing consistent and accurate responses. ### Key Features: - **Self-Adapting Memory**: SodaMem continuously evolves alongside the agent, ensuring facts are updated rather than overwritten. - **Dynamic Entity Profiles**: Profiles are rebuilt on demand, keeping them relevant and accurate. - **Zero LLM Calls**: Answers are always the same, eliminating the need for multiple API calls. ### Tech Stack: Python ### Why it Matters: SodaMem offers a more intelligent and efficient memory layer for AI agents, ensuring consistent and accurate responses without the need for multiple LLM calls. This makes it a valuable tool for enhancing the performance and reliability of AI-driven applications.

Overview: Anything2Image is a powerful tool that allows users to generate images from various inputs using the unified latent space of ImageBind and Stable Diffusion 2.1, making it a versatile solution for image generation. Key features: - Supports ImageBind with a 1024 latent space. - Integrates with the 🤗 Diffusers library. - Provides an online demo using Hugging Face Gradio and Google Colab. - Supports audio and text inputs for image generation. - Offers various image-to-image tasks such as audio-to-image, audio+text-to-image, and more. - Has been integrated into InternGPT and updated with new features and improvements. Tech stack: Python, PyTorch, Hugging Face, Stable Diffusion, ImageBind, Diffusers. Why it matters: Anyone looking to create images from diverse inputs can benefit from Anything2Image, making it a valuable addition to the developer community.

Overview: FlowX is an MCP server, Python SDK, and A2A adapter that enables AI agents to generate, run, and evolve workflows from natural language inputs. It compiles processes into graph-based workflows, Python implementations, and a FastAPI backend, allowing agents to inspect, run, and modify workflows in real-time conversations. Key features: - **Generative Workflow Layer**: Converts intent into executable workflows. - **Python SDK**: Enables integration with various clients and A2A adapters. - **A2A Adapter**: Facilitates seamless interaction with existing A2A systems. - **Node Engine**: Enhanced node engine with generation, audit, test, and repair capabilities. - **Interactive Runtime**: Provides an interactive runtime environment for running and evolving workflows. Tech stack: Python 3.10+ with MCP server, A2A adapter, and FastAPI backend. Why it matters: FlowX transforms AI workflows into reusable, editable processes, enabling continuous improvement and adaptation without the need for re-prompting. This makes it a powerful tool for automating tasks and processes in various applications.

Overview: LooGLE is a comprehensive evaluation benchmark designed for long context understanding in language models, featuring up-to-date and extremely long documents and questions across diverse domains. Key features: - Extensive dataset of up to 24k tokens per document, with over 6,000 questions from various domains. - 7 major tasks to evaluate LLMs' ability to understand both short and long dependency content. - Comprehension, reasoning, computation, timeline reorder, multiple information retrieval, and summarization tasks. - Automatic metrics based on semantic similarity, GPT4-as-judgment, and human evaluation. - 8 representative LLMs evaluated across different models and techniques. Tech stack: Python Why it matters: LooGLE provides a systematic and comprehensive evaluation schema for long-context language models, offering insights into the future development of enhanced models toward "true long-context understanding." Repository: bigai-nlco/LooGLE Primary language: Python GitHub stars: 200 GitHub description: ACL 2024 | LooGLE: Long Context Evaluation for Long-Context Language Models

Overview: BunkaTopics is a comprehensive package designed for data cleaning, textual data visualization, and frame analysis. It assists developers in extracting insights from unstructured data, particularly useful for fine-tuning language models and optimizing them through controlled data cleaning processes. Key features: - Data Cleaning - Topic Modeling Visualization - Frame Analysis Tech stack: Python Why it matters: BunkaTopics simplifies the process of data cleaning, enhancing the precision of fine-tuning language models and optimizing their performance. Its tools enable developers to gain granular insights from unstructured data, making it a valuable asset in the field of data science and artificial intelligence. Repository: charlesdedampierre/BunkaTopics Primary language: Python GitHub stars: 201 GitHub description: 🗺️ Data Cleaning and Textual Data Visualization 🗺️ README excerpt: [](https://pypi.org/project/bunkatopics/) [](https://pypi.org/project/bunkatopics/) [](https://pepy.tech/project/bunkatopics) [](https://pepy.tech/project/bunkatopics) Please read our Documentation: [

### Overview: Diagram of Thought (DoT) is an official implementation of the paper "On the Diagram of Thought" (https://arxiv.org/abs/2409.10038), which introduces a framework that enables a single Large Language Model (LLM) to construct and refine a mental map of its reasoning process. This process is self-contained and auditable, bridging the gap between fluent language and formal, verifiable reasoning. ### Key features: - **Controller-Light, Single-Model Reasoning:** DoT internalizes the propose → critique → refine → summarize loop within one autoregressive model using learned role tokens. This eliminates the need for complex external controllers, multi-agent orchestration, or explicit search algorithms required by other advanced reasoning methods. - **Auditable & Verifiable Traces:** A typed serialization protocol ensures that the reasoning process is recorded as a well-structured Directed Acyclic Graph (DAG). This allows for deterministic extraction, post-hoc analysis, and verification of the model's reasoning path, crucial for safety and debugging. - **Formal Guarantees from Category Theory:** Grounded in a rigorous mathematical foundation using topos theory, DoT guarantees that the synthesis of validated information is principled, consistent, and robust. The final summary is formally a colimit in the information order, ensuring that the result is invariant to the order in which evidence was explored. - **Strict Generalization of Linear Reasoning:** DoT's ability to manage parallel, incomparable lines of validated evidence makes it strictly more expressive than linear

Overview: The Station is an open-world multi-agent environment designed to facilitate autonomous scientific discovery and shared knowledge building among AI agents. Key features: - Explore the Station project page for an overview. - Read the Station v2 paper for the latest architecture and results. - Access the Station v1 paper for the original architecture and findings. - Utilize the Station v2 viewer for mathematical tasks and the Station v1 viewer for other tasks. Tech stack: Python. Why it matters: The Station provides a platform for researchers to pursue autonomous scientific discovery within an open-world environment, fostering collaboration and knowledge sharing among AI agents. Repository: dualverse-ai/station GitHub stars: 201 GitHub description: The Station is an open-world, multi-agent environment that models a miniature scientific ecosystem and helps researchers pursue autonomous scientific discovery.

### Overview: A comprehensive guide to LLM hacking, covering fundamentals, prompt injection, offensive techniques, and defensive strategies. ### Key Features: - Fundamentals of LLM hacking - Techniques for prompt injection - Offensive methods and strategies - Defensive tactics and security measures ### Tech Stack: Python and Jupyter Notebooks ### Why It Matters: This handbook is essential for developers and hackers looking to understand and leverage language models for practical applications, while also learning about the risks and best practices to mitigate them.

### Overview: LogLLM is an open-source implementation of "Log-based Anomaly Detection Using Large Language Models," designed for system log anomaly detection. It leverages large language models to identify irregularities in system logs, providing a robust solution for detecting anomalies in various environments. ### Key features: - **LogParser**: A log parser that extracts meaningful information from system logs. - **DeepLog**: A log anomaly detection model that achieves high precision and recall. - **PLELog**: A model that excels in detecting anomalies in system logs. - **FastLogAD**: A fast anomaly detection model that provides high precision and recall. - **LogBERT**: A model that detects anomalies in system logs with high accuracy. - **LogRobust**: A model that identifies anomalies in system logs with high precision and recall. - **CNN**: A convolutional neural network model that detects anomalies in system logs. - **NeuralLog**: A neural network model that detects anomalies in system logs with high accuracy. - **RAPID**: A model that detects anomalies in system logs with high precision and recall. ### Tech stack: Python ### Why it matters: LogLLM is designed to help system administrators and security professionals quickly identify and address system anomalies, improving the overall security and reliability of their systems. Its ability to detect anomalies in real-time makes it an invaluable tool for monitoring and maintaining system health.

### Overview: This project showcases a custom Retrieval-Augmented Generation (RAG) pipeline for Android, utilizing local models and embeddings to answer natural language questions from PDF/DOCX documents. The app leverages modern Android development practices and recommended architecture guidelines, focusing on efficient offline processing. ### Key Features: - **Local Models**: Utilizes local embeddings and vector databases for efficient offline processing. - **Text Splitter**: Splits text documents into manageable chunks for better retrieval and generation. - **Vector Database**: Stores and retrieves document embeddings for quick question-answer matching. - **LLM (Local)**: Inference engine for generating answers locally, reducing dependency on remote services. ### Tech Stack: - Kotlin ### Why it Matters: This project demonstrates the practical application of RAG in an Android environment, showcasing how to build a fully offline, on-device RAG pipeline with minimal compute and storage requirements. It highlights the importance of efficient local processing and the benefits of leveraging local models and embeddings for improved performance and reduced latency.

### Overview: This project showcases a simplified and cleaned version of the code used by [xuyuan](https://github.com/xuyuan) and [tugstugi](https://github.com/tugstugi) in the Kaggle competition TensorFlow Speech Recognition Challenge. The repository includes a Convolutional Neural Network (CNN) architecture designed for the Google speech commands dataset, achieving a 10th place finish. ### Key Features: - **1x32x32 Mel-Spectrogram as Network Input**: The model uses a 1x32x32 Mel-Spectrogram as the input, simplifying the preprocessing step. - **Single Network Implementation for CIFAR10 and Google Speech Commands Data Sets**: The network architecture is designed to be compatible with both datasets. - **Faster Audio Data Augmentation on STFT**: The model incorporates faster audio data augmentation techniques, enhancing its performance. - **Kaggle Private LB Scores Evaluated on 150,000+ Audio Files**: The repository includes evaluation scores from Kaggle, showcasing the model's effectiveness on a large dataset. ### Tech Stack: - **Framework**: PyTorch - **Language**: Python ### Why It Matters: This project demonstrates the application of PyTorch in speech recognition tasks, achieving a notable performance in the Kaggle competition. The simplified architecture and efficient data augmentation techniques highlight the model's effectiveness and potential for real-world applications.

Overview: This repository is an implementation of VMC, a framework that fine-tunes only the Keyframe Generation Module within hierarchical Video Diffusion Models for motion-customized video generation. It is based on the paper "[VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Models](https://arxiv.org/abs/2312.00845)" presented at CVPR 2024. Key features: - Fine-tunes only the Keyframe Generation Module - Utilizes hierarchical Video Diffusion Models - Supports motion-customized video generation Tech stack: Python Why it matters: This framework offers a novel approach to motion-customized video generation by focusing on the keyframe generation module, potentially improving efficiency and performance compared to full model fine-tuning.

### Overview: ShannonBase is a cutting-edge database designed specifically for AI applications, offering a streamlined and efficient solution for data and AI integration. It eliminates the need for separate infrastructure components, integrating Vector Search, ML training, LLM inference, and RAG pipelines directly into a single, zero-configuration engine. ### Key Features: - **In-Database ML**: Built-in LightGBM and XGBoost for training and prediction. - **Native Vector Search**: First-class `VECTOR` data type with `DISTANCE` functions. - **LLM / ONNX Runtime**: Run Llama, Qwen, DeepSeek, and MiniLM locally via embedded ONNX Runtime. - **RAG Framework**: Full Retrieval capabilities with LangChain and vector DB integration. ### Tech Stack: - **C++**: The primary language for development and execution. ### Why It Matters: ShannonBase simplifies AI application stacks by consolidating multiple components into a single, efficient engine. This eliminates latency, operational overhead, and data synchronization risks, making it ideal for real-time data processing and AI-driven applications. ### Repository: The repository is hosted on GitHub at [Shannon-Data/ShannonBase](https://github.com/Shannon-Data/ShannonBase). ### GitHub Stars: The repository has been starred by 200 developers, indicating high interest and adoption in the community. ### Build from Source: For those interested in building from source, detailed instructions and build workflows are provided in the repository. ### Quickstart: - **Option A (Recommended)**: Run a Docker container for a

### Overview: This repository is a comprehensive collection of free resources for learning Machine Learning, Deep Learning, Python, SQL, CNN, NLP, GAN, GNN, Transformers, Flask, Django, and end-to-end machine learning projects. ### Key Features: - **Python Theoretical**: Resources for theoretical Python concepts. - **DSA In Python**: Python-based data structures and algorithms. - **Machine Learning**: Includes projects, MLflow, TensorFlow, PyTorch, Large Language Models, Neural Networks, and Computer Vision. - **Data Science and Machine Learning**: Resources for data science and machine learning. - **Deep Learning**: Covers TensorFlow, PyTorch, and large language models. - **SQL**: Resources for SQL database management. - **Cloud Computing**: Resources for cloud computing. - **Web Development**: Flask and FastAPI. - **Version Control**: Git. - **Interview Preparation**: Resources for preparing for technical interviews. - **Operating Systems**: Resources for Linux. - **Others**: ChatGPT prompts and a portfolio. ### Tech Stack: Jupyter Notebook ### Why it Matters: This repository serves as a valuable resource for anyone looking to learn and practice machine learning, deep learning, Python, SQL, and related technologies. It provides a wide range of projects and resources that can help learners build practical skills and knowledge in these domains.

### Overview ReMax is a reinforcement learning method designed specifically for reward maximization in Reinforcement Learning from Human Feedback (RLHF). It simplifies the implementation of RLHF, requiring only 6 lines of code. ReMax is memory-efficient, saving 50% of GPU memory compared to PPO, allowing for batch sizes up to 4x40. It significantly speeds up training by up to 2x, achieving 84.22%, 75.28%, and 63.60% win rates over SFT, DPO, and PPO on the AlpacaEval benchmark. ReMax is easy to tune and can be used with various large language models, including OPT and Llama2. ### Key Features - **Simple Implementation**: Requires only 6 lines of code. - **Memory Efficiency**: Saves 50% of GPU memory compared to PPO. - **Training Speed**: Achieves up to 2x faster training speed. - **Easy Tuning**: Easy to tune for optimal performance. - **Benchmark Results**: Achieves win rates of 84.22%, 75.28%, and 63.60% over SFT, DPO, and PPO on the AlpacaEval benchmark. ### Tech Stack - **Python**: The primary language used for the implementation. ### Why It Matters ReMax simplifies the implementation of RLHF, making it accessible for developers. It is memory-efficient, allowing for larger batch sizes, and significantly speeds up training

Overview: MaIN.NET is a NuGet package designed to integrate large language models (LLMs) into advanced AI workflows, making it a versatile foundation for developers. Key features: - Modular Artificial Intelligence Network (MaIN) - Streamlines the integration of LLMs into .NET applications - Supports LLMs, RAG, and Agents Tech stack: C# Why it matters: It simplifies the integration of AI into .NET applications, making advanced AI accessible to developers of all skill levels.