Projects

### Overview: AlpaSim is an open-source autonomous vehicle simulation platform designed for development and testing of end-to-end AV policies. It provides a modular and extensible testbed for realistic sensor data, vehicle dynamics, and traffic scenarios, enabling users to validate algorithms, evaluate safety, benchmark performance, and debug complex behaviors. ### Key Features: - **Sensor Fidelity**: Pluggable renderer service with NuRec and OmniDreams support, high-fidelity camera feeds, and stateful video-model rendering. - **Research Hackability**: Python-based implementation, modular grpc interface, extensive configuration options, and debugging tools. - **Horizontal Scalability**: Microservices architecture, support for distributed computing, and multi-node deployments. ### Tech Stack: Python ### Why It Matters: AlpaSim is essential for researchers and developers looking to test and validate autonomous driving algorithms in a controlled and realistic environment. Its modular design and high-fidelity simulation capabilities make it a valuable tool for both algorithm validation and safety analysis in autonomous driving research.

Overview: This project provides a comprehensive learning path for AI engineers, covering Python, TypeScript, Rust, and Julia. It includes 542 lessons, 20 stages, and a detailed website with interactive charts, videos, and quizzes. The course is available for free, open-source, and uses the MIT license. Key features: - 542 lessons - 20 stages - Python, TypeScript, Rust, and Julia support - Comprehensive website with interactive charts, videos, quizzes, and a detailed website - Free, open-source, and MIT-licensed Tech stack: Python Why it matters: This project offers a complete learning path for AI engineers, covering essential skills and concepts in AI engineering. It's designed to help individuals become proficient in AI agent development, making it a valuable resource for aspiring AI engineers.

Overview: This is a Python package designed to run contextualized topic modeling (CTM). CTMs combine contextualized embeddings with topic models to generate coherent topics. Key features: - CombinedTM: Combines contextual embeddings with the bag of words for better topic coherence. - ZeroShotTM: A multilingual topic model that can be trained with zero-shot learning. Tech stack: Python Why it matters: CTMs offer improved topic coherence by leveraging pre-trained contextual embeddings, making them valuable for applications requiring high topic coherence and coherence across different languages.

## Overview: UniDepthV2: Universal Monocular Metric Depth Estimation Made Simpler UniDepthV2 is a significant advancement in the field of monocular depth estimation, offering a more intuitive and user-friendly approach to depth estimation for monocular cameras. This project introduces a unified framework that can handle various datasets and environments, making it highly versatile and adaptable. The paper, "UniDepthV2: Universal Monocular Metric Depth Estimation Made Simpler," presents a novel method that simplifies depth estimation by leveraging universal metrics, enhancing the accuracy and ease of use for both researchers and practitioners. ## Key Features: 1. **Unified Framework**: UniDepthV2 provides a single, unified framework that can be applied to various datasets, including KITTI, NYU Depth V2, and Kitti Eigen, among others. This simplification allows users to focus on the core problem without being constrained by the specific characteristics of a particular dataset. 2. **Universal Metrics**: The project introduces universal metrics that are applicable across different environments and datasets. These metrics are designed to be robust and accurate, ensuring consistent performance across various applications. 3. **Simplified Interface**: The codebase is designed to be highly intuitive and user-friendly, making it accessible to a wide range of users. The simplicity of the interface reduces the learning curve and allows for easier adoption of the technology. 4. **Versatile Applications**: UniDepthV2 is not limited to a specific application domain. It can be used in various scenarios, including autonomous driving, augmented reality, and more, making it a valuable tool for researchers

### Overview: VOCA is a framework for generating realistic 3D character animations from speech signals. It synthesizes facial animations using a combination of a static mesh and a trained DeepSpeech model, allowing for the creation of dynamic characters that respond to speech input. ### Key Features: - **Realistic Animation Synthesis**: VOCA generates highly realistic character animations that adapt to the speech input. - **Speech-Driven Facial Animation**: It leverages DeepSpeech to process audio signals and create facial expressions. - **Static Mesh and Dynamic Textures**: The system uses a static mesh as a base and integrates real-time textures and animations to enhance realism. ### Tech Stack: Python - The codebase is written in Python 3.6.8, utilizing TensorFlow 1.14.0 for model training and inference. ### Why It Matters: VOCA provides a practical solution for creating dynamic and responsive characters in virtual environments, making it valuable for applications in gaming, animation, and interactive media. The framework's ability to adapt to speech input and integrate with existing facial animation tools makes it particularly useful for creating lifelike characters in various media formats.

### Overview: Secure MCP runtime for AI agents to operate local machines, servers, and containers with multi-device orchestration. ### Key features: - Unified, secure, and controlled file, command, Git, Skill, MCP, browser automation, and task execution across local computers, remote servers, and containers. - Connect multiple AgentDock instances to coordinate work across devices and finish multi-machine workflows in a single conversation. - Manage multiple computers and servers directly from ChatGPT without repeatedly switching SSH sessions. - Write code, modify projects, run tests, and operate Git in the real local or remote environment without depending on a dedicated coding-agent quota. - Manage VPS hosts, Docker services, reverse proxies, and deployment configuration. - Operate authenticated web pages and macOS desktop applications. - Extend capabilities through Skills and dynamic MCP servers. - Persist long-running task state and continue after an interruption. ### Tech stack: Go ### Why it matters: Secure, controlled access to multiple devices without the need for dedicated coding agents or SSH sessions.

Overview: XAI is a machine learning library designed for explainability, developed by The Institute for Ethical AI & ML. It includes tools for analyzing and evaluating data and models. Key features: - Data analysis - Model evaluation - Production monitoring Tech stack: Python Why it matters: XAI provides a framework for machine learning explainability, enabling engineers and domain experts to understand and improve the performance of machine learning models. Repository: EthicalML/xai GitHub stars: 1271 GitHub description: XAI - An eXplainability toolbox for machine learning README excerpt: XAI is a library for machine learning explainability. It includes tools for analyzing data and evaluating models. The library is designed to help engineers and domain experts understand and improve the performance of machine learning models. GitHub description: XAI - An eXplainability toolbox for machine learning. The library is designed with AI explainability in its core. It contains various tools for analysis and evaluation of data and models. The library is maintained by The Institute for Ethical AI & ML and was developed based on the 8 principles for Responsible Machine Learning. The library can be found at [https://ethicalml.github.io/xai/index.html](https://ethicalml.github.io/xai/index.html) and a talk on the idea was presented at the PyData London 2019 Conference. The library can also be found in the community-built "Awesome Machine Learning Production & Operations" list. YouTube video showing how to use XAI to mitigate undesired biases: A video of the talk

Overview: OpenContext is a personal context store designed for AI agents and assistants, allowing you to reuse your existing coding agent CLI (Codex/Claude/OpenCode) with built-in Skills/tools and a desktop GUI to manage, search, and reuse project knowledge across agents and repos. Key features: - Skills-first support: `oc init` generates user-level skills and slash commands for your agent. - Built-in Skills and Tools: Reuses your existing CLI with built-in skills and tools for context management. - Global Context Library: Manages and shares knowledge across all projects. - Lightweight and Easy to Use: Perfect for developers and builders who want to integrate AI into their existing workflows. Tech stack: JavaScript, TypeScript, Node.js, and a minimal backend for server-side operations. Why it matters: OpenContext simplifies context management for AI assistants, ensuring that knowledge is easily accessible and reusable across different agents and projects, making development more efficient and less repetitive.

### Overview: Youtu-GraphRAG is a revolutionary framework that addresses the challenges of graph retrieval and complex reasoning in real-world applications. By vertically unifying agents and integrating seamlessly with graph schemas, Youtu-GraphRAG achieves remarkable improvements in token cost and accuracy, making it a significant advancement in the field. ### Key Features: - **Schema-Guided Hierarchical Knowledge Tree Construction**: Introduces targeted entity types, relations, and attribute types to bound automatic extraction agents. - **Scalable Schema Expansion**: Continuously expands schemas for adaptability over unseen domains. - **Four-Level Architecture**: - **Level 1 (Attributes)**: Entity property information. - **Level 2 (Relations)**: Entity relationship triples. - **Level 3 (Keywords)**: Keyword indexing. - **Level 4 (Communities)**: Hierarchical community structure. - **Quick Adaptation to Industrial Applications**: Seamless domain transfer with minimal intervention on the schema. ### Tech Stack: - Python ### Why It Matters: Youtu-GraphRAG offers a 33.6% lower token cost and 16.62% higher accuracy compared to state-of-the-art baselines, making it a game-changer in complex reasoning tasks. Its ability to adapt seamlessly to various domains, including encyclopedias, academic papers, and private knowledge bases, positions it as a valuable tool for real-world applications.

Overview: A Python library for implementing TextRank, a popular algorithm for ranking the importance of documents in a corpus. Key features: - TextRank algorithm implementation - Supports multiple document types - Easy to integrate into existing projects - Cross-platform compatibility Tech stack: Python Why it matters: TextRank is a highly effective algorithm for text analysis and ranking, making this library a valuable tool for developers looking to improve their natural language processing capabilities.

Overview: Artyom.js is a JavaScript library that provides voice control and speech recognition/synthesis functionalities, allowing developers to create their own versions of Siri, Google Now, or Cortana. Key features: - Voice recognition - Voice synthesis - Dynamic commands - Smart commands with wildcards and regular expressions - Dictation object for converting voice to text - Simulating commands without a microphone - Keyword execution - Pause and resume command recognition - Soundex algorithm for improved command recognition accuracy (default disabled) - Remote command processor service for enhanced performance Tech stack: JavaScript Why it matters: Artyom.js simplifies the implementation of advanced voice control and speech synthesis features, making it easier for developers to create custom versions of popular voice assistants without needing to integrate complex APIs directly into their websites.

Overview: 船长黑板报是关于机器学习、计算机视觉和工程技术的总结和分享平台。 Key features: - 招聘阿里淘系技术部的数据挖掘、计算机视觉和自然语言处理人才。 - 提供详细的技术笔记和项目代码。 Tech stack: C++ Why it matters: 本黑板报是船长在工作和学习中总结和分享技术心得的平台，对提升工程师技能和项目经验有重要意义。

Overview: IoT DC3 is an open-source Industrial IoT Runtime designed to connect the physical world to artificial intelligence, providing a robust platform for physical AI applications. It includes 36 protocol drivers, an MCP tool gateway, and cloud-native microservices for seamless integration and management. Key features: - 36 protocol drivers for various physical devices - MCP tool gateway for AI agent interaction - Cloud-native microservices architecture for scalable and secure operations Tech stack: Java 21, Spring Boot 4.1 Why it matters: IoT DC3 simplifies the integration and management of physical devices with AI applications, enabling developers to build new applications without changing existing devices, and vice versa. This platform provides a unified API for adding or changing devices, ensuring seamless communication and data flow between devices and applications. Repository: pnoker/iot-dc3 GitHub stars: 1287 GitHub description: IoT DC3 — Connect the Physical World to AI. An open-source Industrial IoT Runtime for Physical AI: 36 protocol drivers, edge-to-cloud data delivery, and an MCP tool gateway so AI agents act on devices in a safe, controllable, traceable loop.

### Overview **ktrain** is a lightweight wrapper for the deep learning library [TensorFlow Keras](https://www.tensorflow.org/guide/keras/overview) (and other libraries) to help build, train, and deploy neural networks and other machine learning models. It is designed to make deep learning and AI more accessible and easier to apply for both newcomers and experienced practitioners. With only a few lines of code, **ktrain** allows you to easily and quickly: - Employ fast, accurate, and easy-to-use pre-canned models for `text`, `vision`, `graph`, and `tabular` data: - **Text Classification**: BERT, DistilBERT, NBSVM, fastText, and other models - **Text Regression**: BERT, DistilBERT, Embedding-based linear text regression, fastText, and other models - **Sequence Labeling (NER)**: Bidirectional LSTM with optional CRF layer and various embedding schemes - **Ready-to-Use NER models for English, Chinese, and Russian** - **Sentence Pair Classification**: Paraphrase detection - **Unsupervised Topic Modeling**: LDA ### Key Features - **Ease of Use**: Minimal setup and configuration required - **Model Selection**: Access to pre-defined models for various tasks - **Model Deployment**: Ready-to-use models for deployment in production environments - **Integration**: Seamless integration with existing TensorFlow/Keras projects ### Tech Stack - **Python**: Jupyter Notebook - **TensorFlow K

### Overview: Implementing Flamingo, a state-of-the-art few-shot visual question answering model, in PyTorch, this project provides a comprehensive guide for developers to integrate the model into their existing text-based language models. The implementation includes key components such as the perceiver resampler, specialized masked cross attention blocks, and tanh gating mechanisms, making it highly adaptable for various applications. ### Key Features: - **Flamingo Implementation**: A detailed guide on integrating Flamingo, a model from DeepMind, into PyTorch. - **Perceiver Resampler**: The resampling mechanism for shrinking media sequences to a smaller dimension for efficient processing. - **Specialized Cross Attention Blocks**: Blocks designed specifically for cross attention, enhancing the model's ability to handle complex visual question answering tasks. - **Tanh Gating**: Mechanisms at the end of cross attention and feedforward blocks to control the flow of information. ### Tech Stack: - **Python**: The primary programming language used for the implementation. - **PyTorch**: The deep learning framework used for building and training the model. ### Why It Matters: Flamingo is a groundbreaking model that addresses the challenges of few-shot visual question answering by leveraging attention mechanisms. Integrating it into existing language models can significantly enhance their ability to understand and respond to visual questions, making it a valuable addition to any developer's toolkit. ### Repository: The project is hosted on GitHub at [lucidrains/flamingo-pytorch](https://github.com/lucidrains/flamingo-pytorch). ###

Overview: This server is a WebSocket, gRPC, and WebRTC speech recognition server built on top of the Vosk and Kaldi libraries, providing highly accurate offline speech recognition. Key features: - WebSocket - gRPC - WebRTC - Websocket Tech stack: Python Why it matters: This server is ideal for integrating speech recognition into smart home, PBX systems, chatbots, websites, and telephony applications, offering high accuracy and ease of integration.

Overview: Sage is a chat interface that allows developers to quickly understand and integrate any codebase into their projects. Key features: - Dead-simple setup - Runs locally or on the cloud - Supports both lightweight and traditional retrieval mechanisms - Well-documented experiments Tech stack: Python Why it matters: Sage simplifies the process of learning and integrating new codebases, making it accessible and efficient for developers.

### Overview: typesafe-computer-use is a project that automates computer use tasks on macOS, allowing you to type commands and have the computer perform them for a cost of about 0.0002 cents per step. It uses TypeSafe's decision model to handle the decisions and writes free text when necessary. ### Key Features: - **Decision Making**: Utilizes TypeSafe's decision model to make choices quickly and cheaply. - **Screen Reading**: Reads the screen deterministically and asks for the next action. - **Writing Model**: Only writes free text when necessary, reducing the need for expensive writing models. - **Easy Installation**: Requires macOS 14 or newer, Python 3.12 or newer, and uv. Windows 10 and 11 are experimental. ### Tech Stack: - Python 3.12+ - TypeSafe Decision Model - Ruff for code formatting ### Why It Matters: - **Cost Efficiency**: Each decision costs about 0.0002 cents, making it significantly cheaper than traditional methods. - **Speed**: Models are faster and more efficient, reducing the time needed for each step. - **Accuracy**: The project is built around TypeSafe's decision model, ensuring accurate and efficient decisions. ### GitHub Metadata: - Repository: `awlevin/typesafe-computer-use` - GitHub Stars: 1257 - Primary Language: Python - GitHub Description: Computer use for about $0.0002 a step: OCR the screen, classify the next action with TypeSafe

Overview: HyperPose is a library designed for fast and flexible human pose estimation, offering high-performance pose estimation with CPUs/GPUs and flexibility for custom pose estimation models. Key features: - High-performance pose estimation with CPUs/GPUs - Flexibility for developing custom pose estimation models Tech stack: Python Why it matters: HyperPose provides a powerful and flexible solution for building custom pose estimation applications, offering significant performance improvements over existing libraries like OpenPose, TF-Pose, and OpenPifPaf. Repository: tensorlayer/HyperPose Primary language: Python GitHub stars: 1264 GitHub description: Library for Fast and Flexible Human Pose Estimation README excerpt: HyperPose is a library for building high-performance custom pose estimation applications. It offers two key features: high-performance pose estimation with CPUs/GPUs and flexibility for developing custom pose estimation models. The library contains a C++ inference library and a Python library for model development. Examples are provided for running inference on videos and customizing models.

### Overview: DeText is a deep neural text understanding framework designed for NLP tasks such as ranking, classification, and language generation. Leveraging state-of-the-art deep neural networks, it automates feature extraction and interaction modeling, making it highly flexible and customizable. ### Key Features: - **Natural Language Understanding**: Powered by advanced deep neural networks for semantic matching in search and recommendation systems. - **Flexible Model Architecture**: Configurable components like Word Embedding Layer, Text Encoding Layer, Interaction Layer, and Wide & Deep Feature Processing. - **Customizability**: Offers options for LTR/Classification Layer, MLP Layer, Interaction Layer, Text Embedding Layer, and Continuous Feature Normalization. - **Training**: Supports tf-ranking LTR loss and metrics, making it easy to integrate into existing projects. ### Tech Stack: Python 3.7 ### Why It Matters: DeText simplifies the process of text understanding and interaction modeling, making it a valuable tool for developers working on NLP tasks such as search, recommendation, and language generation. Its flexibility and customization options make it highly adaptable to various use cases, enhancing its utility across different projects.