Projects

### Overview: NeoHorse is TokenRhythm's family of open-weight models designed for agent workflows. NeoHorse-1 provides 4B and 9B language models for tool use, coding, and instruction following. NeoHorse-Jev builds on NeoHorse-1-4B to make structured decisions through prefill-only inference. ### Key features: - **NeoHorse-1**: 4B and 9B language models for tool use, coding, and instruction following. - **NeoHorse-Jev**: Decision model that integrates **Choice**, **Noul**, and **Score** for structured decisions. ### Tech stack: Python ### Why it matters: NeoHorse is a powerful tool for agent workflows, providing structured decisions and improved performance through prefill-only inference.

### Overview: Graph4NLP is a library designed for the easy use of Graph Neural Networks in Natural Language Processing (NLP). It provides both full implementations of state-of-the-art models for data scientists and flexible interfaces for researchers and developers to build customized models. Built upon highly-optimized runtime libraries including DGL, Graph4NLP offers high running efficiency and great extensibility. The library consists of four layers: Data Layer, Module Layer, Model Layer, and Application Layer. ### Key features: - Full implementations of state-of-the-art models for data scientists. - Flexible interfaces for researchers and developers to build customized models. - Support for both full implementations and flexible interfaces. - High running efficiency and great extensibility. - Easy-to-use library for R&D in the intersection of Deep Learning on Graphs and NLP. ### Tech stack: Python ### Why it matters: Graph4NLP simplifies the process of using Graph Neural Networks in NLP, making it accessible to both data scientists and researchers. Its flexibility and ease of use make it an ideal choice for building customized models and integrating with existing workflows.

### Overview: Pixel LLMs is a family of projects aimed at bringing pixel-level, dense grounded understanding to multimodal language models. The official repo for Pixel-LLMs is `bytedance/Sa2VA`, which includes unified models like `Sa2VA`, `SAMTok`, and `VRT` alongside research projects built on top of these models. ### Key Features: - **Sa2VA**: A unified model combining SAM-2 with MLLMs for dense grounded understanding of images and videos. - **VRT**: Object-level grounded reasoning built on Sa2VA, with `VRT-Bench` and `VRT-80k` training data. - **SAMTok**: A unified mask-token interface that allows any MLLM to generate and understand masks. - **Extensions**: **SaSaSa2VA** is a segmentation-augmented extension of Sa2VA, and **Pixel-SAIL** is a single-transformer pixel-level grounding model. ### Tech Stack: The Pixel LLMs projects are built using Python, leveraging the `uv` package for dependency management. ### Why It Matters: Pixel LLMs aim to enhance the capabilities of multimodal language models by providing pixel-level, dense grounded understanding, which is crucial for applications requiring precise and contextually relevant information extraction from images and videos.

Overview: Bub is a tiny agent runtime that allows developers to easily integrate and customize their own Python agents with a composable plugin system. Key features: - Customizable Python agent kernel - Composable plugin system - Support for FastAPI for HTTP endpoints - Embeddable `BubAgent` for reuse in services - Support for macOS, Linux, and Windows Tech stack: Python Why it matters: Bub simplifies the process of integrating and customizing Python agents, making it easier for developers to leverage the power of Bub's agent kernel and plugins in their projects.

Overview: This project introduces a technique called "antialiased CNNs," which improves the stability and accuracy of Convolutional Neural Networks (CNNs) by applying antialiasing to them. The method enhances the performance of CNNs without requiring additional computational resources. Key features: - Antialiasing applied to CNNs - Improved stability and accuracy - Available models for various architectures (e.g., ResNet, VGG, DenseNet) - Easy installation via `pip install antialiased-cnns` - Additional models and variants added in October 2020 Tech stack: Python Why it matters: This technique is crucial for enhancing the performance of CNNs, making them more reliable and accurate in various applications, such as image classification and object detection. Repository: adobe/antialiased-cnns GitHub stars: 1687

Overview: This project implements a lightweight Generative Adversarial Network (GAN) model proposed in the ICLR 2021 conference. The model is designed to generate high-resolution images within a short training period, making it suitable for real-time applications. Key features: - The lightweight GAN model is implemented in Pytorch. - It achieves convergence on a single GPU within a few hours of training, even with 1024 resolution images. - The model can generate images of various sizes, including 512x512 and 256x256 flowers, as well as a pizza image. Tech stack: The project uses Pytorch for the implementation of the GAN model. Why it matters: The lightweight GAN model is particularly useful for applications that require generating high-quality images quickly, such as in the field of computer vision or image synthesis for training models. It can be trained efficiently on a single GPU, making it a practical solution for real-time or low-resource environments.

Overview: Kor is a prototype designed to help extract structured data from text using LLMs, specifically for old-style LLMs that lacked chat interfaces and tool calling capabilities. It uses LangChain's framework and is compatible with both pydantic v2 and v1. Key features: - Supports pydantic v2 and v1 - Compatible with LangChain framework - Handles parsing and JSON modes - Integrates with LangChain for better extraction quality Tech stack: Python Why it matters: Kor is a prototype that simplifies the process of extracting structured data from text using LLMs, making it easier for developers to integrate with existing LLMs without the complexity of tool calling or JSON modes. GitHub stars: 1685 Repository: eyurtsev/kor Primary language: Python Downloads: 1,000+ (as of the last update) Open Issues: 50+ (as of the last update) GitHub description: LLM(😽) Readme excerpt: Kor is a prototype that "helps" you extract structured data from text using LLMs 🧩. Specify the schema of what should be extracted and provide some examples. Kor will generate a prompt, send it to the specified LLM and parse out the output. You might even get results back. So yes – it’s just another wrapper on top of LLMs with its own flavor of abstractions. 😸 Integrated with the [LangChain framework 😽💗 🦜🔗](https://python.langchain.com/en/latest

### Overview: d3rlpy is an offline deep reinforcement learning library designed for practitioners and researchers, offering both offline and online RL capabilities. It supports distributional Q functions and data-parallel distributed training, making it a powerful tool for solving various RL problems. ### Key features: - **Offline RL**: Supports state-of-the-art offline RL algorithms, ideal for scenarios where online interaction is not feasible. - **Online RL**: Supports conventional state-of-the-art online training algorithms, enabling versatility in solving RL problems. - **User-friendly API**: Provides intuitive APIs for zero-knowledge DL library usage, making it accessible to beginners. - **Distributional Q Function**: The first library to support distributional Q functions, enhancing performance. - **Data-parallel Distributed Training**: Supports data-parallel distributed offline RL training, allowing scalability with multiple GPUs or nodes. ### Tech stack: Python ### Why it matters: d3rlpy is a comprehensive and user-friendly library that simplifies the process of implementing and experimenting with deep reinforcement learning algorithms, making it an essential tool for researchers and practitioners alike.

### Overview: This repository contains updated slides, notebooks, and transcriptions for the NYU Deep Learning Spring 2021 course, focusing on the advanced Latent Variable Energy-Based Models (LV-EBMs) module. The slides are redesigned to cater to the current understanding and advancements in the field. ### Key Features: - Updated slides and notes for the LV-EBM module. - Updated notebooks for practical exercises and assignments. - Transcribed lectures for easier understanding. - Updated curriculum and lecture content to reflect recent developments in deep learning. ### Tech Stack: Jupyter Notebook ### Why it Matters: The updated slides and materials ensure that the students are up-to-date with the latest advancements in deep learning, particularly focusing on the advanced LV-EBM module. This is crucial for keeping the students informed and motivated to keep learning and improving in the field of deep learning. ### Repository: Atcold/NYU-DLSP21 GitHub Stars: 1683 ### Primary Language: Jupyter Notebook ### GitHub Description: NYU Deep Learning Spring 2021 (NYU-DLSP21) ### Content New Organisation: - Updated curriculum and lecture content. - Redesigned slides for the LV-EBM module. - Updated notebooks for practical exercises and assignments. ### Previous Releases: - NYU-DLSP20 (major release) - NYU-DLSP19 (previous release) - AIMS-DLFL19 (previous release) - CoDaS-HEP18 (previous release) - NY

### Overview: Cheat Engine MCP Bridge is a powerful tool that enables developers to automate the analysis of game memory and program structures using natural language queries. This feature allows for quick identification of specific functions, structures, and patterns, significantly reducing the time required for reverse engineering and security audits. ### Key Features: - **Memory Analysis**: Instantly read any memory, including integers, floats, strings, and pointers. - **Pointer Chains**: Trace and resolve pointer chains efficiently. - **Structures**: Automatically analyze and document structures with field types and values. - **C++ Objects**: Identify C++ objects using RTTI. - **Function Analysis**: Disassemble and analyze functions. - **Debugging**: Perform debugging invisibly with hardware breakpoints and hypervisor support. - **Automation**: Automate repetitive tasks, such as finding specific hooks, decryptors, and other game updates. ### Tech Stack: The project leverages Lua as the primary scripting language, utilizing Python for the MCP server and Cheat Engine for the GUI and memory access. ### Why It Matters: - **Efficiency**: Automates the tedious process of manual memory analysis, allowing developers to focus on more strategic tasks. - **Accuracy**: Ensures precise identification of memory structures and functions, reducing the risk of errors. - **Speed**: Reduces the time required for reverse engineering and security audits, making the process more efficient and effective. ### Repository: The repository is hosted on GitHub, with 1,682 stars. It includes the MCP server, Lua script, and a detailed README that provides installation instructions and usage examples. ###

Overview: Meet Ava, the WhatsApp agent that turns the Turing Test into a reality, inspired by the film "Ex Machina." By the end of this course, you'll build your own Ava capable of engaging in realistic WhatsApp conversations. Key features: - Receiving and sending WhatsApp messages - Understanding voice - Recognizing images - Sending voice notes - Sharing updates - Sending images of daily activities Tech stack: Python Why it matters: This course provides a practical, hands-on approach to building a modern AI assistant that can mimic human-like interactions, making it a valuable tool for developers looking to enhance their applications with AI capabilities.

### Overview: Mean Teacher is a state-of-the-art semi-supervised learning method that significantly improves accuracy and applicability to large datasets. It achieves remarkable results on ImageNet and CIFAR-10, surpassing traditional approaches like Variational Auto-Encoders and Conditional Generative Adversarial Networks. ### Key Features: - **Semi-Supervised Learning**: Utilizes a student-teacher setup with a shared architecture. - **Consistency Cost**: Adds a consistency cost between the student and teacher outputs to improve accuracy. - **Exponential Moving Average (EMA)**: Updates the teacher weights towards the student weights. - **Improved Accuracy**: Demonstrates superior performance on ImageNet and CIFAR-10 datasets compared to traditional methods. ### Tech Stack: Python ### Why It Matters: Mean Teacher offers a simple yet effective solution for semi-supervised learning, particularly on large datasets. By leveraging shared parameters and introducing a consistency cost, it significantly enhances the accuracy of semi-supervised models, making it a valuable tool for researchers and practitioners in the field of computer vision and machine learning. ### Repository: CuriousAI/mean-teacher ### GitHub Stars: 1681 ### GitHub Description: A state-of-the-art semi-supervised method for image recognition. This repository contains the implementation of Mean Teacher, including both TensorFlow and PyTorch versions, and provides guidelines for choosing hyperparameters and tuning. ### Implementation: The repository includes two implementations: - **TensorFlow**: Follows typical TensorFlow idioms and is suitable for adapting to new datasets. - **PyTorch**: Follow

Overview: Sense2vec is a Python library that provides contextually-keyed word vectors, allowing for the querying of vectors based on part-of-speech tags and entity labels. It is a simplified version of Word2Vec, designed to be easily integrated into spaCy models. Key features: - Query vectors for multi-word phrases based on part-of-speech tags and entity labels. - spaCy pipeline component and extension attributes. - Fully serializable, enabling easy integration into spaCy models. - Optional caching of nearest neighbors for fast "most similar" queries. - Trainable vectors using pretrained spaCy model, raw text, or GloVe/Word2Vec via fastText. - Prodigy annotation recipes for evaluating models, creating lists of similar multi-word phrases, and converting them into pattern-based annotations. Tech stack: Python Why it matters: Sense2vec simplifies the process of obtaining contextually relevant word vectors, making it easier to integrate semantic analysis into various applications, including NLP tasks and rule-based Natural Language Processing (NLP) models.

### Overview: Paddler is an open-source LLM/VLM load balancer and serving platform designed for self-hosting LLMs (and VLMs) at scale. It simplifies the process by providing a developer-friendly experience, reducing the number of moving parts, and enabling simple deployments. ### Key features: - Inference through a built-in LLaMA engine - LLM-specific load balancing - Dynamic agent addition for easy integration with autoscaling tools - Request buffering for scalable scaling from zero hosts - Dynamic model swapping - Built-in web admin panel for management, monitoring, and testing - Observability metrics ### Tech stack: Rust ### Why it matters: Paddler offers a more streamlined approach to running and deploying LLMs at scale, providing a better developer experience and reducing complexity compared to alternative solutions. It is designed for organizations handling sensitive data, ensuring privacy and reliability, and offering cost control and predictability in LLM usage.

### Overview: Ghost OS is a full computer experience for AI agents, allowing them to interact with macOS without needing screenshots or vision models. It provides memory and intelligence to AI agents, enabling them to perform tasks on a Mac. ### Key Features: - **AI Agent Interaction**: Enables AI agents to interact with macOS applications. - **Memory and Intelligence**: Provides AI agents with memory and intelligence. - **Self-Learning Recipes**: Allows AI agents to learn and remember actions through self-learning recipes. - **Persistent AI Agents**: Supports persistent AI agents that can run on dedicated VMs. ### Tech Stack: Swift 6.2, macOS 14+, MCP-compatible. ### Why It Matters: Ghost OS simplifies the interaction between AI agents and macOS, making it easier for AI agents to perform tasks without needing screenshots or vision models. This allows AI agents to operate any application on a Mac, enhancing their capabilities and making them more versatile and efficient.

Overview: Open Source Vizier is a Python-based research interface for blackbox and hyperparameter optimization, built on the Google Vizier service. It provides a reliable and flexible solution for tuning complex objectives. Key features: - A Python-based service for black-box optimization and research. - Supports distributed client-server systems. - Provides a user API for optimization, a developer API for algorithm implementation, and a benchmarking API for objective functions and methods. Tech stack: Python. Why it matters: OSS Vizier offers a reliable and flexible solution for tuning complex objectives, making it suitable for research and development in various fields.

Overview: Voxtype is a voice-to-text tool designed for Linux, offering high-speed transcription with real-time processing capabilities. It supports multiple engines and runs on local systems without cloud services or telemetry. Key features: - **Speed and Engines**: Transcription at 9-11× real-time with models quantized to 1.5 GB (q4f16) and built-in punctuation, capitalization, and normalization. - **Dynamic Model Loading**: Models load on first use and unload when idle, allowing for flexible engine configuration. - **Nine Transcription Engines**: Support for Whisper, Parakeet, Moonshine, SenseVoice, Paraformer, Dolphin, Omnilingual, Cohere, and OpenVINO Whisper. - **Meeting Mode**: Continuous transcription with speaker attribution and export options. - **Native Linux Integration**: Supports Hyprland, Niri, Sway, River, GNOME, KDE, and evdev fallbacks. - **Remote Whisper Servers**: Optional access to remote Whisper servers for additional features. Tech stack: Rust, Rust bindings for C/C++ libraries, and Rust for the CLI and configuration. Why it matters: Voxtype provides a fast, local, and feature-rich solution for voice-to-text transcription on Linux, offering a high-quality transcription experience without the need for cloud services or telemetry. Repository: peteonrails/voxtype GitHub stars: 1666 GitHub description: Voice-to-text with push-to-talk for Wayland compositors

Overview: ICON is an innovative model for generating clothed human images from 3D normals, showcasing its potential in various applications. Key features: - Cloth refinement module - Support for KeypointNeRF and PyTorch Lightning - Evaluation metrics on THuman20 and CAPE datasets - Training instructions provided Tech stack: Python Why it matters: ICON offers a significant advancement in generating realistic clothed human images, enhancing the field of computer graphics and virtual reality.

### Overview: Ragbits is a comprehensive toolkit designed for rapid development of GenAI applications. It provides a reliable and scalable foundation for integrating various language models and vector stores, ensuring type safety and ease of use. ### Key features: - **Flexible LLM Integration**: Supports multiple LLMs via LiteLLM or local models. - **Type-Safe Prompts**: Enforces strict type safety in model interactions. - **Vector Store Support**: Connects to Qdrant, PgVector, or any custom vector store. - **Developer Tools**: Includes vector store management, prompt testing, and CLI access. - **Modular Installation**: Only install the necessary components for improved performance and dependencies. ### Tech stack: Python ### Why it matters: Ragbits simplifies the process of building and deploying GenAI applications, making it easier for developers to integrate cutting-edge language models and vector stores, ensuring reliable and scalable applications.

### Overview: Grad-CAM is a technique that visualizes the activation patterns of a deep neural network, providing insights into how the model makes its predictions. This method was introduced in the paper "[Grad-CAM: Why did you say that? Visual Explanations from Deep Networks via Gradient-based Localization](https://arxiv.org/abs/1610.02391)" by Ramprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Batra. ### Key Features: - **Grad-CAM Implementation**: Provides a way to visualize the activation patterns of a deep neural network. - **Visualization of Activation Patterns**: It highlights the regions of the input image that are most relevant to the model's predictions. - **Versatility**: Works with various types of neural networks, including VGG, AlexNet, and others. - **GPU Support**: Utilizes GPU for faster computation, making it suitable for large-scale applications. ### Tech Stack: - **Programming Language**: Lua - **Backend**: Caffe ### Why It Matters: Grad-CAM offers a powerful tool for understanding the decision-making process of deep learning models. By visualizing the activation patterns, it helps in interpreting the model's confidence and identifying the most important features that influence its predictions. This can be particularly useful in applications like image classification, object detection, and more, where understanding the model's reasoning is crucial. ### Repository: - **Repository**: ramprs/