Projects

### Overview: MIDI-GPT is a powerful transformer model designed for computer-assisted multitrack music composition. It allows users to fill in missing bars, generate new tracks, and control the output through various attributes such as note density, polyphony, note duration, and more. The model is trained on a large dataset and can be easily integrated into existing music arrangements, making it a versatile tool for composers and musicians. ### Key features: - **Fill in missing bars**: Preserve the existing arrangement while adding new music. - **Generate new tracks**: Create entirely new tracks based on musical attributes. - **Steer the output**: Control the output through various attributes, including note density, polyphony, note duration, and more. - **Run a MIDI-GPT API server**: Streamline the process with real-time updates, cancellation, and multiple models in one process. - **One-line setup**: Load pretrained models from HuggingFace Hub without needing a compiler. ### Tech stack: - **Python**: The primary programming language used for the project. ### Why it matters: MIDI-GPT provides a highly efficient and user-friendly solution for multitrack music composition, making it easier for musicians and composers to create complex arrangements with minimal effort. Its ability to generate and control music attributes makes it a valuable tool for both creative and technical users in the music industry. ### Repository: The project is hosted on GitHub at [Metacreation-Lab/MIDI-GPT](https://github.com/Metacreation-Lab/MIDI-GPT). It has received 116 GitHub stars,

### Overview: OceanGPT is a large language model designed for ocean science tasks, providing insights and capabilities for various applications in the field. ### Key Features: - OceanGPT is a comprehensive model for ocean science tasks. - It includes fine-tuning capabilities for custom question-answering applications. - Supports inference using efficient models like sglang, vLLM, ollama, and llama.cpp. ### Tech Stack: Python ### Why It Matters: OceanGPT offers a powerful toolset for researchers and practitioners in ocean science, enabling them to analyze and interpret vast amounts of data related to the ocean environment.

### Overview: AndroLLM is a production-grade AI platform for Android that leverages local `.litertlm` model inference using LiteRT-LM runtime, CPU and GPU acceleration, cloud provider integration, persistent memory, and hands-free voice interaction. ### Key features: 1. **Local Inference Engine**: Runs `.litertlm` language models entirely on-device through Google's LiteRT-LM runtime. 2. **GPU Acceleration**: Utilizes OpenCL GPU delegate with automatic CPU fallback for enhanced performance. 3. **Multi-turn Chat**: KV-cache persistence and diff-based continuation for seamless conversations. 4. **Cloud Providers**: Integrates with any LiteLLM-compatible endpoints for scalability. 5. **Persistent Memory**: Stores vector embeddings and hybrid retrieval for enhanced performance. 6. **Voice Assistant**: Achieves hands-free voice interaction with offline capabilities. ### Tech stack: Kotlin ### Why it matters: AndroLLM provides a complete, production-quality solution for Android developers looking to integrate powerful language models without relying on the cloud or sending data outside their devices. ### Repository: ShadowSafin/AndroLLM ### GitHub stars: 112 ### GitHub description: Open-source Android AI using LiteRT-LM with hardware acceleration, cloud providers, memory, and voice.

### Overview To address the prevalence of low-quality models in the Objaverse dataset, we meticulously curated a collection of high-quality 3D objects and developed an effective classifier capable of scoring the entire dataset. Our extensive annotation system considers geometric structure and texture information, enabling researchers to filter training data according to specific requirements. ### Key Features - **Quality and Attribute Annotations**: We defined quality scores to assess the usefulness of 3D objects for machine learning training, categorizing them into low, medium, high, and superior quality. - **Binary Traits**: We identified and annotated binary traits such as transparency, scene, single color, not a single object, and figure. - **Art Style (Experimental)**: We introduced an experimental art style annotation to further refine the dataset. ### Tech Stack - Python ### Why It Matters By providing high-quality annotations and curated datasets, our work enables researchers to perform generative AI tasks like text-to-3D and image-to-3D more effectively and efficiently. The superior quality of our dataset allows for better performance in image-to-3D generation tasks and faster training loss convergence, while careful curation and rich annotation can compensate for the raw dataset size.

Overview: Omni-SafetyBench is a comprehensive benchmark designed to evaluate the safety of large language models that integrate text, image, video, and audio inputs. It consists of 24 distinct subcategories with 972 samples each, totaling over 23,000 samples. Key features: - Comprehensive evaluation of safety for large language models - 24 distinct subcategories across three modality paradigms: unimodal, dual-modal, and omnimodal - 972 samples per subcategory, totaling over 23,000 samples Tech stack: Python Why it matters: This benchmark is crucial for ensuring the safety of AI models that process multimodal data, providing a standardized way to assess their potential risks and biases.

### Overview: RLCSD, or Reinforcement Learning with Contrastive On-Policy Self-Distillation, is a method designed to improve the performance of reasoning models in reinforcement learning. By leveraging contrastive on-policy self-distillation, RLCSD addresses the issue of premature response-length shrinkage and training instability that can occur during the learning process. This technique focuses on preserving the task correctness while concentrating on the relevant tokens, thereby enhancing the overall performance across various reasoning tasks. ### Key features: - **Privilege-Induced Style Drift**: RLCSD introduces a mechanism to suppress shared stylistic shifts, making the remaining token-level signal more focused on task correctness. - **Teacher–Student Gaps**: RLCSD contrasts the teacher–student gaps under correct and incorrect reference solutions, using the same prompt template to suppress shared stylistic shifts. - **Modulation of Verifier-Derived GRPO Advantage**: The method uses the contrastive token-level signal to modulate the verifier-derived GRPO advantage, preserving its update direction while stabilizing the training process. ### Tech stack: The project is built using Python, leveraging the Qwen framework for development and evaluation. ### Why it matters: RLCSD significantly outperforms other methods across various reasoning tasks, including math and logic reasoning, across different model scales. It achieves the highest math and logic averages among the evaluated methods, demonstrating its effectiveness in improving reasoning model performance.

### Overview: Kindi is a custom inference engine designed for low-cost NVIDIA GPUs, including the Quadro K420, Quadro K1200, GeForce GTX 1080, and RTX 3080. It supports a wide range of language models, including Bonsai 2B to 27B parameters, and is optimized for performance on these GPUs. ### Key Features: - **Custom Inference Engine**: Optimized for low-cost NVIDIA GPUs. - **Supports Multiple GPUs**: Runs on multiple GPUs in tensor parallel mode. - **Prompt Cache**: Enhanced prompt cache for agents and setups with multiple servers. - **OpenAI-Compatible Server**: Offers an OpenAI-compatible server for chat clients, scripts, and agents. - **High Performance**: Writes 22.9 tokens per second on the Quadro K420, and up to 27B parameters on the RTX 3080. - **Token Efficiency**: Generates 13x faster than LLaMA.cpp on the same card with the same model file and context. - **GPU Unlock Module**: Offers experimental support for unlocking NVIDIA GPUs with the `gpu-unlock` module by Zagitta. ### Tech Stack: - Python ### Why It Matters: Kindi is designed to be highly efficient and performant, making it ideal for low-cost GPU setups. Its ability to handle multiple GPUs in tensor parallel mode and its support for a wide range of language models make it a versatile tool for developers and researchers.

Overview: This project is an async Python library designed to automate the solving of ReCAPTCHA v2 using Playwright, making it easier for developers to integrate this functionality into their projects. Key features: - Solves ReCAPTCHA v2 using Playwright - Supports asynchronous operations for improved performance - Provides a trained model for detecting specific classes of ReCAPTCHA images Tech stack: Python Why it matters: This library simplifies the process of automating ReCAPTCHA solutions, making it more accessible and efficient for developers working on projects that require verification checks. Repository: embium/solverecaptchas Primary language: Python GitHub stars: 116

Overview: This project is a comprehensive course on Deep Learning for Speech, covering various tasks such as speech processing, speech recognition, and speaker recognition. The course materials include slides, recordings, and seminars, providing a structured learning experience. Key features: - Digital Signal Processing (DSP) - Speech Recognition (CTC, Beam Search, Rescoring) - Speech Recognition (Encoder-Decoder, Streaming, RNN-T, Decoder-only) - Self-Supervised Learning (wav2vec2.0, HuBERT, BEST-RQ, GigaAM) - Speaker Recognition - Voice Activity Detection, Speaker Diarization, Speaker-attributed ASR - Audio-Conditioned LLMs - Text-to-Speech (Conventional Models, Codecs, Vocoders, Recent Advancements, Speech-to-Speech LLMs) Tech stack: Jupyter Notebook Why it matters: This course is essential for developers looking to enhance their skills in deep learning and speech processing, providing a solid foundation in various applications and techniques used in the field of speech technology. Repository: georgygospodinov/speech_course GitHub stars: 116 GitHub description: Deep Learning for Speech

### Overview: Training an n-gram based Language Model using KenLM toolkit for Deep Speech 2 involves installing necessary dependencies, preprocessing data, and then training and binarizing the model. The resulting binary format provides significant performance improvements for scoring sentences. ### Key features: - Installing KenLM toolkit dependencies - Preprocessing training data - Training an n-gram based Language Model using KenLM - Binarizing the model for faster loading and flexibility - Scoring sentences using the trained model ### Tech stack: Python, KenLM toolkit, Boost, GNU C++ Library, BZip2, LZMA ### Why it matters: This project demonstrates the practical application of KenLM toolkit in training and using an n-gram based Language Model for Deep Speech 2, showcasing the toolkit's capabilities in handling large datasets and providing efficient performance improvements for language modeling tasks.

### Overview: Giga Pisar is a free local Russian dictation tool for macOS, powered by GigaAM v3, a state-of-the-art model for Russian speech recognition. It offers accurate punctuation and capitalization without the need for external dependencies, running fully on-device. Simply hold a key, speak, release, and the text appears wherever your cursor is. ### Key Features: - **Voice Dictation**: Dictate text using voice commands. - **Punctuation and Capitalization**: Automatically adds punctuation and capitalizes dictated text. - **On-Device Recognition**: Runs entirely on macOS without any external dependencies. - **Customizable Menu**: Supports local language settings and can be used in any application. - **Model Download**: Available for macOS 13.4+ and requires a 204 MB download. - **Self-contained App**: Can be built with the model embedded for a more compact size. ### Tech Stack: Swift ### Why It Matters: Giga Pisar is a practical tool for Russian language learners and users who prefer a local solution. It offers a seamless dictation experience without the need for external servers or dependencies, making it ideal for those who want to stay on macOS. ### Repository: - **Repository**: moznoazachem/giga-pisar - **GitHub Stars**: 113 - **GitHub Description**: Гига Писарь: бесплатная локальная русская диктовка для macOS на GigaAM v3. Зажал правый ⌘, сказал, отпустил

### Overview: Ogham MCP is a persistent, searchable shared memory system designed to help AI coding agents retain context across sessions and clients. It uses a structured retrieval engine that can answer path-based questions, unlike traditional vector similarity-based methods. ### Key features: - **Persistent Memory**: Stores context and information across sessions and clients. - **Structured Retrieval**: Hybrid search and typed-edge graph, allowing for path-based answers. - **Cross-Client Support**: Works seamlessly across different clients like Claude Code, Cursor, and OpenCode. ### Tech stack: Python ### Why it matters: Ogham MCP provides a solution to the problem of AI coding agents forgetting context between sessions and clients, ensuring that decisions, gotchas, and the shape of the codebase are remembered and reused across different environments and tools. ### Repository: ogham-mcp/ogham-mcp ### GitHub stars: 116 ### Primary language: Python ### GitHub description: Shared memory MCP server — persistent, searchable, cross-client Claude, Opencode

Overview: This repository implements Speculative Decoding, a decoding technique for transformers that accelerates inference without changing the output distribution. It provides three generation strategies: classic auto-regressive decoding, beam search decoding, and speculative decoding. Key features: - Classic auto-regressive decoding - Beam search decoding with length penalty - Speculative decoding Tech stack: Python Why it matters: Speculative Decoding is a fast and efficient method for accelerating transformer-based models, allowing for faster inference times without the need for additional fine-tuning.

Overview: This project introduces Flipped Learning, a method that enhances language models for zero-shot learning, specifically focusing on improving their performance in NLP tasks. Key features: - Flipped Learning - Zero-Shot Learners - Stronger Language Models Tech stack: Python Why it matters: This project showcases how Flipped Learning can significantly enhance the capabilities of language models, making them more effective in zero-shot learning scenarios, thereby improving their performance across various NLP tasks.

Overview: VT.ai is an open-source multimodal AI chat application designed for dynamic conversation routing, supporting multiple LLM providers with semantic-based routing and comprehensive multimodal capabilities. Key features: - Supports multiple LLM providers - Uses vector-based semantic routing for fast and optimized performance - Provides comprehensive multimodal support including text chat, image generation, visual analysis, voice interaction, and web search - Supports flexible configuration options for API key management and model selection Tech stack: Python Why it matters: VT.ai offers a powerful and flexible platform for building AI chat applications, making it an ideal choice for developers looking to integrate multimodal capabilities into their projects.

### Overview: This project is an implementation of the NeurIPS 2023 paper titled "Towards Generic Semi-Supervised Framework for Volumetric Medical Image Segmentation." The code aims to address the challenge of semi-supervised medical image segmentation methods in more complex real-world scenarios. ### Key features: - **Semi-Supervised Learning (SSL):** Enhances the performance of medical image segmentation by incorporating both labeled and unlabeled data. - **Instance Segmentation (IS):** Segments individual instances within a medical image without relying on a global segmentation mask. - **Instance Segmentation with Label Propagation (ISLP):** Propagates labels from labeled instances to unlabeled instances to improve segmentation accuracy. ### Tech stack: Python ### Why it matters: This work encourages the development of semi-supervised medical image segmentation methods that can handle more realistic and complex scenarios, rather than focusing solely on ideal experimental environments. By consolidating all four settings into a single codebase, the framework allows for the execution of any task using a single bash file, making it highly versatile and efficient.

Overview: OgbujiPT is a comprehensive client-side toolkit designed for using large language models, offering a unified API for storing, retrieving, and managing semantic knowledge across multiple backends. It supports dense vector search, sparse retrieval, hybrid search, and more, ensuring flexibility and efficiency. Key features: - In-memory (`RAMDataDB`, `RAMMessageDB`): Perfect for prototyping and small applications. - PostgreSQL + pgvector: Production-ready persistent storage with advanced indexing and SQL capabilities. - Qdrain: High-performance vector database with distributed capabilities. - Dense vector search: Semantic similarity using embeddings. - Sparse retrieval: Keyword-based exact term matching. - Hybrid search: Combines multiple strategies using Reciprocal Rank Fusion. - Reranking: Cross-encoder reranking for improved precision. Tech stack: Python Why it matters: OgbujiPT simplifies the integration of large language models by providing a straightforward and transparent approach, avoiding the complexities of over-engineered frameworks that often plague the LLM ecosystem. It ensures that every abstraction is justified, making it accessible and efficient for developers. Repository: OoriData/OgbujiPT GitHub stars: 115 GitHub description: Client-side toolkit for using large language models, including where self-hosted.

Overview: This project is a PyTorch implementation of the paper "Multi-mapping Image-to-Image Translation via Learning Disentanglement" presented at NeurIPS 2019. The implementation includes training scripts for Season Transfer and Semantic Image Synthesis, as well as pretrained models and custom experiment capabilities. Key features: - Season Transfer - Semantic Image Synthesis Tech stack: Python Why it matters: This project provides a practical tool for image-to-image translation tasks, demonstrating the effectiveness of learning disentanglement in image-to-image translation. Repository: Xiaoming-Yu/DMIT Primary language: Python GitHub stars: 115 GitHub description: Multi-mapping Image-to-Image Translation via Learning Disentanglement. NeurIPS2019 README excerpt:

### Overview: ComfyUI-AusBoss is a collection of custom ComfyUI nodes designed for image and video editing workflows. Created by AusBoss, these nodes offer a range of advanced features such as mask creation, image cropping and resizing, inpainting, video trimming, and more. The pack is continually updated to include new and improved features, making it a valuable addition to any ComfyUI user's library. ### Key Features: - **Mask by Touch:** Click on an image to generate a mask. - **Mask by Name:** Type a description to get a mask and a cut-out. - **Image Crop + Rotate + Pad:** Drag handles to crop and pad images. - **Stitch Inpaint:** After an inpaint or outpaint, restore original pixels. - **Video Crop + Rotate + Pad → Clip:** Trim and pad video clips. - **Tiled Upscale:** Upscale images in tiles to megapixels. - **Image Folder:** Batch process images in a folder. - **Layout Boxes:** Draw boxes for text and image placement. - **Image Compare A/B:** Compare two images side by side. - **LoRA Loader:** A LoRA stack for advanced image processing. - **Workflow Switches:** Toggle workflow nodes on and off. ### Tech Stack: JavaScript ### Why It Matters: These nodes are essential for anyone looking to enhance their ComfyUI workflows with advanced image and video editing capabilities. Whether you're a professional or a hobbyist, these features can save you time and streamline your editing processes. The pack is constantly updated, ensuring

Overview: CntxtPY is a Python tool designed to enhance the precision and efficiency of large language models (LLMs) by providing a compressed knowledge graph of your project's codebase. This tool minimizes token usage, focusing LLMs on key insights, and supports modern Python frameworks and patterns. Key features: - Deep analysis of Python codebases - Generates detailed knowledge graphs of module relationships, class hierarchies, function signatures, and more - Specially designed for LLM context windows - Built-in visualization capabilities - Supports modern Python frameworks and patterns Tech stack: Python Why it matters: CntxtPY helps LLMs navigate and comprehend your codebase more efficiently, providing clear insights into relationships, dependencies, and implementation patterns, ultimately improving the quality and speed of AI-driven decision-making processes. Repository: brandondocusen/CntxtPY Primary language: Python GitHub stars: 115 GitHub description: A discovery and compression tool for your Python codebase. Creates a knowledge graph for a LLM context window, efficiently outlining your project | Code structure visualization | LLM Context Window Efficiency | Static analysis for AI | Large Language Model tooling #LLM #AI #Python #CodeAnalysis #ContextWindow #DeveloperTools