Projects

Overview: VRCOSC is a modular node-programming language, program creator, animation system, toolkit, router, and debugger designed for VRChat. It simplifies the creation of OSC programs, integrates with other APIs, and provides a central source for useful OSC programs. Key features: - Custom node-programming language (Pulse) - ChatBox animation system - Per-avatar profiles - Built-in router - Automatic updates - Common API interfaces - Drag-and-drop Unity prefabs - Debug tools Tech stack: C# Why it matters: VRCOSC streamlines the development process for VRChat projects, making it easier to create and manage OSC programs, and provides a comprehensive toolkit for developers to enhance their VRChat experiences.

### Overview: Doc Product is a project that uses deep learning language models to answer medical questions. It combines state-of-the-art models like BERT and GPT-2 with TensorFlow 2.0 to extract and generate medical information from text. ### Key Features: - Fine-tuned BERT and GPT-2 models for medical question answering. - Utilizes TensorFlow 2.0 for efficient model training and deployment. - Includes a dataset of over 700,000 medical questions and answers. - Uses FAISS for efficient vector search and negative sampling. ### Tech Stack: - TensorFlow 2.0 - BERT (pre-trained BioBERT weights) - GPT-2 (OpenAI's GPT-2-117M parameters) - Keras.Model for network heads ### Why It Matters: This project demonstrates the potential of deep learning models in scientific encoding and retrieval, providing a novel solution to a difficult problem. It showcases the capabilities of transformer architectures and latent vector search within TensorFlow 2.0, making it a valuable contribution to the field of medical NLP.

Overview: The Alan AI SDK for React Native enables developers to integrate intelligent features into their applications through voice commands, creating a self-coding engine that adapts and evolves automatically. Key features: - Embedding an AI agent into Android applications - Quick creation of AI agents using the Alan AI Platform - Real-time generation of business logic and UI using the Three-Layer AI architecture - Integrating AI-driven interfaces into existing apps in days, not months Tech stack: React Native, JavaScript, Three-Layer AI architecture Why it matters: The Alan AI SDK for React Native simplifies the integration of AI-driven features into applications, making them more adaptive and context-aware.

Overview: Transformers.jl is a Julia implementation of transformer-based models, utilizing the Flux.jl framework. Key features: - Support for various transformer models - Integration with HuggingFace for model loading and fine-tuning - Tokenization and encoding capabilities Tech stack: Julia, Flux.jl Why it matters: This package provides a flexible and efficient way to work with transformer models in Julia, making it easier to experiment with and integrate these models into various applications.

### Overview: LLaVA-Mini is a unified large multimodal model designed to efficiently understand images, high-resolution images, and videos. It significantly improves efficiency while preserving vision capabilities. ### Key Features: - **Efficient Tokenization**: Uses only 1 vision token to represent each image, reducing computational effort by 77% and response latency to 40 milliseconds. - **Reduced VRAM Usage**: Decreases VRAM memory usage from 360 MB/image to 0.6 MB/image, supporting 3-hour video processing. - **Interpretable Model**: Guided by interpretability within LMM, LLaVA-Mini maintains visual understanding while being highly efficient. ### Tech Stack: - **Python** as the primary programming language. ### Why It Matters: - **Efficiency**: Significantly reduces computational effort, response latency, and VRAM usage, making it highly efficient for image and video understanding. - **Interpretability**: Maintains visual understanding while being highly efficient, making it a valuable tool for large multimodal models (LMMs). ### Repository: - **Repository**: ictnlp/LLaVA-Mini - **Primary Language**: Python - **GitHub Stars**: 573 ### Demo: - **Download Model**: Available at [here](https://huggingface.co/ICTNLP/llava-mini-llama-3.1-8b). - **Run Scripts**: Use the following commands to interact with LLaVA-Mini: ```bash # Launch a controller

### Overview: SepLLM accelerates large language models by compressing segments into separator tokens, reducing KV cache and improving inference speed. ### Key Features: - **Efficient Inference**: Reduces KV cache by over 50% on the GSM8K-CoT benchmark. - **Native Sparse Attention**: Aims to be easy-to-use and plug-and-play. - **Comprehensive Results**: Demonstrated effectiveness across various training settings. ### Tech Stack: - Python ### Why It Matters: SepLLM offers a significant performance improvement for large language models, making them more efficient and scalable.

Overview: This project is a C++ implementation of MiniGPT4, focusing on 4-bit, 5-bit, 6-bit, 8-bit, and 16-bit CPU inference using the GGML library. Key features: - Pure C++ implementation - Supports 4-bit, 5-bit, 6-bit, 8-bit, and 16-bit quantization - Inference capabilities for MiniGPT4 models - Compatible with Hugging Face Spaces for quickstart in Colab Tech stack: C++ Why it matters: This project provides a lightweight, efficient alternative to Python-based solutions, making it suitable for applications requiring high performance and low memory usage.

Overview: Tango is a tool designed to help researchers organize and manage their experiments in a structured way, ensuring that results can be easily cached and reused across multiple runs. Key features: - Step-based organization of experiments - Integration with various integrations like PyTorch, Wandb, and datasets - Automatic caching of results to speed up experiments - Support for multiple integrations for different needs Tech stack: Python Why it matters: Tango simplifies the process of managing experiments, making it easier to reproduce results and accelerate research by leveraging cached results. Repository: allenai/tango GitHub stars: 572 GitHub description: Organize your experiments into discrete steps that can be cached and reused throughout the lifetime of your research project.

Overview: Cambrian-S aims to enhance spatial sensing capabilities in video data by leveraging predictive sensing techniques. This project builds upon the existing VSI-590K dataset to create a more powerful model that can better understand and interpret spatial relationships within videos. Key features: - Cambrian-S Weights: Our collection of 3M open-sourced video instruction tuning data. - VSI-590K Dataset: A curated dataset for spatial sensing. - VSI-SUPER Benchmark: A benchmark designed for spatial supersensing. - Cambrian-S Model Weights: We release the weights, training code, and evaluation suite for Cambrian-S. Tech stack: Python Why it matters: Cambrian-S represents a significant advancement in spatial sensing capabilities, offering improved performance in understanding and interpreting spatial relationships within video data. This project is crucial for researchers and developers looking to enhance their models' ability to recognize and analyze spatial information effectively. Repository: cambrian-mllm/cambrian-s Primary language: Python GitHub stars: 572

### Overview: MDFlux is a powerful tool that converts any document into clean, AI-ready Markdown, making it easier for language models to understand and process. It supports a wide range of file formats and can read scanned PDFs, batch folders, and even run offline, significantly reducing the number of tokens compared to vision models. ### Key Features: - **Document Conversion**: Converts any document (PDF, DOCX, PPTX, XLSX, EPUB, HTML, CSV, JSON, XML, TXT, MD, images, audio) into clean Markdown. - **OCR Support**: Reads scanned PDFs and converts them into editable text. - **Offline Functionality**: Runs completely offline, allowing users to work without an internet connection. - **Token Efficiency**: Uses far fewer tokens than vision models, making it more cost-effective and efficient. ### Tech Stack: - **Python**: The primary programming language used for development and conversion. ### Why It Matters: - **Efficiency**: Significantly reduces the number of tokens needed for language model processing, making it more efficient and cost-effective. - **Accessibility**: Allows users to work offline, ensuring they can process their documents without an internet connection. - **Versatility**: Supports a wide range of document types, making it a versatile tool for various industries and applications. ### GitHub Stars: - **GitHub Stars**: 573 ### Primary Language: - **Primary Language**: Python ### GitHub Description: Turn any document into clean, AI-ready Markdown. Local-first desktop app: reads scanned PDFs, batches folders, runs offline,

Overview: This is a TensorFlow implementation of the Siamese Fully-Convolutional Network (SiamFC) tracking method, based on the paper "Fully-Convolutional Siamese Nets for Object Tracking" by Luca Bertinetto et al. Key features: - Improved version of the Siamese tracking method presented in "End-to-end representation learning for Correlation Filter based tracking" by Jack Valmadre et al. - State-of-the-art performance at high frame rates - Pretrained networks available for download and usage - Virtualenv setup for easy project management - Compatibility with various video sequences and evaluation parameters Tech stack: Python Why it matters: This implementation provides a practical tool for researchers and developers interested in object tracking, offering a high-performance solution that can be easily integrated into existing projects. Repository: torrvision/siamfc-tf GitHub stars: 572 GitHub description: TensorFlow port of the tracking method described in the paper "Fully-Convolutional Siamese Nets for Object Tracking".

Overview: This repository contains summaries of papers on deep learning, covering various topics and advancements in the field. Key features: - Summaries of papers on deep learning - Papers from 2016 to 2018 - Reviews and analyses of selected papers Tech stack: Python, Markdown, and GitHub Pages Why it matters: This repository serves as a valuable resource for developers and researchers interested in deep learning, providing a concise overview of important papers and their contributions to the field.

Overview: 源内Web（AIインターフェース）は、デジタル庁が開発・運用する生成AI基盤です。行政職員が業務に活かす生成AIアプリケーションを迅速かつ安全に利用できる環境を提供します。 Key features: - AI アプリケーションの登録と管理機能 - 外部マイクロサービスの実行機能 - 適切なアクセシビリティ対策 - デジタル庁デザインシステムの適用 - サービスの監視とモニタリング機能 Tech stack: TypeScript Why it matters: 源内Webは、デジタル庁が行政業務に効果的に活用できる生成AIの基盤として機能しています。AI アプリケーションの登録と管理、外部マイクロサービスの実行、適切なアクセシビリティ対策、デジタル庁デザインシステムの適用、サービスの監視とモニタリング機能の提供により、行政職員が業務に活かす生成AIの環境を提供します。 Repository: digital-go-jp/genai-web Primary language: TypeScript GitHub stars: 571

### Overview: This project provides a dependency-free and lightweight implementation of CLIP, a widely used model for semantic image search and zero-shot image labeling, using the GGML library. It supports 4-bit, 5-bit, and 8-bit quantization and can be used with models from both OpenAI and LAION. ### Key features: - Dependency-free and lightweight inference thanks to GGML. - Support for 4-bit, 5-bit, and 8-bit quantization. - Inference with text-only, vision-only, and two-tower model variants. - No need for Numpy, TensorFlow, PyTorch, ONNX, or large third-party packages. - Short startup time, suitable for serverless deployments. - Supports batch inference for image encoding. - Updated to use a new model file structure in GGUF format. - Python bindings available without relying on large packages. - Batch inference support for image encoding. - Updated to match Huggingface's zero-shot behavior in the zero-shot pipeline. - Introduces Python bindings. - Batch inference support for image encoding. - Updated to use a new model file structure in GGUF format. - Preconverted models available for download. ### Tech stack: The project is built using C++ and relies on the GGML library for dependency-free inference. ### Why it matters: This implementation is ideal for applications requiring efficient inference on devices with limited computational or memory resources, such as edge devices or servers with cold start issues. The lightweight nature of the project makes it suitable for deployment in serverless environments, reducing

### Overview: MARTI (Multi-Agent Reinforcement Learning) is an open-source framework designed for training Large Language Models (LLMs) with Reinforcement Learning (RL). It combines centralized multi-agent interactions with distributed policy training, enabling powerful, scalable, and adaptive workflows. The framework supports both built-in graph-based workflows and popular third-party multi-agent frameworks. ### Key features: - **MARTI-v2**: Introduces tree search-augmented RL for complex reasoning tasks like code generation, enhancing multi-agent tree search for efficient exploration and high-quality reasoning trajectories. - **MARTI**: A framework for LLM-based Multi-Agent Systems (MAS) with Reinforcement Learning, combining centralized multi-agent interactions with distributed policy training. - **Tree Search-augmented RL**: Enables multi-turn exploration with adaptive node expansion and refinement, supporting ultra-long sequences up to 32K tokens and heterogeneous multi-agent training. ### Tech stack: - Python ### Why it matters: MARTI aims to advance reasoning capabilities beyond individual LLMs or reasoning models, fostering collective intelligence towards the development of general artificial intelligence.

### Overview: This repository is an awesome list of tools, frameworks, and resources for AI-driven development, curated by [sindresorhus/awesome](https://github.com/sindresorhus/awesome). ### Key features: - **AI Code Editors & IDEs**: Tools like Cursor, Bolt.new, Melty, Void, PearAI-App, Onlook, LlamaEdit, CodingIT, Monkey Code, Libra AI, NPC Studio, Claudable, Chara Codes, HumanLayer, NotBadAI IDE, and Kiro. - **Terminal & CLI Agents**: CLI agents for terminal-based development. - **IDE Extensions**: Extensions for enhancing the development workflow. - **Multi-Agent & Orchestration**: Tools for managing and orchestrating multiple AI agents. - **Code Generation & Automation**: Tools for generating and automating code. - **Testing & Security**: Resources for testing and securing AI-driven development. - **MCP Servers & Integrations**: Servers and integrations for managing and integrating AI-driven development. - **Code Review & Collaboration**: Tools for code review and collaboration. - **Project & Knowledge Management**: Tools for project management and knowledge management. - **Language Models & Engines**: Tools for language models and engines. - **Development Workflows & Agents**: Tools for enhancing development workflows. - **Code Analysis & Search**: Tools for code analysis and search. - **Domain-Specific Tools**: Tools for specific domains. - **Dataset Generation & Synthetic Data**: Tools for generating and synthesizing datasets. - **Learning Resources**: Resources for learning and improving AI-driven

### Overview: This repository contains a series of Jupyter notebooks designed to guide developers through the hands-on process of using NLTK for Natural Language Processing (NLP) in Python. ### Key features: - Downloading and checking packages - Text analysis using `nltk.text` - Deriving n-grams from text - Detecting text language by counting stop words - Language identification using word bigrams - Creating bigrams, stemming, and lemmatizing - Finding unusual words for a given language - Creating a parts-of-speech tagger - Exploring WordNet features - Name gender identifier - Text genre classifier - Sentiment analysis - Sentiment analysis with `nltk.sentiment.SentimentAnalyzer` and VADER - Using the `langdetect` and `langid` libraries for language identification - NLTK with the Greek script ### Tech stack: Python ### Why it matters: The repository provides a practical, step-by-step guide for developers to learn and apply NLTK for NLP tasks, making it an essential resource for anyone interested in Python-based NLP projects.

Overview: AgentPilot is a versatile workflow automation platform designed to streamline complex AI-driven tasks. It seamlessly integrates a single LLM with multi-member workflows, offering a seamless chat experience and intuitive interface. Key features: - Create and manage agents - Organize and manage chats - Support for branching workflows - Graph workflows for easy configuration and interaction Tech stack: Python Why it matters: AgentPilot simplifies the creation and execution of AI-driven workflows, making it an essential tool for developers and businesses looking to automate their processes efficiently. Repository: jbexta/AgentPilot Primary language: Python GitHub stars: 570 GitHub description: A versatile workflow automation platform to create, organize, and execute AI workflows, from a single LLM to complex AI-driven workflows. README excerpt: A versatile workflow automation system. Create, organize, and execute complex AI-driven tasks. Seamless chat experience, intuitive interface, and support for branching workflows. Supports scheduled and recurring workflows, making automation flexible and adaptable. 🚀 [](https://discord.gg/ge2ZzDGu9e) [](https://twitter.com/AgentPilotAI) <p align="center"> <img src="docs/demo.gif" align="center" height="25

### Key features: - **Survey**: A comprehensive survey of papers and resources on reasoning in large language models. - **Relevant Survey & Position Paper & Blog**: Links to relevant surveys, position papers, and blog posts on the topic. - **Technique**: Techniques for improving or eliciting reasoning in large language models, focusing on fully supervised finetuning. - **Fully Supervised Finetuning**: A technique that applies to improving reasoning in large language models like GPT-3, with papers listed for reference. ### Tech stack: Python, Markdown, LaTeX, and various programming languages as needed. ### Why it matters: This repository provides a valuable resource for researchers and developers interested in enhancing the reasoning capabilities of large language models, such as GPT-3, by applying techniques like fully supervised finetuning.

### Overview: This project presents a real-time fire detection system utilizing a convolutional neural network (CNN) architecture optimized for non-temporal video imagery. The system achieves high accuracy and real-time performance, surpassing contemporary methods. ### Key Features: - **FireNet Architecture**: A state-of-the-art CNN architecture for fire detection. - **InceptionV1, InceptionV2, and InceptionV3**: Advanced CNN variants for enhanced performance. - **Superpixel Localization Framework**: Accurate localization of fire pixels within superpixels for improved detection accuracy. ### Tech Stack: - Python - TensorFlow 1.15 - TFLearn 0.3.2 - OpenCV 3.x / 4.x ### Why It Matters: This project significantly advances the field of real-time fire detection, offering a compact and efficient solution that outperforms traditional methods. The use of reduced complexity architectures ensures high accuracy and real-time processing capabilities, making it suitable for various applications requiring fast fire detection.