Projects

Overview: KIVI is a new, tuning-free 2-bit KV cache quantization algorithm designed for large language models, specifically targeting LLMs like Llama-2, Falcon, and Mistral. This method optimizes memory usage by quantizing key and value caches to 2-bit, reducing peak memory usage by 2.6 times, and enabling up to 4 times larger batch sizes and increased throughput by 2.35 to 3.47 times in real-world inference workloads. Key features: - KIVI quantizes key cache per-channel and value cache per-token to 2-bit. - Reduces peak memory usage by 2.6 times. - Enables up to 4 times larger batch sizes and increased throughput by 2.35 to 3.47 times. - Hardware-friendly design for LLMs like Llama-2, Falcon, and Mistral. - Supports LLMs with 5-digit passkey examples and 5-shot GSM8K examples. Tech stack: Python, CUDA, transformers, and HuggingFace. Why it matters: KIVI addresses the bottleneck issues in speed and memory usage, making it ideal for large language models that require high throughput and reduced memory footprint.

### Overview: This project presents a novel approach to text-driven video synthesis using a phoneme-pose dictionary and a generative adversarial network (GAN). The method significantly reduces the preprocessing, training, and inference times compared to audio-driven approaches, making it highly effective and versatile. ### Key features: - **Text-driven Video Synthesis**: Generates video from text prompts. - **Phoneme-Pose Dictionary**: Maps text to corresponding phoneme poses. - **Generative Adversarial Network (GAN)**: Trains to synthesize video frames from phoneme poses. - **Reduced Training Data**: Requires significantly less training data than audio-driven methods. - **Flexibility and Stability**: Not subject to speaker variation, reducing preprocessing, training, and inference times. ### Tech stack: Python ### Why it matters: - **Efficiency**: Significantly reduces the time required for data preprocessing, training, and inference. - **Versatility**: Not dependent on speaker variability, making it more stable and reliable. - **Reduced Complexity**: Requires less training data and fewer computational resources compared to audio-driven methods. ### Repository: michaelzhang-ai/Text2Video - **Primary Language**: Python - **GitHub Stars**: 438 - **GitHub Description**: ICASSP 2022: "Text2Video: Text-driven Talking-head Video Synthesis with Phonetic Dictionary".

### Overview: Open-LLaVA-NeXT is an open-source implementation of the LLaVA-NeXT series, designed to facilitate the large multi-modal model community. It is based on the LLaVA codebase with minimal modifications, making it easy to follow. ### Key features: - All training data and checkpoints are open-sourced, friendly for research usage. - Reproduces the results of LLaVA-NeXT. - Based on the LLaVA codebase with minimal modification, easy to follow. ### Tech stack: Python ### Why it matters: It provides a comprehensive and accessible framework for training large multi-modal models, making it easier for researchers and practitioners to experiment and advance the field.

Overview: Locals is a local server designed specifically for running multiple AI agents on macOS, significantly improving performance by up to 3x compared to traditional methods. Key features: - Runs OpenCode, OpenHands, Cline, Aider, Claude Code, and other OpenAI, Anthropic, and Ollama apps. - Supports up to 3x faster performance with up to 3 agents working simultaneously. - Compatible with various macOS models, including Apple Silicon. Tech stack: Python Why it matters: Locals provides a powerful solution for developers working on multiple AI agents, significantly enhancing productivity and performance. Repository: Generated-Computers/Locals Primary language: Python GitHub stars: 356

### Overview: OpenStory 是一个基于大语言模型（LLM）和 Agent-Kernel 开发的多智能体推演与模拟框架，旨在通过生成可交互的故事世界来探索和模拟复杂的人类社会互动。它支持多种智能体类型，包括感知、计划、执行和反思等，以创建高度沉浸式的推演剧情。 ### Key features: - **基于 Agent-Kernel 的动态框架**: 支持在推演过程中动态增删智能体，赋予系统无限的扩展能力。 - **一比一复刻的红楼梦大观园前端**: 精心打造的 1:1 仿真可视化交互界面，直观展示智能体的动态轨迹和人物状态。 - **富有冲击力的推演剧情**: 智能体之间产生深度化学反应，自动生成跌宕起伏、精彩绝伦且极具戏剧张力与冲击力的推演剧情。 - **丰富的插件化机制与高可配性**: 覆盖智能体感知、计划、执行、反思等完整生命周期插件，并支持通过 YAML 文件灵活管理系统、环境、动作与智能体配置。 - **一句话生成世界（WorldKernel）**: 输入一句自然语言设想，系统自动生成角色、地点、路径与规则，并接入可交互的多智能体模拟。 ### Tech stack: Python ### Why it matters: OpenStory 将复杂的多智能体推演与模拟技术与大语言模型（LLM）相结合，为用户提供了一个全新的、高度沉浸式的

Overview: DICe is a digital image correlation (DIC) tool designed for use as a module in external applications or as a standalone analysis code. It is machine portable and can be deployed on Windows, Linux, and Mac platforms. DICe is capable of computing full-field displacements and strains from digital image sequences and tracking rigid body motion. It includes robust optimization methods and a well-posed global DIC formulation. Key features: - Robust optimization method for subsets of arbitrary shape - Robust simplex optimization method for data sets with low contrast or small subset sizes - Well-posed global DIC formulation for addressing instabilities in DIC - Command line interface for advanced features - GUI for basic use cases Tech stack: C++ Why it matters: DICe is a versatile tool for digital image correlation, enabling precise tracking of deformations and displacements in various applications, including material characterization and trajectory tracking. Its robust methods and well-posed formulation make it suitable for a wide range of imaging experiments and applications. Repository: dicengine/dice GitHub stars: 437 GitHub description: Digital Image Correlation Engine (DICe): a stereo DIC application that runs on Mac, Windows, and Linux README excerpt: DICe is an open source digital image correlation (DIC) tool intended for use as a module in an external application or as a standalone analysis code. It is machine portable and can be deployed on Windows, Linux, and Mac platforms. DICe is capable of computing full-field displacements and strains from digital image sequences and tracking rigid body motion. It includes robust optimization

Overview: The `local-llm-function-calling` project is designed to ensure Hugging Face text generation models follow a JSON schema by enforcing compliance with the provided prompt and JSON schema. This tool simplifies the text generation process and allows for precise control over the output. Key features: - Constrains the generation of Hugging Face models to adhere to a JSON schema. - Facilitates the formulation of prompts for function calls, enabling precise data extraction and formatting. - Simplifies the text generation process through a user-friendly `Generator` class. Tech stack: Python Why it matters: This project is crucial for developers and AI enthusiasts who need to ensure the output of text generation models is consistent and compliant with predefined constraints, enhancing the reliability and predictability of their applications.

Overview: TensorPool CLI is a command-line interface designed for managing GPU clusters and running machine learning jobs. Key features: - Job management - Cluster management - SSH access to instances Tech stack: Python Why it matters: It simplifies the process of managing GPU clusters and executing ML training jobs, making it easier for developers to deploy and run their models efficiently.

### Overview: FastTextRank is an optimized algorithm for extracting abstracts and keywords from Chinese text, significantly reducing time complexity and improving accuracy. ### Key features: - Optimized iterative algorithms for improved speed - Selective use of word vectors for enhanced accuracy - Reduced time complexity from O(n^2) to O(n) for text summarization - Faster execution time compared to TextRank4ZH ### Tech stack: Python ### Why it matters: FastTextRank offers a more efficient and accurate method for text summarization and keyword extraction, making it suitable for real-world applications where speed and precision are crucial.

### Overview: Nexus is a unified endpoint for managing and securing your AI stack, enabling seamless aggregation, governance, and control of multiple MCP servers and LLM providers. ### Key Features: - **MCP Server Aggregation**: Connects multiple MCP servers (STDIO, SSE, HTTP) through a single endpoint. - **LLM Provider Routing**: Unified interface for OpenAI, Anthropic, Google, and AWS Bedrock LLM providers with full tool calling support. - **Context-Aware Tool Search**: Intelligent fuzzy search across all connected tools using natural language queries. - **Protocol Support**: Supports STDIO (subprocess), SSE (Server-Sent Events), and streamable HTTP MCP servers. - **Flexible Configuration**: TOML-based configuration with environment variable substitution. - **Security**: Built-in CORS, CSRF protection, OAuth2, and TLS support. - **Rate Limiting**: Multi-level rate limiting with in-memory or Redis backends. - **Docker Ready**: Available as a container image with minimal configuration needed. ### Tech Stack: Rust ### Why it Matters: Nexus simplifies the management and security of your AI stack by providing a unified interface for MCP servers and LLM providers, enabling efficient aggregation, governance, and control. Its built-in security features and flexible configuration options make it a robust solution for modern AI and ML deployments.

### Overview: This project introduces a novel model-based approach to few-shot learning, specifically focusing on embedding adaptation with set-to-set functions. The method, named FEAT (Few-shot Embedding Adaptation with Transformer), leverages the Transformer architecture to adapt instance embeddings for improved task-specific and discriminative embeddings. ### Key Features: - **Model-Based Few-Shot Learning**: Adapt instance embeddings to target classification tasks using set-to-set functions. - **Transformer Optimization**: Employs the Transformer architecture for effective model adaptation. - **Benchmark Results**: Reports average results on MiniImageNet and TieredImageNet datasets with ResNet-12 backbone, achieving 66.78% and 82.05% accuracy for 1-shot and 5-shot setups, respectively. ### Tech Stack: - **Programming Language**: Python - **Framework**: PyTorch ### Why It Matters: This approach significantly improves few-shot learning performance by leveraging the Transformer architecture, making it highly relevant for researchers and practitioners looking to enhance their few-shot learning models.

### Overview: KVQuant is a methodology for efficient KV cache quantization that enables efficient long context length inference with low-precision quantization, making it possible to serve large language models with up to 10 million context length on a single GPU. ### Key features: - **Per-channel, Pre-RoPE Key quantization** to better match outlier channels in keys. - **Non-Uniform Quantization (NUQ)** to better represent non-uniform activations. - **Dense-and-Sparse Quantization** to mitigate the impacts of numerical outliers. ### Tech stack: Python ### Why it matters: KVQuant addresses the memory bottleneck with long context length inference by quantizing the KV cache to low precision, making it possible to serve large language models with up to 10 million context length on a single GPU, significantly improving inference efficiency and resource utilization.

Overview: CAIPE is an open-source AI platform designed for building, governing, and operating AI agents and workflows in platform engineering and beyond. Key features: - Agent creation and runtime execution - Workflow automation - Skills and MCP integrations - Enterprise knowledge management - Identity and authorization - Observability - Multi-channel experiences Tech stack: TypeScript Why it matters: CAIPE provides a comprehensive solution for platform engineering, enabling the creation, management, and monitoring of AI agents and workflows in a scalable and efficient manner.

Overview: Conformer is an implementation of the convolutional module from the Conformer paper, designed to enhance the local inductive bias in Transformers. Key features: - ConformerConvModule: A convolutional module for Transformers. - ConformerBlock: A block of Conformer convolutional modules. - Conformer: A stack of ConformerBlocks for improved performance. Tech stack: Python Why it matters: Conformer improves the local inductive bias in Transformers, making them better suited for tasks requiring local context, such as speech recognition and language modeling. Repository: lucidrains/conformer Primary language: Python GitHub stars: 436 GitHub description: Implementation of the convolutional module from the Conformer paper, for use in Transformers.

### Overview: MLX Swift Chat is a fully native SwiftUI app that enables developers to run local Large Language Models (LLMs) using Apple's MLX framework on Apple silicon devices. This project simplifies the process of integrating LLMs into Swift applications, providing a seamless experience for developers. ### Key Features: - **SwiftUI Integration**: The app is built using SwiftUI, ensuring a smooth and user-friendly interface. - **MLX Framework**: Utilizes Apple's MLX framework for efficient and optimized model execution. - **Model Support**: Currently supports LLMs such as Mistral, Llama, and Phi, with plans to expand to more models in the future. ### Tech Stack: - Swift ### Why it Matters: MLX Swift Chat simplifies the integration of LLMs into Swift applications, making it easier for developers to leverage the power of large language models without the need for complex frameworks. This project is particularly useful for developers working on Apple silicon devices, offering a native solution that runs efficiently on the latest Apple hardware. ### GitHub Stars: 436 ### Primary Language: Swift ### Acknowledgements: - **buh/CompactSlider**: Contributed to the project with the CompactSlider library. - **ml-explore/mlx-swift**: Provided the MLX framework for the project. - **huggingface/swift-chat**: Collaborated with the Hugging Face community for model support. Special thanks to Awni Hannun and David Koski for their early testing and feedback, and to all contributors to the MLX framework for

Overview: This project presents **NerfingMVS**, a method for indoor multi-view stereo reconstruction using neural radiance fields and guided optimization. The work is based on ICCV 2021 Oral Presentation. Key features: - **Guided Optimization of Neural Radiance Fields**: The method uses neural radiance fields to estimate depth maps from multiple views, guided by a set of constraints to improve accuracy and consistency. - **Colmap Integration**: The project integrates COLMAP for pose estimation and sparse depth reconstruction, adding a submodule for better performance. - **Data Preparation**: The project provides a detailed guide on how to prepare data for the project, including downloading and organizing the required data. Tech stack: Python Why it matters: This method offers a significant improvement in indoor multi-view stereo reconstruction by leveraging neural radiance fields and guided optimization, leading to more accurate and consistent depth maps compared to traditional approaches.

Overview: This project introduces the Attention Guided Graph Convolutional networks (AGGCNs) for large-scale sentence-level relation extraction, focusing on the TACRED task. The code is a PyTorch implementation of the paper presented at ACL 2019. Key features: - Attention Guided Graph Convolutional Networks - Implementation of AGGCNs for dependency tree-based relation extraction - PyTorch-based model architecture Tech stack: Python Why it matters: This project provides a practical implementation of AGGCNs, showcasing advancements in relation extraction techniques and their application to large-scale datasets.

### Overview: This repository serves as a comprehensive collection of materials and resources for various Natural Language Processing (NLP) tasks, including data analysis, knowledge graph, models and algorithms, ontologies, question answering systems, search engines, sentiment analysis, shallow discourse parsing, text classification, topic modeling, and word embeddings. It aims to provide a solid foundation for understanding and implementing NLP techniques across different domains. ### Key Features: - **Data Analysis** - **Knowledge Graph** - **Models and Algorithms** - **Ontologies** - **Question Answering System** - **Search Engine** - **Sentiment Analysis** - **Shallow Discourse Parsing** - **Text Classification** - **Topic Modeling** - **Word Embeddings** ### Tech Stack: Python and Jupyter Notebook. ### Why it Matters: This repository is essential for anyone interested in the field of Natural Language Processing, offering a wide range of tools and resources to enhance their understanding and practical skills in NLP.

Overview: ai4j is a Java 8+ agentic SDK that provides unified access to multiple AI model platforms, including OpenAI, Anthropic, DeepSeek, and more. It includes tools for agent calling, RAG, MCP, Skill, sandbox, agent runtime, and a built-in Coding Agent CLI/TUI/ACP. Key features: - Unified access to 12+ AI model platforms - Complete agent orchestration capabilities - Long-running Harness for continuous task execution - Built-in RAG for efficient document retrieval and processing - Extensive skill support and agent blueprint creation - Secure and recoverable Agent execution with checkpointing and event tracing - Open-source and extensible architecture Tech stack: Java 8+ Why it matters: ai4j simplifies the process of integrating AI into Java applications, providing a comprehensive solution for building and managing AI agents. Repository: LnYo-Cly/ai4j Primary language: Java GitHub stars: 434 GitHub description: Java 8+ agentic SDK: unified LLM access (OpenAI/Anthropic/DashScope/Doubao/DeepSeek...), unified AI service(image/video/audio...), Tool Calling, MCP, RAG, SKILLS, Agent Runtime, A2A, agent blueprint, agent harness, and a built-in Coding Agent CLI/TUI/ACP. README excerpt: <p align="center"><img src="https://capsule-render.vercel.app/api?type=waving&color=0:6A5ACD,100:2E8

Overview: UE5-MCP (Model Control Protocol) is a project designed to integrate AI-driven automation into Blender and Unreal Engine 5 (UE5) workflows, enhancing level design, asset management, and gameplay programming. Key features: - **AI-Driven Scene Generation**: Text-to-Scene conversion in Blender, image-to-Scene reference-based generation, material and texture management, automated scene composition, and PolyHaven asset integration. - **Unreal Engine Integration**: Automated scene import, game level conversion, material and lighting transfer, level optimization utilities, and Blueprint-based scene manipulation. - **Asset Management & Creation**: Generates and modifies 3D models, textures, and materials using AI, workflow automation for asset transfers between Blender and UE5. - **Gameplay Programming & Debugging**: Assists in Blueprint scripting, performance profiling, and automated debugging for UE5. Tech stack: Python 3.x, Blender, Unreal Engine 5, Unreal Engine 5 (UE5), Python packages (specified in `dependencies.md`). Why it matters: This project streamlines the workflow for game level creation, asset management, and gameplay programming, potentially reducing time and effort in the development process. Repository: VedantRGosavi/UE5-MCP GitHub stars: 435 GitHub description: MCP for Unreal Engine 5 Primary language: Unknown GitHub stars: 435