Projects

Overview: MeMemo is a JavaScript library that enhances vector search and RAG capabilities for browser environments, enabling efficient high-dimensional vector searches and interactive prototyping. Key features: - Vector Search - RAG (Reinforcement Augmented Generation) - Support for IndexedDB and Web Workers for native browser capabilities Tech stack: TypeScript Why it matters: MeMemo provides a powerful toolset for researchers and developers to explore new design and research opportunities in the field of vector search and AI, particularly in the areas of private and personalized content creation and interactive prototyping.

Overview: PowerServe is a high-speed and user-friendly framework for deploying large language models locally, designed for mobile devices. Key features: - One-click compilation and deployment - NPU speculative inference support - Achieves 40 tokens/s running Smallthinker on mobile devices - Supports Android and HarmonyOS NEXT Tech stack: C++ Why it matters: PowerServe simplifies the deployment of large language models, making them accessible and efficient for mobile devices, enhancing performance and user experience.

Overview: AuralKit is a Swift package designed for real-time speech-to-text transcription on iOS 26 and macOS 26, utilizing Apple's SpeechAnalyzer and SpeechTranscriber APIs. Key features: - End-to-end streaming pipeline - Automatic locale model installation - Configurable voice activation - Audio input monitoring - Async streams for lifecycle status - Device capability helper - Xcode 27-ready file transcription path - Xcode 27-ready live capture and analyzer preheating - ScreenCaptureKit-backed transcription Tech stack: Swift 6.2, iOS 26, macOS 26, Swift Package Manager Why it matters: AuralKit simplifies the process of integrating advanced speech-to-text capabilities into iOS and macOS apps, making it easier for developers to implement real-time transcription without the need for complex setup or additional dependencies. Repository: rryam/AuralKit Primary language: Swift GitHub stars: 167 GitHub description: Swift package for real-time speech-to-text transcription with Apple SpeechAnalyzer and SpeechTranscriber on iOS 26 and macOS 26.

### Overview: MG² is an innovative music generation model that uses melodies to guide the generation process, achieving remarkable results despite its simplicity and limited resources. ### Key Features: - **Melody-Based Generation**: MG² leverages melodies to generate music, offering a fresh approach to music generation. - **Personalized Background Music**: Ideal for creating background music for short videos on platforms like TikTok, YouTube Shorts, and Meta Reels. - **Cost-Effective Fine-Tuning**: Fine-tuning with your own private music dataset is highly cost-effective. ### Tech Stack: Python ### Why It Matters: MG² is a groundbreaking model that significantly improves music generation performance with minimal resources, making it highly valuable for creators seeking high-quality music without high costs or extensive training.

### Overview: This repository is a curated list of high-quality papers on resource-efficient Large Language Models (LLMs). It aims to provide a systematic survey of various approaches and techniques used to enhance the efficiency of LLMs, focusing on computation, memory, energy, and financial cost. ### Key Features: - **LLM Architecture Design:** - **Efficient Transformer Architecture:** Discusses hardware-aware context sharding and attention heads. - **Non-Transformer Architecture:** Covers techniques like mobile LLMs and bilevel positional encoding. - **LLM Pre-Training:** - **Memory Efficiency:** Focuses on distributed training and mixed precision training. - **Data Efficiency:** Includes importance sampling, data augmentation, and training objectives. - **LLM Fine-Tuning:** - **Parameter-Efficient Fine-Tuning:** Discusses parameter-efficient fine-tuning methods. - **Full-Parameter Fine-Tuning:** Covers full-parameter fine-tuning approaches. - **LLM Inference:** - **Model Compression:** Discusses model compression techniques like pruning and quantization. - **Dynamic Acceleration:** Covers input pruning and dynamic acceleration methods. - **System Design:** - **Deployment Optimization:** Discusses deployment optimization strategies. - **Support Infrastructure:** Covers support infrastructure for LLMs. - **Other Systems:** Includes other systems and systems related to LLMs. - **Resource-Efficiency Evaluation Metrics:** - **Computation Metrics:** Measures computation efficiency. - **Memory Metrics:** Measures memory efficiency.

### Overview: `whisply` is a fast, cross-platform CLI and GUI tool for batch processing audio and video files using OpenAI's Whisper. It supports CPU, Nvidia GPU, and Apple MLX for transcription, translation, speaker annotation, and subtitle generation. ### Key features: - **Performance**: `whisply` selects the fastest Whisper implementation based on your hardware: CPU/GPU (Nvidia CUDA) or MLX (Apple M1-M5). - **large-v3-turbo Ready**: Support for `whisper-large-v3-turbo` on all devices. - **Auto Device Selection**: Automatically chooses `faster-whisper` (CPU, Nvidia GPU) or `whisper-MLX` (Apple M1-M5) for transcription and translation. - **Word-level Annotations**: Enables `--subtitle` or `--annotate` using `whisperX` and `pyannote` for word segmentation and speaker annotations. - **Customizable Subtitles**: Specify words per subtitle block (e.g., "5") to generate `.srt`, `.vtt`, and `.webvtt` files with fixed word counts and timestamps. - **Batch Processing**: Handles single files, folders, URLs, or lists via `.list` documents. Supports structured, unstructured, and markup formats. - **CLI / App**: Runs directly from CLI or as a browser app. - **Export Formats**: Structured: `.json`, `.rttm`; Unstructured: `.txt`, `.txt (annotated)`; Markup: `.html`

Overview: This project enhances Claude Code with multi-modal visual capabilities, specifically designed for users using a text-only LLM base, enabling screenshot/UI/chart analysis. It integrates with browser-harness for front-end layout automation checks. Key features: - Multi-modal visual analysis of screenshots, UI, and charts - Integration with browser-harness for front-end layout automation checks - Support for various models including DeepSeek, Qwen, OpenAI, Claude (Anthropic), and custom providers Tech stack: Python Why it matters: This skill allows Claude Code users to easily integrate multi-modal visual analysis directly into their conversations, enhancing the visual understanding and problem-solving capabilities of their models. Repository: xiincs/claude-code-vision-skill Primary language: Python GitHub stars: 166 GitHub description: Enhances Claude Code with multi-modal visual capabilities, designed for users with text-only LLM bases. Supports screenshot, UI, and chart analysis, and integrates with browser-harness for front-end layout checks. Compatible with DeepSeek, Qwen, OpenAI, Claude (Anthropic), and custom providers. Supports various models and custom providers through environment variables. Integrates with browser-harness for front-end layout automation checks. Suitable for users looking to improve their models' visual understanding and problem-solving capabilities. GitHub stars: 166.

### Overview: This repository is an awesome list of curated datasets and models related to Chinese Large Language Models (LLMs). It provides a comprehensive collection of Chinese datasets and models, including 10M Chinese datasets, Chinese book datasets, scientific literature datasets, and more. The repository also includes pre-trained models and resources for Chinese LLMs, such as Baichuan2, Qwen, and TigerBot. ### Key Features: - 10M Chinese Datasets - Chinese Book Datasets - Chinese Scientific Literature Dataset - Chinese Poetry Datasets - CLUECorpus2020 - MNBVC (Massive Never-ending BT Vast Chinese corpus) - MOSS Training Data - News Commentary v13 - NKCorpus - pretrain_zh - THUCNews - WuDaoCorpora Text - Chinese Medical Dialogue Data Set - CSL Dataset - Chinese Public Chat Corpus - TCM Ancient Books - MNBVC (40T Data) - Baichuan2 - Qwen - TigerBot - ChatGLM3 ### Tech Stack: Python, PyTorch, TensorFlow, and others. ### Why It Matters: This repository is crucial for researchers, developers, and enthusiasts looking to explore and utilize Chinese LLMs. It provides a wealth of datasets and models, making it an essential resource for anyone interested in Chinese language processing and NLP.

### Overview SciAtlas is a research map that can be used from the command line. It helps you look up literature, gather graph-backed evidence, and turn the result into readable reports and reusable JSON artifacts. ### Key Features - **Graph-Aware Paper Search**: Combine keywords, semantic matching, title anchors, references, and graph propagation instead of stopping at plain keyword matching. - **Research Workflow Automation**: Run literature review, idea grounding, idea evaluation, idea generation, trend analysis, related-author retrieval, and researcher profiling. - **Agent-Friendly Outputs**: Keep reproducible machine-readable artifacts such as `request.json` and `response.json`, plus user-facing `summary.txt` and `report.md`. - **Editable CLI Skills**: Inspect, copy, modify, and rerun common downstream workflows as reusable JSON skills. - **Portable Agent Skill Pack**: Use `agent-skill/` to migrate SciAtlas retrieval and the current literature-review, automated-review, and idea-generation workflows into end-to-end downstream tasks for tools such as Codex, Claude Code, and other coding agents. ### Tech Stack Python ### Why It Matters SciAtlas is a practical research assistant for researchers and developers looking to automate their scientific research workflows. It connects papers, authors, institutions, venues, keywords, citations, and a four-level research taxonomy from domains down to topics. This makes it a valuable tool for anyone looking to stay up-to-date with the latest research in their field.

### Overview: This repository collects and curates surveys, resources, and papers focused on **Lifelong Learning** with **Large Language Models**. It serves as a comprehensive resource for researchers and practitioners interested in the field, providing a detailed introduction to 12 lifelong learning scenarios and featuring the latest advancements in the field. ### Key Features: - **Surveys**: Collects surveys on various aspects of lifelong learning with large language models. - **Papers**: Includes papers from recent conferences and journals, covering topics such as continual learning, incremental learning, and more. - **Codebase**: Provides a codebase for incremental learning with LLMs, which can be used for practical applications. ### Tech Stack: The repository is built using Markdown, GitHub Pages, and other open-source technologies. ### Why It Matters: This repository is crucial for researchers and practitioners looking to stay updated on the latest advancements in lifelong learning with large language models, providing a valuable resource for both theoretical understanding and practical implementation.

Overview: This repository is a curated collection of data, stats, and AI concepts, covering topics from highly cited research papers. It includes short, runnable notebooks and is managed with Python 3.10-3.12. Key features: - Data, Stats, and AI Concepts - Marimo notebooks for interactive learning - Currently working on TabICLv2: a tabular foundation model for in-context learning (classification and regression) - Notes live in `notebooks/fundamentals/` for general stats, ML, deep learning, and generative-AI primitives - Additional dependencies needed for specific papers, such as Linux + CUDA 12 for `cudf.py` - Contributions welcome, with issues or pull requests for concept, paper, or fixes Tech stack: Python Why it matters: This repository provides a concise and interactive way to learn about data, statistics, and AI concepts, making complex ideas accessible and engaging through practical examples and interactive notebooks. Repository: AIinMinutes/Data-and-AI-Concepts GitHub stars: 166 Primary language: Python GitHub description: Contains Data & AI concepts drawn from highly cited research papers. Live at [learnaiinminutes.com](https://learnaiinminutes.com/).

### Overview: This project uses Tacotron, an end-to-end speech generation model, to tackle speech recognition tasks, demonstrating its versatility in natural language processing. ### Key Features: - **Tacotron Model**: Utilizes Tacotron, an end-to-end speech generation model, for speech recognition. - **Data Source**: The World English Bible text and audio recordings are used for training. - **Training and Evaluation**: Includes hyperparameters, data loading, and evaluation scripts. - **Results**: Provides sample results and a training curve. ### Tech Stack: Python ### Why it Matters: Demonstrates the application of Tacotron to speech recognition tasks, showcasing its potential in NLP applications.

### Overview: RikkaHub Plus is an AI chat client for Android that seamlessly integrates with the SillyTavern game. It offers a lossless import of character cards, lorebooks, and presets, ensuring compatibility with the game's original semantics. The app supports multiple providers, streaming, voice calls, and plugin tools, optimizing memory and cache to enhance performance. ### Key features: - **Prompt Cache Optimization**: Reduces token cost by removing per-turn context divergence. - **Memory & Long Conversations**: Utilizes semantic RAG and three-layer memory with rolling compression. - **Jev Decisioning**: Mounts the TypeSafe System One judge model for quick Q&A. - **Plugin System**: QuickJS-sandboxed plugins, ZIP import, and direct AI callability. - **Deep SillyTavern Compatibility**: Lossless import of character cards, lorebooks, presets, regex, and themes. - **Proxy-Station Compatibility**: Automatically fixes classic Gemini-via-OpenAI proxy pathologies. - **Doubao Voice**: Supports TTS 2.0 and Volcengine ASR. ### Tech stack: Kotlin + Jetpack Compose ### Why it matters: RikkaHub Plus provides a seamless and efficient chat experience, making it a valuable addition to the Android ecosystem. Its compatibility with SillyTavern ensures a smooth transition for players, and its advanced features make it a powerful tool for AI chat applications. ### Repository: MiaoWuNYA/rikkahub-sillytavern-android ### GitHub Stars: 151

### Overview: Axon.MCP.Server is a powerful tool that transforms your entire codebase into an intelligent knowledge base, making it easier for AI assistants like ChatGPT and Claude to understand and analyze your project effectively. ### Key Features: - **Model Context Protocol (MCP) Server**: Provides a deep understanding of your codebase's structure, relationships, and patterns. - **AI-Native Integration**: Built specifically for ChatGPT, Claude, Cursor IDE, and other MCP-enabled tools. - **Semantic Understanding**: Goes beyond grep to understand what code means, not just what it says. - **Relationship Mapping**: Automatically builds call graphs, inheritance trees, and dependency networks. - **Multi-Language Mastery**: Deep analysis for C# (Roslyn), Python, JavaScript, TypeScript. - **Vector-Powered Search**: Find code by meaning using semantic embeddings. - **Architecture Intelligence**: Auto-detects services, APIs, Entity Framework mappings, design patterns. - **Production-Ready Performance**: Handles 10,000+ files with ease, providing <500ms p95 latency. - **Enterprise Security**: JWT auth, RBAC, audit logging, rate limiting. ### Tech Stack: Python ### Why It Matters: Axon.MCP.Server transforms your entire codebase into an intelligent knowledge base, making it easier for AI assistants to understand and analyze your project effectively.

Overview: AXAR AI is a lightweight TypeScript framework designed to simplify the development of production-ready AI applications, focusing on clarity, control, and reliability. Key features: - 🧩 Type-first design: Structured, typed inputs and outputs with TypeScript (Zod or @annotations) for predictable and reliable agent workflows. - 🛠️ Familiar and intuitive: Built on patterns like dependency injection and decorators, making it easy to integrate with existing codebases. - 🎬 Explicit control: Define agent behavior, guardrails, and validations directly in code for clarity and maintainability. - 🔍 Transparent: Includes real-time logging and monitoring tools for insight into agent operations. - 🪸 Minimalistic: Lightweight with minimal overhead for your codebase. - 🚀 Streamed outputs: Streams LLM responses with built-in validation for fast and accurate results. Tech stack: TypeScript. Why it matters: AXAR AI simplifies the development of AI applications by providing a clear, reliable, and efficient framework that focuses on practical use and ease of integration into existing workflows. Repository: axar-ai/axar GitHub stars: 166

### Overview: An LLM OS is an operating system designed to interact with users using large language models, offering intuitive and intelligent assistance. This curated list compiles resources to help understand, develop, and explore the possibilities of LLM OS. ### Key Features: - **Resources and Papers**: A collection of research papers, articles, and blog posts exploring the concept and applications of LLM OS. - **Development Interfaces**: Tools and interfaces for developing and implementing LLM OS. - **Videos and Talks**: Educational videos and talks by experts on the topic. ### Tech Stack: - Python, JavaScript, and other programming languages used for developing LLM OS interfaces and applications. ### Why It Matters: - Enhances human-computer interaction through natural language interfaces powered by advanced AI. - Promotes innovation in operating system development and user experience design. ### Repository: - Repository: [bilalonur/awesome-llm-os](https://github.com/bilalonur/awesome-llm-os) - GitHub Stars: 167 - GitHub Description: A curated list of awesome resources, tools, research papers, and projects related to the concept of Large Language Model Operating Systems (LLM-OS).

### Overview: SuperEdit is a state-of-the-art image editing method that improves supervision quality without requiring extra VLM modules or pre-training tasks. It focuses on providing better supervision signals for instruction-based image editing, offering a direct and efficient solution. ### Key features: - **Improved Supervision Quality:** Enhances the effectiveness of supervision signals in image editing. - **Direct and Efficient Solution:** Avoids the need for additional VLM modules or pre-training tasks. - **Unified Rectification Method:** Guides VLMs based on the four generation attributes of diffusion models to rectify editing instructions. ### Tech stack: Python ### Why it matters: SuperEdit provides a direct and efficient solution for instruction-based image editing, improving the supervision quality without the need for additional VLM modules or pre-training tasks.

Overview: Cerul is an open-source video processing tool that allows users to search and annotate local videos using their own model endpoints. It provides functionalities for searching by meaning or example, finding speech and screen text, and saving useful results. Key features: - Search by meaning or example - Find speech and screen text - Keep useful results - Resume where you left off Tech stack: Rust Why it matters: Cerul simplifies the process of searching and annotating videos, making it accessible without the need for a Cerul account. It supports macOS and Linux, and is compatible with various video formats and tools. Repository: cerul-ai/cerul GitHub stars: 166 GitHub description: Open-source video processing core and CLI. Search and annotate local videos with your own model endpoints. README excerpt: Cerul is a command-line tool for searching videos and saving clips. The download includes the media tools and OCR models it needs. It supports macOS Apple Silicon and Linux x86_64 (Ubuntu 24.04 or newer). It allows users to search by meaning or example, find speech and screen text, and save useful results. It also provides a way to resume where you left off and cleans up after use. GitHub Actions: Build and tests GitHub Releases: Latest release Platforms: macOS and Linux License: Apache-2.0 Discord: Join the Discord Join the Discord English · Chinese (Simplified) · Chinese (Traditional)

Overview: Empirical is a comprehensive platform designed to test and evaluate language models and their configurations across various application scenarios. Key features: - Cross-Scenario Testing - Model Configuration Evaluation - Real-World Application Scenarios Tech stack: TypeScript Why it matters: Empirical provides a robust solution for developers to objectively assess the performance and effectiveness of language models in real-world applications, ensuring that their models are optimized for the specific needs of their projects.

Overview: Block Transformer is a novel architecture designed to address inference bottlenecks in autoregressive transformers, specifically through hierarchical global-to-local language modeling. This approach enables faster inference while maintaining high accuracy. Key features: - **Hierarchical Global-to-Local Language Modeling**: The architecture leverages hierarchical self-attention between coarse blocks at lower layers and decodes fine-grained tokens within each local block at upper layers. - **10-20x Throughput Improvement**: By utilizing both global and local inference modules, Block Transformer achieves significant throughput gains compared to vanilla transformers with equivalent perplexity. Tech stack: Python Why it matters: Block Transformer offers a significant improvement in inference speed without compromising accuracy, making it particularly useful in applications requiring real-time processing and high throughput. Repository: itsnamgyu/block-transformer Primary language: Python GitHub stars: 166 GitHub description: Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)