Projects

### Overview: Axial Attention is a powerful technique for efficient multi-dimensional data attention, implemented in PyTorch. It simplifies the process of attending to multi-dimensional data by incorporating positional encoding and leveraging axial dimensions for efficient computation. ### Key Features: - **Efficient Multi-Dimensional Attention**: Axial Attention efficiently handles multi-dimensional data, such as images, videos, and more, by rotating attention heads across axial dimensions. - **Positional Encoding**: Adds positional information to the data to enhance the model's ability to understand spatial relationships. - **Simple Implementation**: The library provides a straightforward API, making it easy to integrate into existing projects. - **Versatile Applications**: Demonstrated success in applications like weather prediction and all-attention image segmentation. ### Tech Stack: - **Python**: The core programming language used for implementing the library. - **PyTorch**: The deep learning framework used to build and train the models. ### Why It Matters: Axial Attention has shown remarkable performance in various applications, particularly in fields where multi-dimensional data is prevalent, such as image processing and computer vision. Its simplicity and effectiveness have made it a valuable tool for researchers and practitioners alike. ### Repository: - **Repository**: lucidrains/axial-attention - **GitHub Stars**: 396 - **Primary Language**: Python - **GitHub Description**: Implementation of Axial attention - attending to multi-dimensional data efficiently - **Citation**: - **arXiv**: [arXiv:1912.12180](https://ar

Overview: Holmes is a powerful information extraction tool designed for extracting structured data from English and German texts using predicate logic. It is built on top of the Python programming language and utilizes libraries such as spaCy and coreferee for coreference resolution and entity matching, respectively. Key features: - **Information Extraction**: Extracts structured data from text. - **Coreference Resolution**: Handles named-entity matching and coreference resolution. - **Supervised Document Classification**: Trains and classifies documents using supervised learning. Tech stack: Python Why it matters: Holmes is essential for developers and researchers looking to automate the extraction of structured information from unstructured text, making it particularly useful in fields such as natural language processing, information retrieval, and machine learning. Its ability to handle multiple languages and complex text structures makes it a valuable tool in various applications, including chatbots, document analysis, and topic matching.

### Overview: This repository provides end-to-end recipes for optimizing diffusion models using `torchao` and `diffusers`. It showcases a 53.88% speedup on `Flux.1-Dev` and a 27.33% speedup on `CogVideoX-5b` by leveraging `torch.compile()` and `torchao` for quantization. ### Key Features: - **End-to-End Recipes**: Recipes for optimizing diffusion models with `torchao` and `diffusers`. - **Speedup**: Demonstrates significant speedup on inference and experimental training. - **Quantization**: Uses `torchao` to quantize models, achieving 53.88% speedup on `Flux.1-Dev` and 27.33% on `CogVideoX-5b`. - **Compilation**: Utilizes `torch.compile()` for optimized inference. ### Tech Stack: - Python ### Why It Matters: - **Speedup**: Significantly improves the performance of diffusion models. - **Quantization**: Demonstrates the effectiveness of `torchao` for quantizing models, leading to substantial speed improvements. ### Repository: - **Repository**: sayakpaul/diffusers-torchao - **Stars**: 396 - **Description**: End-to-end recipes for optimizing diffusion models with `torchao` and `diffusers` (inference and FP8 training). ### Updates: - **`torchao` Support**: Officially supported as a quantization backend in `

### Overview: Nextcloud MCP Server is a dedicated standalone MCP server designed to connect AI assistants like Claude, GPT, and Gemini to your Nextcloud instance. It allows you to interact with your Nextcloud data through natural language conversations, enabling features such as creating notes, managing calendars, organizing contacts, and working with files. ### Key Features: - **110+ MCP Tools**: Comprehensive API coverage across 10 Nextcloud apps. - **MCP Resources**: Structured data URIs for browsing Nextcloud data. - **Semantic Search (Experimental)**: Optional vector-powered search for Notes, Files, News items, Deck cards, and Mail messages (requires Qdrant + Ollama). - **Document Processing**: OCR and text extraction from PDFs, DOCX, images with progress notifications. - **Flexible Deployment**: Supports Docker, Kubernetes, VM, or local installation. - **Production-Ready Auth**: Basic Auth with app passwords; multi-user via Login Flow v2 — MCP clients authenticate via OAuth, the server handles Nextcloud app passwords transparently. - **Tag-Based File Exclusion**: Hide sensitive files/folders from MCP file tools by tagging them with a configured Nextcloud system tag (`EXCLUDED_TAGS`). See [docs/configuration.md](docs/configuration.md#t). ### Tech Stack: Python ### Why It Matters: This server provides a secure and efficient way to integrate AI assistants with your Nextcloud data, enhancing productivity and accessibility. The comprehensive API coverage and flexible deployment options make it suitable for both self-hosted and managed hosting services, including Astrolabe Cloud

### Overview: This repository hosts a collection of over 50 Thai Natural Language Processing (NLP) libraries, dictionaries, and corpora. The primary languages include Java, Python, Rust, and CoffeeScript. The libraries cover a wide range of functionalities such as Thai character clustering, sentiment analysis, soundex, word segmentation, part-of-speech tagging, and more. ### Key Features: - **Thai Character Cluster**: Java library for Thai character clustering. - **Sentiment Analysis**: Python library for sentiment analysis. - **Soundex**: Python library for soundex. - **Word Segmentation**: Rust library for Thai word segmentation. - **Part of Speech Tagging**: Python library for part-of-speech tagging. ### Tech Stack: Java, Python, Rust, CoffeeScript, JavaScript, Node.js, C, Python 3, Python 2, Python 4, Rust, CoffeeScript, C++, Python, Java, Python, Rust, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python, Python,

### Overview: This project implements a memory-efficient multi-head attention mechanism as proposed in the paper "Self-attention Does Not Need $O(n^2)$ Memory." The implementation is designed to be highly efficient, reducing memory usage significantly compared to traditional approaches. It includes features such as causal masking, cross attention, and is compatible with PyTorch for seamless integration into existing projects. ### Key Features: - **Memory Efficiency**: Utilizes advanced techniques to reduce memory usage, making it suitable for large-scale models. - **Causal Masking**: Ensures that the model can process sequences in a causal manner, which is crucial for autoregressive models. - **Cross Attention**: Supports cross attention layers, allowing for more complex interactions between different parts of the input. - **CUDA Support**: The implementation is optimized for GPU usage, making it highly performant. ### Tech Stack: - **Python**: The primary programming language used for this project. - **PyTorch**: The deep learning framework used to implement the attention mechanism. ### Why It Matters: This project is a significant advancement in the field of attention mechanisms, particularly in terms of memory efficiency. It provides a practical solution that can be easily integrated into existing models, making it a valuable tool for researchers and practitioners looking to improve the performance and efficiency of their neural networks.

### Overview: HotClip是一款免费开源的AI直播剪辑/视频剪辑工具，支持长视频、直播回放和播客的快速切片。通过AI技术，它能自动找到视频中的亮点，生成9:16的竖屏短视频，无需注册、不上传素材、无积分制，提供无水印的本地运行体验。 ### Key features: - **AI自动找爆点**:精准识别视频中的高光时刻。 - **横屏转竖屏**:一键转换视频格式，保持最佳观看体验。 - **自动加动态字幕**:智能添加动态字幕，提升视频可读性。 - **去气口剪口头禅**:自动识别并删除不必要的口语化表达。 - **本地运行无水印不上传**:无需上传素材，直接在本地进行剪辑和处理。 - **逐句稿功能**:支持分段保存、停止和续跑，提供跨句全文搜索和长稿虚拟列表。 - **可试听、确认和撤销的局部时间校准**:提高剪辑的准确性和可操作性。 - **导出自动改善不同语言的字幕阅读节奏**:支持多语言字幕处理。 - **多人对谈说话人识别**:支持说话人识别，提高对话剪辑的准确性。 - **剪切时长、数量与直播品类**:支持1-2分钟、1-12条，以及不同直播品类的剪辑参数。 ### Tech stack: - TypeScript ### Why it matters: - **真免费,

### Overview: This repository is a comprehensive collection of survey papers and resources related to generative models, focusing on simulating the real world. It includes a survey paper published in IEEE TPAMI 2026 and an open collection of state-of-the-art methods for text-to-X transformations. ### Key features: - **Survey Paper Collection**: Our survey paper, "Simulating the Real World: A Unified Survey of Multimodal Generative Models" (IEEE TPAMI, 2026), which provides a comprehensive overview of 2D, video, 3D, and 4D generation. - **Text-to-X Resources**: A collection of open resources and papers focusing on text-to-X transformations, including state-of-the-art methods and datasets. - **GitHub Stars**: 394 stars, indicating the repository's popularity and usefulness. ### Tech stack: The repository uses Markdown for documentation and GitHub Pages for hosting. ### Why it matters: This repository is a valuable resource for researchers and practitioners interested in simulating the real world using generative models, offering a unified survey and a collection of cutting-edge methods and resources. ### Repository: ALEEEHU/World-Simulator ### Primary language: Unknown ### GitHub stars: 394 ### GitHub description: This repository is divided into two main sections: our survey paper collection and text-to-X resources. It includes a survey paper published in IEEE TPAMI 2026 and an open collection of state-of-the-art methods for text-to-X transformations. The repository is maintained and updated regularly, and

Overview: This repository introduces the concept of large language models (LLMs), covering both theoretical and practical aspects, including applications, statistical language modeling, neural language models, conditional language models, evaluation methods, and more. Key features: - Introduction to language models - Applications of language models - Statistical language modeling - Neural language models (NLM) - Conditional language models - Evaluation methods - Transformer-based language models - Practical LLMs: GPT, BERT, Falcon, Llama, CodeT5 - How to generate text using different decoding methods - Prompt engineering - Fine-tuning LLMs - Retrieval Augmented Generation (RAG) - Ask almost everything (txt, pdf, video, etc.) - Evaluating LLM-based systems - AI agents - LLMs for computer vision (TBD) - Further reading Tech stack: Jupyter Notebook Why it matters: Understanding language models is crucial for developing applications in natural language processing, machine translation, speech recognition, and more. Repository: IbrahimSobh/llms GitHub stars: 394 GitHub description: Large Language Models: In this repository Language models are introduced covering both theoretical and practical aspects.

Overview: Transcribe any video URL or audio file into plaintext. No GPU. No cloud. One command. Key features: - Snap any video URL or audio file into plaintext. - Supports YouTube, TikTok, Instagram Reels, and direct media links. - Models download offline once, stay in cache. - Supports ten languages, including French. - Provides sentence-level timestamps and speaker labels. Tech stack: Python. Why it matters: Fast, offline, and reliable for transcribing videos and audio files. Repository: kouhxp/yapsnap Primary language: Python GitHub stars: 363 GitHub description: Transcribe any video URL or audio file into plaintext. No GPU. No cloud. One command.

Overview: 这个项目提供了北京交通大学计算机科学与技术学院（原“计算机与信息技术学院”）研究生期间的课程资料、笔记、回忆和整理的期末考试卷及课程作业。这些资料旨在帮助学生备考和学习，欢迎自由取阅分享。 Key features: - 研究生课程资料 - 博士生资格考试资料 - 期末考试资料 - 课程作业参考 - 实验报告模版 - 科研或企业求职资料 Tech stack: Jupyter Notebook Why it matters: 本仓库为非营利、永久免费的开源项目，旨在帮助学生备考和学习，提供丰富的课程资料和学习资源。

### Overview: 活字通用大模型是哈尔滨工业大学社会计算与信息检索研究中心（HIT-SCIR）推出的一项重大研究成果，旨在为自然语言处理领域提供强大的工具和解决方案。活字3.5模型在32K上下文支持下，继承了活字3.0的综合能力和数学推理、代码生成等能力，同时在指令遵循能力和安全性方面实现了显著提升。 ### Key features: - **活字3.5**: 32K长上下文支持，继承活字3.0综合能力，指令遵循能力和安全性显著提升。 - **活字3.0**: 32K长上下文支持，继承丰富中英文知识，数学推理、代码生成能力强。 - **中文MT-Bench**: 中文MT-Bench数据集，包含多轮对话问题，用于评测模型的指令遵循能力和安全性。 ### Tech stack: Python ### Why it matters: 活字系列模型在自然语言处理领域具有广泛的应用潜力，特别是在指令遵循能力和安全性方面，为实际应用提供了强有力的支持。通过不断优化和增强，活字模型在多个任务上实现了显著的性能提升，为学术研究和工业应用提供了宝贵的技术支持。

Overview: Epona is an official code for the Autoregressive Diffusion World Model for Autonomous Driving, presented at the ICCV 2025 conference. It offers versatile capabilities including generating minutes-long driving videos, controlling diverse trajectories, understanding real-world traffic knowledge, and predicting future trajectories for real-time motion planning. Key features: - Generates consistent minutes-long driving videos. - Supports controlling diverse trajectories. - Understands real-world traffic knowledge. - Predicts future trajectories for real-time motion planning. Tech stack: Python Why it matters: Epona provides a powerful tool for autonomous driving, enhancing the ability to generate realistic driving scenarios and improve the accuracy of motion planning in real-time. Repository: Kevin-thu/Epona GitHub stars: 393

Overview: This repository contains datasets created by the creator (usually automatically, sometimes manually) for solving text-related tasks on the Russian language. Key features: - Russian NLP datasets - Dialogues and replications - Dialogues from chatbots with imageboards - Dialogues from anecdot collections - Dialogues from the Cornell Movie Corpus - Dialogues from the Hulth dataset (flibusta) - Dialogues from Russian literature and other sources Tech stack: C# Why it matters: These datasets are valuable resources for researchers and developers working on natural language processing tasks in the Russian language, providing a rich source of data for training and evaluating models.

### Overview: InfiniteBench is a cutting-edge benchmark designed to evaluate language models' capabilities in processing, understanding, and reasoning over super long contexts, specifically 100k+ tokens. This benchmark challenges models to perform tasks that go beyond traditional datasets, ensuring a comprehensive evaluation of their performance. ### Key Features: - **Loooong Context:** InfiniteBench tests language models with a context length of 100k+, offering unparalleled challenges. - **Diverse Domain:** The benchmark includes 12 unique tasks, each assessing different aspects of language processing and comprehension. - **Specialized Test:** Tasks are designed to test models' abilities when using shorter context, ensuring performance degradation is solely due to context length. - **Real-World and Synthetic Scenarios:** The tasks mix real-world scenarios with synthetic constructs, providing a comprehensive evaluation. ### Tech Stack: Python ### Why it Matters: InfiniteBench pushes the boundaries of language models by testing them against contexts that are 10 times longer than traditional datasets, ensuring models can handle extended contexts effectively.

### Overview: This repository contains curated papers on Large Language Models (LLMs) in the healthcare and medical domain. The list includes popular models and surveys on their applications, aiming to provide a comprehensive overview of advancements in AI-driven healthcare solutions. ### Key Features: - **Large Language Models (LLMs):** Curated papers on various models that excel in medical text and image comprehension. - **Large Vision Models (LVMs):** Papers focusing on models that interpret and understand medical images. - **Survey Papers + Clinical Studies:** Papers that provide insights into the impact of large language models on healthcare and medical research. - **Articles:** Research articles discussing the latest developments and applications of large language models in healthcare. ### Tech Stack: The repository uses Markdown for its structure and includes links to relevant GitHub repositories and Hugging Face models for further exploration. ### Why It Matters: This curated list is essential for researchers, developers, and healthcare professionals looking to stay updated with the latest advancements in AI-driven healthcare solutions. It serves as a valuable resource for understanding the capabilities and applications of large language models in medical fields. ### Repository: The repository is maintained by the Burglarhobbit and has been starred by 393 users, indicating its popularity and usefulness in the community. ### Primary Language: The primary language of the repository is English. ### GitHub Stars: The repository has been starred by 393 users, making it a highly regarded resource in the AI and healthcare community.

### Overview Visual Med-Alpaca is an open-source, parameter-efficient biomedical foundation model designed specifically for the biomedical domain. Built on the LLaMa-7B architecture, it is trained using an instruction set curated collaboratively by GPT-3.5-Turbo and human experts. Leveraging a few hours of instruction-tuning and the inclusion of plug-and-play visual modules, Visual Med-Alpaca can perform a diverse range of tasks, from interpreting radiological images to addressing complex clinical inquiries. The model can be replicated with ease, necessitating only a single consumer GPU. ### Key Features - **Plug-and-Play Visual Extension**: Integrates visual capabilities directly into the model, enabling seamless integration with existing medical experts. - **Parameter-Efficient Training**: Uses a minimal amount of computational resources, making it suitable for consumer-level GPUs. - **Multimodal Capabilities**: Supports multimodal tasks such as image interpretation and clinical inquiries. - **Ease of Replication**: Requires only a single consumer GPU to replicate the model. ### Tech Stack - Python ### Why It Matters Visual Med-Alpaca addresses the challenges of training large language models on biomedical data, making it a cost-effective solution for researchers and institutions. Its plug-and-play visual extension framework ensures that it can be easily integrated with existing medical experts, enhancing its utility in various biomedical applications.

### Overview: Stream-Omni is a GPT-4-like language-vision-speech chatbot that supports interaction across various modality combinations, including text, vision, and speech. It seamlessly integrates intermediate textual results during speech interactions, providing a "see-while-hear" experience. ### Key Features: - **Omni Interaction**: Supports multimodal inputs including text, vision, and speech, and generates both text and speech responses. - **Seamless "see-while-hear" Experience**: Outputs intermediate textual results (e.g., ASR transcriptions and model responses) during speech interactions, enhancing the "see-while-hear" experience. - **Efficient Training**: Requires minimal omni-modal data for training. ### Tech Stack: Python ### Why it Matters: Stream-Omni offers a comprehensive solution for simultaneous multimodal interactions, making it valuable for applications requiring real-time, context-aware responses across different modalities. ### Repository: ictnlp/Stream-Omni ### Primary Language: Python ### GitHub Stars: 393 ### GitHub Description: Stream-Omni is a GPT-4-like language-vision-speech chatbot that simultaneously supports interaction across various modality combinations, with the following features💡: - **Omni Interaction**: Support multimodal inputs including text, vision, and speech, and generate both text and speech responses. - **Seamless "see-while-hear" Experience**: Simultaneously output *intermediate textual results* (e.g., ASR transcriptions and model responses) during

Overview: `nmtpytorch` is a flexible toolkit for training advanced neural machine translation, image captioning, and automatic speech recognition systems. It was inspired by the `dl4mt-tutorial` codebase and received contributions from the Grounded Sequence-to-sequence Transduction Team. Key features: - Neural machine translation - Image captioning - Automatic speech recognition Tech stack: PyTorch Why it matters: `nmtpytorch` is a valuable resource for researchers and developers interested in advanced neural machine translation systems, offering a flexible toolkit with contributions from the Grounded Sequence-to-sequence Transduction Team.

### Overview: Text2Cinemagraph is an innovative method that leverages text descriptions to generate cinemagraphs, which are short videos that show a sequence of still images. This project introduces a fully automated approach for creating cinemagraphs from text prompts, especially useful for generating images with imaginary elements and artistic styles. ### Key Features: - **Automatic Generation**: The method automatically synthesizes image twins from a single text prompt using Stable Diffusion. - **Artistic and Realistic Twin**: It generates an artistic image and a pixel-aligned realistic twin, where the artistic image captures the style and appearance detailed in the text prompt, while the realistic twin simplifies layout and motion analysis. - **Motion Prediction**: The predicted motion is transferred to the artistic image to create the final cinemagraph. ### Tech Stack: Python ### Why it Matters: Text2Cinemagraph is significant because it addresses the challenge of creating cinemagraphs from text descriptions, which are often complex and require interpretation of semantics and motions. The method simplifies the process by leveraging existing natural image and video datasets, making it a valuable tool for artists, designers, and anyone looking to create dynamic visual content from text prompts.