Projects

Overview: This repository provides an overview of the AI-as-a-service landscape, focusing on the key services and tools available for natural language processing and related technologies. Key features: - Entity Recognition - Keyphrase Extraction - Machine Translation - Sentiment Analysis - Speech to Text / Speech Recognition - Text to Speech / Speech Synthesis - Face Detection - Text Recognition Tech stack: Java Why it matters: This repository serves as a comprehensive resource for developers looking to integrate AI services into their applications, providing practical examples and detailed explanations of various AI-as-a-service tools. Repository: sekwiatkowski/awesome-ai-services GitHub stars: 162 GitHub description: An overview of the AI-as-a-service landscape

Overview: This repository is dedicated to providing a comprehensive understanding of diffusion models, a class of generative models that have shown remarkable performance in generating high-quality images. The content is organized into notebooks that can be run on Colab, making it accessible for practical experimentation. Additionally, there is a series of blog posts available, which will be expanded into more detailed explanations in the future. Key features: - Colab notebooks for practical experimentation - Comprehensive blog posts for theoretical understanding - Organized in a series of short blog-posts Tech stack: Python, Jupyter Notebooks, Colab Why it matters: This repository aims to fill the gap in resources available for understanding diffusion models, making it easier for developers to grasp the fundamentals and working principles of these models.

### Overview: Mobile-O is a compact, efficient unified vision–language–diffusion model designed specifically for mobile and edge deployment. It performs both multimodal understanding (VQA, OCR, reasoning) and image generation within a single architecture, achieving real-time performance with a small memory footprint. ### Key Features: - **Unified Model Architecture**: A single architecture for vision, language, and diffusion tasks. - **Real-Time Performance**: Designed for mobile and edge devices, ensuring high performance with minimal latency. - **Small Memory Footprint**: Efficiently utilizes limited device resources, making it suitable for resource-constrained environments. - **Versatile Capabilities**: Capable of image generation, image understanding, and image editing, providing a comprehensive solution for various applications. ### Tech Stack: - Python ### Why It Matters: Mobile-O is a significant contribution to the field of computer vision and natural language processing, offering a unified approach to multimodal understanding and image generation. Its compact design and real-time performance make it ideal for mobile and edge computing environments, addressing the challenges of deploying complex models on devices with limited computational resources.

### Overview: This repository serves as a comprehensive list of papers on game-playing agents and large multimodal models, providing insights into methods, applications, and challenges in the field. It aims to provide a valuable resource for developers and researchers interested in AI and game development. ### Key Features: - **Game Playing Agents**: Papers exploring methods and applications of game-playing agents. - **Large Models**: Research on large multimodal models and their capabilities. - **Methods, Applications, and Challenges**: Comprehensive coverage of techniques and their implications. ### Tech Stack: - **GitHub**: Repository hosting and version control. ### Why It Matters: This repository is essential for anyone looking to stay updated on the latest advancements in game-playing agents and large models. It serves as a valuable resource for developers, researchers, and enthusiasts in the field of AI and game development. ### Repository: - **Repository**: BAAI-Agents/GPA-LM - **GitHub Stars**: 161 - **GitHub Description**: A live list of papers on game playing and large multimodal model - "A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challenges". ### News: - **Creation Date**: 2024/02/06 - **Optimus-1**: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks. - **VideoGameBunny**: Towards vision assistants for video games. - **Autoverse**: An Evolvable Game Language for Learning Robust Embodied Agents. - **Cradle**: Empowering Foundation Agents Towards General

### Overview: HoloTime is a project that integrates video diffusion models to generate panoramic videos and reconstructs them into 4D scenes, offering a fully immersive 4D experience. ### Key Features: - **Panoramic Video Generation**: Seamlessly transforms a single prompt or reference image into a panoramic video. - **4D Scene Reconstruction**: Converts the generated panoramic video into a 4D scene, providing a fully immersive 4D experience. - **360-Degree Dataset**: Curated 360-degree video dataset named 360World. ### Tech Stack: Python ### Why It Matters: HoloTime is a significant advancement in the field of 4D scene generation, offering a more immersive and interactive way to experience panoramic videos.

Overview: SceneScape is an innovative method for generating videos of indoor scenes walkthroughs from text, utilizing a combination of pre-trained text-to-image models and geometric priors learned from monocular depth prediction models. This approach ensures 3D consistency in the generated videos, making it suitable for a wide range of indoor scene walkthroughs. Key features: - Text-driven perpetual view generation - Online generation of videos from text prompts - Combining generative power with geometric priors - 3D consistency through online test-time training - Generation of diverse indoor scenes, including walkthroughs in spaceships, caves, and ice castles Tech stack: SceneScape is implemented using PyTorch, with the addition of PyTorch3D for rendering. Why it matters: SceneScape provides a practical solution for generating realistic indoor scene walkthroughs from text descriptions, offering a significant advancement in the field of video synthesis and scene generation.

Overview: Claude Code is transformed into a Game Master harness, allowing for persistent adventures in any book or world. It provides durable memory, a custom rulebook, and a dynamic living world. Key features: - **Authoring Original Worlds**: Creates a book's worth of original canon and a custom ruleset. - **Importing Existing Books**: Allows you to import PDFs of books and run them as if they were brand-new. - **Custom Rulebook**: Produces a World Bible and a World Kit tailored to your world. - **Persistent Memory**: Keeps track of NPCs, locations, plot threads, and character history. - **Custom Rulebook**: Produces a World Bible and a World Kit tailored to your world. - **Dynamic World**: Pushes context to the model, ensuring every scene arrives pre-loaded. Tech stack: Python Why it matters: This harness turns Claude Code into a powerful Game Master, providing a seamless and immersive experience that goes beyond the limitations of traditional chatbots, making it perfect for those looking to run persistent adventures in any book or world.

Overview: This project focuses on Chinese automatic speech recognition, utilizing a model trained on a combination of a Chinese speech dataset and YouTube video transcripts. The model converts spoken Chinese into text, providing a practical tool for voice recognition and text-to-speech applications. Key features: - Chinese automatic speech recognition - Model trained on a dataset combining speech data and YouTube video transcripts - Outputs text from spoken Chinese - Supports training on GPU and CPU - Available models: 901, 902, 903 - Model 903 available via specific links - Model 902 and 901 available via specific links - Model 903 link: https://pan.baidu.com/s/1NcTN8gojuIBaIFT9FB3EJw (password: 261u) - Model 902 link: https://pan.baidu.com/s/1do7C6Egj6sJO7kn1yHPzBg (password: 9o87) - Model 901 link: https://pan.baidu.com/s/1utz-1Vv4IO9D-3awj3x1QQ (password: pv08) - Model training times: 30 hours for 901, 55 hours for 902, 130 hours for 903 Tech stack: Python 3.6, TensorFlow, PyTorch, and various audio processing libraries. Why it matters:

### Overview: Llamactl is a unified management and routing tool for llama.cpp, MLX, and vLLM models. It provides a web dashboard for easy model management and dynamic resource management, including automatic idle timeout and LRU eviction. The tool supports multiple backend options, including OpenAI-compatible servers, and can be deployed on remote hosts for distributed management. ### Key Features: - **Built-in Model Downloader**: Directly downloads GGUF and Safetensors models from HuggingFace. - **Dynamic Multi-Model Instances**: Serve multiple models from a single instance with on-demand loading. - **Smart Resource Management**: Automatic idle timeout, LRU eviction, instance groups with per-group limits, and configurable instance limits. - **Web Dashboard**: Modern React UI for managing instances, monitoring health, and viewing logs. ### Tech Stack: - **Go**: The primary language used for development and management. ### Why It Matters: Llamactl simplifies the management and routing of llama.cpp, MLX, and vLLM models, making it easier to manage and deploy these models efficiently. Its web dashboard provides a user-friendly interface for monitoring and managing instances, ensuring smooth and reliable operation of the models.

Overview: Vault Curate is a local-first second brain for Obsidian, designed to help you find, connect, and rediscover your notes efficiently. It uses semantic search and relation graphs to find notes even when the exact keyword is not available, and connects related notes through an Obsidian Canvas relation graph. Hot/Cold rediscovery allows you to quickly access your forgotten notes. Key features: - Semantic search for finding notes even when the exact keyword is not available. - Relation graph for connecting related notes. - Hot/Cold rediscovery for quickly accessing forgotten notes. Tech stack: TypeScript Why it matters: Vault Curate offers a powerful way to organize and find your notes locally, ensuring data stays on your machine and avoiding API key usage. It's designed to be user-friendly and efficient, making it a valuable tool for anyone managing their Obsidian notes. Repository: notoriouslab/vault-curate Primary language: TypeScript GitHub stars: 160 GitHub description: Find, connect, rediscover your notes — a local-first second brain for Obsidian. Semantic search with strong Chinese/CJK, relation graphs and semantic paths for connections you never drew, Hot/Cold rediscovery of forgotten notes. Desktop builds the index; phones and tablets search it. No API keys.

### Overview: This project examines how large language models perform across various synthetic regression tasks when given (input, output) examples in their context, without any parameter updates. ### Key features: - **Synthetic Regression Tasks**: Examines regression performance using synthetic data. - **No Parameter Updates**: Models trained without parameter updates. - **Models Tested**: Large Language Models (LLMs) and Traditional Supervised Methods. ### Tech stack: Python ### Why it matters: Despite no parameter updates, LLMs consistently outperform traditional methods such as Gradient Boosting or Random Forest, demonstrating strong performance across open-weights models like DBRX or Mixtral 8x22B.

Overview: Office Oxide is a fast, memory-safe library for extracting text from Office documents, including DOCX, XLSX, PPTX, DOC, XLS, and PPT. It is 5-10 times faster than popular Python libraries like python-docx and openpyxl, and has a 100% pass rate on valid Office files. Key features: - Extracts text from DOCX, XLSX, PPTX, DOC, XLS, and PPT files - Supports Python, Go, C#, Node.js (native and WASM), and Rust bindings - Handles encrypted, truncated, and corrupt files - Benchmarks show it is 5-10 times faster than other libraries - Opens 97.3% of a 10,386-file test collection; the rest are handled Tech stack: Rust Why it matters: Office Oxide is the fastest native Office document library for Python, Rust, Go, C#, and Node.js, making it an ideal choice for developers looking to integrate Office document extraction into their applications without the overhead of a JVM. Repository: yfedoseev/office_oxide GitHub stars: 161 GitHub description: The fastest Office document library for Python, Rust, Go, JS/TS, C# and WASM. DOCX, XLSX, PPTX, DOC, XLS, PPT. Up to 100× faster than python-docx/openpyxl/python-pptx. 10

### Overview: Anomagic is a zero-shot anomaly generation method that produces semantically coherent anomalies without requiring any exemplar anomalies. By unifying both visual and textual cues through a crossmodal prompt encoding scheme, Anomagic leverages rich contextual information to steer an inpainting-based generation pipeline. A subsequent contrastive refinement strategy enforces precise alignment between synthesized anomalies and their masks, thereby bolstering downstream anomaly detection accuracy. ### Key Features: - **Crossmodal Prompt Encoding Scheme**: Unifies both visual and textual cues to generate coherent anomalies. - **Inpainting-Based Generation Pipeline**: Uses rich contextual information to steer the generation process. - **Contrastive Refinement Strategy**: Ensures precise alignment between synthesized anomalies and their masks, enhancing downstream anomaly detection accuracy. ### Tech Stack: Python ### Why It Matters: Anomagic offers a versatile foundation model for anomaly generation, surpassing existing methods in terms of realism and diversity. It can generate anomalies for any normal-category image using user-defined prompts, making it a valuable tool for various applications in anomaly detection.

### Overview: Scumble is a free, open-source desktop image editor that allows users to inpaint, edit, upscale, and generate images using AI models. It supports popular models like FLUX 3 Image, FLUX.2, GPT Image, Nano Banana, Seedream, and Qwen through API keys or ComfyUI. Layers can be created, edited, and exported as part of a PSD file. ### Key features: - **AI-Inspired Image Editing**: Scumble is designed to be AI-native, allowing users to select parts of an image, draw boxes, and prompt the AI to fill in the gaps. - **Layer Management**: Users can create, edit, and blend layers, and even apply filters and text on top of the layers. - **ComfyUI Integration**: Scumble works seamlessly with ComfyUI, allowing users to access the latest models and features directly from their ComfyUI installation. - **Flux 3 Image Support**: Scumble supports multiple Flux 3 Image models, providing a variety of image editing options. - **Comfy Cloud Integration**: Users can access models through Comfy Cloud, expanding the range of available models. - **ComfyUI Compatibility**: Scumble is compatible with ComfyUI, allowing users to leverage the latest features and models. - **Comfy Router Integration**: Scumble integrates with Comfy Router, allowing for seamless communication between different ComfyUI installations. - **ComfyUI Support**: Scumble is compatible with ComfyUI, allowing users to leverage the latest features and models. - **Com

Overview: Glide is a blazing-fast, cloud-native model gateway designed to simplify the development of production GenAI apps. Key features: - Unified REST API across multiple providers - High availability and resiliency with automatic fallbacks and smart retries - Support for popular LLM providers - High performance and low latency - Production-ready observability via OpenTelemetry - Straightforward maintenance and configuration - Centralized API key management Tech stack: Go Why it matters: Glide simplifies the integration and communication with external model providers, allowing developers to focus on their core challenges without worrying about infrastructure complexities. Repository: EinStack/glide GitHub stars: 160 GitHub description: 🐦 A open blazing-fast simple model gateway for rapid development of production GenAI apps README excerpt: <div align="center"> <img src="docs/logo/glide_no_bgd.png" width="300px" alt="Glide GH Header" /> <h1>Glide: Cloud-Native LLM Gateway for Seamless LLMOps</h1> <a href="https://codecov.io/github/EinStack/glide"><img src="https://codecov.io/github/EinStack/glide/graph/badge.svg?token=F7JT39RHX9" alt="CodeCov" /></a> <a href="https://discord.gg/pt53Ej7rrc"><img src="https://img.shields.io/discord/11812814078138

Overview: LangGraph-learn is a learning resource repository designed to help developers understand and master LangGraph and related AI tools. Key features: - Hands-on examples - Tutorials on LangGraph and LangChain - Community-driven project fostering collaboration and learning Tech stack: Python Why it matters: This repository provides practical examples and resources to help developers create and manage language model workflows using LangGraph and related tools, making it a valuable resource for AI and machine learning enthusiasts.

### Overview: AI Video Production Editor is an open-source, local-first film studio designed for filmmakers and AI artists. It offers a seamless workflow from script to final export, including script, director treatment, storyboard, filming, editing, color grading, and export, all managed on your own machine using your own API keys. ### Key Features: - **Script → Director → Storyboard → Filming → Edit → Color → Export**: A complete production loop. - **AI Models for Every Shot**: Choose from a variety of models tailored to different needs (e.g., Nano Banana for dialogue, Krea 2 for creatures, Seedream 5 Pro for environments). - **Local Workflow**: All projects are stored locally, ensuring privacy and security. - **Production-Grade Tools**: Features like DaVinci-style page bar, color wheels, and offline auto-cut tools mimic professional post-production workflows. - **3D Previs**: Turn reference images into 3D models for previs and scene maps. - **Infinite Moodboard**: PureRef/Miro-style moodboard with sticky notes and collaborative features. ### Tech Stack: React + Electron app with node pipelines. ### Why It Matters: AI Video Production Editor provides a production workstation where you can manage your entire project locally, bypassing the limitations of paid AI tools. It offers a seamless workflow, AI models tailored to specific needs, and a production-grade post-production environment. By using it, filmmakers and AI artists can create high-quality videos without the need for expensive software or APIs.

### Overview: AutoMegaKernel (AMK) is an agent harness that compiles models into provably correct, self-tuning CUDA megakernels, significantly improving performance on inference-class GPUs. ### Key Features: - **Provably Correct**: AMK ensures the generated megakernel is correct and robust. - **Self-Targeting**: AMK can dynamically adjust the megakernel to optimize performance for specific GPU architectures. - **Self-Tuning**: AMK continuously improves the megakernel through self-research and optimization. ### Tech Stack: Python ### Why It Matters: AutoMegaKernel leverages the power of GPU megakernels to significantly enhance the performance of language models, making them faster and more efficient on modern GPUs.

Overview: MMToM-QA is a multimodal benchmark designed to evaluate machine Theory of Mind (ToM), focusing on understanding people's minds through both text and video inputs. Key features: - Implementation of the BIP-ALM method - Guidelines for utilizing the MMToM-QA benchmark - SOTA performance on MMToM-QA and four other ToM benchmarks - Human-like confidence estimates - Scalable, robust, and interpretable mental inference Tech stack: Python Why it matters: MMToM-QA provides a comprehensive benchmark for evaluating machine understanding of human minds, making it crucial for researchers and developers in the field of artificial intelligence and cognitive science.

### Overview: PTS, or Pivotal Token Search, is a causal-event search framework designed to analyze model reasoning. It identifies and scores events that change the likelihood of a model solving a task across three scales: latent workspace events, emitted tokens, and reasoning sentences. ### Key Features: - **Latent Workspace Events**: Identifies concepts active in the mid-layer workspace that haven't yet been emitted. - **Emitted Tokens**: Scores the importance of emitted tokens that alter the model's success probability. - **Sentence-Level Thought Anchors**: Captures reasoning sentences that affect the model's success probability. ### Tech Stack: - Python 3.9+ (38B Python 3.9+) ### Why It Matters: PTS provides a comprehensive framework for understanding model reasoning, allowing developers to identify and analyze the critical events that influence model outcomes. This insight can be crucial for improving model performance and enhancing the interpretability of AI systems.