Projects

### Overview: This blueprint by Mozilla.ai provides a comprehensive guide for developers to create and fine-tune their own Speech-to-Text (STT) model, optimizing it for specific languages and use cases. It enables local development and ensures data privacy. The process involves creating a dataset and leveraging either Common Voice or your own data, with support for a wide range of languages. ### Key Features: - **Dataset Creation**: Users can either create a custom dataset or leverage the Common Voice dataset. - **Model Fine-Tuning**: Fine-tuning models using either the Common Voice dataset or custom datasets. - **Performance Optimization**: Fine-tuning for specific languages and use cases, ensuring optimal performance. - **Privacy**: Ensures that data stays private throughout the process. ### Tech Stack: - Python - Hugging Face - Gradio - Common Voice ### Why It Matters: This blueprint simplifies the process of creating and fine-tuning STT models, making it accessible for developers without extensive machine learning expertise. It ensures that the models are optimized for specific languages and use cases, providing better performance and accuracy.

Overview: This project demonstrates how Amazon Bedrock and Neo4j can be used together to build and consume a knowledge graph in Neo4j, showcasing the integration of generative AI with Neo4j through the AWS Bedrock service. Key features: - Integration of Amazon Bedrock with Neo4j using the AWS Bedrock service. - Demonstration of how to leverage Neo4j for knowledge graph creation and querying. - Use of AWS Bedrock to process and extract data from the SEC's EDGAR system. - Implementation of a chat-based UI for non-technical users to interact with Neo4j databases. Tech stack: Python, Neo4j, AWS Bedrock, AWS CLI, Streamlit. Why it matters: This project provides a practical example of how generative AI can be integrated with Neo4j, showcasing the potential for building complex knowledge graphs and making them accessible to non-technical users through a user-friendly interface.

### Overview: This repository is a curated list of recent efficient video generation methods, focusing on various techniques such as auto-regressive, attention-based, caching, VAEs, hierarchical models, pruning, architectures, distillation, quantization, parallelism, and schedulers. ### Key features: - Auto-regressive models - Attention mechanisms - Caching techniques - Variational Autoencoders (VAEs) - Hierarchical models - Pruning strategies - Architectural designs - Distillation methods - Quantization techniques - Parallel processing - Schedulers for efficient generation ### Tech stack: The repository uses Python and a few other programming languages for its implementation. ### Why it matters: This curated list is valuable for developers and researchers interested in efficient video generation techniques, providing a comprehensive overview of the latest advancements in the field.

### Overview: GroundVLP is a zero-shot method that harnesses visual grounding ability from pre-trained models trained on image-text pairs and pure object detection data. This approach enables effective zero-shot transfer for visual grounding tasks. ### Key Features: - **Zero-Shot Visual Grounding**: GroundVLP leverages pre-trained models to achieve zero-shot transfer, enabling it to perform visual grounding tasks without explicit training on the target domain. - **Hybrid Model Architecture**: Combines pre-trained models from ALBEF, TCL, and Detic to enhance performance and adaptability. - **Efficient Training**: Utilizes pre-trained weights to quickly initialize and fine-tune the model, reducing training time and computational resources. ### Tech Stack: - **PyTorch**: Used for the implementation of the model. - **Detectron2**: Framework for object detection and pre-trained models. ### Why It Matters: - **Versatility**: GroundVLP can be applied to various zero-shot transfer tasks, including visual grounding, without requiring extensive domain-specific training. - **Performance**: Demonstrates superior performance compared to existing methods, achieving state-of-the-art results on benchmark datasets. - **Scalability**: Efficiently handles large-scale datasets and diverse tasks, making it suitable for real-world applications. ### Repository: - **Repository**: om-ai-lab/GroundVLP - **Stars**: 74 - **Description**: "GroundVLP: Harnessing Zero-shot Visual Grounding from Vision-Language Pre-training and Open-Vocabulary Object Detection (AAAI 2024)" ### Installation

Overview: Marconi is a Python-based implementation of the "Marconi: Prefix Caching for the Era of Hybrid LLMs" paper presented at MLSys '25. It aims to optimize the performance of hybrid language models by utilizing prefix caching. Key features: - Implementation of prefix caching for hybrid language models - Source code in Python - Tested on Ubuntu 22.04 with Python 3.11.9 Tech stack: Python Why it matters: Marconi offers a practical solution for improving the performance of hybrid language models by efficiently utilizing prefix caching, making it a valuable tool in the field of artificial intelligence and natural language processing.

Overview: Spokestack is an all-in-one solution for Android apps that provides speech processing capabilities, including voice activity detection, wakeword detection, speech recognition, natural language understanding, and speech synthesis. It simplifies the integration of voice functionality into any Android app, ensuring faster results and better privacy. Key features: - Spokestack Android SDK for Android apps - Extensive feature set for speech processing - Supports iOS, React Native, and Python - Free account for training models and testing TTS - Compatible with various Android versions Tech stack: Java Why it matters: Spokestack simplifies the integration of voice functionality into Android apps, providing a comprehensive solution for voice-based interactions, making it easier and faster to implement voice features in applications. GitHub stars: 74

### Overview: LlamaLib is a high-level C++ and C# library designed to run Large Language Models (LLMs) anywhere, from PCs to mobile devices and VR headsets. Built on the llama.cpp library, it offers a self-contained and embedded solution that runs without the need for a separate server, open ports, or external processes. It supports multiple platforms and hardware architectures, making it versatile and accessible to developers. ### Key Features: - **High-Level API**: Intuitive object-oriented design for easy integration. - **Self-Contained and Embedded**: Runs within your application, eliminating the need for a separate server or external dependencies. - **Cross-Platform and Cross-Device**: Works on Windows, macOS, Linux, Android, iOS, and various VR/AR platforms. - **Architecture Detection at Runtime**: Automatically detects and optimizes the best architecture for performance. ### Tech Stack: - **C++ and C#**: High-level implementations for C++ and C#. - **llama.cpp**: The underlying library powering the library. ### Why It Matters: LlamaLib simplifies the process of integrating LLMs into applications, making it easier to deploy and use advanced AI capabilities without the complexity of setting up servers or managing external processes. Its cross-platform and device-friendly nature ensures that developers can leverage the power of LLMs across a wide range of devices and environments, enhancing the user experience and capabilities of their applications.

### Overview: ARIS-Movie-Director is an agentic, long-horizon visual generation project that transforms a fuzzy story into a coherent and faithful image-based movie. The system ensures that the visual elements remain consistent across multiple frames, addressing the challenges of long-range forgetting and linear streaming failures. The key features include an agent workflow that authorizes a source of truth, a multi-agent debate process, and a cross-model-audited rendering pipeline. ### Key Features: - **Cross-Model Audited Movie Generation**: Hand a fuzzy story to an agent, and the system produces a movie with cross-model-audited panels. - **Long-Horizon Consistency**: Ensures that the visual elements remain consistent across multiple frames, addressing long-range forgetting and linear streaming failures. - **Agent Workflow**: Authorizes a source of truth, runs a multi-agent debate process, and assembles accepted panels into the final viewer. ### Tech Stack: - **Python**: The primary language used for the project. ### Why It Matters: ARIS-Movie-Director brings together ARIS's research-wiki and multi-agent debate techniques to create a visually coherent movie generation system. It addresses the challenges of maintaining consistency in long-horizon visual storytelling, making it a valuable tool for creating faithful and engaging visual narratives. ### Repository: - **Repository**: wanshuiyin/ARIS-Movie-Director - **GitHub Stars**: 74 - **GitHub Description**: Agentic, long-horizon visual generation: a fuzzy story → a cross-model-audited image-based

### Overview: PetitGPT is a 124.6M-parameter language model project that explores the capabilities of a small language model under a limited training budget. It includes a custom byte-level BPE tokenizer, approximately 13 billion pretraining tokens, and evaluations that distinguish reference-answer fit from complete generated-answer correctness. ### Key features: - Custom 32,000-token byte-level BPE tokenizer - Approximately 13 billion pretraining tokens - Controlled post-training experiments - Evaluations that distinguish reference-answer fit from complete generated-answer correctness ### Tech stack: Python ### Why it matters: PetitGPT provides insights into what can be learned by building and evaluating a small language model under a limited training budget, exploring fine-tuning, preference optimization, response distillation, and other techniques. The project includes a custom tokenizer and evaluations that demonstrate promising data efficiency under a fixed protocol.

Overview: FLAG is a framework for generating 3D molecules with valid and realistic substructures, designed to tackle the challenges in structure-based drug design. It uses a fragment-based approach and incorporates motif vocabulary to improve the quality of generated molecules. Key features: - Fragment-based Ligand Generation (FLAG) - Motif vocabulary construction - 3D graph neural network for encoding intermediate context - Focal motif selection, prediction, and attachment - Bond length/angle determination using cheminformatics tools - Molecular geometry adjustment - Competitive performance on conventional metrics and realistic substructure generation Tech stack: Python Why it matters: FLAG addresses the limitations of existing methods in generating valid and realistic molecules, making it a significant contribution to the field of structure-based drug design.

Overview: TandemLLM is an environment designed to run multiple open models as agents, utilizing a server and setup guide to manage resources and performance. Key features: - Multiple models can be run simultaneously on a single server. - The engine drafts and verifies its own tokens, ensuring consistency and accuracy. - It supports models like Qwen3.8-27B, Qwen3.8-Flash-Next, and others. Tech stack: Python Why it matters: TandemLLM simplifies the management of multiple models, ensuring efficient and accurate inference, making it suitable for complex AI environments.

```kotlin mavenCentral() } dependencies { implementation "com.github.aatricks:llmedge:0.1.0" } ```

Overview: PortableLM, also known as PrivatePixelPrompt, is a zero-dependency, air-gapped local AI environment that allows you to run large language models, image generation, and text-to-speech functionalities offline on Windows, macOS, Linux, and Android (Termux). It eliminates the need for internet and installation, making it perfect for portable computing. Key features: - Multi-Modal AI Hub: Unified interface for Text Chat, Image Generation, and Text-to-Speech. - Persistent Image Library: Automatically saves generated images to a dedicated folder with metadata. - Aspect Ratio & Live Step Progress: Supports aspect ratios, real-time progress tracking. - Persistent Audio Library: Saves generated TTS speech files with metadata. - Lightweight Model Options: Supports ultra-fast local models like LFM2.5, Gemma 2, Llama 3, and Qwen. - Zero Dependency Setup: Ships with isolated engine binaries, no system permissions required. - Cross-Platform: Uses a shared volume system for native compatibility across Windows, macOS, Linux, and Android (Termux). - Fully Offline: Runs completely air-gapped after initial setup, ensuring data stays on your machine. - Network Proxied UI: Accessible via a custom Python HTTP server for a dark mode UI. Tech stack: Python and isolated engine binaries. Why it matters: PortableLM simplifies AI integration into portable computing environments, allowing users to access advanced AI functionalities without the need for internet or complex installations, making it ideal for remote work, travel, or any scenario where a stable and offline AI solution is required.

Overview: Jarvis-Termux is a Python-based AI chat and voice assistant powered by Google Gemini AI, designed for Termux users to interact with an AI assistant using voice and text commands. Key features: - Voice-based AI assistant - Text-based AI assistant - Natural language conversation - Google Gemini AI-powered - Improved performance and reliability Tech stack: Python Why it matters: The project leverages Google Gemini AI for intelligent responses, enhancing the assistant's capabilities and reliability, making it a valuable tool for Termux users seeking an AI-powered chatbot.

Overview: This project is a repository for the paper "Few-Shot Keyword Spotting with Prototypical Networks," focusing on a few-shot keyword spotting system using prototypical networks. Key features: - Few-Shot Keyword Spotting - Prototypical Networks - Code Repository for the Paper Tech stack: Jupyter Notebook Why it matters: This project offers a practical implementation of the proposed method for keyword spotting in few-shot scenarios, demonstrating its effectiveness and usability in real-world applications. Repository: ArchitParnami/Few-Shot-KWS GitHub stars: 73 GitHub description: Few-Shot Keyword Spotting README excerpt: Code Repository for the paper [Few-Shot Keyword Spotting with Prototypical Networks](https://arxiv.org/abs/2007.14463).

Overview: This project focuses on emotion text classification using the Llama3-8b model, enhanced with LoRA and FlashAttention techniques. The model achieves an accuracy of 92.62%, surpassing other transformer models, including BERT and RoBERTa. Key features: - **Model**: `meta-llama/Meta-Llama-3-8B-Instruct`, fine-tuned via supervised LoRA fine-tuning (SFT). - **Techniques**: LoRA for efficient parameter tuning, FlashAttention-2 for optimized attention computation. - **Dataset**: Six-class emotion text dataset (16,000 train / 2,000 test records). - **Performance**: 92.62% accuracy, surpassing BERT and RoBERTa baselines (Table 3). Tech stack: Python. Why it matters: This project demonstrates the effectiveness of using large language models for more fine-grained sentiment analysis, providing valuable insights for businesses and developers looking to improve customer satisfaction and product development through AI and NLP. Repository: DaoyuanLi2816/llama3-emotion-lora GitHub stars: 73 GitHub description: Emotion text classification using Llama3-8b with LoRA and FlashAttention and LLaMA-Factory.

Overview: Papr Memory is a predictive memory layer designed for AI agents, offering features like multi-tier caching, custom schema support, and advanced semantic search capabilities. It excels in 91% accuracy with <100ms on-device retrieval, making it highly effective for various use cases such as voice agents, B2B applications, and healthcare systems. Key features: - Multi-tier caching - Custom schema support - Semantic search - Graph relationships - User context management - On-device predictions (< 100ms retrieval) Tech stack: Python Why it matters: Papr Memory provides a robust, privacy-first solution for AI agents, offering advanced features and high performance, making it a valuable tool for applications requiring persistent, context-aware memory management. Repository: Papr-ai/memory-opensource Primary language: Python GitHub stars: 74 GitHub description: Predictive memory layer for AI agents. MongoDB + Qdrant + Neo4j with multi-tier caching, custom schema support & GraphQL. 91% Stanford STARK accuracy, <100ms on-device retrieval.

Overview: RenAI Chat aims to provide a fun and interactive experience for users by integrating multiple AI models into a Visual Novel format, allowing for seamless conversations and character customization. Key features: - Use of multiple AI models for conversational AI and Text to Speech - Voice cloning and character customization through TTS and Tortoise TTS modules - Speech recognition and text-to-speech capabilities - Customizable character JSON files - Customizable game appearance with Renpy files Tech stack: Python Why it matters: RenAI Chat offers a unique and engaging way to interact with AI chatbots, making it an interesting project for developers looking to explore AI integration in creative projects.

### Overview: CitrusFarm is a multimodal agricultural robotics dataset designed for localization, mapping, and crop monitoring tasks in citrus tree farms. The dataset comprises nine sensing modalities including stereo RGB, depth, monochrome, near-infrared, and thermal images, as well as wheel odometry, LiDAR, IMU, and GPS-RTK data. It was collected by a wheeled mobile robot in the Agricultural Experimental Station at the University of California, Riverside, during the summer of 2023. The dataset features seven sequences from three citrus tree fields, covering various tree species, planting patterns, and daylight conditions, spanning a total of 1.7 hours and 7.5 km of operation, totaling 1.3 TB of data. ### Key Features: - **Multispectral Images**: Includes stereo RGB, depth, monochrome, near-infrared, and thermal images. - **Navigational Sensor Data**: Wheel odometry, LiDAR, IMU, and GPS-RTK data. - **Sequences**: Seven sequences from three citrus tree fields with diverse tree species, planting patterns, and daylight conditions. - **Total Data**: 1.3 TB of data collected over 1.7 hours and 7.5 km of operation. ### Tech Stack: - **Python** - Used for data processing and analysis. ### Why It Matters: CitrusFarm dataset offers a comprehensive and diverse set of data for researchers and developers working on agricultural robotics, localization, mapping, and crop monitoring. It provides a valuable resource for advancing the field of

### Overview: ContextDiff is a novel and general cross-modal contextualized diffusion model that addresses the inconsistency between forward and reverse processes in text-guided visual generation and editing. It incorporates cross-modal context into both processes to enhance semantic alignment between text conditions and generated samples. ### Key Features: - **ContextDiff** is a cross-modal contextualized diffusion model that integrates cross-modal context into both forward and reverse processes of text-guided visual generation and editing. - It achieves new state-of-the-art performance in two challenging tasks: text-to-image generation and text-to-video editing. - The model generalizes to both DDPMs and DDIMs with theoretical derivations, demonstrating effectiveness in evaluations. ### Tech Stack: Python ### Why it Matters: ContextDiff provides a more comprehensive and accurate representation of text-visual relationships, leading to improved semantic alignment and better quality results in text-guided visual synthesis tasks.