Projects

### Overview: Vo is a macOS CLI tool for real-time on-device transcription and translation, ideal for live meetings and on-the-fly communication. ### Key Features: - **Live Transcription**: Transcribe audio from the mic using Apple's `SpeechTranscriber`. - **Live Translation**: Translate real-time audio using `TranslationSession`. - **Multi-Lingual Support**: Transcribe and translate between multiple languages (e.g., `--src en-US,ja-JP --dst ja-JP,en-US`). - **Separate Channels**: Capture audio from both the mic and system speaker separately. - **Output Formats**: Supports TTY and JSONL output modes, with timestamps and chunk information. - **Confidence Metrics**: Transcription confidence is provided in JSONL format. - **Summary Output**: Provides a summary of the transcription at the end. - **Voice Processing**: Optional voice processing for echo cancellation and noise reduction. - **On-Device Summary**: Transcribe and generate a summary using Apple's `SystemLanguageModel`. ### Tech Stack: Swift ### Why it Matters: Vo simplifies the process of real-time communication by providing a seamless on-device solution for transcription and translation, ideal for live meetings and on-the-fly communication.

Overview: The Personal Voice Assistant is a sophisticated AI-driven tool designed to interact with users through natural language, leveraging a state-of-the-art language model (LLM) to provide a seamless and intuitive experience. Key features: 1. Play Music from YouTube 2. Fix Errors 3. Chatbot Tech stack: Python Why it matters: This assistant offers a personalized and intuitive interaction experience through natural language processing, making it a valuable tool for users seeking assistance or entertainment. Repository: kiritoInd/Personal-Voice-Assistant-Using-LLM-FunctionCalling Primary language: Python GitHub stars: 25 GitHub description: The Personal Voice Assistant is a sophisticated AI-driven tool designed to interact with users through natural language. Leveraging a state-of-the-art language model (LLM), this assistant provides a seamless and intuitive experience by understanding and executing functions. README excerpt:

Overview: Jazz Vector is a local-first vector similarity search tool designed for the Jazz Tools ecosystem, enabling efficient storage and retrieval of high-dimensional vectors directly on-device. Key features: - Local-first vector similarity search - Direct storage and query of high-dimensional vectors - Built for the Jazz Tools ecosystem Tech stack: TypeScript Why it matters: Jazz Vector simplifies vector-based data management, optimizing performance and reducing latency in real-time applications within the Jazz ecosystem. Repository: kysely/jazz-vector Primary language: TypeScript GitHub stars: 25 GitHub description: Local-first vector similarity search for Jazz Tools README excerpt: [](https://jazz.tools/docs/react/using-covalues/covectors)

Overview: web-txt2img is a lightweight, browser-only JavaScript/TypeScript library that generates images from text prompts using open-weights AI models, specifically SD-Turbo and Janus-Pro-1B. Key features: - Unified API: Load a model, generate an image, unload, and purge cache. - WebGPU-accelerated: Leverages modern GPU capabilities for fast inference. - Progress + abort: Phase updates and `AbortController` support. - SD-Turbo: Seeded generation (deterministic latents), 512×512 image size. - Cache aware: Uses Cache Storage for model artifacts where possible. Tech stack: TypeScript Why it matters: web-txt2img simplifies the process of generating images from text prompts directly in the browser, leveraging powerful AI models like SD-Turbo and Janus-Pro-1B, and provides a unified API for efficient and fast inference.

### Overview: Open Fiesta is a comprehensive AI chat application that allows developers to compare and choose the best AI models for various tasks. It features a modern chat interface, real-time AI streaming, and supports multiple models like OpenAI, Gemini, Claude, Perplexity, Deepseek, and Grok. The application is built with TypeScript, Next.js, Tailwind CSS, and Vercel AI SDK, and it supports local-first storage and persistent chat history. ### Key Features: - **Modern Chat Interface**: Real-time AI streaming. - **Multi-Model Chat**: Talk to multiple AI models in parallel. - **Authentication**: Secure with Better Auth. - **API Key Management**: Support for AIMLAPI, OpenRouter, and Vercel AI Gateway. - **Persistent Chat History**: Stored in Postgres. - **Local-First Storage**: IndexedDB for quick access to recent chats. - **Shareable Chats**: Export chats as Markdown or PDF. ### Tech Stack: - Next.js - TypeScript - Tailwind CSS - Vercel AI SDK - Vercel AI Gateway - OpenRouter - PromptKit - Better Auth ### Why It Matters: Open Fiesta simplifies the process of comparing and selecting the appropriate AI models for various tasks, making it a valuable tool for developers looking to integrate AI into their projects efficiently.

Overview: auto_labeler is an all-in-one library designed to automatically label any computer vision dataset with minimal manual effort. Key features: - High abstraction: auto_labeler abstracts away complex SOTA algorithms, making it easy to use. - Modular interface: it provides independent modules for various vision tasks, allowing for easy modification or addition of new algorithms. - Minimal user interaction: the library requires minimal configuration before labeling can begin, with a generic inference wrapper taking care of the rest. - Support for multiple tasks: auto_labeler supports image classification, object detection, instance segmentation, feature matching, visual question answering, and optical character recognition. Tech stack: Python Why it matters: auto_labeler simplifies the labeling process for computer vision datasets, making it accessible to researchers and companies without requiring extensive knowledge of complex algorithms or extensive manual labeling. Repository: mailcorahul/auto_labeler Primary language: Python GitHub stars: 25

Overview: MemFuse is an open-source memory layer designed for large language models, providing persistent and queryable memory across conversations and sessions. Key features: - Lightning-fast buffering with write aggregation - Seamlessly combines vector, graph, and keyword search - Human-inspired layered memory system - Local-first deployment options - Built on TimescaleDB with custom pgai implementation - Compatible with Qdrant, Neo4j, Redis, and expanding backend support - Secure isolation with robust scoping and access controls - Seamless integration with LangChain, AutoGen, Vercel AI SDK, and direct OpenAI/Anthropic/Gemini/Ollama API calls - Comprehensive layered testing framework for reliability at scale Tech stack: Python Why it matters: MemFuse enables AI agents to remember user preferences and context across sessions, recall facts and events from thousands of interactions, and optimize token usage by avoiding redundant chat history resending. This makes it an essential tool for developers building stateful LLM applications.

Overview: STRUCTURE is a framework that leverages limited paired data to build multimodal models, achieving significant improvements in cross-modality zero-shot classification and retrieval tasks. Key features: - STRUCTURE Regularization: Preserves the neighborhood geometry of the latent space of unimodal encoders. - Demonstration that aligning last layers is often suboptimal, with benefits from aligning layers with highest representational similarity. - Strong results: 51.6% average relative improvement in classification and 91.8% in retrieval tasks across 24 benchmarks. - Broad applicability: Can be readily incorporated into existing alignment methods. Tech stack: Python Why it matters: STRUCTURE provides a practical solution for building multimodal models with limited paired data, making it accessible and efficient for various domains where obtaining large amounts of paired data is impractical or costly. GitHub stars: 25

Overview: Katz is a high-performance serving system designed specifically for diffusion model workflows with multiple adapters, aiming to improve inference efficiency while maintaining image quality. Key features: - ControlNet-as-a-Service: Decouples ControlNets from the base model for independent scaling. - Bounded Asynchronous LoRA Loading: Overlaps LoRA loading with base model execution for reduced latency. - Latent Parallelism: Accelerates base model execution across multiple GPUs. - Performance Gains: Up to $7.8 \times$ latency reduction and $1.7 \times$ throughput improvement. Tech stack: Python Why it matters: Katz significantly enhances the efficiency and quality of diffusion model workflows, making it a valuable tool for researchers and practitioners working with complex image generation tasks.

Overview: This is a Python library designed for Modzy Machine Learning Operations (MLOps) Platform, providing developers with tools to interact with and manage the Modzy API. Key features: - Initialize the SDK with an API key for authentication - Submit text-based inferences - Submit file-based inferences - Embed large inputs as base64 - Submit data from a SQL database - Submit data from a cloud storage bucket Tech stack: Python Why it matters: This library simplifies the process of interacting with the Modzy MLOps Platform, allowing developers to easily integrate Modzy's services into their projects without needing to manually configure the API keys or handle authentication.

### Overview: This project presents a self-supervised multimodal opinion summarization framework called MultimodalSum, which addresses the challenge of generating summaries from multiple reviews using self-supervised learning. The framework leverages both textual and non-textual data to improve the quality and relevance of the summaries. ### Key Features: - **Self-Supervised Learning**: The model is trained end-to-end without the need for manual annotation, leveraging the abundant information in non-textual data. - **Multi-Modal Representation**: Each modality (text, image, and table) is represented independently using separate encoders, and the text decoder generates a summary. - **Multimodal Training Pipeline**: A pretraining pipeline that first trains the text encoder-decoder on text-only data, followed by training the non-text modality encoders using the pretrained text decoder as a pivot. - **End-to-End Training**: The entire framework is trained end-to-end to fuse multimodal representations. ### Tech Stack: - Python ### Why it Matters: MultimodalSum significantly improves the quality and relevance of opinion summaries by leveraging the rich information present in non-textual data, such as images and tables, which are often overlooked in traditional self-supervised methods. This approach enhances the overall summarization performance on datasets like Yelp and Amazon, making it a valuable tool for researchers and practitioners in natural language processing and information retrieval.

Overview: This repository contains notebooks for a PyTorch model-serving workshop, focusing on deploying BERT models in production using PyTorch and TorchServe. Key features: - Training a DistilBERT model - Introducing the Hugging Face ecosystem - Demonstrating the impact of quantization and TorchScript - Packaging and serving models with TorchServe Tech stack: Jupyter Notebook Why it matters: This workshop provides practical guidance on deploying PyTorch models in a production environment, specifically using TorchServe, which is essential for developers looking to scale machine learning models in production. Repository: npatta01/pytorch-serving-workshop GitHub stars: 25 GitHub description: Slides and notebooks for serving BERT models in production with PyTorch and TorchServe.

Overview: This project demonstrates a production-style Semantic Search + Retrieval-Augmented Generation (RAG) pipeline using .NET, PostgreSQL `pgvector`, and .NET Aspire. It showcases modern .NET practices such as Clean Architecture, pluggable LLM providers, and Aspire-based container orchestration. Key features: - Semantic search using embeddings - Retrieval-Augmented Generation (RAG) - pgvector integration with PostgreSQL - Clean architecture with strict separation of concerns - Pluggable LLM providers (Ollama / OpenAI) - Aspire-based container orchestration - Scalar UI for API testing Tech stack: .NET 10, ASP.NET Core Minimal APIs, Entity Framework Core, PostgreSQL + pgvector, .NET Aspire, Ollama (local LLM), OpenAI (optional) Why it matters: This project provides a practical example of building a semantic search and RAG pipeline using modern .NET practices, showcasing how to integrate with PostgreSQL and leverage pluggable LLM providers for improved performance and flexibility. Repository: rineshpk/dotnet-rag-pgvector GitHub stars: 25 GitHub description: This project demonstrates how to build a semantic search + RAG pipeline using modern .NET practices: Vector similarity search with pgvector, LLM-based answer generation (Ollama / OpenAI), Clean Architecture, .NET Aspire orchestration, and pluggable LLM providers.

Overview: This AI-powered OSINT (Open-Source Intelligence) tool uses Gemini LLM to analyze images and provide lists of potential locations such as countries, cities, landmarks, addresses, and postal codes. Key features: - Image analysis - Location suggestions - Integration with Vercel Tech stack: TypeScript Why it matters: This tool enhances the accuracy and efficiency of OSINT by leveraging advanced AI capabilities to identify potential locations within images, making it valuable for various applications requiring location-based intelligence.

### Overview: BabyReady is a project that uses a Convolutional Neural Network to predict the reason a baby is crying based on an mp3 recording. It was developed in 2018 by Ang Kang Rong Roy and Kyle Zheng Ching Chan with the help of Tang Yetong. The project won the champion and healthtech award at the Splash Awards 2018 and first runner-up in CodeXtremeApps 2018. ### Key Features: - Predicts the reason for a baby's cry using a Convolutional Neural Network. - Uses Fast Fourier Transform to analyze the audio recording. - Implements a Convolutional Neural Network to differentiate between different crying reasons. - Can play appropriate music to soothe the baby based on the predicted reason. ### Tech Stack: Python ### Why it Matters: BabyReady demonstrates the potential of machine learning in predicting and addressing the emotional needs of infants. By accurately identifying the reason for a baby's cry, the project aims to provide timely and effective care, potentially reducing the stress and discomfort experienced by both the baby and the caregiver. ### Repository: The project is hosted on GitHub at [royangkr/BabyReady](https://github.com/royangkr/BabyReady). It has received 25 GitHub stars and is maintained by the original developers.

Overview: A Hackable speech recognition library designed for developers, focusing on end-to-end models, fine-tuning, and inference support. Key features: - End-to-end speech recognition models - Simple fine-tuning to new languages - Inference support as a first-class feature - Developer-oriented API Tech stack: Python Why it matters: This library provides a comprehensive solution for developers looking to integrate speech recognition into their projects, offering a straightforward API and efficient inference capabilities. Repository: scart97/thunder-speech GitHub stars: 25 GitHub description: A Hackable speech recognition library.

Overview: Imagify is a MERN-stack application that transforms text into visually appealing images with customizable designs, making creativity and technology seamlessly blend. Key features: - Fast processing - Responsive design - Secure image processing - AI-powered images Tech stack: JavaScript Why it matters: Imagify simplifies the process of turning words into images, offering a user-friendly interface and powerful AI capabilities, making it a valuable tool for businesses and individuals looking to enhance their visual content.

Overview: The Houndify SDK for Go is an official Go library that allows developers to interact with the Houndify API, enabling voice and text queries. It includes a fully functional example app demonstrating various SDK features and no third-party dependencies. Key features: - Voice queries - Text queries - Example app demonstrating SDK usage Tech stack: Go Why it matters: This SDK provides a straightforward and efficient way for Go developers to integrate with Houndify's API, making it easier to build voice and text-based applications that can interact with Houndify's services.

Overview: Subgrapher is a local-first semantic browser workspace that leverages semantic references, hybrid RAG + Telegram Remote, and AI agents for context-aware reasoning and remote collaboration. Key features: - Semantic references for reusable knowledge containers - Ambient Notes for evidence-aware drafting - Local and remote models for AI reasoning - Cross-platform sharing and collaboration through private peer sharing and public publishing - Private peer sharing and public publishing - Telegram bot integration for remote model access Tech stack: JavaScript Why it matters: Subgrapher aims to provide a more open and decentralized approach to knowledge sharing and collaboration, avoiding the limitations of centralized platforms and traditional software tools. Repository: srimallya/subgrapher Primary language: JavaScript GitHub stars: 25 GitHub description: Subgrapher is a local-first semantic browser workspace where references become runnable memory apps with graph context, agent execution, and hybrid RAG + Telegram Remote.

### Overview: A free, AI-powered tool for mixing audio and video clips, utilizing WhisperX on an RTX 3080 for voice analysis. ### Key Features: - Sentence mixing for audio and video - Free and anonymous use - Supports multiple languages - Drag and drop functionality for easy file selection - Media trimming for optimal processing - Filtered word generation - Option to generate audio only - Preview and download capabilities ### Tech Stack: Svelte ### Why it Matters: This tool simplifies the process of combining audio and video clips, making it easier for anyone to create unique content without the need for manual editing. The AI-driven approach ensures high-quality results and saves time. ### Repository: stephenswetonic/ytpai ### GitHub Stars: 25 ### GitHub Description: AI powered ytp/sentence mixing for audio and video.