Projects

### Overview: Aila is a high-performance inference engine built with SYCL and oneDNN, designed specifically for Intel Arc GPUs. It leverages hand-optimized kernels for 4-bit quantized models and supports dense models via oneDNN primitives. Key features include dense and hybrid architectures, vision and audio processing, and real-time streaming capabilities. ### Key Features: - **Bitsandbytes 4-bit (NF4) inference** with fused dequant+matmul kernels. - **Bfloat16 inference** for dense models. - **Qwen3.5 Hybrid architecture** with GPU acceleration. - **Qwen3 Dense architecture** with GQA, QK-norm, and SwiGLU FFN. - **YOLO26 detection** with CPU preprocessing and GPU vision transformer. - **Forced Alignment** with CJK tokenization and LIS-based timestamp correction. - **CustomVoice / VoiceDesign** for named speaker presets and text-controlled voice styles. - **Streaming TTS** with real-time raw PCM output. - **Native bf16 GEMV kernel** for faster TTS decoding. - **Stream output** with token-level streaming callback. - **Benchmark mode** for separate prefill and decode throughput measurement. - **C API** for stable C FFI interface (Python, C#, Rust, Go, Java). - **Interactive CLI** for multi-turn conversation with runtime commands. ### Tech Stack: - SYCL - oneDNN ### Why It Matters: Aila is designed for Intel Arc GPUs, leveraging SYCL and

Overview: InferKt is a llama.cpp binding for Kotlin multiplatform API designed for common use on Android and iOS. Key features: - Create an instance of inference - Load a model - Set sampling settings - Perform completion - Engage in chat - Observe events Tech stack: Kotlin Why it matters: This library simplifies the integration of llama.cpp for Kotlin multiplatform, making it accessible for developers working on Android and iOS projects. Repository: Dilivva/Inferkt GitHub stars: 33 GitHub description: A llama.cpp binding for Kotlin multiplatform API for common use on (Android, iOS).

Overview: A Python-based Voice Assistant utilizing WhisperAI for speech recognition. Key features: - WhisperAI speech recognition engine integration - Support for multiple languages - Available in both Linux and Windows versions Tech stack: Python Why it matters: This Voice Assistant offers a practical solution for voice-based communication and language support, enhancing user experience across different platforms.

### Overview: Good Speech is a web application that helps you practice your speech level in any language by using speech recognition from Google's API. ### Key features: - Real-time speech recognition and pronunciation feedback - Support for multiple languages - Live example accessible at [goodspeech.github.io/good-speech-web-client](https://GoodSpeech.github.io/good-speech-web-client) ### Tech stack: JavaScript ### Why it matters: Good Speech provides a practical tool for improving language skills, especially for those who want to enhance their pronunciation in a fun and interactive way.

### Overview: A PHP Client for the Milvus Vector Database This package provides a PHP client for interacting with the Milvus vector database, which is an open-source, flexible, and reliable vector database designed for handling trillions of vectors. ### Key Features: - **Milvus REST API Client**: A PHP client specifically designed for the Milvus REST v2 endpoints. - **Stable Versions**: Compatible with Milvus versions 2.5.21, 2.6.21, and 3.0.0. - **Saloon Integration**: Built on Saloon, a PHP framework for building web applications. - **Stable Runtime Matrix**: Supported by PHP 8.3–8.5 and Laravel 12–13. ### Tech Stack: - PHP 8.3–8.5 - Laravel 12–13 - Saloon ### Why It Matters: Milvus is a powerful vector database that supports adding, deleting, updating, and near real-time search of vectors. This package simplifies interaction with Milvus, making it easier for developers to integrate vector database capabilities into their projects. ### Repository: HelgeSverre/milvus - GitHub Stars: 33 - GitHub Description: A PHP Client for the Milvus Vector Database Rest API ### Installation: - **Composer**: Use `composer require helgesverre/milvus` to install the package. - **Publish Config**: Use `php artisan vendor:publish --tag="mil

### Overview: A Calculadora é uma aplicação simples e intuitiva desenvolvida com JavaScript, focada em suporte a comandos de voz, temas personalizados, e funcionalidades como adição, subtração, multiplicação, e divisão. ### Key features: - Realiza operações matemáticas básicas (adição, subtração, multiplicação, divisão). - Suporte a comandos de voz para interação. - Temas personalizados para a interface. - Responsividade para diferentes tamanhos de tela. - Integração com comandos via teclado. - Shortcuts dinâmicos para troca de temas. - Guia automática ensinando a utilizar a calculadora. ### Tech stack: - JavaScript ### Why it matters: Esta calculadora é uma solução prática e eficiente para quem deseja uma calculadora simples e versátil, sem a necessidade de adicionar bibliotecas externas. A interface personalizável e a funcionalidade de comandos de voz tornam-a uma ferramenta útil para quem precisa de assistência técnica ou em situações onde a interação é feita por voz.

Overview: This project demonstrates a scalable, distributed deployment of Qdrant, a high-performance vector database, showcasing a multi-node cluster setup for efficient vector search operations with high availability and performance. Key features: - Scalable Multi-Node Architecture - Dynamic Sharding - High Availability - Monitoring Stack - Python Client Integration - Docker-Based Deployment - Detailed Configuration Tech stack: Python Why it matters: This project is essential for enterprises looking to deploy Qdrant for semantic text search, image similarity, recommendation systems, anomaly detection, and chatbot knowledge bases, providing a scalable and efficient solution for vector similarity search applications. Repository: GitHub - https://github.com/Mohitkr95/qdrant-multi-node-cluster

### Overview: `magnet` is a small, distributed language model toolkit designed to fine-tune state-of-the-art Large Language Models (LLMs) anywhere and rapidly. It supports a wide range of inference methods, including HuggingFace, vLLM node, GPU, and mlx, making it highly flexible and adaptable to various computing environments. ### Key Features: - **Fast Performance**: `magnet` is exceptionally fast on consumer hardware, Apple Silicon, and ROCm/CUDA, with extremely fast performance on ROCm/CUDA. - **Automatic or Your Way**: `magnet` relies on established transformer patterns to let users do the work, while also allowing users to bring their existing data processing functions. - **100% Distributed**: Processing, embedding, storage, retrieval, querying, or inference can be done from anywhere, with as much or as little compute as needed. - **Choose Inference Method**: `magnet` supports HuggingFace, vLLM node, GPU, and mlx, providing flexibility in choosing the inference method. - **Handle Huge Volumes**: `magnet` can handle enormous amounts of data inexpensively, with fault-tolerant design and decentralized workloads. - **Secure**: `magnet` supports JWT and Basic authentication, ensuring security. - **World-Class Comprehension**: `magnet` logs its own code as it's executed, offering world-class comprehension. ### Tech Stack: Python ### Why It Matters: `magnet` is a powerful tool for developers looking to fine-tune LLMs quickly and

Overview: Quartermaster is a local AI inference platform that runs any model without tuning flags, providing chat, images, and voice capabilities on a single machine. It is designed for easy integration into existing infrastructure and supports multiple compute backends. Key features: - Runs on your hardware - Bring your own models - Chat app with logins - OpenAI + Anthropic API - No telemetry Tech stack: Go Why it matters: Quartermaster simplifies the process of integrating AI models into local infrastructure, making it accessible and easy to use without the need for extensive configuration.

### Overview: SemaDB is a multi-index, multi-vector, document-based vector database / search engine designed for a simple, modern, and efficient search engine. ### Key features: - **Vector search**: leveraging the power of vector search to find similar items and build AI applications. - **Keyword / text search**: search for documents based on keywords or phrases, categories, tags etc. - **Geo indices**: search for documents based on their location either via latitude and longitude or geo hashes. - **Multi-vector search**: search across multiple vectors at the same time for a single document each with own index. - **Quantized vector search**: use quantizers to change internal vector representations to reduce memory usage. - **Hybrid search**: combine vector and keyword search to find the most relevant documents in a single search request. - **Filter search**: filter search results based on other queries or metadata. - **Hybrid, filter, multi-vector, multi-index search**: combine all the above search types in a single query. - **Simple REST API**: JSON or MessagePack based, restful API for interacting with the database. - **Real-time**: changes are visible immediately and search results are returned in milliseconds, even for large datasets. - **Single binary**: the entire database is contained in a single binary. No need to install additional dependencies or services. - **Multiple deployment modes**: standalone, container, or cloud. SemaDB can be deployed in a variety of ways to suit your needs. - **Prometheus metrics**: monitor the health of SemaDB with metrics such as number of

Overview: This project focuses on developing a model capable of detecting multiple cards and identifying their suit and rank, utilizing various YOLOv8 models and datasets. Key features: - Multiple YOLOv8 models for playing cards object detection - Synthetic, "Real", and "Real Augmented" datasets - Combined "Real" and "Synthetic" dataset - Pretrained models available in the final_models directory - Live demo application integrating the best models for card detection using a webcam Tech stack: Python Why it matters: This project offers a comprehensive solution for playing cards object detection, utilizing advanced YOLOv8 models trained on diverse datasets, making it suitable for applications requiring accurate card identification.

Overview: LifelongMemory is a framework designed to access long-form egocentric videographic memory through natural language question answering and retrieval. It leverages the reasoning and contextual understanding capabilities of pretrained large language models to provide precise answers. The approach achieves state-of-the-art performance on benchmarks for question answering and is highly competitive on the natural language query challenge of Ego4D. Key features: - Leverages pretrained large language models for precise answer generation. - Incorporates a confidence and refinement module for improved accuracy. - Supports multiple pretrained models for flexibility and performance optimization. Tech stack: Python Why it matters: LifelongMemory offers a novel solution for accessing long-form video memory through natural language queries, addressing the need for efficient and accurate information retrieval in video-centric applications. Its ability to leverage pretrained models and confidence refinement makes it highly competitive in benchmarks and practical use cases. Repository: agentic-learning-ai-lab/lifelong-memory GitHub stars: 33 GitHub description: Code for LifelongMemory: Leveraging LLMs for Answering Queries in Long-form Egocentric Videos

Overview: nanoCLIP is a lightweight CLIP-like model designed for efficient text-to-image retrieval, enabling users to search their photo collections using natural language. Key features: - **Text-to-Image Retrieval**: Implements an efficient CLIP-based model using the Flickr30k dataset. - **Modern ML Practices**: Provides an opportunity to learn PyTorch Lightning and effective training/validation/tracking pipelines. - **Deployment**: Offers a responsive web application using Gradio for image retrieval. Tech stack: Python Why it matters: nanoCLIP simplifies text-to-image retrieval, making it accessible and practical for personal photo management. Its lightweight design and efficient indexing capabilities make it ideal for both personal use and small-scale deployments. Repository: amaralibey/nanoCLIP GitHub stars: 33 GitHub description: A lightweight Text-to-Image Retrieval model [Web App]

### Overview: Vox is a privacy-focused voice-to-text tool that uses local Whisper transcription and AI correction to transcribe your audio locally, ensuring your data stays on your computer. ### Key features: - **🔒 100% Local Transcription**: Transcribes your voice using whisper.cpp, keeping the audio on your device. - **🤖 AI Correction**: Optionally corrects the transcription with AI to improve grammar and remove filler words. - **⚙️ Custom Prompts**: Tailor corrections for specific workflows, such as medical or technical fields. - **⌨️ Hold or Toggle Modes**: Press-and-hold or toggle recording on/off. - **📋 Auto-Paste**: Text is pasted directly into your focused app. - **🎯 Multiple Models**: Choose between tiny and large models for balance between speed and accuracy. - **☁️ Multiple LLM Providers**: Supports OpenAI-compatible or AWS Bedrock providers. - **🎨 Menu Bar App**: Runs quietly in the background with dark/light mode support. ### Tech stack: TypeScript ### Why it matters: Vox ensures your data stays private by keeping audio on your device, making it ideal for professionals who need to maintain confidentiality. The AI correction feature enhances the quality of the transcribed text, making it suitable for various use cases such as medical professionals and developers.

### Overview: AutoLlama is a JavaScript-first RAG framework that leverages Anthropic's breakthrough contextual retrieval methodology to deliver a more comprehensive understanding of documents. Built with open-source freedom, it offers a simple setup, powerful architecture, and real-time intelligence, making it ideal for developers looking to enhance their document processing workflows. ### Key Features: - **Contextual Retrieval**: Implements Anthropic's advanced methodology for better understanding and context-aware retrieval. - **Quick Start**: Instantly set up with a single command, including global CLI, NPX, Docker, and native installation. - **JavaScript-First Architecture**: Full-stack JavaScript with multiple deployment modes, including local development, hybrid, and production. - **Open WebUI Integration**: Built-in chat interface that automatically discovers and integrates with the RAG pipeline. ### Tech Stack: - JavaScript - HTML - TypeScript - Node.js ### Why It Matters: AutoLlama transforms traditional RAG by providing a more comprehensive understanding of documents, making it essential for developers looking to enhance their document processing workflows. Its real-time intelligence and open-source nature make it a powerful tool for developers seeking to improve their applications' performance and user experience. ### GitHub Stars: 33 ### Repository: autollama/autollama

### Overview: This repository contains a collection of projects developed by a student as part of a Human-Computer Interaction (HCI) course at Tongji University. The projects include a simple voice assistant, an image search tool, a data visualization tool, and an end-of-term project focused on a lost instrument app. ### Key features: - **Voice Assistant Siri:** A basic voice assistant that can play music, open notepad, and open the calculator. - **Image Search Tool:** A tool that allows users to search for similar images based on uploaded images. - **Data Visualization Tool:** A tool that helps users visualize data, including categorizing apps by rating, size, and content rating, and analyzing the relationship between reviews and installs. - **Lost Instrument App:** A project focused on a lost instrument app, allowing users to select and interact with instruments like erhu, zither, and lute. ### Tech stack: The projects are developed using C# and utilize deep learning techniques for image similarity search. ### Why it matters: These projects showcase practical applications of HCI principles in real-world scenarios, demonstrating how technology can be used to enhance user interaction and information retrieval.

### Overview: The emoASR repository is an end-to-end model for Automatic Speech Recognition (ASR) that includes both Encoder and Decoder components. It utilizes various models such as RNN, Transformer, Conformer, and RNN-T for the Encoder, and CTC, RNN-Transducer, LAS, and Transformer for the Decoder. The project also incorporates Language Models (LMs) like RNNLM, Transformer LM, BERT, ELECTRA, and Phone-attentive ELECTRA (P-ELECTRA) with techniques like Rescoring, Shallow Fusion, and Knowledge Distillation. ### Key features: - **ASR Models**: The project includes Encoder models using RNN, Transformer, Conformer, and RNN-T. - **Decoder Models**: The Decoder models use CTC, RNN-Transducer, LAS, and Transformer. - **Language Models**: The project includes models like RNNLM, Transformer LM, BERT, ELECTRA, and P-ELECTRA with various techniques like Rescoring, Shallow Fusion, and Knowledge Distillation. ### Tech stack: The emoASR project is implemented using Python. ### Why it matters: The emoASR project is significant because it provides a comprehensive solution for ASR, combining advanced Encoder and Decoder models with state-of-the-art Language Models. This project is valuable for researchers and developers looking to improve the accuracy and efficiency of ASR systems.

Overview: This project is an AI inference app built using React Native and fal.ai, designed for developers to integrate AI capabilities into their mobile applications. Key features: - AI inference capabilities - React Native integration - fal.ai integration Tech stack: TypeScript Why it matters: This project showcases how to integrate AI into mobile applications using React Native and fal.ai, making it valuable for developers looking to enhance their apps with AI capabilities.

### Overview: Ion is a modern AI voice interface that allows you to interact with a Nano Bot, providing a conversational experience. It is highly customizable, powerful, and hackable, making it a versatile tool for developers. ### Key Features: - **Voice Interface**: Enables direct communication with a Nano Bot. - **Customizable**: Leverages Picovoice for voice recognition, Nano Bots for compatibility with various AI models, and Nano Apps for extension. - **Powerful**: Backed by Nano Bots, compatible with OpenAI ChatGPT and Google Gemini. - **Hackable**: Supports Nano Apps written in Lua, Fennel, or Clojure. ### Tech Stack: Ruby ### Why it Matters: Ion offers a modern, customizable, and powerful way to interact with AI models, providing a unique and engaging conversational experience. Its flexibility and compatibility make it a valuable tool for developers looking to enhance their projects with AI-driven voice interfaces.

Overview: This is a React Native library for speech recognition, wake-up, and synthesis provided by Baidu. Key features: - Speech Recognition - Voice Wake-Up - Voice Synthesis Tech stack: Java Why it matters: It simplifies the integration of speech capabilities into React Native applications, making it easier to implement voice-based features. Repository: gdoudeng/react-native-baidu-asr GitHub stars: 33