Projects

Overview: SrvDB is an open-source embedded vector database for Python and Rust, designed for local, crash-safe, and offline use. It supports semantic search, RAG, and AI memory with no server or cloud. Key features: - **Local, Offline, Crash-Safe**: No server, no need for a database service, no network access. Data is stored on local disk, ensuring data safety and integrity. - **Python and Rust**: Supports Python and Rust, making it versatile for different programming environments. - **True Cosine Scores**: Ensures accurate cosine similarity scores in the range [-1, 1], rejecting bad input before any data is written. - **Thread-Safe**: Supports concurrent access from multiple threads, ensuring data integrity and parallel search capabilities. - **Rust Core**: Uses Rust for its core functionality, providing SIMD distance kernels, typed errors, and no Python runtime dependencies. - **4x/32x Smaller Storage**: Offers different storage modes for varying performance and storage efficiency. Tech stack: Rust. Why it matters: SrvDB is designed for local, offline use, ensuring data safety and integrity. It supports semantic search, RAG, and AI memory, making it suitable for various use cases such as local RAG, AI agent memory, semantic search, and more. Its thread-safe and crash-safe nature make it ideal for edge devices and offline systems, and its small storage footprint makes it suitable for applications where space is a concern. Repository: Srinivas26k/srvdb GitHub stars: 16 GitHub

Overview: ToucanDB is a micro ML-first database engine designed for RAG systems, semantic search, and language model applications. It combines atomic SQLite persistence with FAISS search and does not require a separate database server. Key features: - Supports cosine, dot-product, and Euclidean similarity. - Offers exact flat search and approximate HNSW or IVF indices. - Uses SQLite WAL storage with atomic batch writes and safe vector IDs. - Provides FAISS snapshots with deterministic rebuild after crashes or cross-architecture restores. - Includes a bounded, database-wide LRU cache with automatic tombstone compaction. - Supports optional authenticated encryption with a per-database salt and memory-hard key derivation. - Provides async APIs for moving storage, index, and local-model work off the event loop. - Offers stable upserts that do not re-embed unchanged documents. - Includes a tested RAG pipeline with chunking, namespace isolation, pruning, source attribution, and bounded context. - Provides privacy-safe pipeline traces, reviewed retrieval evaluation, and optional bounded FAISS topic discovery. - Offers lazy Sentence Transformers and OpenAI embedding adapters, plus a generic callable adapter. - Is an embedded component, not a distributed service, owned by a single process at a time. Tech stack: Python Why it matters: ToucanDB is a powerful tool for semantic search, RAG systems, and language model applications, providing efficient storage, indexing, and search capabilities without the need for a separate database server. Its atomic SQLite persistence and FAISS search make it ideal for applications requiring high performance and reliability.

Overview: This repository contains reference implementations for advanced Fine-Tuning and Alignment techniques in Large Language Models, focusing on enhancing performance and reducing computational cost while ensuring alignment with human values. Key features: - Distributed Training Demos - Instruction Tuning Demo - Parameter Efficient Fine Tuning (PEFT) - Model Quantization - Supervised Fine-Tuning Tech stack: Jupyter Notebook Why it matters: This repository provides practical, hands-on examples that developers can use to improve their models, optimize their training processes, and ensure their models are aligned with ethical values, making it valuable for both researchers and practitioners in the field of AI and NLP. GitHub stars: 16

### Overview: Dowse is a powerful local and private semantic search tool for Windows, designed to find files on your computer based on their content. It supports 100+ languages and can search through photos, screenshots, PDFs, documents, recordings, and videos. ### Key Features: - **Local Search**: Dowse searches your local files directly, without uploading them to the cloud. - **EmbeddingGemma 2**: Uses Google DeepMind's open multimodal embedding model to understand the content of files. - **Multi-Language Support**: Dowse can search in any language and find files in another language. - **Quick Search**: Type what you remember, and Dowse finds the file based on the content. ### Tech Stack: Python ### Why It Matters: Dowse is a useful tool for anyone who wants to quickly find files on their computer, regardless of where they are located or what language the files are in. It's particularly useful for professionals who need to quickly find specific information in their work files. ### Overview: Dowse is a local and private semantic search tool for Windows, designed to find files on your computer based on their content. It supports 100+ languages and can search through photos, screenshots, PDFs, documents, recordings, and videos. Dowse uses Google DeepMind's open multimodal embedding model to understand the content of files and can search in any language and find files in another language. Dowse is a powerful tool for anyone who wants to quickly find files on their computer, regardless of where they are located or

Overview: Tangram is a high-throughput LLM serving engine that optimizes non-uniform KV cache compression for multi-turn LLM requests. Key features: - Efficient KV cache compression using non-uniform and uniform compression methods - Seamless integration with vLLM, including paged attention, continuous batching, and CUDA graph mode - Real memory reclamation, turning compression savings into increased serving throughput - Native support for multi-turn serving Tech stack: Python Why it matters: Tangram simplifies LLM serving by optimizing memory usage and throughput, making it a practical solution for high-throughput applications. Repository: aiha-lab/tangram Primary language: Python GitHub stars: 16 GitHub description: A high-throughput LLM serving engine with non-uniform KV cache compression, built on vLLM README excerpt:

Overview: This is the official PyTorch implementation for "Merging and Splitting Diffusion Paths for Semantically Coherent Panoramas," presenting the Merge-Attend-Diffuse operator. The code is tested on Python 3.11.7, CUDA 12.1, and PyTorch 2.1.2. Key features: - Merge-Attend-Diffuse operator - Inference-time pipeline - Panorama generation using Stable Diffusion - Region-based generation using Stable Diffusion - LCM (Learning-based Multidiffusion) inference Tech stack: Python Why it matters: This implementation is useful for generating semantically coherent panoramas using diffusion models, which can be applied in various applications such as image synthesis and augmented reality. Repository: aimagelab/MAD Primary language: Python GitHub stars: 16 GitHub description: Official PyTorch implementation for "Merging and Splitting Diffusion Paths for Semantically Coherent Panoramas," presenting the Merge-Attend-Diffuse operator.

### Overview: DenseSpark is an optimized version of Qwen3.8-27B model, designed to run efficiently on NVIDIA DGX Spark. This project measures and optimizes the model's configuration to make it fast and accessible on a single DGX Spark node. ### Key Features: - **Optimized Configuration:** Removes most of the cost associated with running Qwen3.8-27B on a single DGX Spark. - **Measurable Performance:** Every number below was measured on one machine with the protocol stated beside it. - **Single Command:** Ships a measured configuration that requires just one command to run. ### Tech Stack: Python ### Why It Matters: On the Artificial Analysis Intelligence Index, Qwen3.8-27B ranks 52 points out of 56, making it a very good model. However, it is painfully slow to run at home. DenseSpark optimizes the model to run efficiently on a single DGX Spark, making it fast and accessible. This project measures and optimizes the model's configuration to make it fast and accessible on a single DGX Spark node, making it a valuable tool for developers and researchers.

### Overview: This project involves the development of a team that participated in the Brainhack TIL 2024 competition, focusing on three tasks: Automatic Speech Recognition (ASR), Natural Language Processing (NLP), and Vision-Language Modelling (VLM). The team, consisting of Billy Cao, Ho Wing Yip, Huang Qirui, Marcus Wee, and Ooi Xuan Shan, aimed to control a DJI Robomaster's turret in a simulated environment using the results of their ASR and NLP models. The project was evaluated and ranked 7th in the Finals. ### Key Features: - **ASR**: Improved accuracy using Whisper Medium model with 0.9956923723471317 accuracy score. - **NLP**: Enhanced performance with the gorilla-openfunctions-v2 model achieving 0.99933333 accuracy. - **VLM**: Utilized YOLOv6-L6, RealESRGAN-x4v3, and SigLIP-large-patch16-384 models with an overall accuracy of 0.913. ### Tech Stack: The project primarily uses Jupyter Notebook for data analysis and model training. ### Why It Matters: The project highlights the importance of model generalization and robustness in real-world applications. The team's overfitting issue with their VLM model suggests that overtraining can lead to poor performance in the wild, emphasizing the need for careful model validation and testing across different datasets and

Overview: This project involves building a deep convolutional neural network (CNN) model for breast cancer classification using mammogram images. The model consists of three convolution and max_pooling layers, followed by a dense layer for classification. Key features: - A CNN model with three convolution and max_pooling layers, followed by a dense layer for binary classification. - Transfer learning approach using VGG16 and ResNet50 pretrained networks. - Evaluation metrics: accuracy, precision, recall, F1-score, confusion_matrix, AUC, and ROC curve. Tech stack: Jupyter Notebook. Why it matters: This project is crucial for improving the accuracy of breast cancer detection and treatment, ultimately reducing mortality rates through early diagnosis.

### Overview: This repository contains the official implementation of the paper "Mitigating Hallucinations in Large Vision-Language Models via Summary-Guided Decoding" presented at NAACL 2025 Findings. The project introduces a novel decoding strategy to reduce hallucinations in Large Vision-Language Models (LVLMs) by leveraging summary-based guidance. ### Key Features: - **SumGD-S (Self-Summarization)**: The LVLM generates its own summary before performing guided decoding. - **SumGD-D (Distilled Flan-T5)**: Utilizes a distilled summary model for more efficient summary generation. ### Tech Stack: Python ### Why It Matters: This approach significantly improves model performance on tasks requiring strong visual grounding while effectively reducing reliance on language priors, making it a valuable addition to the field of vision-language models.

Overview: This repository serves as a comprehensive guide to understanding Generative AI, covering its principles, applications, and practical implementation. Key features: - Introduction to Generative AI - Explanation of Generative AI models - Code examples in Jupyter Notebook - Real-world applications of Generative AI - Tips for beginners and advanced users Tech stack: Jupyter Notebook Why it matters: This repository is essential for developers and researchers looking to delve into the world of Generative AI, providing a practical and accessible way to learn and apply this technology.

Overview: This repository contains examples for utilizing the SiLLM framework to train and run Large Language Models (LLMs) on Apple Silicon. It includes various agents, beam search applications, control vectors, LoRA training, DPO-Mix-7K training, MMLU benchmark, MMLU-Pro benchmark, perplexity calculation, and speculative decoding and spelling correction. Key features: - Agents - Beam search applications - Control vectors - LoRA training - DPO-Mix-7K training - MMLU benchmark - MMLU-Pro benchmark - Perplexity calculation - Speculative decoding - Spelling correction Tech stack: Python Why it matters: This repository provides practical examples and demonstrations of how to effectively utilize the SiLLM framework for training and running LLMs on Apple Silicon, making it a valuable resource for developers looking to enhance their model training and inference capabilities.

### Overview: This repository contains the code for image classification using PyTorch, with the addition of MLflow for tracking experiments. The code is designed to be flexible, allowing for training with various models like VGG, SqueezeNet, DenseNet, ResNet, and Inception, and supports datasets like VGG, SqueezeNet, DenseNet, ResNet, AlexNet, and Inception. ### Key Features: - **MLflow Integration**: Tracks experiments, packages code into reproducible runs, and shares and deploys models. - **Fine-Tuning and Feature Extraction**: Option to fine-tune the model (feature_extract = False) or freeze the last layer (feature_extract = True). - **Training Parameters**: Configurable parameters such as model name, number of classes, batch size, number of epochs, and feature extraction. - **Confusion Matrix Generation**: After training, a confusion matrix is generated for evaluation. - **Model Saving**: Trained models are saved using both PyTorch and MLflow. ### Tech Stack: PyTorch, MLflow, Docker ### Why it Matters: This repository provides a comprehensive solution for image classification using PyTorch, with added features for experiment tracking and model sharing. It is designed to be flexible and scalable, making it suitable for various datasets and models.

Overview: Seedance 2.0 Fast API is an OpenAI-compatible endpoint for generating text-to-video and image-to-video content with various resolutions and framing options. Key features: - Supports text-to-video and image-to-video generation with 300+ models - Charges per generated second, billed by resolution tier - No subscription required, only $1 minimum charge - Real-time pricing and automatic task refunds - Uses an asynchronous design for efficient batch processing and retry Tech stack: Python Why it matters: Seedance 2.0 Fast API provides a fast and cost-effective way to generate high-quality video content using the same models as the original Seedance 2.0 route, without the need for a monthly subscription.

### Overview: FLUX Kontext prompts: tested image prompt recipes with example renders, aspect ratios, the model id (`flux-kontext-pro`) and the cost per image. ### Key features: - **Model ID:** `flux-kontext-pro` - **Aspect Ratios:** 1:1, 4:3, 16:9, 8:5, 3:2, 1:16 - **Cost per Image:** $0.032 ### Tech Stack: Python ### Why it Matters: These prompts are designed to test and refine the effectiveness of different image generation models, ensuring that the output is as desired. By providing a structured way to test various aspects of the model, developers can optimize their workflows and achieve better results with minimal effort.

### Overview: This project provides a complete pipeline for converting multi-page documents into structured JSON using Vision LLMs on Amazon SageMaker. It leverages the SWIFT Framework to fine-tune models specifically for document understanding tasks. ### Key Features: - End-to-End Document Processing - SageMaker Integration - Multi-Modal Support - JSON Constrained Decoding - Advanced Evaluation - Deployment Ready ### Tech Stack: Jupyter Notebook ### Why it Matters: This project offers a comprehensive solution for converting documents into structured JSON, leveraging AWS infrastructure for efficient training and inference. The use of SWIFT Framework ensures that models are specifically tailored for document understanding tasks, leading to better performance and accuracy.

Overview: This repository implements Model-Distributed Inference for large language models, specifically for the LLM family, using the litGPT framework. It allows deploying state-of-the-art LLMs on a network of edge devices, optimizing performance with minimal hardware resources. Key features: - Model-Distributed Inference - LitGPT Framework - Distributed Nodes for Inference - Recurrent Pipeline Parallelism - Training and Inference for LitGPT Models - Support for multiple devices Tech stack: Python Why it matters: This approach enables efficient inference of large language models on edge devices, leveraging distributed computing to maximize performance with limited hardware capabilities. Repository: davmacario/MDI-LLM GitHub stars: 16

Overview: This project aims to enhance the search capabilities of an LLM by integrating a Retrieval-Augmented Generation (RAG) pipeline, specifically using a Phoenix_LiveView documentation repository. The goal is to improve the LLM's responses when querying the documentation. Key features: - RAG pipeline integration - Local database and LLM setup - Local repository extraction of documentation - Local SQL Full-Text-Search implementation - Open-source Elasticsearch (Elastisearch) integration - RAG pipeline with LLM for enhanced responses Tech stack: Local Phoenix_LiveView documentation repository, PostgreSQL, Elasticsearch, Livebook Why it matters: This project demonstrates the effectiveness of integrating an RAG pipeline with an LLM, improving the quality of responses to specific queries in the Elixir/Phoenix/Plug/LiveView documentation.

### Overview: A vector database designed for querying meaningfully similar data. ### Key features: - **On-disk database index**, minimizing memory impact for large datasets. - Distance metrics and embedding models provided, with external implementations possible. - Insert, delete, and query vectors. - **No memory-mapped (MMAP) file IO**. - Parallelised database operations; multithreaded reads and writes are safe. ### Tech stack: Rust ### Why it matters: A scalable, mutable vector database that meets the requirements of a content recommendation system, ensuring safe multithreaded database access, efficient CRUD operations, and acceptable recall with multiple distance metrics.

Overview: This project is a high-performance, multi-modal Retrieval-Augmented Generation (RAG) framework that integrates vLLM for rapid inference and Milvus for vector storage, enabling it to process both text and images to deliver grounded, context-aware intelligence. Key features: - Model-Agnostic (Flexible Switching): Swap the rapidly evolving AI models via environment variables. - Multi-modal RAG (Vision-Text Integration): Captions and indexes document images using VLM, transforming visual data into searchable text assets. - High-Performance Inference: Delivers low-latency responses and high throughput powered by the vLLM Async Engine. - Advanced Data Pipeline: Implements Semantic Chunking and Reranker to ensure that only the most relevant context reaches the LLM. - Graph-RAG Capabilities (Structural Intelligence): Supports structured relational search by extracting Entities and Relationships from raw text. Tech stack: Python Why it matters: This project is designed to be highly efficient and capable of integrating various types of data, making it a valuable tool for developers looking to enhance their applications with advanced AI capabilities.