Projects

Overview: aiDAPTIV is a storage-accelerated AI infrastructure platform designed to handle large language model training and inference by utilizing high-performance NVMe SSDs alongside GPU VRAM. Key features: - Storage-accelerated LLM inference and fine-tuning workflows - Installation and deployment guides for aiDAPTIV memory management middleware - Documentation on KV-cache offload and SSD-based memory tiering - Hardware compatibility, benchmarks, and performance guidance - Integration examples and reference architectures Tech stack: Python, PyTorch, vLLM, llama.cpp Why it matters: This platform enables developers, researchers, and enterprises to run larger models, longer context windows, and more demanding workloads on existing on-prem hardware without relying on cloud infrastructure, thereby reducing costs and improving efficiency.

Overview: libstrix is a HIP kernel library optimized for AMD Strix Halo GPUs, designed to enhance the performance of the vLLM model by up to 31x faster prefill and ~2x faster decode. Key features: - HIP kernels tuned for gfx1151 (Strix Halo, Radeon 8060S) - A vLLM plugin that swaps the tuned kernels into stock vLLM, resulting in up to 31x faster prefill and ~2x faster decode at long context. - No vLLM source changes, ensuring compatibility with other GPUs. Tech stack: Python Why it matters: libstrix significantly improves the performance of the vLLM model by optimizing the kernel code for the specific hardware, leading to substantial speedups in both prefill and decode operations. Repository: asf0/libstrix

Overview: This repository is a curated list of open source and commercial feature store tools and platforms, including Amazon SageMaker Feature Store, Butterfree, ByteHub, Databricks Feature Store, Feast, Hopsworks Feature Store, RasgoQL, Tecton, and Vertex AI Feature Store. Key features: The repository includes a wide range of feature store tools and platforms, such as Amazon SageMaker Feature Store, Butterfree, ByteHub, Databricks Feature Store, Feast, Hopsworks Feature Store, RasgoQL, Tecton, and Vertex AI Feature Store. Tech stack: The repository is written in Python, JavaScript, and other programming languages. Why it matters: This repository is essential for developers and data scientists who need to build, manage, and utilize feature stores in their machine learning projects.

### Overview: Spark is a self-hosted AI coding agent that provides a unified interface for developers, offering features like approval, three-OS sandboxing, rollback, and provider resilience. It is designed to be used with OpenAI-compatible endpoints and does not require an API key. ### Key Features: - **Web UI**: The primary interface for developers. - **Desktop Window**: A native desktop window for seamless integration. - **Textual TUI**: Command-line interface for scriptable operations. - **Scriptable CLI**: Command-line interface with a Zed-like interface. - **ACP (IDE)**: Integrated development environment. - **MCP Server**: A server for managing tasks and workflows. - **A2A Server**: An agent-to-agent server for delegating tasks. ### Tech Stack: Python ### Why It Matters: Spark simplifies the process of AI coding by providing a unified interface and robust security features like approval, sandboxing, and rollback. It is designed to be used with OpenAI-compatible endpoints and does not require an API key, making it ideal for developers who want to integrate AI into their workflows without the hassle of managing multiple tools.

Overview: This R package is designed to generate meeting minutes from audio recordings or transcripts using speech-to-text and large language model (LLM) summarization. It supports various speech-to-text APIs and local models for transcribing audio and summarizing transcripts. Key features: - Transcribes audio recordings using OpenAI Whisper API or Azure Whisper model - Summarizes transcripts using either OpenAI GPT-4 or local models like Whisper-CTRANSLATE2 - Supports multiple LLM models for different summarization quality and context window - Provides a customizable workflow template for meeting transcription Tech stack: R, OpenAI Whisper API, Azure Whisper model, Whisper-CTRANSLATE2, llmR package Why it matters: This package streamlines the process of generating meeting minutes, making it easier to quickly summarize large amounts of audio or transcript data. It leverages advanced AI models to ensure accurate and comprehensive summaries, making it valuable for project management and meeting organization.

### Overview: CodexBridge is an OpenAI-compatible bridge that exposes the official Codex CLI/SDK as `/v1/chat/completions`, facilitating seamless integration with various OpenAI-style clients. ### Key features: - **OpenAI-compatible API**: `/v1/chat/completions` (sync + SSE) and `/v1/models`. - **Persistent sessions**: Use `session_id` / `conversation_id` / `thread_id` / `user` to maintain context; omit for ephemeral runs. - **Multimodal input**: Accepts `image_url` and `local_image` blocks, converting them to `local_image`. - **Structured output**: Maps OpenAI `response_format` to Codex `outputSchema`. - **Configurable sandbox**: Environment variables control filesystem access, working directory, networking, and approval policy. - **Self-hosted storage**: Sessions live in `~/.codex/sessions`, and bridge mappings are stored in `.codex_threads.json`. ### Tech stack: Node.js 18+ and npm. ### Why it matters: CodexBridge simplifies integration with various OpenAI-style clients, making it easier to use Codex in local environments or as a drop-in model replacement.

### Overview: MELT is a mobile evaluation framework for language transformers, designed to assess the performance of large language models on mobile devices. The project aims to provide a comprehensive and efficient evaluation tool for developers and researchers. ### Key Features: - **Mobile Evaluation**: Meets the needs of mobile applications by evaluating language models on low-powered devices. - **Framework Support**: Compatible with multiple LLM frameworks, including MLC-LLM and LLMFarm. - **Model Conversion**: Converts models to run on mobile platforms, ensuring compatibility with various hardware configurations. - **Prompt Analysis**: Analyzes input prompts to understand how models interpret and respond to different inputs. - **Model Evaluation**: Provides detailed metrics and performance analysis for each model, helping developers identify strengths and weaknesses. ### Tech Stack: Python ### Why it Matters: MELT offers a practical solution for evaluating language models on mobile devices, ensuring that models perform optimally and efficiently on the target platforms. This is particularly important for developers and researchers working on applications that need to be lightweight and resource-efficient. Repository: brave-experiments/MELT-public GitHub Stars: 21

### Overview: `or-cli.py` is a Python command-line tool designed for interacting with AI models through the OpenRouter API and Cloudflare AI Gateway. It supports direct API calls, request caching via Cloudflare AI Gateway, and local model inference with Ollama, optionally leveraging Microsoft LLMLingua prompt token compression for reduced prompt token sizes. ### Key Features: - Direct API calls to OpenRouter AI - Request caching via Cloudflare AI Gateway - Local model inference with Ollama - Optional Microsoft LLMLingua prompt token compression ### Tech Stack: Python ### Why It Matters: `or-cli.py` simplifies the process of interacting with AI models, providing a non-GUI solution for tasks involving automated text processing. It supports various integration options, including Cloudflare Workers AI, Cloudflare AutoRAG, and Ollama, making it versatile for different use cases.

Overview: Fooocus is an advanced image generation software that allows users to create high-quality images effortlessly for their brand using text input. Key features: - Text-to-Image Generation - Free and User-Friendly - Colab Integration Tech stack: Python, Flask, and other open-source libraries. Why it matters: Fooocus provides a powerful tool for generating images based on text, making it accessible and user-friendly, and it showcases the potential of AI in creating distinct and visually appealing brand representations. Repository: codewithmuh/insatgram-ai-model Primary language: Python GitHub stars: 21 GitHub description: Create high-quality images effortlessly for your brand using Fooocus, an advanced image generation software. README excerpt: # Fooocus: AI-Powered Image Generation  Create high-quality images effortlessly for your brand using Fooocus, an advanced image generation software.

Overview: Fooocus is an advanced image generation software that allows users to create high-quality images effortlessly for their brand using text input. Key features: - Text-to-Image Generation - Free and User-Friendly - Colab Integration Tech stack: Python, TensorFlow, Flask Why it matters: Fooocus provides a powerful tool for generating images from text, making it accessible and user-friendly for various applications, including branding and marketing. Repository: codewithmuh/instagram-ai-model Primary language: Python GitHub stars: 21 GitHub description: Create high-quality images effortlessly for your brand using Fooocus, an advanced image generation software. README excerpt: # Fooocus: AI-Powered Image Generation  Create high-quality images effortlessly for your brand using Fooocus, an advanced image generation software. ## Features - **Text-to-Image Generation:** Generate high-quality images from text input. - **Free and User-Friendly:** Easily create images for various purposes. - **Colab Integration:** Access Fooocus via Google Colab for seamless image creation. ## Get Started Explore the power of Fooocus with the following resources: | Colab | Information | | --- | --- | | [![Open In Colab](https://colab.re

Overview: This repository provides Docker images and scripts for training finetuned or completely personal Kaldi speech models. Primarily used with kaldi-active-grammar. Key features: - Docker image and scripts for training Kaldi models - Supports finetuning and personal model training - Compatible with GPU for faster processing - Includes base model for finetuning - Converts KaldiAG TSV to SCP format for training data - Ensures 16-bit Signed Integer PCM 1-channel 16kHz WAV audio files Tech stack: Shell Why it matters: Ideal for developers and researchers working on speech recognition and language modeling projects, offering a streamlined solution for model training and testing.

### Overview: Capit is a tool designed to help manage and control the cost of large language model (LLM) agents, ensuring that the spending is kept within budget limits. Whether you're managing an LLM agent for a single agent or an entire fleet, Capit provides a simple and effective way to manage your LLM costs. ### Key Features: - **Budget Control**: Capit allows you to set a budget for each agent, ensuring that the agent's costs stay within the specified limits. - **Local Storage**: All data is stored locally, ensuring that no sensitive information is sent to external servers. - **Custom Agents**: Capit supports a wide range of agents, including Claude Code, Cursor, Windsurf, Hermes, OpenClaw, Opencode, and Pi. You can easily add custom agents by following the instructions in the `agents/README.md` file. - **Platform Integration**: Capit integrates with popular platforms like OpenRouter, AI Hub Mix, and Concentrate, making it easy to manage multiple agents on the same platform. ### Tech Stack: - Python ### Why It Matters: By using Capit, you can easily manage and control the cost of your LLM agents, ensuring that you stay within budget limits. This tool simplifies the process of managing multiple agents and provides a simple way to add custom agents, making it an essential tool for anyone managing large language model costs. ### Repository: - **Repository**: day50-dev/limit-model-spending - **Stars**: 21 - **Description**: Stop runaway LLM agent costs

Overview: Deepinfra Node is a TypeScript wrapper for the DeepInfra Inference API, providing a seamless integration for developers. Key features: - Access to the DeepInfra Inference API - TypeScript support for easy integration and maintenance - Integration with various development frameworks and tools Tech stack: TypeScript Why it matters: Deepinfra Node simplifies the process of integrating DeepInfra's AI capabilities into existing projects, making it easier for developers to leverage advanced AI solutions without the need for extensive coding.

Overview: This repository documents a 5-day course on using Google's Generative AI tools and APIs, focusing on prompt engineering, embeddings, agents, domain-specific LLMs, and MLOps for advanced AI applications. Key features: - Comprehensive course on Generative AI using Google's tools and APIs. - Prerequisites: Kaggle account, Google API key, basic Python knowledge, and internet connectivity. - Technical stack: Jupyter Notebook environment. - Why it matters: Provides practical skills in using cutting-edge AI tools for advanced AI applications. Tech stack: Jupyter Notebook. Why it matters: This repository serves as a comprehensive guide for developers looking to enhance their skills in using Google's Generative AI tools and APIs for advanced AI applications.

### Overview: This project is a Python implementation of inference for Llama/Llama2/Llama3 models using NumPy, focusing on the inference mode. It aims to provide a quick and easy way to run these models without the need for C++ or Python bindings. ### Key features: - Inference Llama/Llama2/Llama3 modes in NumPy - Supports conversion of model binaries to NumPy format - Provides a simple CLI interface for running inference - Token count and inference time metrics ### Tech stack: Python ### Why it matters: This project is a valuable resource for developers looking to experiment with Llama/Llama2/Llama3 models without the complexity of C++ or Python bindings. It simplifies the process of running inference, making it accessible for both research and practical applications.

### Overview GRIFFIN is a novel framework designed to address token misalignment in speculative decoding. This repository provides the implementation of GRIFFIN, including its token-alignable training strategy and token-alignable draft model. - GRIFFIN is: - **4.2x** faster than vanilla decoding. - **1.3x** faster than EAGLE-2. ### Key Features - Token Alignable Training Strategy - Token Alignable Draft Model ### Tech Stack - Python ### Why It Matters GRIFFIN accelerates speculative decoding by aligning tokens, significantly improving performance compared to vanilla decoding and EAGLE-2. ### Repository - Repository: hsj576/GRIFFIN - GitHub Stars: 21 - GitHub Description: Official Implementation of "GRIFFIN: Effective Token Alignment for Faster Speculative Decoding"[NeurIPS 2025] ### Primary Language - Python ### GitHub Description - Official Implementation of "GRIFFIN: Effective Token Alignment for Faster Speculative Decoding"[NeurIPS 2025] ### README Excerpt - Figures: - Benchmark: Speed up ratios of GRIFFIN when temperature = 0 and 1. - Acceleration: A GIF demonstrating the acceleration of GRIFFIN for LLaMA3-8B on a 4090 GPU. ### Update - **2025.5.12**: Support for Qwen-2.

Overview: SHIELD is a training-free framework designed to suppress hallucinations in visual language models (LVLMs) by addressing bias and vulnerability. It achieves consistent improvements across multiple LVLM families and benchmarks without requiring fine-tuning. Key features: - Suppresses hallucinations in LVLMs - Fixes statistical bias and inherent bias - Addresses vulnerability without fine-tuning - Consistently outperforms existing methods across 3 LVLM families and 6 benchmarks Tech stack: Python, PyTorch Why it matters: SHIELD provides a practical solution for addressing hallucinations in LVLMs, ensuring that models maintain their capabilities while mitigating potential biases and vulnerabilities, making them more reliable and trustworthy in applications. Repository: hukcc/SHIELD GitHub stars: 21 GitHub description: [ICLR 2026] SHIELD: Suppressing Hallucinations In LVLM Encoders via Bias and Vulnerability Defense

Overview: This project is a testbed for exploring multimodal retrieval augmented generation techniques using FiftyOne, LlamaIndex, and Milvus. Key features: - Multimodal RAG plugin for FiftyOne - Integration with LlamaIndex and Milvus - Chunks and indexes multimodal data - Searches multimodal data together - Experiment with different multimodal RAG techniques - Text embedding model choice - Image representation method - Retrieval results count - Multimodal model for generation Tech stack: Python Why it matters: This project provides a comprehensive solution for working with multimodal data, experimenting with various multimodal RAG techniques, and finding the best approach for specific use cases.

### Overview: A better version of KU's course catalogue, designed and developed by Joshniemela, using Rust and Docker for efficient deployment and management. ### Key Features: - **Rust Backend**: Built with Clojure, responsible for scraping course pages and statistics from STADS, and serving the frontend. - **Vector Store**: Used for semantic searches, providing lower latency compared to trigrams or full-text. - **Rust Parser**: Parses scraped course pages into a database format for efficient searching and frontend rendering. - **Svelte/Typescript Frontend**: Highly responsive SPA displaying courses as cards, with detailed views available on click. ### Tech Stack: Rust ### Why it Matters: This project aims to enhance the accessibility and functionality of KU's course catalogue, providing a more efficient and user-friendly interface.

Overview: CXR-IRGen is an integrated vision and language model designed for generating clinically accurate CXR image-reports pairs, enhancing the performance of deep learning models for various tasks. Key features: - Novel prompt design for the vision module combining text and image embeddings. - A new CXR report generation model leveraging large language models and self-supervised learning. - Improved general quality (FID) and clinical efficacy (AUROC) of generated images. - High-level image-text alignment with a superior $F_1$ score. Tech stack: Python Why it matters: This model addresses the need for high-quality, clinically accurate CXR image-reports, significantly improving the performance of deep learning models in various applications.