Projects

### Overview Directional Textual Inversion (DTI) addresses the issue of embedding norm inflation in pre-norm Transformers, which is a common problem in Textual Inversion (TI). DTI decouples magnitude and direction, fixes embedding magnitude to an in-distribution scale, and optimizes only the direction on the unit hypersphere using Riemannian SGD. This approach leads to superior text fidelity and subject preservation on models like Stable Diffusion XL (SDXL), SANA, and Wan2.1-T2V-1.3B. ### Key Features 1. **Decoupling Magnitude and Direction**: DTI separates the magnitude and direction of the learned tokens, ensuring that the magnitude remains within the expected range. 2. **Spherical Optimization**: It optimizes only the direction on the unit hypersphere, avoiding the drift to out-of-distribution magnitudes. 3. **vMF Prior**: A von Mises-Fisher prior is used to maintain semantic coherence and improve the quality of the generated images. ### Tech Stack - **GitHub Repository**: `kunheek/dti` - **Primary Language**: Jupyter Notebook - **GitHub Stars**: 10 ### Why It Matters DTI improves the fidelity and subject preservation of text-to-image generation models, making them more effective and reliable for applications like personalized text-to-image conversion. By addressing the limitations of standard TI, DTI offers a significant enhancement in the quality of the generated images and text. ### Repository - **Repository**: `kunheek/dti` - **

### Overview: Domicile is a private AI stack that provides a vector database, RAG, and local language model (LLM) solution entirely running in the browser. It ensures zero egress, zero third-party processors, and zero cloud costs, making it ideal for on-prem privacy and data custody. ### Key Features: - **Vector Database**: Persistent client-side storage with a pure-TypeScript HNSW graph index, holding up to 100K documents with cosine, euclidean, or dot metrics. - **Local Embeddings**: Transformers.js with WebGPU acceleration and a WASM fallback, caching model weights on-device for fast repeat queries. - **Dual LLM Runtime**: WebLLM for GPU-fast inference and Wllama for CPU portability, automatic fallback for generation. - **RAG with Citations**: Answers grounded in source documents, with citations linking every claim back to the document that grounded it, ensuring auditable and non-black-box results. - **MCP Integration**: Exposes the custody layer as Model Context Protocol tools, allowing integration into Claude Desktop and team-built agent stacks. - **Resident by Design**: Data domiciled on the device, ensuring no cloud egress, no third-party processors, and no residency drift. ### Tech Stack: TypeScript ### Why It Matters: - **Private**: Data never leaves the device, ensuring zero egress and no third-party processors. - **Fast**: WebGPU acceleration with WASM SIMD fallback for fast queries. - **Free**: Zero cloud costs, no API keys, no per-seat fees

### Overview: This repository contains samples for fine-tuning HuggingFace models with Azure Machine Learning (AzureML). The samples are designed to be easily adapted for various NLP tasks and can be fine-tuned for other applications as well. ### Key features: - AzureML v2 CLI examples for fine-tuning HuggingFace models - Quickstart ARM Templates for fine-tuning HuggingFace models - Fine-tuned HuggingFace models and results available at [huggingface.co/linydub](https://huggingface.co/linydub) ### Tech stack: - AzureML v2 CLI - ARM Templates - HuggingFace models ### Why it matters: These samples provide a practical way to fine-tune HuggingFace models using AzureML, making it easier to adapt the models for different NLP tasks or product scenarios. The samples also include integrations with MLflow and CodeCarbon, which help in benchmarking and carbon accounting, respectively. ### Repository: The repository is hosted on GitHub and has been starred by 10 developers. The project is currently in the preview stage and aims to add more fine-tuned models and benchmark results in the future.

Overview: This is the official repository of "HoliSDiP: Image Super-Resolution via Holistic Semantics and Diffusion Prior". The project aims to address the issues of pre-trained text-to-image diffusion models in Real-ISR, providing improved image quality through semantic guidance. Key features: - Semantic segmentation as text prompts - Dense semantic guidance through segmentation masks - Segmentation-CLIP Map for enhanced spatial control Tech stack: Python Why it matters: By leveraging semantic labels for text prompts and incorporating segmentation masks, HoliSDiP significantly enhances the quality of image super-resolution results, especially in terms of reduced prompt noise and improved spatial control.

**Overview:** This repository contains the ATC '25 Artifact, which is a resource multiplexing approach for tuning and serving large language models. The project aims to optimize the performance of LLMStation, a customized version of PyTorch and vLLM, by leveraging multiple GPUs. **Key features:** - Resource multiplexing approach for LLMStation - Customized PyTorch 2.4.0 and vLLM 0.6.3 - Baselines using vanilla PyTorch 2.4.0 and vLLM 0.6.3 - Installation scripts for Ubuntu 22.04, Conda, CUDA 12.6, cmake 3.26, gcc-13.3.0, and g++-13.3.0 - Hardware requirements: Running these scripts requires 2 GPUs, each with 24 GB HBM **Tech stack:** Python **Why it matters:** This project provides a comprehensive solution for optimizing the performance of large language models, specifically LLMStation, by leveraging resource multiplexing techniques. The detailed installation scripts and hardware requirements ensure that users can effectively run and benchmark the project's performance on their systems. **Repository:** llm-db/llmstation **Primary language:** Python **GitHub stars:** 10

Overview: This repository contains hands-on implementations, research notebooks, and production recipes for fine-tuning Large Language Models (LLMs) and Vision-Language Models (VLMs). It includes examples for supervised fine-tuning, reinforcement learning, preference alignment, dense retrieval, and autonomous multi-agent systems. Key features: - Domain QA factual injection via Wikipedia scraping & QLoRA - Financial sentiment classification with Chain-of-Thought reasoning - Cardiology SFT with Google's Tunix library & JAX/XLA GPU acceleration - Clinical dialogue adaptation with token-level perplexity validation - Multimodal VLM adaptation for handwritten formula LaTeX OCR - Direct Preference Optimization (DPO) pairwise alignment - Mathematical reasoning RL via Group Relative Policy Optimization - Structured tool use and ChatML JSON function calling on Gemma 3 270M - Dense embedding model adaptation with synthetic queries & MNRL loss - Autonomous multi-agent coding system with dynamic LoRA swapping (Unsloth) and (TRL & PEFT) Tech stack: Jupyter Notebook Why it matters: This repository provides a comprehensive set of examples and recipes for fine-tuning LLMs and VLMs, making it valuable for developers looking to enhance their skills in this area.

### Overview: AskMyDocs is an open-source AI hub and intelligent agentic platform designed for enterprise knowledge management. It offers a self-hostable solution that integrates hybrid retrieval-augmented generation, a typed canonical knowledge graph, a streaming chat interface, and an admin operations cockpit. ### Key features: - **Enterprise RAG + Knowledge Graph + Agentic Tool Use**: Self-hostable, MIT-licensed. - **Hybrid Retrieval-Augmented Generation (pgvector + FTS + reranker)**: Enhances AI-powered document retrieval. - **Typed Canonical Knowledge Graph**: Promotes human-gated knowledge through a typed knowledge graph. - **Streaming Chat Surface (Vercel AI SDK)**: Provides a real-time chat interface for collaboration. - **Full Admin Operations Cockpit**: Offers comprehensive management tools for enterprise users. ### Tech stack: Laravel ### Why it matters: AskMyDocs is an open-source, on-prem alternative to Glean, Notion AI, and ChatGPT Enterprise. It eliminates per-seat lock-in, making it accessible and scalable for enterprise-level knowledge management.

### Overview: 本项目是智能路由、任务拆解和链路工程的早期探索。通过使用Qwen Agent、PostgreSQL和MySQL数据库，该项目旨在展示如何通过自动化和优化来提高系统的效率和性能。 ### Key features: - **智能路由**：自动优化任务流程，提高系统响应速度。 - **查询优化**：通过SQL查询优化，提高数据库查询效率。 - **链路工程**：构建和优化数据流，确保信息传递的高效和准确。 ### Tech stack: - 前端: Gradio - 后端: Qwen Agent - 数据库: PostgreSQL/MySQL - MCP: server-postgres/mysql-mcp-server ### Why it matters: 通过这些技术的结合，项目旨在展示如何通过自动化和优化来提高系统的效率和性能，从而在实际应用中减少人工干预，提高响应速度和数据处理质量。 ### Repository: luochang212/agent-project ### Primary language: Jupyter Notebook ### GitHub stars: 10 ### GitHub description: Agent 早期探索：智能路由、任务拆解和链路工程

### Overview: ThemisDB is an actively developed database project that combines multiple data models and AI/LLM capabilities. It supports relational, document, graph, vector, geospatial, and time-series workloads, offering AQL, full-text, and vector search, as well as ACID transactions and hybrid retrieval workflows. The project aims to integrate AI/LLM inference and retrieval-augmented generation integrations, providing a robust foundation for AI/LLM integration in databases. ### Key features: - **Multi-model data**: Supports relational, document, graph, vector, geospatial, and time-series workloads. - **Query and retrieval**: Offers AQL, full-text, and vector search, and hybrid retrieval. - **Transactions and distribution**: Provides ACID transactions, replication, sharding, and recovery workflows. - **AI/LLM**: Integrates inference and retrieval-augmented generation capabilities. - **Operations**: Includes metrics, tracing, security controls, and deployment tooling. ### Tech stack: C++ ### Why it matters: ThemisDB is designed to be a high-performance database system that integrates multiple data models and AI/LLM capabilities, making it suitable for applications requiring both traditional data management and advanced AI/LLM integration. ### Repository: makr-code/ThemisDB ### GitHub stars: 9 ### GitHub description: Themis Database System - High-performance C++ hybrid-database (graph-vector-relational-file) with AQL support and MVCC. Support us on kickstarter https://www.kickstarter.com/projects/mak

Overview: This repository provides a comprehensive guide on implementing efficient and lightweight model serving strategies, covering basic PyTorch implementations to ONNX Runtime in both Python and Rust. Key features: - PyTorch-based model serving - ONNX Runtime-based model serving - Comparison of performance metrics Tech stack: Python Why it matters: This project illustrates how different serving strategies impact model service load capacity and evaluates secondary metrics such as image size and container startup time. Repository: martynas-subonis/model-serving GitHub stars: 10

Overview: This project aims to build a scalable infrastructure for hosting machine learning operations, specifically focusing on fine-tuning chatbots for university courses. The system will be tested at [this link](https://boysc-brioschi.streamlit.app/). Key features: - Fine-tuning chatbots on specific material of specific courses - Splitting traditional monolithic ML pipeline into three processes: Feature Pipeline, Training Pipeline, and Inference Pipeline - Feature Pipeline: Downloads new files, generates transcripts, uploads to Hopsworks, and generates Instruction Set for fine-tuning - Training Pipeline: Retrieves Instruction Set, sets up trainer instance, runs training, uploads result to HuggingFace - Inference Pipeline: Streamlit-based chatbot interface, downloads transcripts, computes embeddings, retrieves matching material, augments chatbot's response using RAG Tech stack: Jupyter Notebook Why it matters: This project is crucial for students preparing for university exams, providing a study buddy system for specific course material, and improving the scalability and efficiency of machine learning operations in educational settings.

Overview: This project is an agentic AI-powered helpdesk platform designed to autonomously resolve customer support tickets using a structured multi-agent architecture. Key features: - Multi-agent orchestration with explicit roles - Autonomous ticket classification and prioritization - Decision-driven resolution or escalation - Tool-augmented problem solving (KB, APIs, scripts) - Solution verification before ticket closure - Learning from past failures and escalations - End-to-end observability with metrics and logs Tech stack: Python Why it matters: This platform prioritizes correctness, cost efficiency, and controlled escalation over generic AI responses, making it suitable for reliable automation and transparent decision-making in customer support systems.

Overview: A personal AI knowledge assistant powered by RAG, semantic search, and document understanding for context-aware answers. Key features: - Smart Note Storage - AI Q&A (Hybrid RAG System) - Semantic Search with Embeddings - Document Understanding (PDF / DOCX / TXT) - AI Summarization - Multi-Question Handling - System Stats Tech stack: TypeScript Why it matters: This project combines personal knowledge retrieval with general AI intelligence, using semantic search and a hybrid system to provide context-aware answers, making it a valuable tool for anyone looking to enhance their knowledge and productivity.

### Overview: Estimate the environmental footprint of LLM generation requests with the EcoLogits API, part of the EcoLogits project maintained by the Code Carbon non-profit. Use the `/estimations` endpoint to estimate impacts based on model, token count, and latency. ### Key Features: - **Estimate Environmental Footprint**: Quickly estimate the environmental impacts of your LLM generation requests. - **Provider and Model Selection**: Specify the provider (e.g., "openai") and model (e.g., "gpt-4o-mini") to get accurate estimates. - **Electricity Mix Zone**: Choose the appropriate electricity mix zone for accurate calculations. - **Live Documentation**: Access detailed documentation and tutorials on the EcoLogits website. ### Tech Stack: Python ### Why It Matters: The EcoLogits API helps developers understand and mitigate the environmental impact of their AI models, promoting more sustainable practices in the tech industry.

Overview: This project is a deep learning model implementation using Keras, TensorFlow, and NLTK for image and text classification tasks. Key features: - Image and text classification models using CNN, LSTM, and BERT. - Utilizes Keras, TensorFlow, OpenCV, NLTK, and Scikit-learn. - Dataset includes images and captions for both training and testing. Tech stack: Keras, TensorFlow, OpenCV, NLTK, Scikit-learn, Python 3.7.12, Tensorflow 2.7.0, OpenCV 4.1.2.30, NLTK 3.2.5, Tensorflow-hub 0.12.0. Why it matters: This project demonstrates the application of deep learning models for image and text classification tasks, showcasing the integration of various libraries and techniques in practical applications.

### Overview: Enhancing Automatic Speech Recognition (ASR) performance through semantic audio filtering. ### Key features: - **Synthetic Audio Generation**: Creating high-quality, contextually relevant synthetic audio for ASR fine-tuning. - **Model Fine-Tuning**: Fine-tuning pre-trained models on synthetic data to improve ASR performance. - **Performance Metrics**: Measuring improvements in ASR accuracy and efficiency. ### Tech stack: Python ### Why it matters: Improving ASR performance by leveraging semantic audio filtering techniques, enhancing model accuracy and efficiency.

Overview: This project is a comprehensive medical chatbot application built with Flask, Gemini AI, Pinecone, and PostgreSQL. It features real-time chat, document-based RAG, source citations, conversation history, and advanced query processing. Key features: - User Authentication - Multi-Document Upload - Real-time Chat - Source Citations - Conversation History - User Feedback - Advanced RAG Tech stack: Flask, Gemini AI, Pinecone, PostgreSQL Why it matters: This application provides a secure and efficient way for users to interact with medical information, including document-based RAG for enhanced query processing and transparent citation attribution, making it valuable for healthcare professionals and patients looking for accurate and up-to-date medical advice.

Overview: RRI is a self-hosted platform designed for researchers, engineers, and teams to discover, organize, and interact with academic papers and repositories using AI-powered chat, semantic search, and automated trend analysis. Key features: - Semantic search across 10,000+ papers & repos with vector similarity - Chat with your documents — upload PDFs, DOCX, or ingest GitHub repos - Auto-collect from 11+ sources (ArXiv, GitHub, HuggingFace, OpenReview...) - Built-in CLI — run OSINT tasks directly from the terminal - Trending & analytics — Tech Radar, trend charts, community insights - Personal library — bookmarks, folders, weekly research digests Tech stack: Python, FastAPI, Next.js, Docker, PostgreSQL, Elasticsearch Why it matters: RRI helps researchers, engineers, and teams automate the discovery and analysis of academic papers, open-source repositories, and AI/ML community discussions in one place.

Overview: PromptCast is a browser-based, peer-to-peer 3D spell duel game that uses natural language prompts to generate structured spells. Players cast spells in the form of free-form prompts, which are interpreted by an in-browser LLM and validated against a Zod schema. Key features: - Peer-to-peer multiplayer over WebRTC - Next.js + React Three Fiber + Rapier engine - Zod schema validation for spell structure - `@sipphq/sipp` LLM client for natural language to structured spell conversion - Real-time spell generation and resolution Tech stack: TypeScript Why it matters: PromptCast demonstrates the potential of AI-driven, natural language-based game mechanics, showcasing how AI can be integrated into traditional 3D games to create more immersive and interactive experiences. Repository: noumena-labs/PromptCastGameExample GitHub stars: 11

Overview: Imagen 4.0 API (`imagen-4.0-apimart`) is a powerful tool for generating high-quality images from text prompts, compatible with OpenAI's API. It offers flexible pricing and easy integration, making it a valuable asset for developers and designers. Key features: - Supports multiple models and resolutions - Flexible pricing model with per-image cost - Easy integration via API key - Assynchronous processing for faster turnaround times Tech stack: Python Why it matters: - Provides access to cutting-edge AI image generation capabilities - Simplifies image creation from text prompts - Offers a cost-effective solution for image generation Repository: o96wzy7/imagen-4-api-pt Primary language: Python GitHub stars: 10 GitHub description: Imagen 4.0 API (`imagen-4.0-apimart`): model id, pricing, parameters and copy-paste examples for the OpenAI-compatible APIMart endpoint.