Projects

Overview: This project presents a method for language-driven affordance-pose detection in 3D point clouds, addressing the challenge of detecting open-vocabulary affordances and generating specific poses for them. Key features: - 3DAPNet: A new method for joint learning of affordance and pose detection. - Language-conditioned: The model can detect affordances based on natural language inputs. - 3D point cloud: The input is a 3D point cloud of an object. - Pose generation: The model generates 6-DoF poses that facilitate the detected affordances. Tech stack: Python. Why it matters: This project addresses a significant challenge in computer vision and robotics, enabling more intuitive and efficient interaction with 3D objects through language-based instructions.

Overview: KubePilot SDK is a Python toolkit designed to simplify the deployment of AI workloads on Kubernetes, making it easier for practitioners, MLOps engineers, and platform architects to orchestrate AI workloads without the complexity of YAML manifests. Key features: - Declarative Task Graphs - Multi-Runtime Backends - Accelerator Autodiscovery - Volume Whispering - Secrets Stewardship Tech stack: Python Why it matters: KubePilot SDK provides a more intuitive and efficient way to manage AI workloads on Kubernetes, streamlining the process and making it more accessible to non-Kubernetes experts. Repository: Himanshu-giri-01/k8s-ai-runtime Primary language: HTML GitHub stars: 53 GitHub description: Streamline AI Workloads on Kubernetes: Universal Python SDK Guide 2026 README excerpt:  [](https://Himanshu-giri-01.github.io/k8s-ai-runtime/)

Overview: This repository contains the code for a paper that proposes an efficient LLM-serving system using a proxy model-based sequence length prediction. The key idea is that a small proxy model can predict the verbosity of an LLM with low latency overhead, leading to a speculative shortest-job-first (SSJF) scheduler for improved performance. Key features: - SSJF scheduler for LLM serving - Proxy model-based sequence length predictor for execution time estimation - Lightweight BERT-based proxy model architecture Tech stack: Jupyter Notebook Why it matters: This approach can significantly reduce the non-determinism of generative models and the head-of-line blocking issue in FCFS scheduling, leading to improved performance and lower latency overhead compared to traditional methods. GitHub stars: 53 Repository: James-QiuHaoran/LLM-serving-with-proxy-models

### Overview: ImmunoStruct is a project that leverages multimodal deep learning to predict immunogenicity, a critical factor in vaccine design and cancer therapy. It integrates various data sources to enhance predictive accuracy and personalization. ### Key Features: - **Multimodal Data Integration**: Combines genomic, proteomic, and immunological data to provide a comprehensive view of the immune response. - **Deep Learning Models**: Utilizes advanced neural networks to analyze complex patterns and predict immunogenicity with high precision. - **Personalized Vaccine Design**: Tailors vaccine components to optimize immune response, enhancing efficacy and reducing side effects. ### Tech Stack: Python ### Why It Matters: ImmunoStruct offers a powerful tool for researchers and clinicians aiming to enhance the design and effectiveness of personalized vaccines and cancer treatments. By integrating diverse data sources and utilizing advanced machine learning techniques, it promises to revolutionize the field of immunotherapy and personalized medicine.

### Overview: tllm is an opinionated training library designed for teams and solo researchers who want to shape large language models through instruction tuning without wrestling with fragmented scripts, lost hyperparameters, or half-documented utilities. It aims to streamline the process by providing a studio-like workflow where every experiment is a canvas, every dataset is pigment, and every checkpoint is a signed piece. ### Key Features: - 🧩 Composable training recipes: Mix and match loss functions, schedulers, and adapters without forking the codebase. - 🗂️ Dataset curation suite: Deduplication, length bucketing, toxicity filtering, and prompt-response balancing. - 📊 Live training telemetry: Streamline metrics with drift detection and anomaly detection. ### Tech Stack: HTML ### Why It Matters: tllm is a fresh interpretation of the instruction-tuning idea, built around three convictions: 1. **Reproducibility is a feature, not a chore.** Every run produces a manifest that can be replayed byte-for-byte. 2. **Data quality outranks data quantity.** tllm ships with curators, deduplicators, and toxicity screens that run before a single gradient step. 3. **Human oversight stays central.** Evaluation dashboards are designed for reading, not just for logging. ### Repository: GitHub: Lucas5913/tllm-instruct-forge Stars: 53 Description: Open Source Instruction Tuning LLM Training Library 2026 License: MIT License ### Project

### Overview: TD-Pipe is a novel pipeline parallelism framework designed for high-throughput large language model (LLM) inference on GPU clusters with limited interconnect bandwidth. It addresses two major sources of pipeline bubbles: Prefill-Decode Imbalance and Inter-batch Imbalance, significantly improving throughput over conventional pipeline and tensor parallelism approaches. ### Key Features: - **Temporal Disaggregation**: Decouples prefill and decode phases to reduce switching overhead and phase interference. - **AI-based Greedy Prefill Strategy**: Dynamically determines optimal switching points between prefill and decode phases. - **Inter-batch Work Stealing**: Redistributes decode workloads on the fly to mitigate Inter-batch Imbalance. - **Global Communication Context**: Ensures efficient inter-stage data exchange while minimizing coordination overhead. ### Tech Stack: Python ### Why it Matters: TD-Pipe is designed to optimize the performance of high-throughput LLM inference on GPU clusters, ensuring efficient GPU utilization and improved throughput compared to conventional pipeline and tensor parallelism approaches.

### Overview: CURLoRA is a novel approach to fine-tuning large language models (LLMs) that leverages CUR matrix decomposition in the context of Low-Rank Adaptation (LoRA). Our method addresses two critical challenges in LLM fine-tuning: mitigating catastrophic forgetting during continual learning and reducing the number of trainable parameters. We propose a unique modification to the CUR decomposition process to enable a more efficient and stable way to adapt LLMs to new tasks without compromising any existing knowledge. We demonstrate through experiments on multiple datasets that CURLoRA outperforms standard LoRA in mitigating catastrophic forgetting. It maintains model stability and performance across tasks while significantly reducing the number of trainable parameters. Our results show that CURLoRA achieves superior accuracy and perplexity scores compared to LoRA, particularly in scenarios with limited data. ### Key features: - **CURLoRA**: A novel method for fine-tuning large language models that addresses the challenges of catastrophic forgetting and reduces the number of trainable parameters. - **CUR Decomposition**: Utilizes CUR matrix decomposition to adapt LLMs more efficiently. - **Low-Rank Adaptation (LoRA)**: Implements a low-rank adaptation technique to stabilize and improve model performance. ### Tech stack: - **Python**: The primary programming language used for implementing the CURLoRA approach. ### Why it matters: CURLoRA provides a more stable and efficient way to fine-tune LLMs, particularly in scenarios with limited data, by mitigating catastrophic forgetting and reducing the number of trainable parameters. This results in superior performance and better

Overview: Dive into the world of fine-tuning Transformers, a hands-on laboratory where you can experiment with different fine-tuning paradigms, multilingual support, and more. Key features: - Multiple fine-tuning paradigms: full fine-tuning, parameter-efficient tuning (adapters, LoRA-style), instruction tuning, and light prompt-based adaptation. - Curated notebooks: self-contained, heavily commented, and designed to be read like a short essay. - Multilingual support: datasets and experiments that span languages, allowing for knowledge transfer across linguistic boundaries. - Responsive experiment UI: adapts to desktop, tablet, and handheld screens. - Model-agnostic design: works with encoder-only, decoder-only, and encoder-decoder architectures in the Hugging Face ecosystem. - Reproducible experiments: seeds, configs, and environment notes recorded alongside every result. - Safety and ethics notes: discusses bias, dataset provenance, and responsible deployment. - Metrics dashboard ideas: guidance on which metrics matter for which tasks, and why accuracy alone is rarely sufficient. Tech stack: HTML Why it matters: This repository provides a comprehensive, hands-on approach to understanding fine-tuning Transformers, making it a valuable resource for learners, researchers, hobbyists, and engineers looking to deepen their understanding of modern machine learning. Repository: STEAM-DROPS/finetune-lab-notes GitHub stars: 53

Overview: Transcribe.js is a browser-only JavaScript library that transcribes speech to text using a wasm build based on whisper.cpp, enabling real-time transcription in the browser. Key features: - Transcribes speech files and streams - Supports both FileTranscriber and StreamTranscriber - Built with SIMD (Single Instruction, Multiple Data) for better performance - Compatible with Node.js (not supported) - Available in various JavaScript environments including Svelte, Vue, Angular, and others Tech stack: JavaScript, WebAssembly (Wasm), SIMD Why it matters: Transcribe.js simplifies speech-to-text transcription, making it accessible and performant in the browser, suitable for real-time applications and web-based content delivery.

Overview: Seedance 1.5 Pro API (`seedance-1.5-pro`) is an OpenAI-compatible endpoint for accessing model-based video generation services. It offers a base URL `https://api.apimart.ai/v1` and a single key for both the Seedance 1.5 Pro model and 300+ other models, providing smooth motion, 4K quality, and precise control. Key features: - One key for all models, including Seedance 1.5 Pro - Minimal cost with immediate payments - Summaries returned in task completion - Default asynchronous processing Tech stack: Python Why it matters: Seedance 1.5 Pro API simplifies access to high-quality video generation services, making it easier and more cost-effective for developers to integrate AI into their projects. Repository: a1dbdgioyt/seedance-1-5-pro-api-ru Primary language: Python GitHub stars: 52

Overview: Grok Imagine Video 1.5 API is a powerful tool for generating high-quality videos from text or images, compatible with OpenAI's API endpoint. It offers a single endpoint for both text-to-video and image-to-video tasks, with flexible pricing and easy integration. Key features: - Text-to-video and image-to-video capabilities - Compatible with OpenAI's API endpoint - Supports multiple resolutions and aspect ratios - Flexible pricing model with per-second billing - Assynchronous processing with task tracking Tech stack: Python Why it matters: This API streamlines the process of creating engaging videos from text descriptions or images, making it ideal for developers looking to enhance their applications with dynamic video content.

Overview: This project is a comprehensive API for accessing MiniMax H3, an AI model designed for video generation. It provides a key for both text-to-video and image-to-video conversions, allowing for professional-grade performance and flexible pricing. Key features: - Access to MiniMax H3 (`MiniMax-H3-Context-IR`) via API - Pricing based on usage, with a minimum of $1 per minute - Asynchronous processing, with task IDs for tracking progress Tech stack: Python Why it matters: This API offers a cost-effective and efficient way to integrate AI video generation into projects, with a key that can be used across multiple resolutions and models, making it versatile and powerful for various applications. Repository: abovqa/minimax-h3-context-ir-api-de Primary language: Python GitHub stars: 31 GitHub description: MiniMax H3 API (`MiniMax-H3-Context-IR`): model id, pricing, parameters and copy-paste examples for the OpenAI-compatible APIMart endpoint.

Overview: The Kling 3.0 Motion Control API, available on APIMart, allows developers to transfer motion from video to images with per-second billing, saving up to 20% as a member. Key features: - Supports 300+ models across video, image, and text-to-video/image-to-video - Pricing is per generated second, with a minimum of $1 and pay-as-you-go - Returns the exact cost upon completion of each task - Asynchronous by design, allowing for efficient batch processing Tech stack: Python Why it matters: This API provides a cost-effective solution for developers looking to integrate motion transfer functionality into their applications, offering a key feature that simplifies the process of converting video to images while keeping costs manageable.

### Overview: Veo 3.1 Fast (`veo3.1-fast`) is a low-latency image and video generation API service that supports both text-to-video and image-to-video conversions. It offers a single API key for a catalog of 300+ models, with pricing per generated second, making it ideal for teams looking to integrate Google Veo functionality into their projects without the need for a monthly subscription. ### Key Features: - **Single Key, Entire Catalog**: Access to a catalog of 300+ models via a single API key. - **$1 Minimum, Pay As You Go**: No subscription required; pay as you go, with no free quota to burn through. - **Charge Backs**: The cost is reported in the response, ensuring accurate billing. - **Async by Design**: Submit a task, retrieve the `task_id`, and poll for progress. ### Tech Stack: Python ### Why It Matters: - **One Key, Entire Catalog**: The same API key reaches all Veo 3.1 Fast models, simplifying integration. - **$1 Minimum, Pay As You Go**: No subscription required, allowing for flexible pricing based on usage. - **Charge Backs**: Real-time billing ensures accurate tracking of costs. - **Async by Design**: Simple integration with real-time progress updates. ### Repository: - **Repository**: ahxjt/veo-3-1-fast-api - **Stars**: 36 - **Description**: Veo 3.1 Fast (`veo3.1-fast`

Overview: A demo project for speech recognition or wake word detection using the Maixduino framework and PlatfomIO on a K210 MCU on a Sipeed Maix dev board. Key features: - Speech Recognition - Wake Word Detection Tech stack: Maixduino, PlatfomIO Why it matters: This project provides a practical example of speech recognition and wake word detection using the Maixduino framework, demonstrating its capabilities for real-world applications.

### Overview: ConsciousDB is a graph-database-agnostic Agentic GraphRAG framework that seamlessly integrates with various graph databases, offering a complete 4-phase pipeline and continuous evaluation. It eliminates the need for hard-wiring GraphRAG logic to a single database, making it a versatile and efficient intelligence layer. ### Key Features: - **Graph Database Independence**: Completely decouples the AI decision layer from the storage layer, allowing seamless integration with Neo4j, Memgraph, Apache AGE, and Kùzu. - **4-Phase Pipeline**: Identical 4-phase pipeline (Ingestion → Pre-Retrieval → Traversal → Post-Retrieval) across different databases, ensuring consistent evaluation and evaluation runs. - **Continuous Evaluation**: Active edge and relationship scoring, verification, and pruning throughout the lifecycle of data. - **System One Models**: Swappable models (local Laya / cloud Jev) using three deterministic mathematical primitives (Score, Noul, Choice) to handle semantic chunking, intent routing, and A* traversal. - **Customizable Evaluation**: Allows switching the AI decision model and graph database independently with a single environment variable each. ### Tech Stack: Python ### Why It Matters: By eliminating the need for slow generative LLM calls and ensuring edge verification, ConsciousDB significantly reduces latency and context window issues. It abstracts the database layer, making it a versatile and efficient intelligence layer that can be seamlessly integrated with various graph databases, ensuring consistent evaluation and evaluation runs across different databases.

### Overview: ConsciousDB is a graph-database-agnostic Agentic GraphRAG framework, designed to enhance any graph database with intelligence. This framework allows for a complete 4-phase pipeline, continuous evaluation, and custom A* traversal, making it a versatile and efficient intelligence layer. ### Key Features: - **Continuous Evaluation**: Every edge and relationship is continuously evaluated across the entire lifecycle of the data. - **4-Phase Pipeline**: Ingestion → Pre-Retrieval → Traversal → Post-Retrieval. - **Swappable Models**: Decouples the AI decision layer from the storage layer, allowing the same pipeline to run across different graph databases (Neo4j, Memgraph, Apache AGE, Kùzu). - **Edge Verification**: Edge validity is evaluated to purge hallucinations and disambiguate entities. - **Semantic Routing**: Custom A* traversal is used for intelligent semantic routing, with early termination and context gating. - **Context Reranking**: Reranks context to maximize token density and resolves contradictions for accuracy. - **Citation Verification**: Cites are verified before LLM synthesis to ensure the final output is accurate. ### Tech Stack: - Python - PyTorch - Apache License 2.0 ### Why It Matters: ConsciousDB provides a production-ready intelligence layer that enhances any graph database, making it fully agentic. It eliminates the need for hard-wired GraphRAG logic, replacing it with a complete 4-phase pipeline that runs identically across different graph databases. The framework continuously evaluates every edge and

### Overview: Inference Snaps are a lightweight, portable, and hardware-optimized solution for deploying and running AI models on various devices. These snaps bundle pre-optimized model weights and runtimes, ensuring consistent performance across different hardware types. With a single command, you can install and run AI models locally, making deployment and experimentation more accessible and efficient. ### Key Features: - **Plug-and-Play AI Models**: Each snap includes a range of optimized model weights and runtimes. - **Hardware-Aware**: Automatically selects and deploys silicon-optimized runtimes and model weights for your machine. - **API-First Design**: Exposes local standard APIs for seamless integration. - **Offline-Ready**: Once installed, the model runs entirely on your machine. - **Reproducible & Isolated**: Snaps guarantee consistent environments across supported Linux distributions. ### Tech Stack: Inference Snaps are built on the following technologies: - Ubuntu-based driver installation - Pre-optimized model weights and runtimes - Local-first design - Consistent environments across supported Linux distributions ### Why It Matters: - **Ease of Use**: Simplifies the deployment process by providing a single command for installation and running AI models. - **Performance Optimization**: Automatically selects and deploys the most efficient runtimes for your hardware, ensuring optimal performance. - **Portability**: Ensures that AI models can be run on any compatible device without dependency hell or optimization nightmares. - **Scalability**: Designed to be easily scalable and can be integrated into various applications and workflows. ### Available Inference Snaps:

Overview: This repository is a curated list of End-to-End speech recognition resources, including papers, codes, and other materials. It covers a wide range of recent advancements in the field, from 2014 to 2018. Key features: - Papers: A collection of highly recommended papers on end-to-end speech recognition. - Papers (2019-2018): Detailed lists of papers from 2019 to 2018. - Papers (2017-2016): Comprehensive lists of papers from 2017 to 2016. - Papers (2015-2014): Detailed lists of papers from 2015 to 2014. Tech stack: Python, TensorFlow, PyTorch, and various deep learning frameworks. Why it matters: This repository serves as a valuable resource for researchers, developers, and enthusiasts interested in improving speech recognition technology. It provides a comprehensive overview of the latest advancements and techniques in the field, making it an essential tool for anyone looking to stay updated on the latest developments in end-to-end speech recognition.

### Overview: ShiftDDPMs is an official PyTorch implementation of the paper "Shiftddpms: Exploring conditional diffusion models by shifting diffusion trajectories" (AAAI 2023). This repository aims to provide a comprehensive and accessible way for developers to experiment with and understand the capabilities of ShiftDDPMs. ### Key features: - **Conditional Diffusion Models**: ShiftDDPMs are designed to be conditional diffusion models, allowing for the manipulation of specific conditions within the diffusion process. - **Shifting Diffusion Trajectories**: By shifting the diffusion trajectory, ShiftDDPMs can explore different aspects of the diffusion process, such as the impact of different conditions on the final output. - **Official Implementation**: The repository provides an official PyTorch implementation, ensuring that developers can easily integrate and experiment with the model. ### Tech stack: Python ### Why it matters: ShiftDDPMs offer a unique approach to diffusion models by allowing for the manipulation of conditions within the diffusion process. This can lead to more flexible and adaptable models, making them valuable for various applications in image generation and other diffusion-based tasks.