Projects

Overview: Taiwan-LLM Tutor: A platform designed for Taiwanese secondary education, leveraging large language models to provide educational support. Key features: - Access to a comprehensive dataset including GSAT social and question bank datasets. - Preprocessed and structured data for efficient training and testing. - Fine-tuning of Chinese-BERT and Taiwan-LLM models for multiple choice questions. - Support for both instruction tuning and multiple choice fine-tuning. - Pre-trained Taiwan-LLM-7B model available for download. Tech stack: Python, Conda, Pyenv, and various bash scripts for data preprocessing and model training. Why it matters: This platform offers educational support tailored to Taiwanese secondary education, utilizing advanced language models to enhance learning outcomes and provide tailored assistance. Repository: jwliao1209/TWLLM-Tutor Primary language: Python GitHub stars: 21

### Overview: The dsh-visual-plugin is a local-first visual media plugin designed for DeepSeek Harness, a platform for processing and analyzing video and images. It allows users to validate, transcode, and extract video keyframes locally, understand images and videos using the current DSH vision model, view image history, and play videos directly in the right panel. ### Key features: - **Native image understanding**: Uploaded images stay on DSH's native attachment and model path; the plugin does not configure or call a separate vision model. - **Copyable image history**: The right panel records the current DSH model's final answer beside each image thumbnail, with expandable history and one-click copy. - **Plugin-owned video upload**: Accepts MP4, M4V, MOV, AVI, MPG/MPEG, MKV, and WebM only when extension, signature, and FFprobe agree. - **Scene-aware video analysis**: Normalizes to H.264/yuv420p MP4, extracts keyframes with PySceneDetect, and sends ordered timestamped images to the current DSH vision model. - **Right-side panel**: Switch between image/video views, play normalized videos directly, and stage a selected video in the chat draft. - **Advanced video settings**: Tune upload size, storage quota, duration, output size, FPS, CRF, and keyframe count from the plugin settings card. ### Tech stack: - JavaScript ### Why it matters: This plugin simplifies the workflow for users by centralizing media processing and analysis capabilities within the D

### Overview: QDHF (Quality Diversity through Human Feedback) is a project that enhances Quality Diversity algorithms by inferring diversity metrics from human judgments of similarity. It surpasses state-of-the-art methods in automatic diversity discovery in robotics and reinforcement learning tasks, significantly improving performance in open-ended generative tasks. ### Key Features: - **Enhanced Diversity Discovery**: QDHF improves the diversity in text-to-image generation results compared to state-of-the-art methods. - **Automatic Metrics**: Infers diversity metrics from human judgments of similarity. - **State-of-the-Art Performance**: Outperforms existing methods in both robotics and reinforcement learning tasks. - **Open-Ended Tasks**: Successfully applied to open-ended generative tasks, demonstrating versatility. ### Tech Stack: - Python ### Why It Matters: QDHF is crucial for enhancing the quality and diversity of outputs in various applications, including robotics, reinforcement learning, and generative AI tasks. By leveraging human feedback, it significantly improves the performance and diversity of generated outputs, making it a valuable tool for researchers and practitioners in these fields. ### Repository: Repository: `ld-ing/qdhf` GitHub Stars: 21 GitHub Description: "Quality Diversity through Human Feedback: Towards Open-Ended Diversity-Driven Optimization (ICML 2024)" ### Updates: - **2024-06-24**: Gradio Demo on Hugging Face - **2024-03-14**: Tutorial on pyribs - **2023-12-13**:

### Overview: 在 RTX 5090 笔记本上，我们跑通了 15 条最新开源生成模型线、30 组实测配置（图像 / 图像编辑 / 视频 / 视频+音频 / 角色·动作迁移 / 音乐 / 图生 3D），并用受控实验证明 NVFP4 是这台机器上的最佳量化格式 —— 比 INT8 快 2.6 倍、小 27%。 ### Key features: - **15 条最新开源生成模型线**：包括图像、图像编辑、视频、视频+音频、角色·动作迁移、音乐、图生 3D 等。 - **30 组实测配置**：涵盖了图像、图像编辑、视频、视频+音频、角色·动作迁移、音乐、图生 3D 等多个场景。 - **受控实验**：通过受控实验证明 NVFP4 是这台机器上的最佳量化格式，比 INT8 快 2.6 倍、小 27%。 ### Tech stack: HTML ### Why it matters: 在 24 GB 显存的笔记本 GPU 上，我们跑通了 15 条最新开源生成模型线、30 组实测配置（图像 / 图像编辑 / 视频 / 视频+音频 / 角色·动作迁移 / 音乐 / 图

### Overview: CoCoA: Collaborative Chain-of-Agents for Parametric-Retrieved Knowledge Synergy is a project designed to enhance the integration and synergy of knowledge retrieval and collaboration among agents. This framework leverages the strengths of both internal and external knowledge to improve the performance of large language models (LLMs) by integrating and optimizing their decision-making processes. ### Key features: - **CoCoA-zero**: A multi-agent collaboration framework that integrates internal and external knowledge through knowledge induction and decision-making. - **Trajectory Training**: Combines the trajectories of different agents into long chains to train and enhance the integration ability of the LLM. - **Framework Details**: The top part is CoCoA-zero, a multi-agent collaboration framework. It integrates internal and external knowledge in a collaborative manner by first performing knowledge induction and then making decisions. The bottom part is the training strategy, which is based on CoCoA-zero and combines the trajectories of different agents into long chains to train and enhance the integration ability of the LLM. ### Tech stack: - Python ### Why it matters: CoCoA aims to improve the performance and integration of large language models by leveraging the synergy between internal and external knowledge through a collaborative approach. This project is particularly relevant in the field of artificial intelligence and natural language processing, where enhancing the ability to integrate and leverage knowledge is crucial for better decision-making and problem-solving.

Overview: Sovereign Shield is a production-grade AI defense framework built with deterministic filters, N-Model LLM cryptographic hash consensus, and self-learning adaptive rules. It is hardware-sealed with zero dependencies, ensuring zero false positives and zero false negatives. Key features: - Deterministic Filters: Blocks obvious attacks instantly. - AdaptiveShield: Self-learning keyword engine that learns from reported attacks and builds its own ruleset. - N-Model Consensus Veto: Canonical-JSON SHA-256 agreement between multiple independent LLMs on a structured verdict. - Integrity Seals: Hashes its security modules at import time and re-verifies seals on every action call. Tech stack: Python. Why it matters: Sovereign Shield is the defense system Mattijs Moens uses in his autonomous AI agent, ensuring zero false positives and false negatives in a live production environment. The architecture is fundamentally deterministic, and even the semantic intelligence of the LLMs is mathematically constrained into a deterministic state via cryptographic verification. GitHub stars: 21.

### Overview: Fine-tuning open-source large and small instruct/chat language models (LLMs & SLMs) from the Hugging Face Model Hub using public datasets. ### Key features: - Fine-tuning both small and large instruct/chat language models. - Primary example dataset: `medalpaca/medical_meadow_medical_flashcards`. - Available models: SmolLM (135M Instruct, medical_meadow_medical_flashcards, 10 epochs). - Supports quantization for fine-tuning large models on modest hardware. - Adaptability to other datasets through configuration file modifications. ### Tech stack: Jupyter Notebook ### Why it matters: Fine-tuning is crucial for teaching an LLM new skills or enhancing its understanding in specific domains. This project fine-tunes both small and large models, including SmolLM and Mistral, to improve their performance in medical and instruct/chat tasks.

Overview: This project is an implementation of the Deep Research Agent using the Neuron PHP framework, designed for creating comprehensive research reports using large language models. Key features: - Modular Workflow Architecture - Automated Research - Structured Reports - Performance Monitoring - Streaming Results Tech stack: Neuron PHP framework Why it matters: This project provides a structured approach to generating research reports using large language models, focusing on modularity, extensibility, and real-time results. It offers a standard toolkit for AI Agent development in PHP, reducing vendor lock-in and making it easy to switch between different LLMs, vector stores, and embedding providers.

Overview: Neighborly is an open-source vector database built with C# that efficiently handles large volumes of high-dimensional vector data, offering two deployment options: a gRPC API in a Docker container and a lightweight NuGet client library. Key features: - Disk-backed Storage: Efficiently handle large volumes of vector data with memory caching and disk storage for persistence. - High Performance: Optimized for fast read and write operations. - API (gRPC): Access Neighborly's functionality through a gRPC API hosted in a Docker container. - Client Library (NuGet): Integrate Neighborly as a minimal library into your projects, similar to SQLite. - Cross-Platform Compatibility: Leverage Neighborly on various platforms, including Windows, Xbox, iOS, macOS, Android, and Linux. - Advanced Search Algorithms: Utilize k-NN, ANN, range search, and cosine similarity search for efficient vector queries. - Unit Testing: Ensure reliability and stability with a comprehensive test suite. Tech stack: C# Why it matters: Neighborly is a versatile open-source vector database designed to efficiently store and retrieve high-dimensional vector data, making it suitable for a wide range of applications across various platforms, including Windows, Xbox, iOS, macOS, Android, and Linux.

### Overview: This repository contains the DeepSeek-V4-Flash-0731 recipe optimized for high-concurrency agentic serving on four RTX PRO 6000 Blackwell cards. It includes the public prebuilt image, serving configuration, and benchmarks to help developers optimize their own models. ### Key Features: - Public prebuilt image for DeepSeek-V4-Flash-0731 on 4x RTX PRO 6000 Blackwell (SM120) with 1M context. - Optimized for high-concurrency agentic serving with DP attention for better KV capacity. - Measured performance at ~2,460 out tok/s at 64 concurrent and 102-112 tok/s at batch size 1 on real prose. - Includes patches for DSPARK crash-loops, tool-call parser hardening, and opt-in logprobs. ### Tech Stack: Dockerfile-based reproducible SGLang recipe. ### Why it Matters: This recipe is optimized for high-concurrency agentic serving, making it suitable for many parallel sessions, tool calling, and long contexts. It includes the necessary patches and public image to help developers achieve similar performance on their own models. Repository: ombori/deepseek-v4-flash-0731-sglang-4x-rtx-pro-6000 GitHub stars: 21 GitHub description: This is the config we run in production: DeepSeek-V4-Flash-07

Overview: Click2Mask is an official PyTorch implementation for the paper "Click2Mask: Local Editing with Dynamic Mask Generation," presented at AAAI 2025. This project aims to enable free-form editing of images by dynamically generating masks during the diffusion process. Key features: - Dynamically generated masks for local editing - Real-time localization through clicks - No reliance on existing objects or detailed masks - Supports both PyTorch and Hugging Face models Tech stack: Python Why it matters: Click2Mask provides a novel approach to local editing that allows for free-form manipulation without the need for precise object placement or detailed masks, making it particularly useful for creative and artistic applications. Repository: omeregev/click2mask Primary language: Python GitHub stars: 21 GitHub description: Official PyTorch Implementation for "Click2Mask: Local Editing with Dynamic Mask Generation" Paper.

Overview: This project is a comprehensive guide on fine-tuning large language models, combining theoretical learning with practical micro-tuning. It is designed for beginners and can be starred/forked. Key features: - Fine-tuning of large language models - Practical micro-tuning experience - Self-learned and summarized understanding - Innovative applications Tech stack: Jupyter Notebook Why it matters: This project offers a practical and educational approach to fine-tuning large language models, making it useful for beginners and researchers alike. Repository: phbst/fine-tunning GitHub stars: 21 GitHub description: 大模型学习--从模型部署到模型微调，此项目是经过训练营学习后，结合训练营项目，自我理解消化总结，以及创新型应用。可star/fork

### Overview: LLMOps is a comprehensive collection of tools, frameworks, and resources designed to streamline the deployment, monitoring, and continuous improvement of large language models in production environments. It covers the entire lifecycle of LLM applications, from development and training to deployment, monitoring, and optimization. ### Key Features: - **Model Development**: Training, fine-tuning, and optimizing large language models. - **Inference & Serving**: High-performance inference engines, platforms, and frameworks for deploying models. - **Orchestration**: Application frameworks, agent frameworks, and workflow management tools for managing AI agents. - **Training & Fine-Tuning**: Training frameworks and fine-tuning tools for model development. - **Prompt Engineering**: Techniques for crafting effective prompts and responses. - **Vector Search & RAG**: Implementations for vector search and retrieval-based reasoning (RAG). - **Observability & Monitoring**: Tools for monitoring and analyzing LLM performance and agent behavior. - **Security & Safety**: Measures to ensure the security and safety of LLMs and AI agents. - **Data Management**: Best practices for managing and optimizing data used in LLM operations. - **Optimization & Performance**: Techniques for improving the efficiency and performance of LLMs and AI agents. - **Development Tools**: Tools and resources for developing and testing LLM applications. - **LLMOps Platforms**: Platforms and tools for deploying and managing LLM operations. - **Resources & Learning**: Comprehensive guides, tutorials, and community resources for developers and practitioners. ### Tech Stack: The tech stack for LLMOps is

Overview: Writekin is a Swift-based tool that ingests your writing, curates it into a training corpus, and fine-tunes a local language model to draft and rewrite text that sounds like your own voice. Key features: - Ingests your writing from Mail, Messages, documents, and chat exports. - Fine-tunes a local QLoRA model on Apple Silicon. - Drafts and rewrites text in your voice. - Provides a live loss chart for model training. - Supports local model library and auto-update check. Tech stack: Swift. Why it matters: Writekin is designed to enhance your writing by making it sound more like your own style, improving readability and authenticity. It's a powerful tool for anyone looking to refine their writing skills or improve their communication. Repository: scouttyg/writekin GitHub stars: 22 GitHub description: Writing that's kin to yours Privacy model: Everything happens on your Mac; no telemetry or account required. Requirements: Apple Silicon Mac with 16 GB unified memory minimum, macOS 14 or newer, Full Disk Access. Install: Download DMG, drag to Applications, and follow the app's guide. Build from source: Use `xcodegen` to generate the project and open the Writekin scheme. License: Source-available, free for personal (noncommercial) use under the PolyForm Noncommercial 1.0.0 license.

Overview: TServe is a local inference server designed for foundation models, enabling efficient and scalable time series forecasting. Key features: - Loads models once and keeps them in memory. - Supports Chronos, TimesFM, Moirai, TTM, and TiRex models. - Provides a single request interface for different model families. - Supports Python, JSON, and native tables for input and output. - Offers a dashboard for easy visualization of forecasts. Tech stack: Python Why it matters: TServe simplifies the process of integrating time series models into applications, making it easier to deploy and scale time series forecasting pipelines without the need for complex setup or hosting services. Repository: sktime/tserve Primary language: Python GitHub stars: 20 GitHub description: Time Series serving (TServe): local inference server for foundation models

### Overview: Memcortex is a lightweight semantic memory layer designed for LLMs using open-source embedding models and Weaviate for vector storage and retrieval. It enables persistent, long-term memory for conversational agents and LLM applications, transcending context-window limitations. ### Key Features: - **Persistent Memory Layer**: Stores user/application memories as both text and vectors in Weaviate. - **Memory-RAG Architecture**: Implements a Memory-RAG (Retrieval-Augmented Generation) model to recall past interactions and specific data points indefinitely. - **Weaviate Vector Storage**: Embeds incoming text using `nomic-embed-text` embeddings and stores memories in a `Memory_idx` class on Weaviate. - **Vector Searches**: Retrieves top-K relevant memories for a user and injects them into the prompt before it reaches the LLM. - **Background Persistence**: Optionally persists new memories asynchronously into Weaviate. ### Tech Stack: Go 1.24 ### Why it Matters: Memcortex is essential for building chatbots, agents, and personalization layers that need to "remember" details across sessions. It enhances the capabilities of conversational AI by providing a persistent, long-term memory layer. ### Repository: sobowalebukola/memcortex ### GitHub Stars: 21 ### GitHub Description: Memcortex is a Proof of Concept (PoC) designed to equip conversational agents and LLM applications with persistent, long-term memory. By implementing a Memory-RAG (Retrieval-Augmented Generation) architecture, Memcortex

Overview: ComfyUI custom nodes for 180+ hosted AI models, including Seedance, Kling, Wan, MiniMax H3, Suno, GPT Image, Nano Banana, and more. No GPU needed, pay per request. Key features: - ComfyUI custom nodes for 180+ models - Supports 104 video, 69 image, and 16 audio models - Runs on SpicyAPI's servers, no GPU or model downloads required - Pay per request, no model weights needed - Supports text models in a Chat node - Picked up automatically, rebuilds the node list on start Tech stack: Python Why it matters: Simplifies integration of AI models into ComfyUI workflows, reducing dependencies and costs. Repository: spicyapi-ai/comfyui-spicyapi Primary language: Python GitHub stars: 21

### Overview: This repository showcases the performance of the "elpis" model pair on an AMD RX 7900 XTX, utilizing Qwen3.8-27B and DFlash2 speculative decoding. The model is optimized for high throughput and low latency, achieving impressive speedups over LLM models. ### Key Features: - **Performance Metrics:** - Code and math tasks show significant speedups (2.1x on code, 1.8x on math, 1.5x on code and math with thinking off, 1.1-1.2x on prose). - Speedup against LLM models: The 7900 XTX reaches 53-71% of the 3090's speed. - Raw benchmarks and results are available in the `results` directory. - **Benchmark Scripts:** - The repository includes benchmark scripts to measure performance in various scenarios. ### Tech Stack: - Python 3.12 - ROCm 7.2.4 - TabbyAPI v1.07131c ### Why It Matters: This setup demonstrates the potential of ROCm and Qwen models for high-performance computing tasks, achieving significant speedups over traditional LLM models. The results highlight the effectiveness of speculative decoding and optimized model configurations for real-world applications. ### GitHub Stars: 17 ### Repository: - Repository: thefahien/elpis-on-7900xtx - Primary Language: Python - GitHub Stars

### Overview: _vailá_ is a comprehensive multimodal toolbox designed for video processing and analysis. It includes features for equalizing video events, manual cuts, and synchronization by events with interpolation, all accessible through a user-friendly GUI and command-line interface. ### Key Features: - **Equalize by Events (RIFE)**: Opens a CUDA frame-interpolation tool for reference cameras. - **Manual Cuts Side by Side (M)**: Allows previewing and trimming videos side by side. - **Sync by Events + Interpolation**: Preserves original frames and inserts Farnebäck estimates. - **Sync by Events Physical Time Calibration**: Calibrates physical time after review. ### Tech Stack: Python 3.12.x ### Why it Matters: _vailá_ provides a powerful and user-friendly solution for video processing tasks, making it an essential tool for researchers, engineers, and professionals in the field of computer vision and video analysis.

### Overview: This repository contains a Keras implementation for the ISIC 2019 Challenge Task 1: classify dermoscopic images among nine different diagnostic categories without meta-data. The approach achieved a Balanced Multiclass Accuracy of 0.505, ranking 25th out of 64 unique teams. The key features include the use of ODIN for out-of-distribution detection and the implementation of the Seven-Point Checklist Dermatology Dataset. ### Key features: - Implementation of the Seven-Point Checklist Dermatology Dataset - Use of ODIN for out-of-distribution detection - Balanced Multiclass Accuracy of 0.505 - Ranking 25th out of 64 unique teams ### Tech stack: Keras with TensorFlow backend ### Why it matters: This project showcases the effectiveness of using out-of-distribution detection methods in medical image classification tasks, particularly in challenging scenarios where meta-data is not available.