Projects

### Overview: arcint is an inference server designed for Intel Arc GPUs, specifically targeting the Arc A770 and B60. It utilizes the Qwen hybrid model up to Qwen3.8-Flash-Next, providing a range of capabilities including text-to-image, vision, and speech services. The server runs on OpenCL kernels and supports various features such as context checkpoints and speculative decoding, making it highly efficient and versatile. ### Key Features: - **Model Support:** Supports Qwen3.8-27B and Qwen3.8-Flash-Next models. - **Model Size:** Can run models up to 512 experts on one card and host RAM. - **Inference Speed:** Utilizes OpenCL kernels and OpenAI-compatible endpoints for a short list of models. - **Image Generation:** Supports Z-Image Turbo, Qwen-Image, and FLUX.2 with image editing capabilities. - **Voice and Vision:** Provides image input to the chat endpoint, speech to text, and German text to speech with voice cloning. - **Context Checkpoints:** Ensures that follow-up or edited messages resume instead of re-reading the conversation. - **OpenCL Kernels:** Uses Intel's published series of OpenCL kernels for efficient model inference. - **OpenVINO Engine:** Retired in 0.8.0, but its design and record are available at the tag `openvino-history`. ### Tech Stack: - **Programming Language:** C++ - **OpenCL Kernels:** Intel's published series of OpenCL

Overview: This package provides a Swift SDK for Cactus, a low-latency inference engine for mobile and wearables, allowing developers to run high-performance AI locally in their applications. It supports both Apple platforms and Android, with a focus on cross-platform compatibility. Key features: - Cross-platform Swift SDK for Cactus inference - Supports hybrid cloud inference for when local inference is insufficient - Automatically performs hybrid cloud inference behavior - Compatible with all Apple platforms, Android, and Linux on ARM Tech stack: Swift Why it matters: This SDK enables developers to integrate powerful AI capabilities directly into their Swift applications, enhancing user experience and performance without the need for cloud-based inference.

Overview: A developer-friendly AI tool designed to quickly explore and analyze files, extracting insights and saving them in a text file for easy reference. Key features: - Visual filetree navigation - One-click file exploration - Integration with Tauri for cross-platform compatibility Tech stack: TypeScript Why it matters: Plux Agent GUI MVP is a valuable tool for developers looking to streamline their file management and data analysis processes, providing a quick and efficient way to explore and extract insights from files.

### Overview: A low-level Metal inference engine for Apple M4: universal quantization formats at MLX parity, 1M-token out-of-core contexts, and speculative decode off flash storage. ### Key features: - **Universal Quantization Formats (Q4_0, Q4_K, Q8_0, Q8_1, Q8_2, Q8_3, Q8_4, Q8_5, Q8_6, Q8_7, Q8_8, Q8_9, Q8_10, Q8_11, Q8_12, Q8_13, Q8_14, Q8_15, Q8_16, Q8_17, Q8_18, Q8_19, Q8_20, Q8_21, Q8_22, Q8_23, Q8_24, Q8_25, Q8_26, Q8_27, Q8_28, Q8_29, Q8_30, Q8_31, Q8_32, Q8_33, Q8_34, Q8_35, Q8_36, Q8_37, Q8_38, Q8_39, Q8_40, Q8_41, Q8_42, Q8_43, Q8_44,

### Overview: This project explores a two-stage training pipeline for chord recognition using knowledge distillation as a regularizer. It includes a pure-transformer architecture with dual encoders and a smoothing pipeline for prediction. ### Key Features: - Pure-Transformer ChordNet architecture - Dual encoders for frequency and temporal information - Smoothing pipeline for improved prediction accuracy - Knowledge distillation as a regularizer - Implementation of the paper "Enhancing Automatic Chord Recognition via Pseudo-Labeling and Knowledge Distillation" by Nghia Phan, Rong Jin, Gang Liu, and Xiao Dong ### Tech Stack: Python ### Why it Matters: This project provides a practical solution for enhancing automatic chord recognition through the use of pseudo-labeling and knowledge distillation, offering a valuable approach to improving the accuracy of chord recognition systems.

### Overview LLMs have gained immense popularity in recent months. This workshop focuses on building and deploying LLM-based applications using Ray and Anyscale. You will learn how to scale your application locally using Ray and then deploy it in the cloud using Anyscale. Additionally, you will explore modern tools for running your application on production-grade platforms. ### Key Features - **Ray and Anyscale Integration**: Learn how to integrate Ray with Anyscale to deploy and scale your LLM-based applications. - **Question Answering Service**: Build a local QA service using Ray and HuggingFace libraries. - **Scaling Techniques**: Understand the trade-offs involved in fine-tuning and batch inference of LLMs. - **Modern Deployment Tools**: Explore tools like LangChain and embedding models to run your application on production-grade platforms. ### Tech Stack - **Ray**: A distributed computing library for building and deploying machine learning models. - **HuggingFace**: A library for working with pre-trained models and fine-tuning them. - **LangChain**: A library for building conversational AI applications. - **Anyscale**: A cloud platform for deploying and scaling machine learning models. ### Why It Matters - **Efficiency**: Ray and Anyscale provide a robust and scalable solution for deploying and scaling LLM-based applications. - **Flexibility**: With Ray, you can easily integrate with other libraries like HuggingFace and LangChain, making it easier to build and deploy complex AI applications. - **Scalability**: Ray and Anyscale allow you to scale your application both locally and in the cloud, ensuring

Overview: Transcribe Critic is a comprehensive automated pipeline for producing accurate speech transcripts from video URLs, merging multiple sources including ASR transcripts, YouTube captions, and external human-edited transcripts using an LLM-based adjudication system inspired by textual criticism. Key features: - Critical text merging: Combines 2-3+ transcript sources into the most accurate version using blind, anonymous presentation to an LLM. - wdiff-based alignment: Uses longest common subsequence alignment to keep chunks properly aligned across sources of different lengths. - Multi-model ASR ensembling: Optionally runs multiple ASR models from diverse architectures and resolves disagreements via LLM with anonymous A/B/C labels. - Anti-hallucination: Prevents cascading hallucination by using `condition_on_previous_text=False` and other flags. - External transcript support: Merges in human-edited transcripts as an additional source. - Structured transcript preservation: Preserves speaker labels and timestamps from external transcripts. - Slide extraction and analysis: Automatically detects slide extraction and analyses with optional vision API descriptions. - Make-style DAG pipeline: Each stage checks whether its outputs are newer than its inputs, skipping unnecessary work. - Checkpoint resumption: Saves checkpoints and resumes after interruption. - Cost estimation: Shows estimated API costs before running. - Local-first LLM: Uses Ollama by default for free, local operation. - Speaker diarization: On by default, identifies who is speaking using pyannote.audio, with automatic or manual speaker naming. - Transcript summarization: Generates a structured summary. Tech stack: Python. Why

Overview: Khronos is an open-source intelligent personal assistant that uses speech recognition to perform commands and synthesize speech in response. Key features: - Speech recognition for command execution - Speech synthesis for responses Tech stack: C Why it matters: Khronos is a valuable tool for anyone looking to automate tasks through voice commands or enhance their digital assistant capabilities. Repository: syb0rg/Khronos GitHub stars: 26 GitHub description: The open source intelligent personal assistant

### Overview: `goinfer` is a pure-Go, no-cgo implementation of LLM inference, designed to run Gemma, Qwen, Llama, and friends directly from safetensors or GGUF in a single static binary. It guarantees output types, ensuring that invalid tokens are unreachable, and provides a single static binary with the model baked into it. Additionally, it supports offline operation, with no telemetry or network interactions beyond fetching a model from HuggingFace. ### Key Features: - **Output Types Guarantee**: Constrains generation to a Go struct or a JSON Schema, ensuring invalid tokens are unreachable. - **One Static Binary**: Runs an open-weight LLM inside a Go program, making it easy to integrate with existing Go projects. - **43 Model Families**: Supports 43 model families, each with a HuggingFace logit-parity gate. - **Cross-Platform Support**: Runs on CPU, CUDA, and Metal, with a cgo-free WebGPU opt-in build. - **Offline Operation**: Runs completely offline, without network interactions beyond fetching a model from HuggingFace. - **Tool Calls**: On the Qwen families, Nemotron-3-Nano, Mellum2, and Granite 4.2, tool calls cannot be malformed or name a tool you did not send, with any number of tools, and a turn that answers in prose runs as fast as it would unconstrained. ### Tech Stack: Pure-Go, no cgo, static binary. ### Why It Matters: - **Guaranteed Output Types**:

Overview: MAYA is a private AI companion web app designed for emotionally aware conversations, voice notes, memories, shared activities, and continuous continuity. It uses TypeScript and React for the frontend and Express for the backend, powered by Supabase for user authentication and Supabase Realtime for real-time updates. Key features: - Streamed text responses over Server-Sent Events (SSE) - Emotion labeling and English/Hinglish-friendly prompting - Server-side transcription of voice notes - Browser speech synthesis and recognition - Persistent history and personal context preservation - Durable memory candidates for personal facts, relationship context, recurring mood, and meaningful topics - Voice game sessions - Co-watching of YouTube videos with optional titles and notes - Persistent user records and preferences - Passwordless Supabase email verification - User-scoped server procedures Tech stack: TypeScript, React, Express, Supabase Why it matters: MAYA provides a private, emotionally aware AI companion that preserves user's personal context and shared activities, making it a valuable tool for maintaining relationships and emotional well-being. Repository: vincenzo-afk/MAYA GitHub stars: 26 GitHub description: A private AI companion web app for emotionally aware chat, voice notes, memories, shared activities, and gentle continuity.

Overview: This project provides a comprehensive implementation of GraphRAG, a powerful tool for graph-based reasoning and analysis. It includes detailed setup instructions and usage guidelines for developers. Key features: - GraphRAG Implementation - Pre-requirements setup - Installation and deployment of LLM and embedding models in Azure OpenAI - Configuration of OPENAI_API_KEY - Changes to `settings.yaml` file for LLM and embeddings Tech stack: Python 3.10 Why it matters: This project is essential for developers looking to implement advanced graph-based reasoning and analysis solutions, leveraging powerful AI models for improved performance and accuracy. Repository: ApexIQ/End-to-End-Graphrag-implementation GitHub stars: 25 GitHub description: End-to-End-Graphrag-implementation

Overview: This project is an official Python implementation of Fourier domain structural relationship analysis for unsupervised multimodal change detection, focusing on the Fourier domain structural relationship analysis framework for unsupervised multimodal change detection (FD-MCD). Key features: - Fourier domain structural relationship analysis framework for unsupervised multimodal change detection - Implementation of local and nonlocal structural relationships on the graph Fourier domain - High-order vertex information modeling using graph spectral convolution with Chebyshev polynomial Tech stack: Python Why it matters: This framework addresses the challenge of comparing multimodal images by exploiting both modality-independent local and nonlocal structural relationships, making it particularly useful in applications requiring time-sensitive analysis, such as disaster response. Repository: ChenHongruixuan/FDMCD Primary language: Unknown GitHub stars: 25 GitHub description: [ISPRS P&RS 2023] Official Python Implementation of Fourier domain structural relationship analysis for unsupervised multimodal change detection README excerpt: # FDMCD Python Official Implementation of [Fourier domain structural relationship analysis for unsupervised multimodal change detection.](https://www.sciencedirect.com/science/article/pii/S092427162300062X) <img src="./figures/FDMCD.jpg">

### Overview: This repository contains the evaluations for the "ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models" paper. It provides a framework for personalized text-to-image generative models and includes evaluation metrics for concept alignment and composition alignment. ### Key Features: - Concept Alignment Evaluation - Composition Alignment Evaluation - Data and Model Checkpoints Downloaded - Scripts for Generating and Evaluating Images ### Tech Stack: Python ### Why it Matters: This repository offers a comprehensive evaluation framework for personalized text-to-image generative models, helping researchers and practitioners assess the concept learning abilities of these models.

Overview: Fin-Fact is a benchmark dataset designed for multimodal financial fact-checking and explanation generation, consisting of 3562 claims across multiple sectors. Key features: - Multimodal representation: Includes textual claims and visual elements like images and captions. - Comprehensive labels: Includes Claim, Author, Posted Date, Sci-digest, Justification, Evidence, Evidence href, Image href, Image Caption, Visualisation Bias Label, Issues, and Claim Label. - Extensive dataset size: Contains 3562 claims across multiple financial sectors. Tech stack: Python Why it matters: Fin-Fact provides a valuable resource for researchers, data scientists, and fact-checkers in the financial domain, enhancing the accuracy and effectiveness of fact-checking systems through multimodal analysis.

Overview: LocalSpeech is a one-command script for deploying AI-powered speech models on MacOS, supporting various platforms like Sesame, Kokoro, Spark, Zonos, and Whisper. Key features: - Supports multiple voice models: Whisper, Kokoro, Spark, Zonos, and Sesame. - Provides a fully configured OpenAI-compatible API server. - Includes detailed instructions for setting up the environment and running both backend and playground services. Tech stack: TypeScript Why it matters: This script simplifies the process of deploying AI models on MacOS, making it accessible for developers without extensive setup knowledge.

### Overview: UAVReason is a multimodal aerial scene reasoning and generation dataset designed for UAV-view images. It includes VQA / caption annotations, image-to-image generation JSONL files, and additional depth data. The dataset is used for tasks such as UAV visual question answering, scene captioning, spatial reasoning, temporal reasoning, heading reasoning, depth-aware perception, and cross-modal generation. ### Key features: - **VQA / Caption Annotations**: Single-frame, two-frame, heading reasoning, and scene caption generation. - **Image-to-Image Generation**: RGB to depth, RGB to segmentation, depth to RGB, segmentation to RGB, and depth to segmentation to RGB. - **Depth Data**: Original depth array, grayscale depth visualization, and depth statistics. ### Tech stack: Python ### Why it matters: UAVReason provides a comprehensive dataset for researchers and developers interested in aerial scene understanding and generation, making it valuable for various applications in computer vision and artificial intelligence.

### Overview: This project presents an end-to-end speech command recognition system utilizing capsule networks. The system is trained on the Google Speech Command Dataset and demonstrates superior performance on both clean and noise-added test datasets compared to baseline CNN models. ### Key Features: - **Capsule Network**: Captures spatial and temporal relationships in speech spectrograms. - **Keyword Spotting (KWS)**: Recognizes speech commands with high accuracy. - **Noise Addition**: Adds noise to the dataset for robust testing. - **Feature Generation**: Extracts speech features from raw audio files and saves them as .npy files. - **Training and Testing**: Implements both training and testing scripts for Capsule Network and various KWS models. ### Tech Stack: Python ### Why it Matters: This project showcases the effectiveness of capsule networks in speech command recognition, achieving significant improvements over traditional CNN models. The inclusion of various KWS models provides a comprehensive solution for real-world applications where noise is present. Repository: JaesungBae/Speech-Command-Recognition-with-Capsule-Network GitHub Stars: 25 GitHub Description: Speech command recognition with capsule network & various NNs / KWS on Google Speech Command Dataset.

Overview: This project is a code repository for the paper "Semantic Diversity-aware Prototype-based Learning for Unbiased Scene Graph Generation" presented at ECCV 2024. It includes the official implementation and dataset preprocessing instructions for the paper. Key features: - Code for Semantic Diversity-aware Prototype-based Learning - Dataset preprocessing instructions for VG and GQA datasets - Pretrained models for VG and GQA datasets - Training instructions for Scene Graph Generation tasks (PredCls, SGCls, SGDet) Tech stack: Jupyter Notebook Why it matters: This project provides a comprehensive solution for scene graph generation, demonstrating the effectiveness of the proposed method in the ECCV 2024 paper. Repository: JeonJaeHyeong/DPL GitHub stars: 25 GitHub description: Code for paper "Semantic Diversity-aware Prototype-based Learning for Unbiased Scene Graph Generation (ECCV 2024)"

Overview: This repository contains the PyTorch code for an attention-based backend designed to efficiently fine-tune transformer models for speaker verification. It is based on the VoxCeleb dataset and includes pre-trained models for WavLM. Key features: - Efficient fine-tuning of transformer models - Speaker verification using an attention-based backend - Baseline model: WavLM Base Plus + MHFA - Large margin fine-tuning (LMFT) model Tech stack: Python Why it matters: This implementation provides a practical solution for speaker verification tasks, leveraging efficient fine-tuning techniques to improve performance and speed.

Overview: TextFlow is a framework that converts flowchart images into text, addressing the limitations of end-to-end vision-language models (VLMs) in flowchart understanding tasks. Key features: - Users can select the type of text representations (e.g., Graphviz, Mermaid, PlantUML) or convert them into executable graph objects for enhanced performance and controllability. - It improves explainability by attributing errors more clearly to visual or textual processing components. - It promotes modular solution design, allowing advanced LLMs to be used in the reasoner stage when VLMs underperform in end-to-end fashion. Tech stack: Python Why it matters: TextFlow addresses the limitations of end-to-end VLMs in flowchart understanding tasks by providing a framework that enhances explainability and performance, making it a valuable tool for researchers and practitioners in the field. Repository: JunyiYe/TextFlow Primary language: Python GitHub stars: 25 GitHub description: [NAACL 2025] Beyond End-to-End VLMs: Leveraging Intermediate Text Representations for Superior Flowchart Understanding