Projects

Overview: This project demonstrates an end-to-end Fondant pipeline for preparing data to fine-tune a ControlNet model, specifically for interior design. Key features: - Prompt Generation - Image URL Retrieval - Download Images - Add Captions - Add Segmentation Maps Tech stack: Jupyter Notebook Why it matters: This pipeline provides a comprehensive solution for generating and processing data for fine-tuning a ControlNet model, focusing on interior design, allowing users to generate realistic images of their desired interior spaces.

Overview: Auricle is a streaming ASR, audio captioning, and speech QA toolkit built on a whisper-style encoder with pluggable LLM backends. Key features: - Whisper-style encoder - Streaming ASR - Audio captioning - Speech QA - Pluggable LLM backends - Evaluation suite - CLI Tech stack: Python Why it matters: Auricle provides a comprehensive solution for speech understanding, enabling real-time transcription, captioning, and question answering, making it valuable for applications requiring continuous speech processing.

Overview: Enhance Prompt Editor (EPE) is a full prompt-writing studio integrated into a single ComfyUI node, powered by local Ollama models, offering AI editing, style targeting, prompt browsing, and wireless prompt injection without cloud APIs or keys. Key features: - Enhanced prompt editor with up to 10 tabs for multi-tab editing - Send to tab feature to share results across tabs - What's in the prompt box is what renders, ensuring consistency - Editor: multi-tab prompt editor with undo/redo, find & replace, case/sort/dedupe/trim, cleanup, word wrap, and live word + token counts - Synonyms: suggests plain and creative alternatives for selected words - Flag words: highlights empty quality words and offers replacements Tech stack: JavaScript Why it matters: No cloud APIs, no keys, making it perfect for local workflows and security-conscious environments.

Overview: P2P TOKENS is a decentralized network designed for LLM inference, enabling peers to seed and leech completions without a central authority. It uses a barter ratio model based on the BitTorrent protocol, ensuring honest behavior and trustless interactions. Key features: - Distributed, peer-to-peer LLM inference - Seed tokens, leech tokens, no middleman - OpenAI-compatible API - Built with Rust, libp2p, and licensed under MIT Tech stack: Rust, libp2p Why it matters: P2ptokens offers a decentralized and trustless solution for GPU inference, making it ideal for those looking to avoid middlemen and ensure fair and honest interactions in the decentralized computing space.

### Overview: 在 DeepSeek Harness 会话中，只需轻触麦克风即可开始语音输入、编辑草稿并按句朗读，支持实时字幕和发声打断。识别在宿主端进行，无需额外语音 API Key。 ### Key features: - 🎤 实时出字（流式识别） - 🔊 按句朗读 - ⏸ 发声打断 ### Tech stack: JavaScript ### Why it matters: 无需额外语音 API Key，识别在宿主端本地推理，支持本地 VITS 和 Kokoro 中英混读，无需依赖额外服务。

### Overview: AESTETIK is a convolutional autoencoder model designed to learn accurate spot representations from spatial transcriptomics and morphology data. It integrates transcriptomics and morphology information on a spot level and topology on a neighborhood level, capturing biological complexity. ### Key Features: - **Spot Representation Learning**: AESTETIK learns accurate spot representations that capture biological complexity. - **Multi-Modal Integration**: It integrates spatial transcriptomics, morphology, and topology information to provide a comprehensive understanding of biological processes. - **Dimensionality Reduction**: The model uses dimensionality reduction techniques like PCA to preprocess the spot representations. - **Neighborhood Fusion**: The modality-specific representations are fused through concatenation, and the grid per spot is built as input for the autoencoder. ### Tech Stack: - **Python**: The primary programming language used for implementing AESTETIK. ### Why It Matters: AESTETIK offers a powerful tool for researchers to gain a multi-modal understanding of key biological processes through spatial transcriptomics, making it an essential resource for those working in the field of spatial transcriptomics and related disciplines.

Overview: The LLM Hotel Recommender is a Streamlit app utilizing Redis and the OpenAI API to generate hotel recommendations based on user preferences. It allows users to search for hotels in the US by specifying state, city, positive qualities, and negative qualities. Key features: - Semantic search capabilities in Redis - Hypothetical Document Embeddings (HyDE) approach for generating fake reviews - Redis vector search for semantic similarity - Streamlit app for easy deployment Tech stack: Python Why it matters: This project showcases the integration of Redis, OpenAI, and Streamlit to create a powerful hotel recommendation system, leveraging semantic search and LLMs for personalized recommendations.

Overview: This Streamlit project provides a user-friendly interface for localizing and using an LLM (Large Language Model) on Ollama, making it accessible for coding and everyday queries. Key features: - Streamlit webapp UI for local LLM implementation - Three Python libraries required: torch, streamlit, and ollama - Works with Ollama Windows and DuckDuckGo browser - Supports both GPU and CPU - Available models from Ollama library Tech stack: Python Why it matters: This project simplifies the process of integrating an LLM into a local environment, making it accessible for developers and users alike.

### Overview: This Go library provides a comprehensive client for interacting with the Perplexity AI API, supporting chat completions, async jobs, streaming, multimodal messages, structured outputs, and web search integration. It simplifies the interaction with the Perplexity models and provides features like authentication, model management, and request validation. ### Key features: - **Chat Completion Requests**: Simple and easy-to-use interface for chat completion requests. - **Search API Support**: Direct access to Perplexity's real-time web index without the generative LLM layer. - **Model Support**: Handles all Perplexity models, including online LLMs. - **Authentication and Key Management**: Supports authentication and API key management. - **Request Validation**: Comprehensive request validation. - **Concurrent Requests**: Supports concurrent request handling with thread-safe operations. - **Context-Aware Handling**: Handles context-aware request handling with cancellation support. ### Tech stack: The library is built using Go, leveraging its simplicity and efficiency for rapid development and deployment. ### Why it matters: This library is essential for developers looking to integrate Perplexity AI capabilities into their projects, providing a robust and easy-to-use interface for interacting with the Perplexity models and accessing the Perplexity Search API. The lightweight and thread-safe nature of the library makes it suitable for both small and large-scale applications.

Overview: This project provides a native implementation of the DeepSeek-V4-Flash-0731 model running on a single laptop CPU, utilizing pure C programming language with no GPU, CUDA, PyTorch, or weight conversion. It is designed for efficient inference with a prompt-lookup TPOT of 0.892 seconds per token on a 32 GB x86 laptop, making it suitable for applications requiring high performance and minimal resource usage. Key features: - Pure C implementation - No GPU, CUDA, PyTorch, or weight conversion - Supports prompt-lookup TPOT of 0.892 seconds per token on a 32 GB x86 laptop - 8 GB minimum system RAM requirement - Runs on a single laptop CPU Tech stack: Pure C programming language Why it matters: This project offers a lightweight and efficient solution for running large language models on a single CPU, making it ideal for applications requiring high performance and minimal resource usage. The absence of GPU and weight conversion ensures optimal performance without the need for additional hardware resources, making it suitable for environments with limited computational power. Repository: shyringo/deepseek-v4-flash-0731-in-c Primary language: C GitHub stars: 28 GitHub description: Run the native 284B-A13B DeepSeek-V4-Flash-0731 LLM locally on one laptop CPU: pure C, 8 GB RAM minimum, no GPU, best TPOT 0.892 s

Overview: Jarvis, a voice assistant powered by GPT-3.5/GPT-4, is designed to enhance user interactions by providing personalized and intelligent responses. Key features: - Voice recognition and synthesis - Natural language processing - Customizable voice commands - Integration with various devices Tech stack: Python Why it matters: Jarvis leverages cutting-edge AI technology to offer a seamless and intuitive user experience, making it a valuable tool for anyone looking to enhance their communication with technology.

Overview: This repository contains the code for the Landing Page of a Resume Matcher website. Key features: - Landing Page for Resume Matcher - Built with Astro - Uses Astrowind for development Tech stack: Astro Why it matters: This project showcases the use of Astro for building a website and demonstrates the benefits of using a specific development toolchain for a practical application.

### Overview: A bot designed to record and transcribe audio fragments from Discord. ### Key features: - Record and transcribe audio from Discord channels. - Transcribe speech to text. - Replay audio files. - Record and replay conversations. - Play lastest audio file with transcription. ### Tech stack: Python ### Why it matters: This bot is useful for automating the recording and transcription of conversations in Discord, making it easier to manage and archive communications.

### Overview: Formant features extraction from WAV files using Python. ### Key features: - Extracts frequency, power, width, and dissonance of formants from WAV files. - Supports speech recognition and music analysis applications. - Provides formant features in a structured numpy array format. - Supports customizable parameters for formant extraction and windowing. ### Tech stack: - Python 3.7 or later - Numpy 1.16 or later - Scipy v1.3.1 - H5py v2.9.0 - Numba v0.45.1 - Wavio v0.0.4 ### Why it matters: - Useful for speech recognition and music analysis applications. - Provides formant features in a structured numpy array format, making it easy to integrate into existing projects. ### Repository: tabahi/formantfeatures ### Primary language: Python ### GitHub stars: 28 ### GitHub description: Extract frequency, power, width and dissonance of formants from WAV files

### Overview: EPDServe: Disaggregated Serving System for Large Multimodal Models EPDServe is designed to efficiently serve large multimodal models (LMMs) by implementing the Encode-Prefill-Decode (EPD) Disaggregation architecture. It is accepted at ICML 2025 and aims to reduce the time-to-first-token (TTFT) and enable efficient inference of large models. ### Key Features: 1. **Disaggregated Inference Pipeline**: EPDServe splits the inference pipeline into three independent stages: Encoding, Context (Prefill), and Decoding. Each stage operates independently with its own compute resources, scheduler, cache manager, and GPU workers. 2. **Independent Scaling**: It supports independently deployed encoder, prefill, and decode workers, including multimodal embedding transfer and caching. 3. **Intra-Request Parallelism**: It allows for intra-request parallelism to distribute heavy encoding loads. 4. **Monitoring Delays/Bottlenecks**: It monitors delays and bottlenecks at various stages, such as the load estimator, to optimize performance. 5. **CUDA-Level IPC Utilities**: It utilizes CUDA-level IPC utilities for fast and asynchronous token and model weights transfer. 6. **Dynamic Resource Reallocation**: It dynamically reallocates GPU resources and switches roles between workers. 7. **Minimal Latency and Maximized Throughput**: It achieves minimal latency and maximized throughput (TTFT / E2ETP). ### Tech Stack: - **Python**: The primary language used for the implementation. ### Why It Matters: EPDS

### Overview: Loki Agent 𖤍 The Loki Agent is a self-improving AI agent built by WunderCorp, Inc. It learns from your interactions, improving its skills and knowledge over time. Run it on a $5 VPS, a GPU cluster, or serverless infrastructure that costs nearly nothing when idle. It can be used with any model, including OpenRouter, OpenAI, or your own endpoint. ### Key Features: - **Built-in Learning Loop**: Creates skills from experience, improves them during use, and persists knowledge. - **Cross-Session Recall**: Uses FTS5 session search with LLM summarization for cross-session recall. - **Dialectic User Modeling**: Compatible with the agentskills.io open standard. - **Scheduled Automations**: Built-in cron scheduler for scheduled tasks. - **Closed Learning Loop**: Agent-curated memory with periodic nudges. - **Delegated Work**: Spawns isolated subagents for parallel workstreams. - **Serverless Persistence**: Your agent's environment hibernates when idle and wakes on demand, costing nearly nothing between sessions. ### Tech Stack: Python The Loki Agent is built using Python, leveraging the power of AI and open-source technologies. ### Why It Matters: The Loki Agent is the only AI agent with a built-in learning loop, making it uniquely capable of adapting to your needs over time. Whether you're using it for personal productivity, creative projects, or any other purpose, it's designed to evolve with you, providing a seamless and efficient experience.

### Overview: OpenOmni Framework is a multimodal open-source framework designed for the research and development of conversational agents. It aims to provide a balanced platform for researchers to test their contributions and collaborate effectively. ### Key Features: - **Multimodal End-to-End Conversational Agents**: Aims to develop advanced AI models that can handle complex multimodal inputs, including audio and images, and generate contextually appropriate responses. - **Balanced Performance**: Focuses on achieving high accuracy, low latency, and efficient cost, addressing the challenges faced by traditional approaches. - **Open-Source Framework**: Provides an open-source environment for researchers to contribute and collaborate, facilitating the research and development of multimodal conversational agents. ### Tech Stack: Python ### Why It Matters: The development of multimodal large language models (LLMs) is crucial in the field of conversational agent research and applications. While GPT-4 has demonstrated the capability of handling complex multimodal inputs and generating contextually appropriate responses, achieving the right balance between cost, accuracy, and latency remains a significant challenge. OpenOmni Framework provides a solution by offering an open-source platform that allows researchers to easily test their contributions and collaborate effectively, accelerating the research and development process towards more efficient and capable multimodal conversational agent systems.

Overview: Infera is a distributed, SLA-aware serving mesh optimized for large-scale deployments, designed to scale goodput from frontier models with a focus on agentic AI applications. Key features: - Route: KV-Aware Routing - Specialize: Prefill-Decode Disaggregation - Retain: KV-Cache Tiering Tech stack: Python Why it matters: Infera addresses the inefficiencies in traditional inference orchestration by coordinating engine instances across three dimensions—KV-aware routing, prefill-decode disaggregation, and KV-cache tiering—improving inference goodput and ensuring consistent performance across different instances, especially for agentic workloads.

### Overview: Diffusion-SDPO is a safeguarded direct preference optimization rule designed for diffusion models, ensuring that updates do not increase the winner's loss to first order while preserving the preferred output. ### Key features: - **Safeguarded Direct Preference Optimization (SDPO)**: Computes an adaptive scale for the loser branch based on the alignment between winner and loser output-space gradients, preventing the winner's loss from increasing. - **Model Agnostic**: Works with any diffusion model without additional training or modifications. - **Minimal Overhead**: Drops into Diffusion-DPO, DSPO, and DMPO with negligible performance impact. ### Tech stack: Python ### Why it matters: SDPO ensures that updates to diffusion models do not increase the loss of the winner, preserving the preferred output while maintaining a wide preference margin. This makes it a valuable tool for maintaining the quality and relevance of generated images in various applications.

Overview: This repository is a Multimodal Semantic RAG system designed to process documents with multimodal content, generating factual, context-aware answers based on textual, visual, and tabular information. Key features: - Semantic chunking - Image and table detection - Summarizing images and tables - Embedding - Semantic retrieval - Re-ranking - Answer generation Tech stack: Python Why it matters: This system aims to provide accurate, context-aware answers by integrating various data types, making it highly versatile and useful for applications that require handling diverse information sources. Repository: AhmedAl93/multimodal-semantic-RAG GitHub stars: 27 GitHub description: A RAG system designed to process documents with multimodal content. It can generate factual, context-aware answers to user queries, based on the documents texts, tables, figures, ...