Projects

Overview: The Arcee Python client is a tool designed for managing and executing various language model routines on the Arcee Platform, including CPT, SFT, DPO, and Merge models. Key features: - CLI (Command Line Interface) for easy execution - SDK for programmatic use - Support for uploading context, finetuning datasets, and generating text completions Tech stack: Python Why it matters: The Arcee Python client simplifies the process of managing and executing language model routines on the Arcee Platform, making it accessible to developers and researchers alike. Repository: arcee-ai/arcee-python GitHub stars: 29

### Overview: This project showcases a speech-to-text system utilizing the Wav2Vec 2.0 framework and Hugging Face's Transformers, focusing on self-supervised learning for improved accuracy. ### Key features: - Speech-to-Text Conversion - Utilizing Wav2Vec 2.0 for Improved Accuracy - Leveraging Hugging Face's Transformers for Enhanced Model Performance ### Tech stack: - Python - Jupyter Notebook ### Why it matters: This project demonstrates the application of advanced machine learning techniques in speech recognition, highlighting the effectiveness of self-supervised learning in improving speech-to-text conversion accuracy.

Overview: This project is an AGI interface for Asterisk PBX servers that utilizes Google for speech recognition and Text-to-Speech capabilities. It integrates with Home Assistant to control home devices. Key features: - Speech recognition - Text-to-Speech - Intent recognition for Home Assistant integration Tech stack: Python Why it matters: This project provides a flexible and efficient solution for integrating voice assistants with Asterisk PBX, enabling automation and control of home devices through speech commands. Repository: bkbilly/asterisk-assistant Primary language: Python GitHub stars: 29 GitHub description: 📞 AGI interface with python for speech recognition

Overview: This project, "No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models," is a research paper that introduces a novel approach called SAGE for optimizing the learning rate in large transformer models. The paper was presented at ICLR 2022 and is designed to improve the efficiency and performance of training large transformer models. Key features: - SAGE: A sensitivity-guided adaptive learning rate optimization method for large transformer models. - Implementation of Adamax-SAGE and AdamW-SAGE for fine-tuning pre-trained models. - Scripts provided for fine-tuning BERT and RoBERTa models on various tasks. Tech stack: Python Why it matters: SAGE offers a significant improvement in the training process of large transformer models by dynamically adjusting the learning rate based on the sensitivity of the model parameters, leading to better performance and efficiency. Repository: cliang1453/SAGE Primary language: Python GitHub stars: 29 GitHub description: No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models (ICLR 2022)

Overview: LocalFlow is a fast, private, on-device dictation tool for Apple silicon Macs, designed to transcribe speech into text without sending audio or transcripts to a cloud service. Key features: - Streams speech recognition locally with Apple SpeechTranscriber. - Uses Apple Foundation Models for optional two-second smart cleanup. - Falls back to the raw transcript if cleanup misses its deadline. - Pastes safely into the currently focused text field. - Keeps listening and processing in a 170 × 38 waveform-only notch. - Dictation auto-pastes into a focused text field; without an editable target, the notch expands with Copy and History. - Keeps unlimited, searchable transcript history in a user-private local SQLite database. - Supports click-to-start/stop, push-to-talk, Reduce Motion, and a persistent 220–420 pt expanded-HUD width. - Never persists audio, prompts, or clipboard contents. Transcript history is stored only on your Mac and can be cleared at any time. Tech stack: Swift Why it matters: LocalFlow provides a fast, private, and secure dictation solution for Apple silicon Macs, eliminating the need for cloud-based transcription services and ensuring user privacy and data security.

Overview: This repository contains hands-on labs designed to demonstrate the capabilities of Microsoft Foundry NextGen, Azure's unified PaaS for enterprise AI operations, model builders, and application development. Key features: - Provisioning - Agents (incl. hosted Copilot SDK + REST) - MCP (Model Management Center) - Foundry IQ knowledge bases - Guardrails - Red-teaming - Fine-tuning - Observability Tech stack: Jupyter Notebook Why it matters: These labs provide a comprehensive overview of Foundry NextGen, showcasing how it unifies agents, models, and tools under a single Azure resource provider namespace, with built-in tracing, monitoring, evaluations, and a single RBAC/networking/policy surface. This platform is ideal for enterprise AI operations, model builders, and application development. Repository: corticalstack/awesome-foundry-nextgen GitHub stars: 29 GitHub description: Hands-on labs for Microsoft Foundry NextGen - Azure's unified PaaS for enterprise AI operations, model builders, and application development. Each lab demonstrates a specific Foundry pattern: provisioning, agents, hosted Copilot SDK agents, MCP, Foundry IQ knowledge bases, guardrails, red-teaming, and fine-tuning.

Overview: This project visualizes the agent loop mechanism, showing how an agent's chat box operates step by step. It provides a more intuitive and visual representation compared to traditional reading, allowing users to understand the entire mechanism laid out in front of them. Key features: - Detailed step-by-step visualization of the agent loop - Support for multiple models, including Google's Gemma-4 and Qwen/Qwen3.5 - Option to run models on different platforms, including vLLM, Docker, and local machines - Integration with the Agent Atlas course, matching sections with the reading material - Support for two languages, English and Traditional Chinese Tech stack: Python Why it matters: This project offers a valuable learning tool that simplifies the understanding of complex agent loop mechanisms by providing a visual representation. It helps users grasp the underlying logic of how an agent's chat box operates in a clear and intuitive way. Repository: cyyeh/agent-loop-visualizer GitHub stars: 29 GitHub description: This project is for learning: it shows how an agent loop works underneath, in a way more intuitive and visual than reading about it. Behind an agent's chat box runs a loop: step after step, the model proposes an action and the harness carries it out, until it stops. A chat app shows only the question and the answer; this page draws every step as it happens, down to each token the model writes, so the whole mechanism is laid out in front of you. Each part of the page matches a section of the Unit 01 reading

### Overview: Qwen Lens Studio is a multimodal AI studio powered by Qwen3.6-35B-A3B, offering a single model with visual reasoning, image captioning, and document understanding tools. It provides a web app for end-to-end integration, allowing users to ship screenshots, invoices, and images for structured JSON outputs, and compare documents side by side with a markdown diff. ### Key features: - **Visual Reasoning**: Answers with chain-of-thought reasoning and optional thinking traces. - **Multilingual Describe**: Vivid descriptions in the requested language. - **Document IQ**: Structured JSON extraction from invoice, receipts, IDs, and forms. - **Code Lens**: HTML / React / Vue / Svelte implementation from UI screenshots. - **Dual Compare**: Markdown diff with similarities, differences, and verdicts. ### Tech stack: TypeScript ### Why it matters: Qwen Lens Studio simplifies the integration of multimodal AI tools, providing a unified interface for visual reasoning, document understanding, and code analysis. The single model and streamlined architecture make it accessible and efficient for developers and users alike.

Overview: This repository serves as a webpage for keeping track of publicly available courses in the fields of Deep Learning, Machine Learning, Reinforcement Learning, Vision, Natural Language Processing, and Optimization. Key features: - A comprehensive list of courses - Updated information regularly - Easy navigation for developers Tech stack: HTML Why it matters: This resource is valuable for developers looking to stay updated on the latest courses and educational materials in the field of artificial intelligence and machine learning.

Overview: `xaidr` is a runtime security tool designed to detect and prevent malicious activities in AI agents, including prompt injection, known jailbreak patterns, and destructive tool calls, without requiring any backend or network dependencies. Key features: - Real-time detection of AI agent behavior - Zero dependencies and minimal latency - Measures against 186 shell attacks with 89.8% accuracy - Includes OWASP Agentic Top 10 coverage and limitations Tech stack: Python Why it matters: `xaidr` provides a crucial layer of security for AI agents, ensuring that they do not execute malicious commands or engage in unauthorized actions, thereby protecting systems from potential threats. Repository: delphisecurity/xaidr GitHub stars: 29 GitHub description: Runtime security for AI agents. In-process, zero dependencies, less than 1ms latency.

### Overview: This repository is a curated list of vector database solutions, libraries, and resources for AI applications. It serves as a comprehensive guide for developers and researchers interested in vector databases, providing insights into various types of vector databases, their applications, and best practices. ### Key Features: - A wide range of vector database engines, including popular solutions like FAISS, VectorDB, and others. - Resources for machine learning models, including LLMs (Large Language Models) and their integration with vector databases. - Research and surveys on vector database technologies and their impact on AI applications. - Managed and serverless vector databases, ideal for cloud-based deployments. - Tools and frameworks for integrating vector databases with LLMs and other AI models. - A curated list of libraries and resources for working with vector databases. - A section dedicated to benchmarking and evaluation tools for vector databases. - Resources for cloud-based vector databases, including managed and serverless options. - A list of curated resource lists for additional learning and inspiration. - A section for cloud-managed vector databases, providing managed services for vector database management. - A collection of data integration and migration techniques for vector databases. - A section on embedded and edge vector databases, suitable for real-time applications. - A library of evaluation and observability tools for vector databases. - A section on experimental and learning vector databases, ideal for developers looking to experiment with new technologies. - A list of federated vector databases, useful for distributed systems. - A section on full-text vector search engines, essential for text-based applications. - A collection of graph

### Overview: Markdown Vector Database (mdvdb) is a filesystem-native vector database built around Markdown files. Designed for AI agents and memory management, it offers three search modes: hybrid, semantic, and lexical. Built with Rust for performance, it simplifies indexing and search, making it ideal for knowledge bases, agent memory, and personal note-taking. ### Key Features: - **Hybrid Search**: Combines semantic and lexical search for comprehensive results. - **Semantic Search**: Uses embeddings for similarity-based search. - **Lexical Search**: Supports full-text search using BM25. ### Tech Stack: Rust, a language known for its performance and reliability. ### Why It Matters: mdvdb provides a fast, flexible, and efficient way to search and manage local knowledge bases, making it a valuable tool for AI agents and developers looking to optimize their workflows.

Overview: This project utilizes embeddings to create a memory system that abstracts away the details of managing indexes saved on disk, making it easier to manage and query large amounts of text data. Key features: - Abstracts away the details of managing indexes saved on disk. - Provides a simple interface for creating and managing indexes. - Assists in querying the memory in two steps, first identifying the appropriate index and then running the query against the most relevant index. Tech stack: Python Why it matters: This project simplifies the process of managing and querying large volumes of text data by abstracting away the complexities of indexing and memory management, making it easier for developers to implement and utilize in their projects.

Overview: UniServe is a unified serving engine that combines a Rust server for HTTP admission, scheduling, and response assembly with Python workers for running models, designed for the fastest FastH3 serving. Key features: - Unifies HTTP server, scheduling, and response assembly with Python model execution. - Supports Linux x86-64 and aarch64, NVIDIA Hopper or Blackwell GPUs, CUDA 13, Python 3.12, and a stable Rust toolchain. - Built-in FFmpeg for media processing and reference media resolution. - Supports serving with up to eight GPUs, Python workers, and multiple media resolutions. Tech stack: Rust, Python, FFmpeg, CUDA, Docker, and NVIDIA GPUs. Why it matters: UniServe simplifies serving by integrating multiple components into a single engine, optimizing performance and ease of deployment.

Overview: PRISM is a photonic block selection method designed to reduce the O(N) cost of KV cache scans in long-context language model inference, specifically for models like Qwen2.5-7B. It offloads the selection to a photonic co-processor, eliminating the need for expensive HBM reads and significantly reducing energy and latency. Key features: - Photonic block selection for O(1) KV cache scan elimination - Offloads the selection to a photonic co-processor - Signatures are stored locally in MRR weight banks, reducing HBM traffic - Signatures are computed in a single optical pass, eliminating the need for GPU scans - Signatures are stored in local memory, reducing energy and latency Tech stack: Python, PyTorch Why it matters: PRISM significantly reduces the energy and latency required for long-context language model inference, making it highly efficient and practical for large models like Qwen2.5-7B. The elimination of the O(N) scan and the reduction in HBM traffic make it an ideal solution for memory-bound inference tasks.

### Overview: Chough is a fast, memory-efficient audio-to-text transcription tool that processes audio in chunks, supports multiple formats, and requires no setup. It’s designed for real-time transcription and is compatible with various languages, making it a versatile solution for developers. ### Key Features: - **Fast**: Transcription speeds up to 6-21x real-time, including startup. - **Memory-Efficient**: Processes audio in chunks, reducing memory usage. - **Any Format**: Supports any audio format supported by `ffmpeg`. - **No Setup**: Automatically downloads models on first run. - **Multiple Formats**: Outputs in text, JSON, and VTT formats. - **CPU Only**: No GPU required. - **Server Mode**: Transcribes audio via a remote server, eliminating startup time. ### Tech Stack: Chough leverages Parakeet TDT 0.6b V3 via Sherpa-Onnx for ASR, ensuring fast and efficient processing of audio files. ### Why It Matters: Chough is designed for developers who need real-time transcription capabilities without the need for complex setup. Its fast processing speed and memory efficiency make it suitable for applications requiring high performance and low resource usage.

### Overview: Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment Immune is a novel inference-time defense framework designed to enhance the safety of Multi-modal Large Language Models (MLLMs) against jailbreak attacks. By leveraging a safe reward model during decoding, Immune effectively rejects harmful queries while preserving the model's original utility and performance. ### Key Features: - **Safe Reward Model**: Immune dynamically adjusts token probabilities to suppress harmful responses, ensuring ethical outputs while maintaining the model's performance and utility. - **Inference-Time Alignment**: Immune leverages a safety alignment framework during inference, combining model logits with a safety score generated by a reward model. - **Provable Guarantees**: Immune provides rigorous mathematical characterization, offering provable guarantees against jailbreaks. ### Tech Stack: Python ### Why It Matters: Immune addresses a critical safety gap in MLLMs, enhancing their performance and utility while effectively defending against jailbreak attacks. Its approach ensures that the model remains safe and ethical, making it essential for the development of robust and responsible AI systems.

### Overview: This project introduces the concept of Parts of Speech–Grounded Subspaces in Vision-Language Models, focusing on improving the disentanglement of visual representations in vision-language models. The paper proposes a model that separates visual representations by leveraging the association between parts of speech and specific visual modes of variation, such as nouns and appearance. ### Key Features: - **Model Architecture**: The model is designed to learn subspaces that disentangle the visual representations of different parts of speech, such as nouns and adjectives. - **Disentanglement**: By learning these subspaces, the model provides disentangled representations of visual properties, which can be useful for tasks like text-to-image synthesis and improving zero-shot classification. - **Visualization**: The model's performance is validated through both qualitative visualizations and quantitative metrics, including class invariance and improvements in zero-shot classification. ### Tech Stack: - **Programming Languages**: The project is implemented in Jupyter Notebook, utilizing Python for its computational tasks. ### Why It Matters: - **Improved Disentanglement**: The proposed model helps in separating visual representations, making them more interpretable and useful for downstream tasks. - **Enhanced Representation Learning**: By learning subspaces corresponding to specific visual appearances, the model facilitates the selective removal of entire visual themes, enhancing the quality of text-to-image synthesis. ### Repository: - **GitHub Repository**: [James Oldfield/PoS-subspaces](https://github.com/james-oldfield/PoS-subspaces) - **GitHub Stars**: 29 - **GitHub Description**: [NeurIPS

Overview: The `heyshiny` package introduces a new Shiny input called `speechInput()` that allows Shiny apps to translate audio to text. Key features: - Introduces a new Shiny input for voice interaction - Recognizes speech and converts it to text - Compatible with Google Chrome - Works with JavaScript library `annyang` Tech stack: R Why it matters: This package enhances Shiny apps with voice interaction capabilities, making them more interactive and user-friendly. Repository: jcrodriguez1989/heyshiny GitHub stars: 29

Overview: This repository is a reproduction of the original Crystal Graph Convolutional Neural Networks (CGCNN) framework, designed to predict material properties. It includes updates for deprecated components and additional features for smooth operation. Key features: - Training a CGCNN model using a custom dataset - Predicting material properties with a pre-trained CGCNN model - Fine-tuning a pre-trained CGCNN model on a new dataset - Extracting structural features as descriptors for downstream tasks Tech stack: Python Why it matters: Despite its age, CGCNN remains a straightforward and fast deep learning framework that is easy to learn and use, making it valuable for researchers and developers working on material property prediction. Repository: jcwang587/cgcnn2 GitHub stars: 29 GitHub description: Reproduction of CGCNN for predicting material properties