Projects

### Overview: In today's fast-paced environment, customers expect quick, accurate, and personalized responses. Managing large volumes of emails, categorizing them, crafting appropriate replies, and ensuring quality consumes significant time and resources, often leading to delays or errors, which can harm customer satisfaction. ### Key Features: - **Email Inbox Management with AI Agents**: Continuously monitors the agency's Gmail inbox, categorizes emails into predefined categories, and automatically handles irrelevant emails. - **AI Response Generation**: Quickly drafts emails for customer complaints and feedback using Langgraph, and uses RAG to answer product/service-related questions accurately. - **Quality Assurance with AI**: Ensures every response meets high standards before reaching the client through quality and formatting checks. ### Tech Stack: Python ### Why It Matters: This system enhances customer communication for businesses by automating customer support emails, improving efficiency, and ensuring high-quality responses.

### Overview: This project presents a minimal hardware-software architecture for giving large language models a closed-loop physical embodiment. The system includes an ESP32 microcontroller that senses its surroundings, including environment, light, motion, and sound, as well as direct contact through touch pressure and skin temperature. Optionally, it can classify experimental smells using a gas sensor, with three output channels for haptic feedback, an OLED display, and a piezo buzzer. The system's self-perception loops are characterized through a series of experiments, providing a detailed validation dataset for the model's performance. ### Key Features: - **ESP32 Microcontroller**: Sensing and control hardware for the embodiment. - **Self-Perception Loops**: Model can hear and feel itself through haptic feedback, OLED display, and buzzer. - **Validation Dataset**: 660-trial validation dataset for the system's performance. - **GitHub Repository**: Stars: 281, Description: "A minimal hardware-software architecture giving large language models a closed-loop physical embodiment with self-perception loops." ### Tech Stack: - **Node.js**: Bridge service implementation. - **Arduino**: Firmware for the ESP32 microcontroller. ### Why It Matters: This project provides a practical demonstration of how to give large language models a physical embodiment, enabling them to perceive and interact with their environment in a more tangible way. The self-perception loops allow the model to learn from its own actions and experiences, enhancing its understanding and adaptability.

### Overview: CodeQA is a powerful code search and query system that utilizes natural language processing to explore codebases. It leverages LanceDB and OpenAI models to provide context-aware answers and improved performance. ### Key Features: - **Optimized Token Management**: Reduced HYDE max tokens to 400 and HYDE-v2 to 768, resulting in ~5-10 seconds saved. - **Enhanced Context Processing**: Implemented SambaNova Llama 3.1 8b for context filtering and changed gpt4o to SambaNova Llama 3.1-70b for chat responses, improving context relevance and processing speed, resulting in ~20 seconds saved. - **Performance Improvement**: The optimized branch runs 2.5x faster than the previous branch, achieving 10-20 seconds range under 20 seconds. ### Tech Stack: - Python ### Why It Matters: CodeQA simplifies the process of understanding complex codebases by providing efficient and context-aware answers. It optimizes performance and reduces the time required to explore and analyze code, making it an invaluable tool for developers and engineers. ### Repository: - Repository: [sankalp1999/code_qa](https://github.com/sankalp1999/code_qa) - GitHub Stars: 281 - GitHub Description: RAG on codebases using treesitter and LanceDB - Primary Language: Python - GitHub Description: RAG on codebases using treesitter and LanceDB

Overview: Qwen2.5 Sex 是基于 Qwen2.5-1.5B-Instruct 微调的模型，专为处理成人文学和敏感数据集设计，尤其擅长中文文本的处理。 Key features: - 处理成人文学和敏感数据集 - 专为中文文本优化 - 遵守当地法律法规 Tech stack: Python Why it matters: 由于数据集以中文为主，Qwen2.5 Sex 模型在处理中文文本时表现更佳，适用于研究和测试。 Repository: ystemsr/Qwen2.5-Sex Primary language: Python GitHub stars: 280 GitHub description: A fine-tuned model from Qwen2.5-1.5B-Instruct, capable of handling sensitive topics. / 从 Qwen2.5-1.5B-Instruct 微调，主要擅长处理敏感话题 README excerpt: [English](README.en.md)

### Overview: TriForce is a cutting-edge framework designed to accelerate long sequence generation by leveraging hierarchical speculative decoding and lossless acceleration techniques. This project is part of the 2024 COLM competition and focuses on optimizing the performance of large language models, specifically long-context LLMs, by offloading computations to accelerate the generation process. ### Key Features: - **Hierarchical Speculative Decoding**: A novel decoding strategy that allows for speculative decoding steps to be performed ahead of the actual generation, significantly reducing the generation latency. - **Lossless Acceleration**: TriForce achieves this by offloading computations to offloading GPUs, utilizing the GPU's parallel processing capabilities to speed up the generation process without compromising the quality of the output. - **On-Chip Results**: Demonstrates the effectiveness of offloading by generating samples using a LWM-Text-Chat-128K model with offloading on 2x RTX 4090s, showcasing a 2.2x speedup. ### Tech Stack: The framework is built using Python and leverages the Flash Attention library for efficient computation. It supports both on-chip and offloading settings, with the latter being particularly effective for large models. ### Why It Matters: TriForce is significant because it provides a practical solution for accelerating long sequence generation tasks, which are crucial in applications such as text completion, dialogue systems, and more. By reducing the generation latency and improving the overall performance of large language models, TriForce enables faster and more efficient processing of long sequences, making it a valuable tool for developers and

Overview: Phi-3-MLX is a comprehensive framework for AI tasks on Apple Silicon, combining the Phi-3-Vision multimodal model and the Phi-3-Mini-128K language model, optimized for performance and ease of use. Key features: - Integration with Phi-3.5-Vision and Phi-3.5-mini models - Optimized performance on Apple Silicon using MLX - Batched generation for multiple prompts - Flexible agent system for various AI tasks - Custom toolchains for specialized workflows - Model quantization for improved efficiency - LoRA fine-tuning capabilities - API integration for extended functionality Tech stack: Jupyter Notebook Why it matters: Phi-3-MLX simplifies the integration of advanced AI models and tools for developers and researchers, making it easier to create and deploy AI solutions on Apple Silicon platforms. Repository: JosefAlbers/phi Primary language: Jupyter Notebook GitHub stars: 279 GitHub description: Phi-3, -3.5, and -4 for Mac: Locally-run Vision and Language Models for Apple Silicon

### Overview: Build your own LLM-native WIKI (knowledge library). Search, extract, summarize, Q&A with contextual RAG, layered knowledge graph, and reinforced memory. Empowered by Claude subagents and CodeAct pipeline, the library is locally stored and can be used offline. Authored by the latest Claude in-process or via the Claude Code CLI as a subprocess, skills are evaluated by a deterministic rubric panel and then reviewed by human reviewers before being added to the library. ### Key features: - **Agentic Knowledge Graph**: A `deepagents` agent that saves passages, extracts entities and typed relations, canonicalizes duplicates, nests topics, and writes synthesis pages. - **Contextual RAG**: Anthropic-style contextual retrieval over a two-layer HNSW index (section summaries + chunks), with an LLM re-ranker (precision) or document-aware MMR (multi-doc recall). - **Memory**: A cross-session store recalled before every answer and written back after (observations, 👍/👎, corrections that supersede stale notes); it improves from use, not just ingestion. - **Agent Skills**: Turn selected context into a reusable, evaluated skill via a sub-agent pipeline (understand → analyze → author → eval → gate → refine). Authored by the latest Claude in-process or via the Claude Code CLI as a subprocess, skills are scored by a deterministic + rubric panel, and gated behind human review before joining the library. - **MCP, both directions**: Connect agents to external MCP servers (Supabase, GitHub, ...) and

### Overview: GeoVista is an official repository for "GeoVista: Web-Augmented Agentic Visual Reasoning for Geolocalization," designed for researchers and developers interested in geolocalization and visual reasoning. This project aims to provide a comprehensive framework for geolocalization tasks using web-augmented agentic visual reasoning techniques. ### Key Features: - **Web-Augmented Agentic Visual Reasoning**: Utilizes web-based search APIs to enhance the accuracy of geolocalization tasks. - **Global Coverage (GC)**: Ensures that images cover diverse regions worldwide. - **Reasonable Localizability (RC)**: Filters out non-localizable or trivially localizable images to maintain meaningful difficulty. - **High Resolution (HR)**: Requires all images to have at least 1 million pixels for reliable visual clue extraction. - **Data Variety (DV)**: Includes multiple image modalities and sources to test generalization. - **Nuanced Evaluation (NE)**: Checks for precise coordinates to enable fine-grained distance-based metrics. ### Tech Stack: - **Python**: The primary programming language used for development and experimentation. ### Why It Matters: - **Enhanced Geolocalization Accuracy**: Utilizes web-based search APIs to improve the accuracy of geolocalization tasks. - **Robustness and Scalability**: Ensures that the model can handle a wide range of geolocalization challenges. - **Versatility**: The dataset and benchmarks are designed to evaluate the general geolocalization ability of agentic models across various dimensions. ### Repository:

Overview: WARC-GPT is an experimental retrieval augmented generation pipeline designed for exploring Web Archive Collections using AI. Key features: - Retrieval Augmented Generation pipeline for WARC files - Highly customizable, can interact with multiple LLMs, providers, and embedding models - REST API - Web UI - Embeddings visualization Tech stack: Python Why it matters: WARC-GPT provides a powerful tool for researchers and enthusiasts to analyze and explore Web Archive Collections using AI, making it easier to extract meaningful insights and generate contextually relevant content.

Overview: Aimybox is an open-source, embeddable custom voice assistant built on the Aimybox SDK, designed to be integrated into Android applications for enhanced voice interaction. Key features: - Modular and independent from speech-to-text, text-to-speech, and NLU vendors - Ready-to-use UI components for quick app development - Speech-to-text and text-to-speech implementations like Google Platform Speechkit, Google Cloud Speechkit, Houndify, or Snowboy wake word trigger - Supports any NLU providers such as Aimylogic, Rasa, or Dialogflow - Fully customizable and extendable - Open source under Apache 2.0, written in pure Kotlin - Embeddable into any Android application or device Tech stack: Kotlin Why it matters: Aimybox simplifies the development of voice-activated applications, providing a robust and customizable solution that can interact with various speech-to-text, text-to-speech, and NLU services, making it ideal for enhancing user experience in Android applications. Repository: just-ai/aimybox-android-assistant GitHub stars: 280 GitHub description: Embeddable custom voice assistant for Android applications

### Overview: Pinecone TypeScript SDK is the official client for the Pinecone vector database, providing full-text and vector search capabilities with TypeScript support. ### Key features: - **Document & Full-Text Search**: Store and search documents with text queries. - **Document Operations**: CRUD operations for documents within namespaces. - **Vector Operations**: Store, query, and manage high-dimensional vectors with metadata. - **Index Management**: Create and manage indexes, both serverless and pod-based. - **Integrated Embedding**: Embed documents with hosted models for semantic search. - **Inference**: Hosted embedding and reranking models. - **Pinecone Assistant**: AI assistants powered by vector database capabilities. - **Type Safety**: Full TypeScript support with generic type parameters for metadata. ### Tech stack: TypeScript/Node ### Why it matters: Pinecone is a powerful vector database for storing and searching high-dimensional data, making it ideal for applications requiring semantic search, recommendation systems, and more. Repository: pinecone-io/pinecone-ts-client

### Overview: This project introduces a novel paradigm for reasoning that leverages both visual and linguistic information to bridge the "Reference Gap" in complex reasoning tasks. By integrating spatial markers directly into the reasoning trajectory, it addresses the challenge of natural language ambiguity and logical collapse, providing a more grounded and precise approach to understanding and reasoning about complex structures. ### Key Features: - **Spatial Reasoning Integration**: The model learns to point while it reasons, anchoring abstract concepts to concrete spatial coordinates. - **Minimal Units of Thought**: Spatial markers (points and bounding boxes) are integrated as minimal units of thought, ensuring precise and logical reasoning. - **Grounded Task Reasoning**: This approach helps in understanding and reasoning about complex structures by grounding linguistic concepts in physical coordinates. ### Tech Stack: - **Makefile**: Used for project management and automation. ### Why It Matters: - This paradigm shift addresses the limitations of current large language models in complex reasoning tasks, particularly in understanding and reasoning about complex structures. - By integrating spatial markers directly into the reasoning process, it provides a more grounded and precise approach to understanding and reasoning about complex structures. ### Repository: The repository is archived and can be accessed at [AILNTX/Thinking-with-Visual-Primitives](https://github.com/AILNTX/Thinking-with-Visual-Primitives). ### GitHub Stars: - **Stars**: 279 ### Primary Language: - **Primary Language**: Makefile ### Why It Matters: - This paradigm shift addresses the limitations of current large language models in complex reasoning tasks, particularly

Overview: Uteke is a local-first memory engine for AI agents, designed to store, recall, and search memories using semantic embeddings. It is a single Rust binary that requires no configuration and works fully offline. Key features: - Local-first memory storage and retrieval - Works with Claude, Cursor, Copilot, and MCP-compatible agents - 98.2% recall on LongMemEval-S, ~45 ms warm query, 0 LLM tokens per query - Zero config, no need for cloud or API keys - Downloads the embedding model (~200MB) on first run Tech stack: Rust Why it matters: Uteke provides a reliable, efficient memory solution for AI agents, ensuring that their knowledge never leaves their machine. It is designed to work seamlessly with various AI agents and offers high recall and low latency, making it a valuable tool for AI development and deployment.

Overview: This repository contains the official code and data for a NeurIPS 2023 paper on "Monitor-Guided Decoding of Code LMs with Static Analysis of Repository Context." It introduces a method called Monitor-Guided Decoding (MGD) that uses static analysis to guide the decoding of language models, improving compilation rates and matching ground truth at various granularities. Key features: - Monitor-Guided Decoding (MGD) for code generation using Language Models. - Datasets: PragmaticCode and DotPrompts. - Evaluation scripts for metrics like Compilation Rate (CR), Next-Identifier Match (NIM), Identifier-Sequence Match (ISM), and Prefix Match (PM). - `multilspy`: A cross-platform library for querying various static analyses from language servers. Tech stack: Python. Why it matters: MGD improves the compilation rate of code generated by language models across different scales, leading to a 19-25% increase in compilation rate without any training/fine-tuning required.

### Overview: This project showcases various chatbot implementations using Langchain and Streamlit, providing a comprehensive guide on how to integrate language models into applications. ### Key Features: - **Basic Chatbot** - **Context-Aware Chatbot** - **Chatbot with Internet Access** - **Chat with Documents** - **Chat with SQL Database** - **Chat with Websites** ### Tech Stack: Python ### Why It Matters: This project serves as a practical example of how Langchain can be used to create sophisticated chatbot applications, highlighting the integration of language models with Streamlit for easy deployment and interaction.

### Overview: FinSight is a multi-agent research system designed to automate the entire financial research process—from data collection and analysis to generating publication-ready reports with professional charts and deep insights. It aims to provide a one-click solution for financial research, rivaling human experts in factual accuracy, analytical depth, and presentation quality. ### Key Features: - **📊 Professional-Grade Report Generation**: Generates 20,000+ word financial reports with high factual accuracy, analytical depth, and presentation quality. - **🤖 Programmable Code Agent (CAVM)**: Implements the *Code Agent with Variable Memory* architecture, allowing agents to operate in a unified variable space and execute Python code for dynamic analysis. - **📈 Automated Charting with VLM Feedback**: Corrects missing legends, wrong scales, or low information density through visual feedback loops. - **🔍 Deep Research with Evidence Tracing**: Provides a transparent *Chain-of-Analysis* with strict citations linking back to original data sources, ensuring high textual faithfulness and verifiable insights. - **⚡ Comprehensive Market Intelligence**: Access real-time stock quotes, financial statements, and macro indicators across A-share, HK, and US markets, powered by a robust Python-based tool ecosystem. - **🇺🇸 US Market Support**: Full US equity coverage via yfinance: company profiles, OHLCV price data, income statements, balance sheets, cash-flow statements, and institutional holder data. US macro indicators via FRED: CPI, GDP, unemployment, interest rates. Market indices: S&P 500, DJIA, NASDAQ. ### Tech

### Overview: SecurityClaw is a modular, skill-based autonomous Security Operations Center (SOC) agent designed to monitor OpenSearch/Elasticsearch data, build RAG-based behavioral memory, and validate real-time anomalies using LLMs. This tool is built with Python and supports versioned launchers for local and minimal-container deployments, making it portable and easy to set up. ### Key Features: - **Skill Modularity**: Capabilities are isolated in folders with `logic.py` (Python) and `instruction.md` (LLM guidance). - **Heartbeat Loop**: A cron-like scheduler that watches for anomalies and builds RAG-based behavioral memory. - **Provider Agnostic**: OpenSearch ↔ Elasticsearch configuration allows seamless swapping. - **RAG-Based Memory**: Vector embeddings stored in OpenSearch for context-aware threat analysis. - **LangGraph Orchestration**: Multi-step DECIDE→EXECUTE→EVALUATE supervisor loop implemented as a `StateGraph`. - **Manifest-Grounded Planning**: Supervisor planning and retry steps are repaired against manifest contracts. - **Working Memory**: Interactive chat working memory checkpointed in `data/conversations.db`. - **Conversation-based Investigations**: Investigate threats through an interactive chat interface. - **Web Interface**: Modern React-based UI for chat, memory visualization, and skill dispatch. ### Tech Stack: - Python ### Why It Matters: SecurityClaw simplifies the process of building and managing a SOC, making it easier to monitor, analyze, and respond to security threats. Its modular design and use of LLMs for threat validation make it highly effective

Overview: SmartLLMLoader is a lightweight Python package that efficiently transforms documents into LLM-ready chunks, streamlining the preprocessing of documents for AI applications. Key features: - Converts documents to clean markdown - Built-in OCR for scanned documents and images - Smart, context-aware text chunking - Seamless integration with LangChain and LlamaIndex - Ready for vector stores and LLM ingestion Tech stack: Python Why it matters: SmartLLMLoader simplifies the process of preparing documents for AI applications, allowing developers to focus on creating exceptional AI-based solutions without the hassle of manual preprocessing. It offers a more efficient and cost-effective approach compared to traditional methods, particularly when paired with advanced LLM models like Google's Gemini Flash.

### Overview: This repository serves as a benchmark for evaluating complex visual document retrieval systems using the ViDoRe v3 dataset. It includes a community results repository and guidelines for submitting pipeline evaluation results to compare different approaches and track progress on the ViDoRe v3 datasets. ### Key features: - Multi-stage retrieval systems - Hybrid approaches - Custom preprocessing pipelines - Arbitrary retrieval logic ### Tech stack: Python ### Why it matters: This repository provides a standardized way to evaluate and compare complex visual document retrieval systems, making it easier for researchers and practitioners to share and compare their work on the ViDoRe v3 dataset.

Overview: SwarmZero is a Python SDK designed for building and managing AI agents and swarms of agents. It simplifies the process of integrating various LLM providers into your projects. Key features: - Easy integration with multiple LLM providers - Support for AI agents and swarms - Configuration through environment variables or configuration files - SDKContext for managing configurations and resources Tech stack: Python Why it matters: SwarmZero simplifies the integration and management of AI agents and swarms, making it easier to deploy and scale AI solutions using various LLM providers. Repository: swarmzero/swarmzero GitHub stars: 278 GitHub description: SwarmZero's SDK for building AI agents, swarms of agents and much more. README excerpt: [](https://badge.fury.io/py/swarmzero) [](https://github.com/swarmzero/swarmzero/actions/workflows/test.yml) [](https://pypi.org/project/swarmzero/) [](https://opensource.org/licenses/Apache-2.0) [](https://