Projects

### Overview: This project is an Android app utilizing Alibaba's MNN (Mobile Neural Network) technology to integrate a full multimodal language model (LLM). It supports various tasks including text-to-text, image-to-text, audio-to-text, and text-to-image generation, showcasing exceptional performance on CPU benchmarking. ### Key Features: - **Multimodal Support:** Enables functionality across diverse tasks, including text-to-text, image-to-text, audio-to-text, and text-to-image generation. - **CPU Inference Optimization:** Demonstrates significant improvements in CPU benchmarking, achieving prefill speed improvements of 8.6x over llama.cpp and 20.5x over fastllm. - **Broad Model Compatibility:** Supports multiple leading model providers, including Qwen, Gemma, Llama (TinyLlama and MobileLLM), Baichuan, Yi, DeepSeek, InternLM, Phi, ReaderLM, and Smolm. - **Privacy First:** Runs entirely on-device, ensuring complete data privacy with no information uploaded to external servers. ### Tech Stack: C++ ### Why it Matters: This app leverages cutting-edge MNN technology to provide a seamless, privacy-focused experience for users, making it ideal for applications requiring high performance and data security.

### Overview: Turn your Google Search Console data into a vector database and analyze search performance with AI. Use Gemini embeddings to analyze your data and gain insights into search performance. This project integrates with Google Search Console API, ChromaDB, and various AI models for comprehensive analysis. ### Key Features: 1. **Extracts**: Retrieves the last 16 months of search performance data from Google Search Console. 2. **Embeds**: Converts queries, pages, and metrics into local ChromaDB vector database using Gemini embeddings. 3. **Analyzes**: Utilizes AI with semantic retrieval (Gemini Flash 3, Grok 4.1, Claude Opus) to analyze your data. 4. **Audits**: Scrapes individual pages and compares them with competitors using Parallel.ai. ### Tech Stack: - **Vector Database**: ChromaDB (local) - **Embeddings**: Gemini (768d, MRL) - **Analysis LLMs**: Gemini Flash 3 (default), Grok 4.1 (2M context window), Claude Opus (200K tokens) ### Why It Matters: - **Comprehensive Analysis**: Seamlessly integrates with Google Search Console, ChromaDB, and AI models for detailed search performance analysis. - **Versatile AI Models**: Utilizes Gemini Flash 3 for quick, free analysis, Grok 4.1 for deep insights, and Claude Opus for nuanced recommendations. - **Customizable Analysis**: Options for different AI models based on specific needs, including parallel page audits and competitor analysis. ### Repository

Overview: DocsFlow is a multi-tenant RAG reference architecture designed for document intelligence, offering isolated team workspaces, hybrid AI search, multi-model AI failover, confidence scoring, and source attribution. It ensures privacy and accuracy in AI-driven document analysis. Key features: - Isolated team workspaces - Hybrid AI search - Multi-model AI failover - Confidence scoring - Source attribution Tech stack: TypeScript Why it matters: DocsFlow provides a secure, efficient, and accurate way to manage document intelligence, ensuring that each team's data is kept separate and that every answer includes the original source. Repository: nicuk/docsflow Primary language: TypeScript GitHub stars: 20 GitHub description: Multi-tenant RAG reference architecture (Next.js, Supabase, Pinecone): per-tenant isolation, hybrid search, cited answers, LLM failover, and a public hallucination benchmark.

Overview: SlicedLlama is a lightweight, AI-powered text generation software that leverages the power of large language models (LLMs) to generate text. It offers a web-based user interface and a server-based alternative, allowing for easy model management and parameter adjustment. Key features: - Text Completion WebUI - OpenAI-compatible API (work in progress) - Layer Slicing: Instant Franken-self-merges, no need to reload the model - Top Logprobs: See the top probabilities for each token Tech stack: Python Why it matters: SlicedLlama simplifies the process of working with large language models, making it accessible and easy to use for developers and researchers alike. Repository: silphendio/sliced_llama GitHub stars: 20 GitHub description: Simple LLM inference server

### Overview: Use Vim as a tool for efficiently designing and running, debugging, and saving Large Language Models (LLMs) prompts. ### Key features: - Prompter.vim transforms the Vim editor into an efficient prompt engineering environment. - It effectively replaces proprietary web playgrounds like Azure OpenAI Service Playground or OpenAI Playground. - Supports a vast list of different LLM providers through the use of LiteLLM. ### Tech stack: Python ### Why it matters: This plugin aims to be a general-purpose replacement for web text completion playgrounds, intended for prompt engineers who want to test and debug natural language prompts.

Overview: This repository is a comprehensive collection of resources and guides for the ComfyUI ecosystem, focusing on video generation, face and portrait mastery, image enhancement, 3D creation, and workflow productivity. It includes the latest models and tools for AI art and custom diffusion pipelines. Key features: - Video Generation Revolution - Face & Portrait Mastery - Image Enhancement & Effects - 3D Generation & Depth - Quick Start & Essential Setup - Latest Models (2024-2025) - Learning Hub - Community Favorites Tech stack: Python Why it matters: This collection provides a one-stop solution for developers looking to leverage the latest advancements in ComfyUI, including cutting-edge video generation, face and portrait mastery, and 3D creation tools. GitHub stars: 20 Repository: umitkacar/awesome-comfyui

### Overview: The repository "Awesome Post-Training Atlas" is a structured, continuously updated atlas of post-training research across various domains including language, reasoning, agents, multimodal models, generative models, and embodied intelligence. It aims to provide insights into the post-training pipeline, feedback mechanisms, and method evolution in these fields. ### Key features: - **Structured Indexing**: Organized by direction, year, month, and paper. - **Curated Papers**: Papers are categorized and ranked based on their relevance. - **Academic Discovery**: Papers are filtered and marked with a discovery candidate marker. - **Method Evolution View**: Includes method families, precedents, and explicit idea surfaces. - **Coverage Matrix**: Shows sparse months and directions, making academic gaps visible. - **Coverage Report**: Tracks the academic-search window, query counts, and unresolved backlog. ### Tech stack: Python ### Why it matters: This repository aims to provide a comprehensive overview of post-training research, helping researchers and practitioners understand the current state and future trends in various AI domains. By focusing on the post-training pipeline, it addresses three key questions: where the work changes, what feedback mechanisms are used, and how it relates to previous and subsequent methods.

Overview: This repository contains the official code release for the paper "diff History for Neural Language Agents" by Piterbarg, Pinto, and Fergus, presented at ICML 2024. The project aims to improve the quality of language model generations for decision-making settings by leveraging low-resource instruction tuning through the use of `diff` history. Key features: - Implementing `diff` history for neural language agents. - Extending model context lengths. - Increasing the length of the history used to train/tune models. - Preprocessing observations in history with the Unix `diff` command. Tech stack: Python Why it matters: This project showcases how small language models can be effectively trained and fine-tuned using low-resource instruction tuning, demonstrating the potential of `diff` history in improving the performance of neural agents in decision-making scenarios.

Overview: This repository contains comprehensive documentation for Weaviate, including details on the database, cloud, and query agent. The documentation is built using Docusaurus 3 and is available in multiple languages, including Python. Key features: - Weaviate Database documentation - Deployment documentation - Cloud documentation - Query Agent documentation Tech stack: Docusaurus 3 Why it matters: This repository serves as a crucial resource for developers looking to understand and utilize the capabilities of Weaviate, including its database, cloud services, and query agents. Repository: weaviate/docs GitHub stars: 20 GitHub description: The documentation repo for Weaviate Database, Cloud, Agents and much more!

Overview: This TypeScript file provides an inference implementation for Llama2-like Transformer models, achieving around 70/25/10 tokens per second for 15/45/110M models on a 15/45/110M model. Key features: - Binary compatibility with the C version - Performance: 70/25/10 tokens per second for 15/45/110M models - Can run the full 7B model with 0.16 tokens per second on a laptop Tech stack: JavaScript Why it matters: This implementation allows developers to understand and implement Llama2 models, providing educational value while showcasing performance and efficiency. Repository: wizzard0/llama2.ts GitHub stars: 20 GitHub description: Llama2 inference in one TypeScript file

### Overview: This repository provides an easy way to train models on the datasets of DCASE Task 1, focusing on Acoustic Scene Classification. The example system won the Judges' Award for DCASE 2024 Challenge Task 1 and is accepted by the DCASE Workshop 2024. ### Key Features: - **Simple YAML Configuration**: All configurations of model, dataset, and training can be done via a YAML file. - **PyTorch Lightning Implementation**: The entire system is implemented using PyTorch Lightning. - **TensorBoard Logging**: Logging is implemented using TensorBoard. - **Task-Specific Techniques**: Includes 3 Spectrogram Extractors, 3 High-performing Backbones, 4 Plug-and-play Data Augmentation Techniques, and 2 Model Compression Methods. ### Tech Stack: PyTorch Lightning ### Why it Matters: This repository simplifies the training process for DCASE Task 1, making it accessible for developers and researchers. The easy-to-use YAML configuration and PyTorch Lightning framework enable efficient model training with minimal setup, making it suitable for both academic and industrial applications.

### Overview: AI Wattch is an open-source browser extension that tracks the energy use and carbon emissions of prompts on AI chatbots like ChatGPT, Gemini, and Claude. Powered by Antarctica’s One Token Model (OTM), it detects the model used, estimates real inference impact, and provides daily summaries and prompt-level tips to enable more carbon-aware AI usage. ### Key Features: - Real-time tracking of energy use and carbon emissions. - Estimates the energy and carbon footprint of AI interactions. - Provides daily summaries and prompt-level tips for better carbon-aware AI usage. - Tracks the model used, such as ChatGPT, Gemini, and Claude. - Estimates the energy and carbon emissions of inference based on the model and prompt. ### Tech Stack: TypeScript ### Why It Matters: - **Transparency:** Makes invisible energy costs visible. - **Efficiency:** Helps people prompt more efficiently and choose more efficient models. - **Privacy:** Prioritizes privacy; no chat text leaves the browser. - **Scalability:** Built for extensibility: multi-model, multi-region, multi-browser roadmap. ### Repository: AIWattch/AI-Wattch ### GitHub Stars: 19 ### GitHub Description: AI Wattch is an open-source browser extension that tracks the energy use and carbon emissions of prompts on AI chatbots like ChatGPT, Gemini and Claude. Powered by Antarctica’s One Token Model (OTM), it detects the model used, estimates real inference impact, and provides daily summaries and prompt-level tips to enable more carbon-aware AI usage.

Overview: This project is an AI-powered YouTube Q&A Assistant that extracts video transcripts and allows users to chat, summarize, and search the content using a RAG pipeline with LangChain, ChromaDB, and MiniLM embeddings. Key features: - Chat with video content - Generate summaries - Extract key takeaways - List topics discussed - Support for multiple languages Tech stack: Streamlit, LangChain, ChromaDB, MiniLM embeddings, OpenRouter LLMs, YouTube Transcript API Why it matters: This project leverages advanced AI techniques to provide accurate, contextually grounded answers to YouTube video questions, enhancing user experience and reducing the risk of misinformation.

### Overview: AI-Inference-Managed-by-AI: A single Go binary designed to manage AI inference on edge devices. It detects hardware, resolves optimal configurations from a YAML knowledge base, deploys inference engines via K3S, and exposes 61 MCP tools for AI Agents to operate everything. ### Key Features: - **Zero-Config Hardware Detection** — Automatically discovers NVIDIA, AMD, Huawei Ascend, Hygon DCU, MetaX, Moore Threads, Houmo, and Apple accelerators, plus CPU and RAM. - **Knowledge-Driven Deployment** — YAML catalog of hardware profiles, engines, models, and partition strategies; no engine-specific code branches. - **Multi-Runtime** — K3S (Pod) for clusters, Docker for single-node containers, Native (exec) for bare-metal inference. - **Recoverable Native-Engine Lifecycle** — Verifies, imports, installs, upgrades, rolls back, and restores native deployments across restarts. - **61 MCP Tools** — Full programmatic control for AI Agents over hardware, models, engines, deployments, fleet, and more. - **Fleet Management** — mDNS-based auto-discovery of LAN peers; remote tool execution across heterogeneous devices. - **Offline-First** — All core functions are designed to work without requiring a network connection. ### Tech Stack: Go ### Why It Matters: This AI-inference management solution simplifies the deployment and management of AI inference on edge devices, making it easier for developers to integrate AI capabilities into their projects without the need for extensive configuration.

### Overview: ChromaDB UI is a modern web interface for ChromaDB, providing a visual experience for browsing collections, inspecting records, running queries, and editing documents, metadata, and embeddings. ### Key features: - **Collection Management**: Browse and manage collections. - **Record Inspection**: Inspect and edit collection metadata. - **Query Execution**: Run semantic text search and embedding-based nearest-neighbor search. - **Vector Editor**: Edit and visualize embeddings. - **Record Import/Export**: Import and export JSON records with or without embeddings. - **Metrics Viewer**: View collection metrics and quality audit findings. - **Activity Log**: Review recent workspace activities. - **Bulk Actions**: Perform bulk operations like delete, metadata patch, and CSV export. - **Full Vector Viewer**: View large embeddings in detail. - **Export Table View**: Export the current table view as CSV. ### Tech stack: Vue.js ### Why it matters: ChromaDB UI simplifies the management and exploration of a ChromaDB instance, making it easier to interact with the database through a user-friendly interface.

Overview: IALab-Suite is a comprehensive tool suite designed for testing Large Language Models (LLMs) using llama.cpp. It offers a modern interface, offline functionality, and supports various models and tools. Key features: - 🔒 Total Privacy: Works offline without sending data to third parties. - 🌈 Modern Interface: Responsive React frontend with a modern design. - 🔌 Modular Architecture: Auto-discoverable tools with dynamic registration. - 🔝 Real-Time Communication: Bidirectional communication via WebSockets. - 🧠 Multiple Models: Supports any GGUF model. - 📚 Integrated RAG: Retrieval Augmented Generation system included. - 🖼️ AI Tools: Complete set of tools for the assistant with automatic discovery. Tech stack: Python, React, Flask, llama.cpp. Why it matters: IALab-Suite simplifies the process of testing and evaluating different LLMs without requiring coding, making it ideal for developers and researchers looking to explore and evaluate AI models in a controlled environment. Repository: BorjaOteroFerreira/IALab-Suite GitHub stars: 19 GitHub description: Tool for test diferents large language models without code.

### Overview: Dakera is a self-hosted AI agent memory server designed specifically for multi-agent pipelines. It provides shared cross-agent memory, session isolation, and importance-ranked recall, all while keeping your data on your infrastructure. Dakera is particularly effective in scenarios requiring long-term context, such as code review or multi-agent pipelines. ### Key Features: - **Cross-Agent Memory**: Enables shared cross-agent memory across multiple AI agents. - **Session Isolation**: Ensures that each agent has its own isolated session, preventing data leakage. - **Importance-Ranked Recall**: Prioritizes recall based on the importance of the data, ensuring relevant information is always accessible. ### Tech Stack: - **JavaScript**: The primary language used for the frontend and backend components. - **Docker Compose**: Used for defining and managing multi-container Docker applications. - **Kubernetes**: Provides orchestration and automation for managing the deployment and scaling of the Dakera server. - **Helm**: Offers a simple and declarative way to manage Kubernetes applications. ### Why It Matters: Dakera is particularly effective in scenarios where long-term context is crucial, such as in multi-agent pipelines or code review. Its ability to provide shared cross-agent memory and prioritize recall based on importance makes it a powerful tool for maintaining and improving the performance of AI agents. The self-hosted nature of Dakera ensures that your data remains on your infrastructure, providing full control and security. ### GitHub Stars: 19 GitHub stars indicate that the project has gained some traction and interest among developers and users. This suggests that Dakera

Overview: DiscovAI Crawl API is a web scraping solution designed for AI tools and vector databases, providing clean HTML, LLM-friendly content, and embeddings from any URL. Key features: - Clean HTML extraction - LLM-friendly Markdown conversion - Ad-free, cookie-free, and dialog-free content - Website screenshots (auto-saved to AWS S3 or Cloudflare R2) - SEO-friendly LLM-generated content - Key information LLM-extracted - Ready-to-use embeddings for vector database integration Tech stack: TypeScript Why it matters: This API streamlines the process of extracting and processing data for AI tools and vector databases, making it easier to integrate clean, SEO-friendly content and key information into these systems. Repository: DiscovAI/DiscovAI-crawl Primary language: TypeScript GitHub stars: 19 GitHub description: 🕷️ DiscovAI Crawl API(🚧 Work in Progress 🚧): A powerful web scraping solution for AI tools and vector databases. Extract clean HTML, generate LLM-friendly content, and create embeddings from any URL.

Overview: This project is a generic model serving implementation using Apache Flink, designed to handle both keyed and non-keyed streams. The implementation leverages dynamically controlled streams to serve models based on data types, making it scalable and adaptable to various use cases. Key features: - ModelServingKeyedJob: A RichCoProcessFunction-based application for serving multiple models for different data types, with scaling based on data type. - ModelServingFlatJob: A RichCoFlatMapFunction-based application for serving a single model for a single data type, with scaling based on input parallelism. - Clientr: A small application for emulating continuous data stream and publishing PMML model definitions. - ModelStateQuery: A small application for external access to a Flink queryable data, demonstrating model state in the context of ModelServingKeyedJob. - Wine Serving: An application leveraging the underlying machinery to build a model serving application. Tech stack: Java Why it matters: This project provides a flexible and scalable solution for model serving, making it suitable for various applications that need to handle both keyed and non-keyed streams, and for serving models based on data types.

Overview: This repository contains an explanation of how the RAG (Retrieval-Augmented Generation) architecture works and provides some examples, utilizing PostgreSQL, Ollama, and Go. Key features: - RAG architecture explanation - Google's AI Overview example - Seed diagram for data storage - Request diagram for user interaction Tech stack: Go Why it matters: RAG is a flexible alternative to fine-tuning large language models, allowing for more flexible content generation by leveraging pre-trained models and a retrieval mechanism. It is particularly useful for scenarios where the data being fetched from search results is constantly changing, such as in Google's AI Overview tool, where it is impossible to train the model on new data each time someone makes an update.