Projects

### Overview: DeepSeek-V4.1-Flash is a high-performance model serving system designed for eight NVIDIA DGX Spark nodes. It utilizes tensor parallelism and DSpark speculative decoding, achieving 1 million context serving on a 200GB RoCE fabric. This system is a four-node recipe scaled to eight nodes, with the companion job to the original glm-5.3-uncensored-8x-dgx-spark. The model is optimized for vision and tool calling, with measurements and a detailed boot log available. ### Key features: - **Tensor Parallelism**: Utilizes 8 nodes for tensor parallelism. - **DSpark Speculative Decoding**: Enhances decoding efficiency. - **CUDA Graphs**: Optimized for faster inference. - **Vision and Tool Calling**: Integrated for improved performance. - **Context Serving**: Achieves 1 million context serving on 8x DGX Spark nodes. ### Tech stack: Shell-based system optimized for performance and efficiency. ### Why it matters: This system is designed for high-performance computing environments, leveraging advanced GPU technology to achieve significant improvements in processing speed and resource utilization. It is particularly useful for applications requiring large-scale data processing and real-time analysis, such as in AI research and development. ### GitHub stars: 9 stars ### Repository: im0xMagnus/deepseek-v4.1-flash-uncensored-8x-dgx-spark

Overview: This project demonstrates how to build an AI-powered dinner menu generator using Inngest and Weaviate, utilizing RAG (Retrieval-Augmented Generation) to create personalized dinner menus based on cuisine preferences and dietary requirements. Key features: - Cuisine-based recipe suggestions - Dietary restrictions and allergies handling - Wine pairing recommendations - Detailed shopping list generation Tech stack: TypeScript Why it matters: This project showcases the integration of AI and machine learning with traditional database and search technology, providing a practical application of these technologies in a real-world scenario. Repository: inngest/weaviate-agentic-workflow GitHub stars: 9

### Overview: This plugin allows you to semantically search through text blocks extracted from Optical Character Recognition (OCR) in your dataset using Qdrant, Sentence Transformers, and FiftyOne. ### Key features: - Create a Qdrant index for text fields within detections labels. - Use the `semantically_search_documents` operator to search through your dataset for semantically similar text. - Supports multiple indexes for multiple detections with text blocks. - Compatible with the PyTesseract OCR plugin for text extraction. ### Tech stack: Python ### Why it matters: This plugin enhances the semantic search capabilities of datasets, making it easier to find relevant text within your OCR-documented content.

Overview: Lorebrain is a local-first AI knowledge layer that utilizes vector embeddings and retrieval for efficient knowledge management. Key features: - Local-first storage - Vector embeddings for semantic similarity search - Integration with Go ecosystem Tech stack: Go Why it matters: Lorebrain simplifies AI knowledge management by providing a fast and efficient way to store, retrieve, and search for documents using vector embeddings, making it ideal for applications requiring local-first data management and real-time querying. Repository: jmerelnyc/lorebrain GitHub stars: 9 GitHub description: Local-first AI knowledge layer with vector embeddings and retrieval

Overview: rag-forge is a framework designed to fine-tune Large Language Models (LLMs) specifically for retrieval-augmented generation tasks, addressing the issue of poor domain-specific knowledge retrieval by injecting retrieved context into the model's training process. Key features: - Fine-tuning on RAG-specific tasks to teach the model how to synthesize retrieved documents, cite sources accurately, and distinguish between relevant and irrelevant passages. - Automated data preparation, training loop, and evaluation pipeline for specialized fine-tuning. - Supports both encoder-decoder and decoder-only architectures. - Integrates with popular retrieval backends like FAISS, Elasticsearch, and Pinecone. Tech stack: Python Why it matters: This framework is crucial for developers looking to enhance the capabilities of their LLMs by improving their ability to handle domain-specific knowledge retrieval, making them more effective in tasks requiring context-aware reasoning. Repository: jmerelnyc/rag-forge GitHub stars: 9 GitHub description: Framework for fine-tuning LLMs on retrieval-augmented generation tasks

Overview: Claude Code Memory Cache is a persistent, token-efficient memory system for Claude Code, designed to remember your past sessions and answer questions about your codebase. It includes five memory layers: vector memory, fact files, Obsidian vault, code graphs, and brain files. Key features: - Persistent memory layers that survive session ends. - Token-efficient search and retrieval. - Local, no-cloud, no-API-key implementation. - Five memory layers tailored to different types of questions. Tech stack: Python. Why it matters: This system ensures continuity across sessions, reduces token costs, and provides a more efficient way to remember and recall information about your codebase. Repository: jushayden/claude-code-memory-cache GitHub stars: 9

### Overview: AdaMSS (Adaptive Multi-Subspace Approach) is a parameter-efficient fine-tuning method designed to improve the expressiveness-efficiency trade-off in fine-tuning of large models. ### Key features: - **Parameter-Efficient Fine-Tuning (PEFT)**: AdaMSS optimizes the fine-tuning process by adapting the number of parameters used for each layer, enhancing model expressiveness without significant performance degradation. - **Adaptive Multi-Subspace Approach**: The method decomposes the model into multiple subspaces, allowing for efficient parameter sharing and fine-tuning. ### Tech stack: Python ### Why it matters: AdaMSS provides a significant improvement in parameter efficiency, making it particularly useful for fine-tuning large models without compromising on performance. The adaptive approach ensures that the model retains its expressiveness while being more efficient in terms of parameters used.

Overview: PromptMill is a self-contained local-first AI prompt generator that uses uncensored Dolphin LLMs to generate optimized prompts for video, image, audio, 3D, and creative AI tools. It runs without API keys or cloud dependencies, using GPU VRAM to scale models. Key features: - Smart GPU Detection - 8 LLM Tiers (1B to 24B parameters) - 169 Specialized Targets (video, image, audio, 3D, creative) - Category Filter and Per-Category Examples - Session History - REST API for generating prompts Tech stack: Python, Gradio, Docker Why it matters: It's a self-contained, GPU-based AI prompt generator that ensures local performance and uncensored model usage, making it ideal for developers and content creators looking to integrate AI into their workflows without relying on external APIs or cloud services.

Overview: Ventu is a Python library designed to make it easy to serve deep learning models, focusing on simplicity and ease of use. Key features: - Only need to implement Model (`preprocess`, `postprocess`, `inference`, or `batch_inference`) - Request & response data validation using Pydantic - API document using SpecTree (when run with `run_http`) - Backend service using Falcon, supports both JSON and msgpack - Dynamic batching with Unix domain socket or TCP - Errors in one request won't affect others in the same batch - Load balancing - Support all the runtime - Health check - Monitoring metrics (Prometheus) - Inference warm-up Tech stack: Python Why it matters: Ventu simplifies the process of serving deep learning models, making it accessible and easy to use for developers.

Overview: DeepRecall is a recursive reasoning engine designed for AI agents and vector databases, powered by MIT's RLM. Key features: - Integrates with various vector databases and LLM frameworks - Allows LLMs to search, reason, search again, and repeat - Supports multiple backend options (Chroma, Milvus, Qdrant, Pinecone, FAISS, Redis, OTel) - Provides rich console output for verbose debugging - Supports multiple LLMs and models - Offers a cost tracking feature for real USD cost extraction Tech stack: Python Why it matters: DeepRecall enables LLMs to perform recursive reasoning and search operations, making it a powerful tool for AI agents and vector databases. Repository: kothapavan1998/deeprecall GitHub stars: 9

### Overview: BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained model that has revolutionized natural language processing. This repository collects various sources, visualizations, and code examples of BERT, providing a comprehensive understanding of the model's architecture and applications. ### Key Features: - **BERT Model Architecture**: Detailed explanations of the Transformer model and its components. - **Tokenization Techniques**: Insights into how BERT tokenizes input text. - **Fine-Tuning Examples**: Practical demonstrations of how to fine-tune BERT for specific tasks like classification and sequence classification. - **BERT Implementation**: Code examples in Jupyter Notebooks for hands-on learning. ### Tech Stack: - Jupyter Notebooks ### Why It Matters: BERT has significantly improved the performance of machine learning models in natural language processing tasks. By understanding BERT, developers can enhance their models and achieve better results in tasks such as sentiment analysis, language translation, and text classification. ### Repository: kryvokhyzha/how-to-understand-bert - **GitHub Stars**: 9 - **Primary Language**: Jupyter Notebook - **GitHub Description**: A collection of resources, visualizations, and code examples of BERT, including attention mechanisms, transformers, tokenization, and fine-tuning. This repository aims to provide a comprehensive understanding of BERT for developers.

### Overview: AI Engineer is a repository containing runnable AI engineering modules, covering a wide range of functionalities such as agents, MCP, memory, RAG, prompting, deployment, and training. Each module is accessible through its own folder, making it easy to start with the most relevant content. ### Key Features: - **Agents**: Implementing agents in Python, TypeScript, and LangGraph. - **Model Context Protocol (MCP)**: Servers and clients for MCP in Python and Docker. - **Memory and Retrieval**: Short-term and long-term memory for LangGraph agents on Redis. - **Prompting and Context Engineering**: Official prompting guides and context engineering. - **Deployments**: AWS, GKE, and Kubernetes deployment options. - **Training**: Fine-tuning models with Hugging Face. - **Contributing**: Guide for developers to contribute to the project. ### Tech Stack: Python ### Why It Matters: - **18 Modules**: Accessible through 8 areas, each with a level tag for easy entry. - **Real-World Applications**: Includes modules like agents, MCP, memory, and deployment, ensuring practical use cases. - **Practical Guides**: Official prompts and context engineering guides, making it easier to understand and implement. - **Community Involvement**: Contributing to the project can help improve the overall quality and functionality of the modules.

Overview: SkillLoop is a governed learning layer designed to help AI agents learn from their execution traces, ensuring that the system can safely and effectively improve without silently mutating its global state. Key features: - Normalizes execution traces - Evaluates traces for evidence and context - Proposes memory and skill artifacts - Gated through human review - Prepares datasets and training configs for future model improvement Tech stack: Python Why it matters: SkillLoop provides a structured way for AI agents to learn from their actions, ensuring that the system can evolve safely and improve its performance without altering the agent's global state. Repository: lamenting-hawthorn/SkillLoop GitHub stars: 9 GitHub description: A governed learning layer for AI agents — turns execution traces into reviewed memories, reusable skills, and evidence-backed training data, without silently mutating the agent's global state.

### Overview: This book provides a comprehensive design-first exploration of AI as an infrastructure, focusing on the lifecycle of capabilities from raw compute to deployed and governed behavior. It covers history, design decisions, and foundational theories, aiming to bridge the gap between systems and machine learning. ### Key features: - **Design-first Approach:** Treats AI as an infrastructure, explaining design decisions and trade-offs. - **Continuous Arc:** Tracks the lifecycle of a capability from raw compute to a deployed and governed behavior. - **Interactive Figures and Runnable Cells:** Step through pipelines, optimize operations, and evaluate models interactively. ### Tech stack: TypeScript ### Why it matters: This book offers a unique perspective on AI as infrastructure, providing a foundational understanding for software engineers transitioning into machine learning. It emphasizes the importance of understanding the underlying design decisions and trade-offs that shape AI capabilities, making it a valuable resource for anyone involved in AI development.

### Overview: This repository contains a curated collection of academic papers focused on using Large Language Models (LLMs) for software vulnerability detection, analysis, and remediation. The papers cover various aspects such as prompt engineering, knowledge-augmented reasoning, fine-tuning, evaluation, code generation, code understanding, and static-analysis query/rule synthesis. ### Key Features: - **Prompt Engineering & Knowledge-Augmented (pe)**: Papers discussing the use of LLMs for software vulnerability detection, classification, and repair. - **Agent-based Systems (agent)**: Research on autonomous agents, multi-agent systems, and self-improving agents. - **Fine-Tuning (ft)**: Papers on fine-tuning LLMs on vulnerable code for better detection accuracy. - **Evaluation & Benchmarks (ev)**: Papers on benchmarking and evaluation frameworks for LLMs in software security. - **Code Generation (gen)**: Research on using LLMs for secure code generation. - **Code Understanding (understand)**: Papers on using LLMs to understand and analyze software code. - **Surveys & Overviews (st)**: Papers providing surveys and overviews on LLMs in software security. - **Others**: Miscellaneous approaches not directly related to vulnerability detection. ### Tech Stack: The repository uses a variety of technologies and frameworks, including LLMs, static analysis tools, and various research papers and methodologies. ### Why It Matters: LLMs have the potential to revolutionize the field of software security by providing a more efficient and accurate way to detect vulnerabilities. The papers in

Overview: LM-WebUI is a unified workspace for running and managing AI workloads, combining local inference, multimodal chat, retrieval, image generation, and agent workflows in one privacy-first interface. Key features: - Runs models, inference runtimes, agents, tools, retrieval, and usage from one workspace across local and cloud environments. - Manages local inference engines, model formats, and configuration. - Supports Smart-Modality™ for intent routing, reducing unnecessary processing and context. - Provides seamless integration with cloud APIs and local data storage. - Offers easy setup, reproducibility, and infrastructure-level control with open-source code. Tech stack: Python. Why it matters: LM-WebUI simplifies the execution layer of AI workloads, making local AI inference easier to run and use, and provides a unified workspace for managing models, runtimes, and agents. Repository: lm-webui/lm-webui GitHub stars: 9 GitHub description: Sovereign Local AI Workspace & LLM Runtime Manager. Easily Run GGUF, MLX, OpenAI, Gemini, etc. Under One Unified Interface ✨

### Overview: 本项目专注于三种流行的微调框架：LLaMA Factory、trl 和 unsloth，旨在展示如何使用这些框架对大模型进行监督微调。通过实际操作，本文旨在帮助开发者快速理解和应用这些技术。 ### Key features: - 使用医疗问答数据集对Qwen模型进行监督微调。 - 介绍三种微调框架：LLaMA Factory、trl 和 unsloth。 - 比较三种框架的优缺点，特别是unsloth因其高效性而备受推荐。 ### Tech stack: 使用Jupyter Notebook进行实验和代码展示。 ### Why it matters: 在GPU服务器上认真微调时，LLaMA Factory和trl可能需要几个小时甚至更长时间，而unsloth只需几分钟，大大提高了微调效率。尽管如此，不同框架的量化方法和LoRA参数不同，因此需要根据具体情况进行选择。

Overview: DocQA Stream is a self-hosted PDF Q&A API that uses FastAPI, Weaviate, PostgreSQL, Azure OpenAI, OpenAI-compatible models, and Vertex AI for processing and answering questions. It provides streaming responses with citation metadata and supports various backend options for vector stores and model providers. Key features: - Streaming Q&A endpoint - Document upload and indexing - Citation metadata included in answers - Minimal browser demo - Support for multiple backend options Tech stack: Python, FastAPI, Weaviate, PostgreSQL, Azure OpenAI, OpenAI-compatible models, Vertex AI, and local embeddings. Why it matters: DocQA Stream simplifies the process of building a self-hosted Q&A API, making it accessible for developers and businesses looking to integrate AI-powered knowledge retrieval into their applications without the need for cloud-based services.

### Overview: A semantic font search system that utilizes natural language processing and vector embeddings to find fonts based on descriptions, characteristics, and use cases. This project was a significant achievement in HackSC '24, securing 1st position in the Artists track. ### Key Features: - **Natural Language Font Search:** Enables users to search for fonts using textual descriptions. - **Semantic Understanding of Font Characteristics:** Analyzes and categorizes font features like boldness, serif vs sans-serif, etc. - **Vector-Based Similarity Matching:** Uses FAISS for efficient similarity searches. - **Real-time Preview of Fonts:** Displays font previews as users type. - **Dynamic Card-Based Results Interface:** Provides a card-based interface for easy navigation. - **Persistent Search Index:** Ensures that search results are saved and can be accessed later. - **RESTful API Interface:** Allows for easy integration and access to the search system. ### Tech Stack: - **Frontend:** Jupyter Notebook - **Backend:** Flask - **Search Engine:** Sentence Transformers, FAISS - **Architecture:** Mermaid diagram showing frontend, backend, and search system components. ### Why It Matters: This project showcases the integration of NLP, vector embeddings, and efficient search algorithms to create a powerful font search tool. It demonstrates the potential of semantic search and real-time previews in the context of fonts, which can be applied to various design and creative projects. ### Repository: The project is hosted on GitHub under the repository name `fontsearch`. It has been starred by 9 developers, indicating its significance in

### Overview: Infer compact schema descriptions from any JSON value, reducing the size of API responses and improving AI/LLM tool-use contexts. ### Key features: - Infer schema descriptions from JSON values - Compact representation without including actual values - Typically 80% smaller than raw JSON - Supports TypeScript and ESM - Measured 58.7% reduction in context window and 51.2% cost savings on API calls ### Tech stack: TypeScript ### Why it matters: - Reduces API payload size, improving performance and efficiency - Optimizes AI/LLM tool usage, reducing token costs - Simplifies data exploration and context management Repository: markadelnawar/json-schema-sketch GitHub stars: 9