Projects

Overview: Fine-tune local LLMs with LoRA/QLoRA: PEFT training scripts, dataset formatting, 4-bit quantization, Ollama/GGUF export and an inference server. Key features: - Chat-template aware data prep. - QLoRA / LoRA training via 🤗 `peft` + `transformers.Trainer`. - Reproducible runs. - Evaluation: perplexity, generation pass with n-gram overlap metrics. - Export: merge LoRA into base weights, quantize to GGUF q4_k_m / q5_k_m / q8_0. - Inference server: FastAPI + Uvicorn, OpenAI-compatible `/v1/chat/completions`, token-streaming via SSE, paged-kv-cache via `vLLM` engine when available. Tech stack: Python. Why it matters: The provided scripts and tools enable developers to fine-tune local large language models (LLMs) using LoRA/QLoRA techniques, ensuring reproducibility, evaluation, and efficient model export for deployment.

Overview: TapirusDB is a Rust-based embedded database that combines SQL, vector search, and AI document memory, making it suitable for local applications that require these functionalities. Key features: - SQL support - Vector search - Graph and JSON document APIs - AI source retrieval with BM25, embeddings, recency, and importance - Indexed memory with HNSW, upsert, revision checks, atomic batches, source versions, TTL, and configurable budgets - `SessionDatabase` for independent transaction ownership - Offline IoT capsule bridge for committed Micro slot import - First-party SQL Lite, Document, Graph, and Vector entry points - Edition binaries share the `.tapir` format but are API boundaries Tech stack: Rust Why it matters: TapirusDB provides a robust, in-process database solution for local applications that need SQL, vector search, and AI document memory, offering a safer and more secure alternative to SQLite or SQLite-like solutions. Repository: tapiruslab/TapirusDB GitHub stars: 9 GitHub description: The 100% Safe-Rust Embedded Quad-Model AI Database & Cognitive Memory Engine (SQL, Vectors, GraphRAG, Documents) Build process: `cargo build --release -p tapirus --no-default-features --features lite --bin tapirus-lite` Asset list: Check the asset list for the exact published release before depending on a prebuilt edition binary.

Overview: This repository contains a fine-tuned version of the ChatGPT language model for finance-related tasks, pre-trained on financial documents and news articles to improve performance on tasks such as financial news generation, stock price prediction, and financial statement analysis. Key features: - Financial news generation - Stock price prediction - Financial statement analysis Tech stack: Python Why it matters: This fine-tuned model can be used in various financial applications, enhancing the accuracy and relevance of financial information and predictions.

Overview: This project demonstrates the implementation of modern search engines using Text Embeddings and a Vector Database, showcasing a straightforward structure for efficient data retrieval. Key features: - Text Embeddings Interface - CommonCrawl Data Worker - WebApp for Search Tech stack: TypeScript Why it matters: This project provides a practical example of how to build a search engine using Huggingface Text Embeddings and a vector database, making it useful for developers looking to enhance their search functionalities.

### Overview A Text-To-Image Generator using the Stable Diffusion model from Hugging Face, running on Fly GPUs and Tigris as storage. This project allows for quick image generation and easy deployment. ### Key Features - Text-to-image generation using Stable Diffusion model - Inference using Hugging Face and Fly GPUs - Storage using Tigris for image hosting and metadata - Deployable on Fly Cloud ### Tech Stack - Python - Stable Diffusion model from Hugging Face - Fly GPUs - Tigris as storage ### Why it Matters This project provides a simple and efficient way to generate images from text descriptions, leveraging advanced AI models and cloud resources for deployment. It simplifies the process of creating and managing image generation applications, making it accessible to developers and users alike.

Overview: LoFT is a low-rank adaptation method that mimics full fine-tuning, enabling efficient and effective fine-tuning on limited data. Key features: - Demonstrates synthetic experiment - Provides instructions for applying method using Huggingface models Tech stack: Python Why it matters: This approach offers a practical solution for fine-tuning models on limited data, making it valuable for researchers and practitioners working with limited computational resources.

### Overview: AetherMind is a local-first personal AI memory OS designed to remember your entire life. It leverages Git, notes, calendar, and location data to create semantic memories and answers natural language questions about your past. With 100% offline functionality and no cloud storage, it ensures privacy and cost-effectiveness. ### Key Features: - **Automatic Data Collection**: Collects data from Git commits, notes, Google Calendar, and location history. - **Incremental Sync**: Only imports new data, ensuring deduplication via content hashing. - **Semantic Memory Search**: Answers "when was I most productive?" with actual history data. - **Daily AI Reflection**: Generates daily summaries, insights, and pattern detection. - **100% Local**: Runs on your machine, no data leaves your computer. - **CUDA-accelerated**: Embeddings run on your GPU for speed. - **Streamlit UI**: Provides Timeline, Ask Memory, Reflections, and Stats views. - **One-Command Setup**: `python setup.py` handles everything. ### Tech Stack: - Python ### Why It Matters: - **Local-first & Privacy-first**: Ensures your data is kept secure and never leaves your machine. - **Zero Cloud & Subscriptions**: No cloud storage or subscription fees. - **Open Source**: Built with community contributions, ensuring reliability and continuous improvement. ### Repository: - **GitHub**: [tomaszwi66/AetherMind](https://github.com/tomaszwi66/AetherMind) - **Stars**: 9

Overview: This project is an implementation of model serving using Apache Beam, focusing on processing data and models from Kafka streams. Key features: - Data Stream: Delivers data records as protobuf buffers. - Model Stream: Delivers models as protobuf buffers. - Stateful Processing: Uses state to store intermediate model values. Tech stack: Scala Why it matters: This project demonstrates the use of Beam for model serving, showcasing its capabilities in handling streaming data and stateful processing. Repository: typesafehub/fdp-beam-modelServer Primary language: Scala GitHub stars: 9

Overview: This package allows developers to create embeddings for Eloquent models in Laravel, enabling the use of OpenAI's text embeddings with their applications. Key features: - Automatically saves embeddings for Eloquent models in a PostgreSQL vector database. - Supports both OpenAI and custom drivers for embeddings. - Configurable for different database connections and table names. - Provides options for chunk sizes for mass importing data. - Includes a trait to add the Embeddable functionality to models. Tech stack: Laravel, PHP, PostgreSQL, OpenAI. Why it matters: This package simplifies the integration of text embeddings into Laravel applications, making it easier to leverage advanced AI capabilities for text analysis and retrieval.

### Overview: Control your in-house LLM outputs by generating robust and accurate JSON outputs faster than ever. llm_template utilizes schema tokens to ensure the LLM generates only the necessary values, making it an efficient solution for developers. ### Key Features: - **Schema Injection**: Injects schema tokens into the response to ensure only the required information is generated. - **Robust JSON Outputs**: Produces accurate and structured JSON outputs from any instruction model. - **Intricate Schemas**: Creates intricate schemas to enhance the precision and accuracy of the generated JSON. ### Tech Stack: Python ### Why It Matters: By leveraging schema tokens, llm_template streamlines the generation process, ensuring that the LLM focuses on the specific information needed, thereby improving the quality and efficiency of the generated JSON outputs. This makes it an invaluable tool for developers looking to automate and streamline their workflows.

Overview: LLMFS is a filesystem-based persistent memory solution designed for LLMs and AI agents, offering an unlimited context window without losing information. Key features: - Supports unlimited context windows without lossy summarization - Stores memories at filesystem paths for easy retrieval and search - Persistent and searchable, organized like a filesystem - Compatible with LangChain, OpenAI Function Calling, and more Tech stack: Python Why it matters: LLMFS provides a solution for LLM agents to overcome the context window limitation, ensuring zero information loss and an effectively unlimited context window, even over thousands of turns. Repository: viditraj/llmfs Primary language: Python GitHub stars: 9 GitHub description: Filesystem-based persistent memory for LLMs and AI agents — unlimited context windows without lossy summarization

Overview: InkChatGPT is a Streamlit application that enables users to upload PDF documents and engage in a conversational Q&A with a language model based on the content of those documents. Key features: - Upload any documents and start asking key information about it, currently supports: PDF, TXT, DOCX, EPUB - Limit 200MB per file - Conversational Q&A with LLM (powered by gpt-3.5-turbo model) - HuggingFace embeddings to generate document embeddings with all-MiniLM-L6-v2 model - VectorDB for document vector retrieval storage Tech stack: Python Why it matters: This application leverages the power of language models and advanced embeddings to provide users with valuable insights and answers directly from their documents, enhancing the efficiency and accuracy of information retrieval.

### Overview: This repository showcases various approaches to structured data extraction using language models and frameworks. It includes examples of generating JSON outputs for name and age extraction from text prompts, as well as handling multiple prompts and structured outputs from multimodal models. ### Key features: - **Multiple Framework Support**: Examples for LangChain, vLLM, Outlines, Ollama, OpenAI, and more. - **Pydantic Integration**: Type-safe structured outputs with validation. - **Batch Processing**: Efficient handling of multiple prompts. - **Vision Model Support**: Structured outputs from multimodal models. - **Flexible Backends**: Support for local models, API services, and GGUF formats. ### Tech stack: Python ### Why it matters: This repository demonstrates how to ensure language models return well-formatted, schema-compliant responses, making it valuable for developers looking to integrate structured data extraction into their projects.

Overview: Seedream 5.0 Flash Prompts offers tested image prompt recipes with example renders, aspect ratios, the model ID (`seedream-5-0-flash`), and the cost per image. This repository provides a catalog of 8 recipes, including a live pricing model and compatibility with OpenAI's API. Key features: - 8 tested recipes with example renders, aspect ratios, and the model ID. - The cost per image is $0.0137, estimated for the entire set. - The API key can be obtained via a live pricing model. Tech stack: Python. Why it matters: Seedream 5.0 Flash (`seedream-5-0-flash`) is used through APIMart for its efficient pricing and compatibility with OpenAI's API, offering a cost-effective alternative to traditional image generation services. Repository: w4c1mqh4329/seedream-5-0-flash-prompts-es Primary language: Python GitHub stars: 9

Overview: This project is a Weaviate multimodal search demo built with TypeScript, ImageBind, Vertex AI, and Next.js. It allows users to run a Next.js web app, start a Weaviate instance, import images, audio, and videos into a Weaviate database, and search for related information. Key features: - Running a Next.js web app - Starting a Weaviate instance using Docker - Importing data into Weaviate - Searching for related information Tech stack: TypeScript Why it matters: This project showcases the integration of multimodal AI with Weaviate, ImageBind, Vertex AI, and Next.js, providing a practical example for developers looking to build similar applications.

### Overview: FLUX 2 Max API, also known as `flux-2-max`, is an OpenAI-compatible endpoint that provides high-resolution image generation services at a cost per megapixel. This API is designed to be efficient and cost-effective, with a starting price of $0.056 per image, ranging from $0.056 for 1MP to $0.104 for 3MP. The API supports various image resolutions and allows for up to 1 image per task. ### Key Features: - **Model ID and Pricing**: FLUX 2 Max offers pricing based on the megapixels of the output image, with a default tier of $0.1 per image, $0.07 for 1MP, $0.08 for 2MP, and $0.104 for 3MP. - **Billing Model**: The API operates on a per-image basis, with the cost per image varying based on resolution. - **Asynchronous Nature**: Tasks are processed asynchronously, with the `task_id` returned and the status checked via `GET /v1/tasks/{task_id}`. - **Pricing Snapshot**: Prices are a snapshot and are updated periodically by CI, with the exact amount reported in the `cost` field of completed tasks. ### Tech Stack: - Python ### Why It Matters: - **Cost Efficiency**: FLUX 2 Max offers a high-resolution image generation service at a cost per megapixel, making it ideal for applications requiring detailed and high-quality images. - **OpenAI

### Overview: FLUX 2 Max Prompts is a comprehensive resource for developers looking to enhance their image generation workflows. This repository provides tested recipes and example renders, along with the associated costs and aspect ratios for the `flux-2-max` model. ### Key Features: - **8 Recipes**: Detailed prompts for various products, food photography, and cinematic street scenes. - **Aspect Ratios**: 1:1, 4:3, 85mm, 24mm, etc., to ensure optimal output quality. - **Costs**: $0.056 per image, $0.08 per 2MP, $0.104 per 3MP, providing accurate pricing for each recipe. ### Tech Stack: Python ### Why It Matters: This repository offers a practical guide for developers to optimize their image generation processes, ensuring that their models produce high-quality, consistent results without unnecessary costs.

Overview: This project provides comprehensive documentation and examples for the Gemini Nano Banana 2.1 API, which is compatible with OpenAI. It includes pricing details, model parameters, and a quickstart guide for developers. Key features: - Detailed API documentation and examples - Pricing information - Model parameters and parameters for text-to-image and reference-image editing - A quickstart guide for developers Tech stack: Python Why it matters: This project is crucial for developers looking to integrate Gemini Nano Banana 2.1 into their applications, offering a cost-effective and flexible solution for image generation and editing. Repository: xou-fm9md3t/gemini-nano-banana-2-1-api-ko Primary language: Python GitHub stars: 9 GitHub description: 나노바나나 / 나노 바나나 — Gemini Nano Banana 2.1 API (`gemini-nano-banana-2.1`): model id, pricing, parameters and copy-paste examples for the OpenAI-compatible APIMart endpoint.

### Overview: Seedance 2.0 Mini Prompts: 8 Tested Recipes with Outputs Seedance 2.0 Mini Prompts offers a collection of 8 tested recipes designed to help developers create compelling images quickly and efficiently. These recipes are crafted with the OpenAI-compatible endpoint `https://api.apimart.ai/v1`, ensuring a seamless integration without the need for monthly subscriptions or prepaid tariffs. ### Key Features: - **Model ID:** `seedance-2.0-mini` - **Prompt Structure:** Each recipe includes the subject, light, framing, setting, and finish details. - **Cost:** $0.066 per second, with a minimum cost of $1 for the entire set. ### Tech Stack: Python Seedance 2.0 Mini Prompts is built using Python, making it accessible and easy to integrate into existing projects. ### Why It Matters: - **Key:** A single key allows access to the entire Seedance 2.0 Mini catalog, including 300+ models. - **Minimum Cost:** The cost is $1 per minute, allowing users to start using the service without any upfront commitments. - **Real-Time Pricing:** The cost is reflected in the response, making it straightforward to track and manage expenses. - **Asynchronous Processing:** Tasks can be submitted, and the `task_id` is returned, enabling efficient handling of tasks without the need for complex integration. ### Seedance 2.0 Mini (`seedance-2.0-mini`) Over APIMart - **Key Access:**

Overview: Seedance 1.5 Pro API is an OpenAI-compatible endpoint offering model id, pricing, parameters, and copy-paste examples for image and video generation. It provides a seamless and cost-effective solution for developers using ByteDance's AI models. Key features: - One key, entire catalog access via `https://api.apimart.ai/v1` - $1 minimum, pay-as-you-go pricing with no subscription required - Charge comes back in the response, reporting cost per call - Async by design, submit, take `task_id`, poll `GET /v1/tasks/{id}` Tech stack: Python Why it matters: Seedance 1.5 Pro API simplifies the integration of ByteDance's AI models, providing a straightforward and cost-effective way to access high-quality video and image generation without the need for a monthly plan or upfront investment. GitHub stars: 9 Repository: y1i87cdabvzwil/seedance-1-5-pro-api-gateway