Projects

### Overview: Kitsune-Tales is a project that fine-tunes LoRA models on the Gemma 4 E4B dataset to generate original fantasy light-novel stories in Japanese and English. The models are trained on synthetic data and validated using a LLM judge and safety audits, resulting in high-quality, general-audience content. ### Key Features: - **Japanese Light-Novel Prose**: Kitsune-Tales-E4B-JP generates Japanese light-novel stories. - **English Prose with Anime Themes**: Kitsune-Tales-E4B-EN produces English prose with Japanese anime themes. - **Filtered Synthetic Data**: The models are trained on filtered synthetic data from two Apache-2.0 models. - **95% Bootstrap CI Evaluation**: The models are evaluated using 95% bootstrap confidence intervals. - **Pairwise LLM Judge Validation**: A validated LLM judge is used to ensure the quality of the generated content. - **Safety Suite and JGLUE Audit**: The models undergo a safety suite and JGLUE audit to ensure they are safe for use. - **Bootstrapped CI Evaluation**: The models are evaluated using 95% bootstrap confidence intervals. - **Validation Perplexity**: The models are validated using validation perplexity, with Kitsune-Tales-E4B-JP having lower perplexity in both languages. ### Tech Stack: - **Python**: The project is written in Python. - **LoRA Fine-Tuning**: The models are fine-tuned using LoRA. - **SFT (Sequence-to-

Overview: PyDoctor is a Python CLI tool designed to automatically generate docstrings for your codebase, focusing on concise and descriptive summaries. It leverages local LLM models to improve the quality of generated docstrings. Key features: - Automatically generates docstrings for Python functions, methods, and classes - Preserves code formatting while extracting necessary information - Fine-tuned model for generating accurate and concise docstrings - Supports local installation and reusing cached GGUF model Tech stack: Python, LibCST, SmolLM2-1.7B-Instruct, llama.cpp Why it matters: PyDoctor simplifies the documentation process by automating the creation of docstrings, making it easier for developers to maintain clean, well-documented codebases.

### Overview: BYLM is a desktop application designed to train and fine-tune Large Language Models (LLMs) locally, without the need for cloud services or external APIs. It simplifies the process by offering a user-friendly interface and a step-by-step guide, allowing users to train their own models from scratch or fine-tune existing ones directly on their machines. ### Key features: - **Training from Scratch**: Design and train a GPT-2 style decoder from scratch, including layer configuration, width, heads, and context window. - **Fine-tuning a Real Model**: Adapt a model using LoRA or QLoRA, either by pointing it at a HuggingFace repository or an earlier checkpoint. - **Bring Any Data**: Support for datasets from HuggingFace Hub, local text files, CSV, JSON, JSONL, and Excel. - **Clean Data Visually**: Preprocess text data with live preview of cleaner effects. - **Train a Tokenizer**: Clean and prepare text data for tokenization using BPE, WordPiece, or Unigram. - **Watch It Learn**: Real-time loss and perplexity curves, checkpoints, and running best metrics. - **Talk to It**: Interact with the model with adjustable parameters like temperature, top-k, top-p, and repetition penalty. - **Take It with You**: Export the model to `.pt` or `.safetensors` formats, and keep the `defs.py` file for further use. - **Work Offline**: Cache the dataset and tokenizer for offline training, and switch to offline mode once the

Overview: This project showcases a production-style end-to-end machine learning pipeline implemented with Python, scikit-learn, FastAPI, MLflow, Docker, and GitHub Actions. It includes data ingestion, validation, transformation, training, evaluation, packaging, inference, and CI/CD workflows. Key features: - Data Ingestion - Data Validation - Data Transformation - Model Training - Model Evaluation - Model Packaging - Real-time Inference API - Docker Support - CI/CD Workflows - Unit Tests Tech stack: Python, scikit-learn, FastAPI, MLflow, Docker, GitHub Actions, Pytest, Joblib, PyYAML Why it matters: This project demonstrates best practices for designing and deploying a production ML pipeline, ensuring scalability, reliability, and maintainability.

Overview: This is a wiki dedicated to discussing FlinkML concepts, providing a platform for community discussions and documentation. Key features: - Discussion forum for FlinkML concepts - Roadmap document for Flink ML - Model serving document for Flink ML Tech stack: Unknown Why it matters: This wiki serves as a central resource for developers and researchers discussing FlinkML concepts, facilitating collaboration and knowledge sharing.

### Overview: This project is an abandoned fine-tuning experiment using Qwen3.5-0.8B for Jev-like judgments. It includes datasets, LoRA adapters, evaluations, and a retrospective on the project's failure. The project is documented in detail, covering setup, evaluation, training, and exporting the model. ### Key features: - Fine-tuning experiments with Qwen3.5-0.8B - Jev-like judgments with Qwen3.5-0.8B - Evaluation reports and datasets - LoRA adapters and fine-tuning contributions - Model card and model export - Evaluation metrics and calibration - Training data preparation and validation ### Tech stack: Python ### Why it matters: The project showcases the capabilities and limitations of Qwen3.5-0.8B in making judgments and fine-tuning experiments, providing valuable insights into the model's performance and potential improvements.

Overview: pegainfer is a lightweight inference engine built with Rust and CUDA, designed to run large language models (LLMs) efficiently on Windows machines using Nvidia graphics cards. It focuses on fast, efficient processing without complex setup, making it suitable for both developers and users. Key features: - Fast and efficient inference using GPU acceleration (CUDA) - Written in Rust for safety and speed - Supports popular LLM formats - Minimal setup without complex dependencies - Works offline after initial download - Lightweight and runs on everyday Windows PCs Tech stack: Rust Why it matters: pegainfer simplifies the process of running LLMs on Windows, providing a straightforward solution for developers and users alike, while leveraging the power of Nvidia GPUs for fast and efficient inference.

Overview: AI-Trains-AI is a framework designed to build, train, evaluate, and deploy AI assistants with specific roles, using fine-tuning, retrieval-augmented generation, and multi-layer defenses. Key features: - Fine-tuning of small open models like Qwen2.5 1.5B and 3B - Retrieval-augmented generation for improved accuracy and context - Multi-layer defenses to prevent hallucinations and off-topic behavior - Role-specific off-topic regex to block out-of-scope queries - Early exit mechanism to prevent context-stuffing attacks Tech stack: Python Why it matters: AI-Trains-AI provides a structured approach to building AI assistants that can be fine-tuned for specific roles, ensuring they remain focused and accurate, even when faced with adversarial inputs. Repository: Kishalll/ai-trains-ai Primary language: Python GitHub stars: 4 GitHub description: Framework for building, fine-tuning, and deploying role-locked AI assistants with guardrails, local RAG, and tool calling.

Overview: A high-performance deep learning model inference server built on TensorRT, designed for fast inference of Embedding, Reranker, and NLI models. Key features: - Optimized TensorRT inference - Dynamic batching for improved efficiency - Support for multiple models: Embedding, Reranker, and NLI - RESTful API interface - Compatible with OpenAI SDK - Efficient GPU memory management Tech stack: Python Why it matters: This server provides a fast and efficient way to run deep learning models, especially for tasks like embeddings and natural language inference, making it ideal for applications requiring real-time inference. Repository: LLMSystems/TensorrtServer GitHub stars: 4 GitHub description: A high-performance deep learning model inference server based on TensorRT, supporting fast inference for Embedding, Reranker, and NLI models.

Overview: Julia Medical Reasoning is a fine-tuned version of Google's Gemma-3 model optimized for clinical reasoning, diagnostic support, and medical question answering in English. It has been trained using a curated dataset of medical case studies, question-answer pairs, and evidence-based medicine protocols. Key features: - Fine-tuned from Google's Gemma-3 model - Optimized for clinical reasoning and diagnostic support - Capable of generating answers to patient queries and common clinical questions - Adaptable for integration into clinical decision support tools and chatbot interfaces Tech stack: Python, PyTorch, Transformers, PEFT Why it matters: This model enhances medical education and training, assists clinicians in reasoning through differential diagnoses, and supports hospitals in providing accurate patient answers and clinical guidance.

Overview: Pantograph is a Rust-based workflow framework designed for local AI workflow orchestration and runtime management. Key features: - Connect processing and inference nodes - Manage execution with a scheduler - Inspect task status, diagnostic events, and retained outputs - Use a headless workflow service Tech stack: Rust Why it matters: It provides a robust framework for managing and orchestrating AI workflows locally, ensuring efficient execution and easy inspection of tasks. Repository: MrScripty/Pantograph GitHub stars: 4 GitHub description: Local AI workflow orchestration and runtime management framework

Overview: Zephyr is a pure C++ LLM inference engine built from scratch, designed for hands-on learning. It uses its own lightweight tensor library and has no external dependencies. Key features: - Tensor parallelism: Single-node, multi-GPU inference in one process with a worker thread per GPU. - Paged KV cache: Prefix caching and automatic GPU memory budgeting. - Continuous batching: Dynamic request batching, chunked prefill, and preemption under cache pressure. - Python APIs: Synchronous and asynchronous generation, streaming, and request cancellation. Tech stack: C++20 and CUDA. Why it matters: Zephyr is a unique project that provides a hands-on learning experience in building an LLM inference engine from scratch, focusing on efficiency and performance. It emphasizes the importance of understanding the underlying components and performance optimizations. Repository: Poseidon-fan/zephyr GitHub stars: 4 GitHub description: A pure C++ LLM inference engine built from scratch in C++20 and CUDA as a hands-on learning project. It uses its own lightweight tensor library and has no PyTorch dependency. Python provides a thin wrapper for tokenization and serving; the inference engine runs entirely in C++.

### Overview: Mini-SciBERT aims to demonstrate the effectiveness of domain-specific pre-training on improving BERT performance in scientific text processing, specifically for Named Entity Recognition (NER) and Citation Intent Classification. ### Key Features: - 🧠 **Scientific Domain Focus**: Optimized for biomedical and scientific literature. - 📝 **Dual-Task Pre-training**: Implements both Masked Language Modeling (MLM) and Next Sentence Prediction (NSP) objectives. - 📊 **Two Downstream Tasks**: Named Entity Recognition (NER) and Citation Classification (SciCite). - 📈 **Comprehensive Evaluation**: Detailed performance metrics and visualizations. - 🔄 **Reproducible Pipeline**: Complete workflow from data preparation to evaluation. - 📦 **Model Artifacts**: Integration with Weights & Biases for experiment tracking. - 🚀 **Mixed Precision Training**: FP16 for efficient training on GPUs. - 📊 **Visual Analytics**: Performance comparison charts and statistical analysis. ### Tech Stack: Python, PyTorch, Transformers, Jupyter Notebook ### Why it Matters: This project showcases how domain-specific pre-training can significantly enhance the performance of BERT on specialized scientific tasks, providing a valuable educational resource and practical application for scientific text analysis.

### Overview: This repository contains a YOLO Serving Cookbook based on Triton Inference Server, designed to streamline the deployment and inference process for real-time applications. ### Key Features: - **Architecture**: The repository includes Docker, Docker Compose, and Kubernetes configurations for deploying YOLO models. - **Server**: The server architecture supports both synchronous and asynchronous processing, with options to replicate the Triton Inference Server for improved performance. - **Client**: The client can be configured to use either a serial, concurrent, or random method for calling the FastAPI endpoint. - **Experimental Setup**: The experimental setup includes various configurations, such as different numbers of replicas and ensemble models, to test the performance under different conditions. ### Tech Stack: - **Python**: The primary language used for the server and client. - **FastAPI**: A microservices framework for building APIs. - **Triton Inference Server**: A server-side inference engine. - **OpenCV**: A popular open-source computer vision library. - **Docker**: Containerization technology for easy deployment. - **Kubernetes**: A container orchestration platform. - **Traefik Proxy**: A reverse proxy for managing traffic. - **ONNX**: An open-source neural network format. - **Triton Inference Server**: A server-side inference engine. ### Why It Matters: This cookbook provides a comprehensive solution for deploying YOLO models using Triton Inference Server, making it easier to integrate and deploy real-time applications. The configurations and experimental setups offer insights into optimizing performance and handling different scenarios, ensuring that

Overview: This repository provides reference implementations for various applied AIML service patterns, including training pipelines, inference APIs, testing, CI, and infrastructure scaffolding. Key features: - Model training and artifact management - Typed inference services using FastAPI - Integration testing for ML-backed APIs - Reproducible local development workflows - CI validation via GitHub Actions - Terraform scaffolding for deployment conversations Tech stack: Python Why it matters: This repository offers practical, production-adjacent approaches that can be easily reused in consulting, interviews, and portfolio discussions, making it a valuable resource for hiring teams, clients, and engineers looking to demonstrate their understanding of AI/ML service delivery. GitHub stars: 4 Repository: aimlsolutions-nyc/aiml-service-patterns

### Overview: InferDeck is a self-hosted AI gateway for Windows that manages multiple models using a single GPU, providing a local hosted web dashboard for real-time monitoring and control. It features hot model swapping, request history, and cost tracking, aiming to be a reliable and efficient solution for AI inference on a single-GPU workstation. ### Key Features: - **One GPU, Multiple Models**: Manages multiple models on a single GPU using a single process. - **Local Dashboard**: Provides a live React dashboard for real-time monitoring and control. - **Hot Model Swapping**: Automatically switches between models without downtime. - **Request History**: Tracks and displays request history for better understanding and optimization. - **Cost Tracking**: Tracks costs associated with model requests for efficient resource management. ### Tech Stack: C++ ### Why It Matters: InferDeck offers a practical, efficient solution for AI inference on a single-GPU workstation, making it ideal for developers and AI enthusiasts who prefer to keep their data and infrastructure under their own control.

Overview: AI-RECON is a specialized reconnaissance tool designed to identify, fingerprint, and enumerate AI/ML infrastructure, going beyond traditional scanners by detecting model-serving frameworks, vector databases, and LLM runtimes. Key features: - Port-agnostic HTTP/HTTPS fingerprinting - More scan coverage with smart, common, all-TCP-port, and explicit port profiles - Non-HTTP protocol fingerprints for SSH, FTP, MySQL, Redis, PostgreSQL, TLS, and HTTP/2 - Expanded AI/ML discovery signatures for model-serving APIs, OpenAI-compatible endpoints, local-model runtimes, model registries, vector stores, notebooks, metrics, workflow interfaces, and selected AI web UIs - Opt-in metadata enumeration for read-oriented service API queries - Safer transport handling with certificate verification enabled by default - More cautious exposure flags with no claims of exploitability or vulnerability presence Tech stack: Python Why it matters: AI-RECON is designed to complement traditional port scanning tools by providing a more comprehensive and evidence-driven approach to identifying and enumerating AI/ML infrastructure, including services that may not be reachable through conventional ports. Repository: giriaryan694-a11y/ai-recon GitHub stars: 4

### Overview: This repository includes all the essential scripts and notebooks required to fine-tune the Parakeet RNNT 0.6B model for Automatic Speech Recognition (ASR) in Urdu. ### Key features: - Fine-tuning process of NVIDIA's Parakeet RNNT 0.6B model on the Urdu dataset from Mozilla's Common Voice. - Training on a NVIDIA A100 GPU over approximately 5 hours. - Utilizes the Hugging Face Datasets Library for the Urdu dataset. - Achieved a Word Error Rate (WER) of 25.513% for the fine-tuned model. ### Tech stack: Jupyter Notebook ### Why it matters: This repository showcases the potential of fine-tuning models for ASR in Urdu, demonstrating impressive results even with limited training time.

### Overview: The micro-kiki project is a comprehensive adaptation of the Qwen model, specifically tailored for a micro-environment. It includes 41 adapter artefacts and a cognitive layer optimized for Qwen3.6-35B-A3B, featuring 256 experts and 3 billion active tokens. The project is deployed on a Mac Studio, leveraging MLX for inference and Q4_K_M for the cognitive pipeline. ### Key Features: - **Cognitive Layer**: Enhanced with Aeon memory, CAMP negotiator, and KnowBias. - **Adapter Artefacts**: 41 adapter artefacts for 35 domain-expert LoRAs. - **Deployment**: Deployable artefact of the *dreamOfkiki* research program, part of Hypneum Lab. - **Status**: 35/35 V4 adapters trained and verified healthy. - **Router V4**: Top-1 accuracy 46.4%, top-3 accuracy 69.6%. - **Stress Test**: 10/10 queries pass, 8/8 after restart, all domains routed correctly. - **Post-Pivot MoE-LoRA Adapters**: Pre-pivot MoE-LoRA adapters pass the adapter-health validator; post-pivot MoE-LoRA adapters were archived as dead weights after an `lora_B = 0` audit. ### Tech Stack: Python ### Why It Matters: The micro-kiki project is crucial for adapting large language models to micro-environments,

Overview: This repository contains the code for a paper that debiases deep chest X-ray classifiers using intra- and post-processing methods. The methods aim to mitigate bias in healthcare applications where protected attributes like race or gender might affect model performance. Key features: - Fine-tuning and pruning of a trained neural network to debias it. - Implementation of intra-processing techniques for debiasing. - Post-processing approach to debiasing without knowing the protected attribute during development. Tech stack: Python Why it matters: The debiasing methods are crucial for ensuring fair and unbiased healthcare models, addressing concerns about demographic disparities and discrimination in medical decision-making. Repository: i6092467/diff-bias-proxies Primary language: Python GitHub stars: 4 GitHub description: Pruning and fine-tuning for debiasing an already-trained neural network with applications to deep chest X-ray classifiers README excerpt: This repository holds the official code for the paper ["*Debiasing Deep Chest X-Ray Classifiers using Intra- and Post-processing Methods*"](https://proceedings.mlr.press/v182/marcinkevics22a.html) presented at the [the 7^th Machine Learning for Healtcare Conference (MLHC), 2022](https://www.mlforhc.org/) and [ICLR 2022 Workshop on Socially Responsible Machine Learning](https://iclrsrml.github.io/).