Projects

### Overview: DISCO is a multimodal generative model that co-designs protein sequences and 3D structures, enabling sequence-based objectives to inform structure generation and vice versa. It achieves state-of-the-art performance in silico for diverse biomolecular targets, including new-to-nature catalysis. ### Key Features: - **Multimodal Co-Design**: Simultaneously generates sequences and structures while co-folding with biomolecules like small molecules, DNA, and RNA. - **Fine-Grained Control**: Achieves high performance on 178/179 evaluated ligands, DNA, and RNA. - **Novel Active-Site Geometries**: Designed enzymes catalyze new-to-nature reactions, including alkene cyclopropanation, spirocyclopropanation, B-H and C(sp³)–H insertions. - **Evolvable Design**: Random mutagenesis of a selected design yields a fourfold activity gain, indicating evolvability. ### Tech Stack: DISCO uses Python for its implementation. ### Why It Matters: DISCO offers a novel approach to protein design by enabling sequence-based objectives to inform structure generation, leading to improved performance and the design of diverse and evolvable enzymes with novel active-site geometries.

### Overview: LLAMATOR is a comprehensive Python framework designed for red teaming and testing chatbots and generative AI systems. It provides a robust environment to simulate various attacks and assess the resilience of AI-based applications. ### Key features: - **Custom Attacks and Datasets**: Users can easily integrate custom attacks and datasets to tailor the testing scenarios. - **Wide Range of Attacks**: Offers a variety of attacks on LLMs, RAGs, Agents, and VLMs in both English and Russian. - **Custom Configuration**: Allows for the customization of chat clients, enhancing flexibility in testing environments. - **Attack History**: Stores and documents the history of attack requests and responses, providing valuable insights. - **Test Report**: Generates a test report in DOCX format, facilitating easy documentation and analysis. ### Tech stack: Python ### Why it matters: LLAMATOR is essential for developers and red team members looking to thoroughly test and secure AI-based systems. Its comprehensive features and wide range of attacks make it a powerful tool for identifying vulnerabilities and improving the overall security posture of chatbots and generative AI applications. Repository: LLAMATOR-Core/llamator Primary language: Python GitHub stars: 224 GitHub description: Red Teaming python-framework for testing chatbots and GenAI systems.

### Overview: TokenPrint is an interactive 3D visualization platform designed for exploring transformer architectures, tensors, and real-time language model inference. It provides a live debugger for models, allowing developers to inspect attention, RoPE, KV cache, and residual streams as they happen. ### Key features: - **Real Data Visualization:** Every value in the model is derived from a real forward pass, named parameters, or the GGUF header, ensuring no fake data reaches the application layer. - **Grouped-Query Attention:** Renders attention heads and KV groups as geometric shapes, accurately reflecting their configuration. - **Debugging Tools:** Offers breakpoints, watch expressions, activation ablation, logit lens, induction-head lab, and trace recording, replay, and branching capabilities. - **Multiple Run Modes:** Supports Python backend, quantized GGUF, and GPT-2 browser execution. ### Tech stack: TypeScript ### Why it matters: TokenPrint simplifies the complex world of transformer architectures and language model inference, making it easier for developers to understand and debug their models in real-time. Its real data visualization and debugging tools make it an indispensable tool for anyone working with transformer-based models.

### Overview: This project provides a pure Zig implementation of the Llama 2 model architecture, focusing on inference capabilities. It supports temperature control, top-p sampling, prompt handling, sequence length control, and more. The code is designed for simplicity and readability, leveraging SIMD for a significant speed increase. The project aims to create a fast, portable, and user-friendly implementation of the Llama 2 model. ### Key features: - Inference of Llama 2 model checkpoints - Temperature control - Top-p (nucleus) sampling - Prompt handling (bpe tokenization) - Sequence length control - Custom tokenizers - Multiquery support - Running very fast ### Tech stack: Pure Zig ### Why it matters: The project prioritizes simplicity and readability without sacrificing performance. It aims to create a fast, portable, and user-friendly implementation of the Llama 2 model architecture. The code is designed to be single-language, focusing on single-threaded and multi-threaded implementations for different performance metrics.

Overview: Kiln is an open-source MCP server designed for 3D printing, allowing AI agents to design, slice, and print parts on various printers, including Bambu Lab, Creality, Prusa, Elegoo, Voron, Sovol, AnkerMake, Artillery, FlashForge, QIDI, RatRig, and SparkX. Key features: - AI-driven 3D printing - Compatible with OctoPrint, Moonraker/Klipper, PrusaLink, and Direct USB - Supports custom MCP clients - Real-time monitoring and error recovery - Integrates with Claude Desktop for easy setup Tech stack: Python Why it matters: Kiln simplifies the complex process of 3D printing by automating design, slicing, and printing, making it accessible to anyone with an AI agent and a compatible printer. Repository: codeofaxel/Kiln Primary language: Python GitHub stars: 224 GitHub description: The open-source MCP server for 3D printing, enabling AI agents to design, slice, and print on various printers.

Overview: Transcribe any audio or video file into high-quality text using Whisper ASR. Key features: - Wide range of supported file formats - Local transcription engine using Whisper ASR - Timestamps for each line of transcription - Transcribe multiple files at once - Background transcription functionality Tech stack: TypeScript Why it matters: This plugin provides a seamless and efficient way to convert audio and video files into readable text, making it invaluable for developers who need to quickly summarize or extract information from audio content. Repository: djmango/obsidian-transcription Primary language: TypeScript GitHub stars: 224

### Overview: This project introduces a simple yet powerful training procedure for conditioning image-generating diffusion models on (possibly incomplete) multiclass segmentation masks. The method outperforms existing segmentation-guided image generative models in terms of faithfulness to input masks, anatomical realism, and flexibility in image generation. The optional mask-ablated training algorithm allows conditioning on incomplete segmentation masks, enabling more flexible image generation and adjustable anatomical similarity. ### Key Features: - **Segmentation-Guided Diffusion Models**: Train diffusion models on input segmentation masks to generate more faithful and anatomically precise images. - **Mask-Ablated Training**: Conditioning on segmentation masks with missing classes, allowing for more flexible image generation and adjustable anatomical similarity. - **Easy to Use**: Primarily written in Python, making it accessible for developers and researchers. ### Tech Stack: - Python 3.11 - PyTorch 2.1.2 - CUDA 12.2 ### Why It Matters: - **Improved Image Quality**: Generates images that are more faithful to input segmentation masks. - **Flexibility**: Allows for more flexible image generation and adjustable anatomical similarity. - **Ease of Use**: Simple to use and train, making it accessible for developers and researchers. ### Repository: mazurowski-lab/segmentation-guided-diffusion - GitHub Stars: 224 - GitHub Description: "Easy and Precise Segmentation-Guided Diffusion Models" for medical images and beyond. - Primary Language: Python - GitHub Link: [mazu

### Overview: Weam AI is a comprehensive AI platform designed for teams of up to 20 members. It connects to major LLMs via API, allowing for seamless integration with various AI models. The platform features a modern, fully responsive stack built with Next.js and Node.js, making it easy to deploy and use. It includes features like chat systems, productivity tools, and the ability to create AI apps, agents, and workflows. ### Key Features: - **Multi-LLM Support:** Connects to OpenAI, Anthropic, Gemini, Llama, Perplexity, DeepSeek, OpenRouter, Hugging Face, and more. - **Local Model Support (Ollama):** Supports self-hosted models via Ollama. - **Conversation Management:** Allows for forked chats to explore different directions. - **Voice Input:** Enables voice-to-text functionality. - **Web Scraping:** Extracts content from any URL to provide context. - **Web Search:** Real-time information retrieval during conversations. ### Tech Stack: - Next.js - Node.js ### Why It Matters: - **Modern Teams Need:** Easy access to the latest AI models. - **Centralized AI Interactions:** All AI interactions are managed in one centralized platform. - **Customizable Brains:** Organize AI interactions into intelligent folders based on departments. - **Customizable AI Apps:** Rapidly build custom AI apps using the library of open-source AI apps. - **Scalable and Expandable:** Ready to deploy today, can grow with your needs. ### Repository: -

Overview: This repository is a curated map of credit assignment methods in reinforcement learning for large language models, organized by reasoning, agentic, and multi-agent settings. It includes 47 methods published between 2024 and early 2026, with 29 newer entries and related resources added since the snapshot. Key features: - Surveys and overviews - Classical RL foundations - RL algorithms for LLMs - PRM foundations - Token-level methods - Segment-level methods - Step-level methods - Turn-level process reward models - Hindsight & counterfactual methods - Critic-free step-level methods - Hierarchical methods - Information-theoretic methods - Implicit & DPO-based methods - Infrastructure & practical methods - Benchmarks - Citation Tech stack: Python Why it matters: This repository provides a comprehensive overview of credit assignment methods in reinforcement learning for large language models, addressing a fundamental question in the field of large language model reasoning and agentic reinforcement learning.

### Overview: This repository contains the code and resources for building a large language model (LLM) from scratch, as guided by Sebastian Raschka's book "Build a Large Language Model (from Scratch)." The project aims to demystify the process of creating, training, and fine-tuning LLMs, providing a hands-on approach to understanding the underlying mechanics of these powerful AI models. ### Key Features: - Implementing a GPT Model from Scratch to Generate Text - Pretraining on Unlabeled Data - Fine-tuning LLMs ### Tech Stack: Jupyter Notebook ### Why It Matters: By following the steps in this repository, you will gain a comprehensive understanding of LLMs and develop your own functional model. This project serves as a practical guide to the process of building and fine-tuning LLMs, making it an invaluable resource for developers and researchers interested in deep learning and natural language processing.

Overview: Bespoke Automata is a comprehensive GUI and deployment pipeline for creating and deploying complex AI agents, enabling the integration of large language models with various instruments for database IO, dictionaries, arrays, logic, APIs, and more. This tool simplifies the process of designing and deploying AI brains via a Directed Graph GUI and a single user-friendly API. Key features: - Integration of large language models with various AI instruments - Directed Graph GUI for designing and testing brains - Single API endpoint for deploying and managing AI brains - Model Context Protocol (MCP) tools for accessibility to any MCP-compatible AI assistant - Standalone MCP server for running the server without the web UI - Optional LLM backend for local text inference using llama-cpp-python Tech stack: JavaScript, Node.js, NPM, Python Why it matters: Bespoke Automata simplifies the process of creating and deploying complex AI agents, making it easier for developers to integrate various AI instruments and design powerful AI brains. The single API endpoint and Directed Graph GUI make it accessible to any MCP-compatible AI assistant, enhancing the flexibility and usability of the tool. Repository: C0deMunk33/bespoke_automata Primary language: JavaScript GitHub stars: 223 GitHub description: Bespoke Automata is a GUI and deployment pipline for making complex AI agents locally and offline. README excerpt: # Bespoke Automata ![image](https://github.com/C0deMunk33/bespoke_automata/assets/13264637/d

Overview: SVDB is a Swift Vector Database designed for building next-generation user experiences, providing on-device intelligence for Swift apps. Key features: - Local on-device vector database - Built for fast performance - Supports Swift 5.7+ - Easy to use with Swift Package Manager Tech stack: Swift Why it matters: SVDB empowers developers to create more intelligent and personalized user experiences on-device, enhancing the overall user experience. Repository: Dripfarm/SVDB Primary language: Swift GitHub stars: 223 GitHub description: Swift Vector Database. On-device, local vector database for building the next-generation of user experiences

Overview: This project showcases a scalable RAG (Relevance-Augmented Generation) and multi-agent workflow system utilizing Amazon Bedrock models, Amazon Opensearch Serverless as a vector database, and Bedrock AgentCore. It includes features such as document chat, and multi-agent workflows. Key features: - Document Chat - Multi-Agent Tech stack: Python Why it matters: This project demonstrates the integration of advanced AI models and vector databases to create a robust and scalable RAG system, suitable for various applications requiring document-based interactions and multi-agent workflows. Repository: aws-samples/serverless-rag-demo Primary language: Python GitHub stars: 223 GitHub description: Amazon Bedrock Foundation models with Amazon Opensearch Serverless as a Vector DB README excerpt: The project includes features like document chat and multi-agent workflows, and utilizes Amazon Bedrock models, Opensearch Serverless, and Bedrock AgentCore. It also details the architecture, tech stack, prerequisites, and deployment process. Tech Stack: Hosting - S3 + CloudFront (OAC), Auth - Amazon Cognito, Vector DB - OpenSearch Serverless NextGen (scale-to-zero), Indexing - Bedrock Knowledge Base + Data Automation parser, Embeddings - Amazon Titan Embed Text V2 (1024 dims), LLM - Claude Sonnet 4.6 / Opus 4.6 (global inference profiles), Agents - Bedrock AgentCore Runtimes (ARM64 containers), Multi-Agent - Strands Graph pattern (classifier → specialist nodes), IaC

Overview: BioChatter is a Python library designed for connecting biomedical applications to conversational AI, enabling the use of generative AI models in the biomedical domain. Key features: - A generic backend for biomedical AI applications - Seamless integration with multiple LLM providers - Native connection to BioCypher knowledge graphs - Extensive testing and evaluation framework - Living benchmark of specific biomedical applications Tech stack: Python Why it matters: BioChatter simplifies the integration of AI models into biomedical applications, making it easier to develop and deploy conversational AI solutions in the field of biomedicine. Repository: biocypher/biochatter GitHub stars: 223

### Overview: video-SALMONN 2 is a powerful audio-visual large language model (LLM) designed for generating high-quality video captions. Developed by the Department of Electronic Engineering at Tsinghua University and ByteDance, it significantly enhances caption generation throughput and latency. ### Key features: - **Caption Generation**: Generates detailed and accurate video captions. - **Audio-Visual Alignment**: Ensures captions are aligned with the audio and visual content. - **Versatile Models**: Supports 4B, 8B, and 32B models with improved performance. ### Tech stack: Python ### Why it matters: video-SALMONN 2 is a crucial tool for enhancing the accuracy and efficiency of video captioning, making it indispensable for various applications requiring precise and contextually relevant video summaries.

### Overview: This repository is a curated collection of resources focused on the Mechanistic Interpretability (MI) of Large Multimodal Models (LMMs). It aggregates surveys, blog posts, and research papers that explore how LMMs represent, transform, and align multimodal information internally. ### Key features: - Surveys - Blog posts - Papers on input-level attribution - Sparse Autoencoder - Probing - Beyond Logit Lens - Causal Tracing - Neuron Analysis - Steering - Representation - Attention ### Tech stack: Python, Markdown, and GitHub Pages. ### Why it matters: This repository serves as a valuable resource for researchers, developers, and practitioners interested in understanding and improving the interpretability of large multimodal models. It provides insights into how these models process and represent different modalities, which can be crucial for developing more transparent and explainable AI systems.

### Overview: Kalavai is an open-source platform that aggregates and coordinates spare GPU capacity, enabling teams to efficiently use computing resources more effectively. ### Key features: - **Multi-Platform Support**: Aggregates resources from ARM64, AMD64 CPUs, and GPUs (NVIDIA, AMD). - **GPU Utilization**: Increases GPU utilization from devices. - **Multi-Node Deployments**: Supports multi-node, multi-GPU deployments. - **Ready-Made Templates**: Provides templates for common AI building blocks like model inference, GPU clusters, automation workflows, evaluation, and monitoring tools. - **Custom Workloads**: Easy to expand to custom workloads. ### Tech stack: TypeScript ### Why it matters: Kalavai is designed to help teams maximize their computing budget by leveraging spare GPU capacity effectively. It simplifies the process of using computing resources across different platforms, making it easier to deploy AI models and workflows. The project is open-source and free to use, making it accessible to both commercial and non-commercial purposes.

### Overview: Generative Reinforcement Learning (GenerativeRL) is a Python library designed to solve reinforcement learning problems using generative models, such as diffusion and flow models. This library aims to provide a framework for combining the power of generative models with reinforcement learning algorithms, offering support for training, evaluation, and deployment of diverse generative models, and integration with popular RL algorithms tailored for generative models. ### Key Features: - **Training, Evaluation, and Deployment of Generative Models**: Supports various generative models including diffusion and flow models. - **Integration of Generative Models**: Integrates generative models for state representation, action representation, policy learning, and dynamic model learning in RL. - **Popular RL Algorithms**: Implements Q-guided policy optimization (QGPO) and other popular algorithms tailored for generative models. - **RL Environments and Benchmarks**: Supports various RL environments and benchmarks. - **Easy-to-Use API**: Provides an easy-to-use API for training and evaluation. ### Tech Stack: Python ### Why It Matters: GenerativeRL aims to bridge the gap between generative models and reinforcement learning, making it easier to train and evaluate generative models in RL environments, leading to more effective and efficient learning algorithms.

Overview: This repository contains a comprehensive collection of papers and repositories related to various aspects of large language models (LLMs), including zero-shot learning, world models, visual prompt engineering, and more. It serves as an excellent resource for developers interested in the latest advancements in AI and LLM research. Key features: - Zero-shot learning for robot manipulation - Leveraging pre-trained LLMs to construct and utilize world models - Learning and leveraging world models in visual representation learning - Language models meeting world models - Learning to model the world with language - Diffusion world model - Learning to model the world with language - Visual prompt engineering for large multimodal models - Open-vocabulary robotic manipulation through mark-based visual prompting - Set-of-mark prompting unleashes extraordinary visual grounding in GPT-4V - Memory-augmented large multimodal model for long-term video understanding - Scaling video foundation models for multimodal video understanding Tech stack: Python, JavaScript, and various LLMs and frameworks. Why it matters: This repository is essential for developers looking to stay updated with the latest advancements in AI and LLM research. It provides a wealth of papers and repositories that can be used to build, improve, and innovate in the field of AI and LLMs.

### Overview: MERGE is a unified diffusion model designed for image generation and depth estimation. Leveraging streamlined converters and rich visual priors, it expands the zero-shot depth estimation capability of generative image models. ### Key features: - **Unified Model**: Combines image generation and depth estimation into a single model. - **Streamlined Converters**: Simplifies the conversion process between image and depth. - **Rich Visual Prior**: Utilizes visual information from generative image models to enhance depth estimation. ### Tech stack: - Python ### Why it matters: MERGE simplifies the process of zero-shot depth estimation by integrating generative image models with depth estimation, making it easier and more efficient for researchers and practitioners to apply depth estimation capabilities to a wide range of image data. ### Repository: - **Repository**: H-EmbodVis/MERGE - **GitHub Stars**: 222 - **GitHub Description**: NeurIPS 2025 | More Than Generation: Unifying Generation and Depth Estimation via Text-to-Image Diffusion Models ### Primary Language: - **Primary Language**: Python ### GitHub Description: - **Arxiv**: [2510.23574](https://arxiv.org/abs/2510.23574) - **Homepage**: [project-orange](https://h-embodvis.github.io/MERGE) - **Code License**: [Apache 2.0](https://github.com/h-embodvis/MERGE/main/LICENSE) ### Installation: - **Requirements**: