Projects

### Overview: This repository focuses on evaluating the feasibility of detecting and fixing bugs using prompt engineering and fine-tuning techniques in large language models (LLMs). The project utilizes a custom dataset and innovative methods to enhance LLMs' capabilities in understanding and correcting code errors. ### Key Features: - Custom dataset for bug detection and fixing. - Model description: [Unsloth Meta-Llama 3.1-8B]. - Prompt engineering and fine-tuning techniques. - User-friendly interface for evaluation and performance metrics. ### Tech Stack: Jupyter Notebook. ### Why it Matters: This project aims to demonstrate the effectiveness of prompt engineering and fine-tuning in improving the performance of large language models for bug detection and fixing, highlighting the potential of these techniques in enhancing software development efficiency and quality.

**Overview:** This repository provides the official resources for the **CulturalPangea** model and the **CulturalGround** dataset, which contains 30 million high-quality VQA pairs from 42 countries and 39 languages, curated from Wikidata. The dataset is specifically designed to address the common issue of MLLMs misinterpreting long-tail cultural entities by grounding the model in diverse cultural knowledge. **Key features:** - **CulturalGround Dataset**: Contains 30 million high-quality VQA pairs from 42 countries and 39 languages, curated from Wikidata. - **CulturalPangea Model**: An open-source, multilingual, multimodal large language model fine-tuned to understand culturally significant entities from around the world. **Tech stack:** Python **Why it matters:** The **CulturalPangea** model and dataset are designed to address the common issue of MLLMs misinterpreting long-tail cultural entities by grounding the model in diverse cultural knowledge, making it a valuable resource for researchers and practitioners in the field of cultural understanding and language modeling. **GitHub stars:** 13 **Primary language:** Python **Repository:** neulab/CulturalGround

Overview: This project allows users to generate illustrations from novels by highlighting specific sentences using Stable Diffusion. The app is designed for self-hosting and can be integrated into Google Colab and publicly used. Key features: - Live novel: A backend that leverages Stable Diffusion to create fantasy and beautiful images from novel text prompts. - Self-host: Smooth installation for self-hosting, Google Colab free tier, and public use. - Integration: Ready for integration and production, built on top of DocArray data structure, and supports gRPC/HTTP/websockets and TLS. - Installation: Easy installation via Docker, including pre-built image and Docker commands for self-hosting and customization. Tech stack: Python. Why it matters: This tool can help authors promote their work and readers visualize scenes without needing to create illustrations themselves. Repository: nhtlongcs/live-novel GitHub stars: 13 GitHub description: Self-host application can generate illustration from a novel by highlighting certain sentences.

Overview: This project is an Elixir library for building RAG (Retrieval Augmented Generation) systems, featuring multi-LLM support, modular architecture, and advanced features like GraphRAG and advanced chunking. Key features: - Multi-LLM Provider Support: Gemini, Claude, Codex (OpenAI-compatible), and Ollama - Smart Routing: Fallback, round-robin, and specialist routing strategies - Streaming Responses: Real-time streaming for supported providers - Modular RAG Architecture: Pluggable retrieval, vector storage, and reranking - GraphRAG Support: Entity extraction, knowledge graph storage, and community detection - Advanced Chunking: Behavior-based, byte positions, character-based, sentence-based, paragraph-based, recursive, semantic, and format-aware - Agent Framework: Tool-using agents with session memory and built-in tools Tech stack: Elixir Why it matters: This library provides a comprehensive solution for building RAG systems, addressing common issues with traditional language models and enhancing their capabilities with external knowledge sources and advanced chunking techniques. Repository: nshkrdotcom/rag_ex GitHub stars: 13 GitHub description: Elixir RAG library with multi-LLM routing (Gemini, Claude, OpenAI, Ollama), GraphRAG, knowledge graphs, modular retrievers/rerankers, composable pipelines, pgvector integration, advanced chunking, and tool-using agents. Fork of bitcrowd/rag. ``` The README excerpt provides an overview of the library, including its

Overview: This Discord bot emulates Midjourney's capabilities by utilizing the DALL·E API from OpenAI, enabling users to create unique AI-generated images from textual descriptions directly within Discord. Key features: - Text-to-Image Generation - Midjourney-style Commands - Custom Image Sizes - Interactive Art Creation Tech stack: Python Why it matters: This bot provides a user-friendly interface for AI-driven image creation, making it accessible to Discord users without requiring advanced knowledge of AI or programming. Repository: ooovenenoso/Dall-E-3-Discord-Bot Primary language: Python GitHub stars: 13 GitHub description: Discord Bot for AI-Powered Art Creation, emulating Midjourney's capabilities with OpenAI's DALL·E API, enabling user-driven, text-to-image generation. README excerpt:

Overview: This project automates the process of fact-checking using state-of-the-art language models and fine-tuning them on specific datasets to classify claims. Key features: - Fine-tuning of pre-trained language models (BERT, RoBERTa, XLNet) on annotated datasets (FEVER, MultiFC, Liar, COVID19, ANTiVax) for claim classification. - Achieving an accuracy of 98% and F1-score of 98% in the COVID19 & ANTiVax datasets, and 64% and 63% in the FEVER dataset. - Utilizing Hugging Face's Transformers API for easy model fine-tuning. - Tracking training and validation metrics using Weights & Biases (wandb) for real-time monitoring. Tech stack: Python, Transformers API, Hugging Face library, Weights & Biases (wandb). Why it matters: This project addresses the need for automated fact-checking, particularly in the context of fake news and misinformation, by leveraging state-of-the-art language models to classify claims efficiently and accurately. Repository: othmanelhoufi/LM-for-FactChecking Primary language: Python GitHub stars: 13 GitHub description: An automated solution for fact-checking using available claims and fake-news datasets to fine-tune state-of-the-art language models published recently for NLP tasks (BERT, RoBERTa, XLNet, ConvBERT...) in order to classify unseen claims.

Overview: Crucible is a MoE expert compression toolkit designed to optimize large Mixture-of-Experts language models for efficient local inference on consumer hardware. It scores experts based on their contribution to calibration data and prunes them to create specialized models that are quantized and ready for deployment. Key features: - Prune - Merge - Forge specialist models Tech stack: Python 3.12+ Why it matters: Crucible helps developers create more efficient and lightweight models by removing unnecessary weights, making them suitable for low-power devices like consumer hardware. Repository: peonist-ai/crucible GitHub stars: 13

Overview: This repository contains the official code for the paper "Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagation" (ICLR'24), which proposes GreenTrainer, an extension to LLMs that adaptively selects trainable tensors for backpropagation to achieve user-specified finetuning FLOPs reduction while maintaining accuracy. Key features: - GreenTrainer is an extension to LLMs that adaptively selects trainable tensors for backpropagation. - It is designed to reduce finetuning FLOPs while maintaining accuracy. - The code includes implementations for both decoder structures and encoder-decoder structures. Tech stack: Python Why it matters: GreenTrainer is a significant contribution to the field of AI optimization, offering a practical solution for reducing the computational cost of fine-tuning large language models without compromising performance.

Overview: This repository contains the official code for the paper "Modality Plug-and-Play: Elastic Modality Adaptation in Multimodal LLMs for Embodied AI". It includes the implementation of mPnP-LLM, a system that enables fully elastic modality adaptation for LLMs via trainable latent connections. Key features: - mPnP-LLM: A system for elastic modality adaptation in multimodal LLMs. - NuScenes-QA dataset: A mini-split dataset used for evaluation. - ViT-small for RGB camera views, pre-trained RangeViT for LiDAR inputs. Tech stack: Python. Why it matters: This project provides a practical solution for adapting LLMs to different modalities, enhancing their performance in embodied AI applications. Repository: pittisl/mPnP-LLM Primary language: Python GitHub stars: 13 GitHub description: Code for paper "Modality Plug-and-Play: Elastic Modality Adaptation in Multimodal LLMs for Embodied AI"

Overview: Chalchitra.ai is a powerful tool that transforms text into captivating, synchronized animated videos, offering multilingual support for Indian languages. Key features: - Text to animated video conversion - Multilingual support for Indian languages - Versatile use cases including digital advertising, e-learning, language learning, entertainment, news sharing, and personalized greetings Tech stack: Python, GPT-3.5 Turbo, Stable Diffusion, Google Cloud Translation API, Google Cloud Text to Speech API, SadTalker Why it matters: Chalchitra.ai simplifies the creation of engaging and effective animated videos, making it easier for creators, educators, and businesses to communicate their messages and capture the attention of their audiences.

Overview: This sample skill demonstrates the new Alexa Presentation Language (APL) and showcases a basic fact skill compatible with the Alexa Presentation Language (APL). It selects a random fact about monkeys and provides an image when invoked. Key features: - Selects a random fact about monkeys - Provides an image and the fact when invoked - Compatible with devices with a display Tech stack: JavaScript Why it matters: This skill is a template for a basic fact skill in the Alexa Presentation Language (APL) and demonstrates how to create a compatible skill with Alexa's new language, showcasing the new multi-modal functionality.

Overview: Marlin is a speculative decoding orchestrator designed for LLM serving, featuring a draft-model ensemble with rejection sampling, KV-cache-aware admission control, and self-tuning draft lengths. Key features: - Ensemble composition - Rejection sampling - Drift detection - Autotuner - Cache admission - Deterministic plans Tech stack: Go Why it matters: Marlin accelerates inference by leveraging multiple draft models, ensuring efficient and cost-effective processing. Repository: rodney-rate/MLN GitHub stars: 13 GitHub description: Speculative decoding orchestrator for LLM serving: draft-model ensemble with rejection sampling, KV-cache-aware admission control and self-tuning draft lengths - the substrate for faster, cheaper inference.

### Overview: IKEA-Dataset is a comprehensive multilingual-multimodal machine translation dataset containing textual and visual data of IKEA and Under Armour products from 2017. It is used for training and evaluating models in the field of multimodal translation. ### Key Features: - **Multilingual Multimodal Translation**: The dataset includes both English and French descriptions for IKEA products and German and French descriptions for Under Armour products. - **Image and Text Pairing**: Each sample consists of a bilingual pair of text descriptions and corresponding images. - **Normalization and Tokenization**: The dataset is normalized and tokenized, and the text is converted to lowercase and processed through byte-pair encoding. ### Tech Stack: - **GitHub Repository**: samchengcs/IKEA-Dataset - **GitHub Stars**: 13 ### Why It Matters: IKEA-Dataset provides a rich resource for researchers and practitioners in the field of multimodal machine translation, enabling the development of models that can effectively handle the complexities of multimodal data and improve translation quality across different languages and modalities.

### Overview: This repository contains the Pytorch code for the "BAH Dataset for Ambivalence/Hesitancy Recognition in Videos for Digital Behavioural Change" (ICLR 2026). The dataset consists of 1,427 videos with a total duration of 10.60 hours, collected from 300 participants across Canada. The dataset is designed to mirror real-world digital behavior change interventions and is annotated by three experts, providing timestamps and frame-level annotations for Ambivalence/Hesitancy (A/H). ### Key Features: - **Data Collection**: 1,427 videos with a total duration of 10.60 hours from 300 participants across Canada. - **Annotation**: The dataset is annotated by three experts, providing timestamps indicating where A/H occurs and frame-level annotations with A/H cues. - **Code**: The code is written in Pytorch version 2.2.2. - **Benchmarking**: The dataset includes benchmarking results using baseline models on A/H recognition for frame- and video-level recognition, zero-shot prediction, and personalization with source-free domain adaptation methods. ### Tech Stack: Pytorch ### Why it Matters: This dataset is crucial for the development of machine learning models to recognize Ambivalence/Hesitancy (A/H) in digital health interventions. It provides a valuable resource for researchers and practitioners aiming to personalize and cost-effectively personalize digital behavior change interventions. The annotated data helps in understanding the subtle and conflicting emotions that individuals may experience, making it essential for

Overview: Verdict is a high-performance serving engine designed for System One decision models, providing up to 66x the throughput of the model authors' own servers with the same answers as official SDKs. Key features: - GPU acceleration for text, image, and video processing - GPU-only throughput test results - CPU throughput and latency test results - ModelScope integration for seamless model loading Tech stack: Rust Why it matters: Verdict offers significant performance improvements over traditional servers, making it an essential tool for developers working with System One models, ensuring faster and more accurate model responses. GitHub stars: 13

Overview: This repository provides a Dockerized inference server and benchmark suite for the Google Gemma 4 family of models on the NVIDIA DGX Spark (GB10). The models, including E2B, E4B, 26B-A4B, and 31B, are optimized for ARM64 CUDA 13 builds using llama.cpp. Key features: - Dockerized inference server - Benchmark suite for single-sequence throughput, context window scaling, multi-user concurrency, and chain-of-thought timing - ARM64 CUDA 13 builds using llama.cpp Tech stack: Shell Why it matters: This setup allows for efficient and high-performance inference of the Gemma 4 models on the NVIDIA DGX Spark, leveraging unified memory and optimized CUDA architecture for optimal performance. Repository: shamily/gemma4-llama-dgx-spark GitHub stars: 13 GitHub description: Dockerized inference server and benchmarks for quantized Gemma 4 family on the NVIDIA DGX Spark (GB10). Features ARM64 CUDA 13 builds using llama.cpp.

Overview: Alisia is a conversational AI application that integrates web search capabilities, providing users with access to the internet's latest knowledge through a chat-based interface. Key features: - Interactive chat interface - Web search integration - Learning and fun features Tech stack: JavaScript, Python, React, FastAPI, Vite Why it matters: Alisia leverages AI techniques like RAG and VectorDB to provide insightful and relevant responses, making it a valuable tool for anyone looking to engage in real-time conversations or access up-to-date information.

### Overview: ScratchFormers is a project that focuses on building various transformer model architectures and modules from scratch. The repository includes modules for attention mechanisms, positional embeddings, and low-rank adaptation techniques. The project aims to provide a comprehensive understanding of transformer models and their implementation from the ground up. ### Key features: - **Attention Mechanisms:** Implementations of multi-head causal attention, multi-head cross attention, and multi-head grouped query attention. - **Positional Embeddings:** Rotary positional embeddings (RoPE) for efficient model training. - **Low-Rank Adaptation (LoRA):** Implementation based on the tutorial by Sebastian Raschka. - **KV Cache:** Support for KV Cache that works with RoPE. - **Models:** - LLaMA: A model for process implementation and inference. - Simple Vision Transformer: Model implementation and inference. - GPT2: Model implementation and inference. - OpenAI CLIP: Model implementation and inference. - Encoder-Decoder Transformer: Model implementation. - BERT - MLM: Model implementation for masked language modeling. ### Tech Stack: Jupyter Notebook ### Why it Matters: ScratchFormers provides a deep dive into the implementation of transformer models from the ground up, making it an excellent resource for developers looking to understand and implement transformer architectures. The project's focus on low-rank adaptation and KV caching techniques adds practical value to the transformer model implementation.

### Overview: A single-agent system for insurance damage claim verification, designed to be auditable, deterministic, and reproducible. It leverages a vision model to interpret photos and a deterministic codebase to enforce safety rules, ensuring fair and transparent decision-making. ### Key features: - **Safety Gate in Code, Not in Prompt:** The safety gate is enforced through deterministic rules rather than in the prompt, ensuring that model decisions cannot be overridden. - **Handling Non-Original Images:** Non-original images are flagged for manual review rather than being automatically downgraded, reflecting how a human reviewer would handle such cases. - **Structured Output:** The system produces a fixed 14-column output for each claim, including evidence standard met, risk flags, issue type, object part, claim status, supporting fields, and more. - **Image Attachment Up Front:** All images are attached upfront, allowing the model to decide when it has seen enough information to commit. ### Tech Stack: Python ### Why it Matters: The system ensures fairness, transparency, and audibility in insurance damage claim verification, making it a valuable tool for insurers and claimants alike.

Overview: Vectra is a unified Ruby client designed for vector databases, allowing developers to switch providers seamlessly by writing once. Key features: - Supports multiple providers including Pinecone, Qdrant, Weaviate, PostgreSQL, and Memory. - Provides a unified API for vector database operations. - Simplifies the process of switching between different vector database providers. Tech stack: Ruby Why it matters: Vectra simplifies the integration with various vector database providers by offering a single, unified client, making it easier for developers to switch providers without changing their code. This is particularly useful in environments where multiple vector database solutions are being used or when transitioning to new ones.