Projects

### Overview: Transform your PDF experience with MultiPDF Chat AI App! 🚀 This Streamlit-based web application allows users to upload multiple PDF documents, extract text information, and engage in real-time conversations with a chatbot powered by Langchain, Google Gemini Pro, and FAISS Vector DB. Seamlessly integrate with Google Gemini OpenSource language models for instant, accurate responses. ### Key Features: - **Adaptive Chunking**: Dynamically adjusts window size and position for RAG, balancing fine-grained and coarse-grained data access based on data complexity and context. - **Multi-Document Conversational QA**: Supports simple and multi-hop queries across multiple documents simultaneously, breaking the single-document limitation. - **File Compatibility**: Supports both PDF and TXT file formats. - **LLM Model Compatibility**: Supports Google Gemini Pro, OpenAI GPT 3, Anthropic Claude, Llama2 and other open-source LLMs. ### Tech Stack: Python ### Why it Matters: Transform your PDF experience with MultiPDF Chat AI App! 🚀 Seamlessly integrate with Google Gemini OpenSource language models for instant, accurate responses. 📚💬 Repository: GURPREETKAURJETHRA/Multi-PDFs_ChatApp_AI-Agent GitHub stars: 138

Overview: gensokyo-llm is an open-source project that provides a unified API for integrating various large language model interfaces, including support for Onebotv11, enabling seamless integration with multiple platforms such as QQ, Discord, Kook, Telegram, and Gensokyo. Key features: - Support for multiple large language model interfaces including OpenAI, Google, and Alibaba. - Ability to generate keyboard-style bubbles for chat interactions. - Streamlined and context-aware agent for Onebotv11. - Provides a unified API structure for seamless integration with multiple platforms. - Supports multiple Onebotv11 standard frameworks. - Integrates with Gensokyo framework for seamless integration with QQ, Discord, Kook, Telegram, and Gensokyo. Tech stack: Go Why it matters: gensokyo-llm simplifies the integration of large language models across multiple platforms, providing a unified API that enhances user experience and simplifies development for developers working with various chatbots and platforms. GitHub stars: 138

Overview: SlideFormer is an efficient heterogeneous co-design framework for fine-tuning models on a single GPU, achieving up to 6.27x higher throughput compared to offloading baselines while reducing peak GPU and CPU memory usage by 50% and 40%, respectively, and enabling full-parameter fine-tuning of 100B+ models. Key features: - Layer-sliding execution - Lightweight asynchronous engine - Heterogeneous memory hierarchy - Memory-efficient implementation Tech stack: Python Why it matters: SlideFormer is essential for accelerating fine-tuning on single-GPU setups, offering significant performance improvements and reduced memory usage, making it ideal for large models on commodity hardware.

Overview: Anti-Vocale is an Android app that transcribes voice messages locally on-device, offering full language support and no internet requirement. It works with various messaging apps and supports recordings of calls and meetings, exporting results as subtitles or timestamped text. Key features: - Transcribes voice messages from WhatsApp, Telegram, and other apps. - Supports multiple languages and community fine-tunes. - Transcribes recordings beyond a model's limit automatically. - Handles aggressive vendor customizations to keep transcription open. - Integrates with WhatsApp voice notes and Forkgram as built-in offline transcribers. Tech stack: Kotlin Why it matters: Anti-Vocale fills the gap in language support for major messaging apps, offering an alternative to partial or no transcription provided by the big players. It's designed for compatibility across a wide range of devices and inference backends, ensuring real-world performance and user experience. Repository: RisorseArtificiali/anti-vocale Primary language: Kotlin GitHub stars: 136 GitHub description: Android app for transcribing voice messages locally on-device, with no internet required.

Overview: RA-Bench is a benchmark designed to evaluate the ability of current AI-generated video detection systems to identify realistic generated videos anchored in real footage of crisis events. It includes 17,886 public videos, 1,830 real anchors, 16,056 generated videos, and 9 generators, covering 10 crisis categories and 44 subcategories. Key features: - Real videos as anchors - 1,830 real anchors - 16,056 generated videos - 9 generators - 10 crisis categories and 44 subcategories - 633 human-deceptive generated videos - 93.8 GB public media - 25,575 public media files Tech stack: Python Why it matters: RA-Bench is crucial for assessing the effectiveness of AI-generated video detection systems in identifying realistic generated videos, which can be misleading to human reviewers and pose a threat in real-world crisis settings. Repository: Shuo-Liang-0111/RA-Bench Primary language: Python GitHub stars: 138 GitHub description: Real-event-anchored benchmark for detecting AI-generated videos in real-world crisis settings.

Overview: Sketch-of-Thought (SoT) is an innovative prompting framework designed to enhance the efficiency and accuracy of language models by combining cognitive-inspired reasoning paradigms with linguistic constraints. Key features: - Conceptual Chaining: Connects essential ideas in logical sequences through structured step links. - Chunked Symbolism: Organizes numerical and symbolic reasoning into structured steps with equations, variables, and arithmetic operations. - Expert Lexicons: Leverages domain-specific shorthand, technical symbols, and jargon for precise and efficient communication. Tech stack: Python Why it matters: SoT offers a more efficient and accurate way to reason by minimizing output token usage while preserving the accuracy of commonsense, multi-hop inference, and fact-based recall tasks. It also excels in mathematical problems and technical calculations, and it can communicate in a highly efficient manner suitable for technical disciplines. Repository: SimonAytes/SoT Primary language: Python GitHub stars: 138 GitHub description: Official code repository for Sketch-of-Thought (SoT) README excerpt: # Sketch-of-Thought (SoT) [](LICENSE) [](https://www.python.org/downloads/) [](https://pytorch.org/) [![Hugging Face](https://img.shields.io

### Overview: `last_layer` is a security library designed to protect LLM applications from prompt injection attacks, jailbreaks, and exploits. It acts as a robust filtering layer to scrutinize prompts before they are processed by LLMs, ensuring only safe and appropriate content is allowed through. ### Key Features: - **Ultra-fast scanning**: Achieves >=2ms latency for prompt injection/jailbreak scanning, ensuring minimal impact on user experience. - **Privacy-focused**: Designed with privacy in mind, `last_layer` operates without tracking or making network calls, ensuring data stays within your infrastructure. - **Serverless-ready**: Compatible with serverless platforms like Vercel or AWS Lambda. - **Advanced detection mechanisms**: Utilizes a combination of a closed, pruned AutoML model, heuristic analysis, and regular expression datasets to accurately identify threats with an accuracy of 92%. ### Tech Stack: Python ### Why it Matters: `last_layer` significantly reduces the risk of prompt-based attacks and exploits but cannot guarantee complete protection against all possible threats.

Overview: This repository contains tutorials, projects, and notebooks for Large Language Models (LLMs), Generative AI, and Transformer architectures, designed for engineers, data scientists, and developers. Key features: - Foundations of neural networks and transformer architecture - Fine-tuning techniques for modern models like LoRA and PPO - Implementations of semantic search and Retrieval-Augmented Generation (RAG) using ChromaDB - Deployment guides for LLMs using FastAPI and AWS - Front-end applications using Streamlit - Practical AI agent (email drafter) using FastAPI - GPU acceleration with NVIDIA CUDA and RAPIDS - General deep learning concepts with Keras - Monitoring LLM services with Grafana and Prometheus Tech stack: Jupyter Notebook Why it matters: This repository provides a comprehensive guide for mastering the full lifecycle of LLM development, from fundamental concepts to advanced deployment and monitoring, making it valuable for engineers, data scientists, and developers looking to enhance their skills in Generative AI and Transformer architectures.

### Overview: Vektori is a memory system designed for AI agents that remembers the story, not just the facts. It uses a three-layer sentence graph to understand how preferences and patterns evolve over time, providing context-aware insights. ### Key Features: - **Memory Graph Layers:** - **FACT LAYER (L0):** Short, crisp statements. - **EPISODE LAYER (L1):** Patterns discovered through graph traversal. - **SENTENCE LAYER (L2):** Raw conversation history with sequential NEXT edges. - **SQLite by Default:** SQLite is used for simplicity and scalability. Users can switch to Postgres, Neo4j, Qdrant, or Milvus for advanced scaling. - **Context-Oriented Insights:** Agents understand the trajectory of preferences and patterns, providing context-aware insights. ### Tech Stack: - Python ### Why It Matters: - **Contextual Understanding:** Unlike traditional systems, Vektori remembers the story, not just the facts, by understanding the trajectory of preferences and patterns over time. - **Scalability:** It scales well with the addition of more layers, making it suitable for complex systems. - **Real-World Applications:** Vektori has been tested on long-horizon memory benchmarks, showing high recall rates and context-aware insights, making it useful for applications requiring understanding of user behavior over time. ### GitHub Stars: 138 ### Primary Language: Python ### GitHub Description: Memory that remembers the story not just the facts. File System Memory and Three layer sentence

### Overview: Auto-improve is a text improvement tool that uses a Generative Adversarial Network (GAN) to enhance any text artifact. It automatically mutates the input file, grades each candidate with a separate model, filters out unverified improvements, and keeps only the verified ones. The git history serves as the improvement log, ensuring every commit is a genuine gain. ### Key Features: - **Mutate and Grade**: The model mutates the input file and grades each candidate against a rubric. - **Pairwise-Judge**: Each candidate is evaluated against the current champion using a debiased pairwise gate, keeping only the improvements that win. - **Revert Rejected**: Any mutation that fails to win is reverted, ensuring only verified improvements are kept. - **Monotonic Improvement**: The git history is the improvement log, making every commit a real gain. ### Tech Stack: - Python 3.9+ ### Why It Matters: Auto-improve eliminates the "LLM slop" of unverified rewrites by using a separate judge model and a pairwise keep/discard gate. This ensures that every commit in the git history is a verified improvement, providing a clear and trackable history of the project's progress.

### Overview: WhisperClip simplifies your life by automatically transcribing audio recordings and saving the text directly to your clipboard. With just a click of a button, you can effortlessly convert spoken words into written text, ready to be pasted wherever you need it. Powered by OpenAI's Whisper model via [faster-whisper](https://github.com/SYSTRAN/faster-whisper), it provides fast, free, and fully local transcription — your audio never leaves your machine. ### Key features: - **Record audio with a simple click or global hotkey (`Alt+Shift+S`).** - **Fast, local transcription using OpenAI's Whisper model with GPU acceleration (CUDA).** - **Option to save transcriptions directly to the clipboard.** - **Transcribe existing audio files via the file picker.** - **Optional LLM context prefix — prepends a note explaining the text was generated via speech-to-text.** - **Real-time audio visualizer showing recording and transcription states.** - **Remote transcription API — transcribe audio from your phone using your PC's GPU via an iOS Shortcut or Android app.** ### Tech stack: Python ### Why it matters: WhisperClip simplifies the process of converting audio to text, making it a valuable tool for anyone who frequently needs to transcribe audio recordings. The application's integration with OpenAI's Whisper model and its fast, free transcription capabilities make it an efficient solution for those on a budget or who need to transcribe large amounts of audio quickly.

Overview: Repilot is a patch generation tool introduced in the ESEC/FSE'23 paper "Copiloting the Copilot: Fusing Large Language Models with Completion Engines for Automated Program Repair". It leverages the synergy between a semantics-based code completion engine and an auto-regressive large language model to generate more efficient valid patches. Key features: - Repilot leverages the synergy between a semantics-based code completion engine and an auto-regressive large language model for efficient valid patch generation. - It is implemented for Java patch generation as a complex hybrid system combining a Modified Eclipse JDT Language Server and Python's huggingface/transformers interface. - It is provided as a Docker image for quick start, which includes a `meta_config.json` file for proper operation. Tech stack: Python, Docker, Eclipse JDT Language Server, huggingface/transformers. Why it matters: Repilot is a significant advancement in the field of program repair, offering a more efficient and effective way to generate patches for software issues.

### Overview: This project exports large language models (LLMs) like LLaMA, Baichuan, Qwen, ChatGLM2, and Bloom to ONNX files without modifying the original models. It supports various models and provides usage examples for different export methods. ### Key features: - **Export LLMs to ONNX**: Converts models like LLaMA, Baichuan, Qwen, ChatGLM2, and Bloom to ONNX files. - **Supported Models**: Includes LLaMA, Alpaca, Baichuan, Qwen 1.5/2, ChatGLM2, and Bloom. - **Usage Examples**: - **LLaMA (Hugging Face Format)**: Converts models to multiple ONNX files using `export_llama.py`. - **Qwen**: Converts models to single ONNX file using `export_qwen_naive.py`. - **ChatGLM2**: Converts models to ONNX files using `export_chatglm2.py`. - **Bloom**: Converts models to ONNX files using `export_bloom_naive.py` or `export_bloom.py`. ### Tech stack: Python ### Why it matters: This project is essential for developers who need to deploy large LLM models efficiently. The ability to export models to ONNX format allows for better compatibility with existing deployment frameworks and tools, ensuring smoother integration into production environments.

### Overview: TokenCompose is a text-to-image diffusion model that leverages token-level supervision for enhanced multi-category instance composition and photorealism. It was presented at CVPR 2024 and is a part of the CoMat framework. ### Key Features: - **Token-Level Supervision**: TokenCompose uses token-level consistency terms to ensure multi-category instance composition. - **Enhanced Photorealism**: The model achieves high accuracy in photorealistic image generation. - **Multi-Category Instance Composition**: It provides high accuracy in multi-category instance composition tasks. ### Tech Stack: - **Model**: Stable Diffusion v1.4 - **Supervision**: Token-level supervision ### Why It Matters: TokenCompose is a significant advancement in text-to-image diffusion models, offering improved multi-category instance composition and photorealism. Its successful application in the CoMat framework highlights its potential for enhancing image generation tasks. ### GitHub Stars: 138 ### Repository: - **Repository**: mlpc-ucsd/TokenCompose - **Primary Language**: Jupyter Notebook ### Updates: - Incorporated into CoMat for enhanced text-to-image attribute assignments. - Accepted to CVPR 2024. - Used as a base model for enhanced compositionality in the RealCompo paper.

### Overview: An application that converts live speech or audio recording into text and displays relevant Indian Sign Language images or GIFs using Natural Language Processing and Machine Learning. ### Key features: - Speech recognition using Google Speech API and Sphix for offline use. - Text preprocessing using NLP. - Dictionary-based machine translation. ### Tech stack: Python. ### Why it matters: This application aims to bridge the communication gap between deaf people and others, providing a solution for effectively communicating through sign language, especially for those who find it difficult to learn the language.

Overview: VisorGPT is a NeurIPS 2023 paper that introduces a method for customizing spatial layouts in conditional image synthesis models, specifically focusing on ControlNet and GLIGEN. This project leverages Generative Pre-Training to learn visual priors, enabling more effective control over image synthesis. Key features: - Customizable spatial layouts for ControlNet and GLIGEN models - Utilizes Generative Pre-Training to enhance visual prior learning - Supports three types of data: box, keypoint, and mask Tech stack: Python Why it matters: This project showcases innovative techniques in image synthesis and control, demonstrating how generative pre-training can be applied to improve the quality and control of conditional image synthesis models, particularly in fields like art and design where precise control over image elements is crucial.

### Overview: This repository contains code for the paper "On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines," authored by Marius Mosbach, Maksym Andriushchenko, and Dietrich Klakow from Saarland University and EPFL. The paper addresses the instability of fine-tuning pre-trained BERT models and proposes a strong baseline to improve stability. ### Key features: - Fine-tuning BERT, RoBERTa, and ALBERT on three commonly used GLUE datasets. - Shows that observed instability is due to optimization difficulties leading to vanishing gradients. - Demonstrates that differences in generalization can explain remaining variance in downstream task performance. - Presents a simple but effective baseline to stabilize fine-tuning of BERT-based models. ### Tech stack: Python ### Why it matters: The paper addresses a significant issue in the field of fine-tuning pre-trained transformer-based models, providing a strong baseline that significantly improves the stability of these models.

Overview: This is a pytorch implementation of Denoising Diffusion Implicit Models (DDIM), a method for diffusion-based models. Key features: - DDIM implementation in PyTorch - Uses the original paper's implementation as a reference - Supports training and generating images Tech stack: PyTorch Why it matters: DDIM is a powerful technique for diffusion models, offering improved stability and performance compared to other methods.

Overview: Heimdall is a lightweight CPU-only memory approach that indexes everything you touch, providing persistent memory across repositories and projects. It simplifies the process of finding previously solved problems, reducing orientation time, and eliminating token spend. Key features: - Persistent memory across repositories and projects - Reduced orientation time to seconds - Zero token spend on memory maintenance - Semantic memory verification for high confidence results Tech stack: JavaScript Why it matters: Heimdall makes it easy to find previously solved problems, reducing orientation time and token spend, and providing a more reliable and efficient way to manage knowledge across projects. Repository: ArihantDeva/heimdall GitHub stars: 137

Overview: DocuMind is a document-based Retrieval-Augmented Generation (RAG) application that converts your documents into dynamic knowledge sources, providing accurate and fact-based answers to your questions. Key features: - Ask questions about your documents and get instant, accurate responses. - Provides relevant, fact-based answers using advanced RAG technology. - Documents are processed securely on your local machine with no data leakage. Tech stack: Axum Rust server for backend, Ollama for inference, and Qdrant vector database for storage. Why it matters: DocuMind simplifies access to information by turning your documents into a dynamic knowledge source, making it easier to find and understand the relevant data quickly and accurately.