Projects

Overview: This project utilizes LangChain and Databutton to create RAG-enabled chatbots, focusing on PDF parsing and indexing for enhanced context retrieval. Key features: - PDF parsing and indexing - Session state storage - LangChain integration for retrieval and prompt augmentation - Streamlit deployment for easy integration Tech stack: Python Why it matters: This project leverages LangChain and Databutton to enhance the retrieval and prompt generation capabilities of chatbots, making them more effective and contextually relevant. GitHub stars: 168 Repository: avrabyt/RAG-Chatbot

Overview: This project presents a novel extension to TS-TCC, focusing on self-supervised contrastive representation learning for semi-supervised time-series classification. The work is accepted in TPAMI and includes two new training modes: "gen_pseudo_labels" and "SupCon". Key features: - Extends TS-TCC with self-supervised contrastive learning - Two new training modes: "gen_pseudo_labels" and "SupCon" - Fine-tuning and linear evaluation of pre-trained models Tech stack: Python Why it matters: This project offers an innovative approach to semi-supervised time-series classification, leveraging self-supervised learning techniques to improve model performance on limited labeled data. Repository: emadeldeen24/CA-TCC Primary language: Python GitHub stars: 168 GitHub description: [TPAMI 2023] Self-supervised Contrastive Representation Learning for Semi-supervised Time-Series Classification

Overview: RunGPT is an open-source cloud-native framework designed to simplify the deployment and management of large language models (LLMs) on a distributed cluster of GPUs. Key features: - Scalable architecture for handling high traffic loads - Optimized for low-latency inference - Automatic model partitioning and distribution across multiple GPUs - Centralized model management and monitoring - REST API for easy integration with existing applications Tech stack: Python Why it matters: RunGPT aims to make it a one-stop solution for a centralized and accessible place to gather techniques for optimizing LLMs and making them easy to use for everyone. Repository: jina-ai/rungpt

### Overview: Feel how fast a local LLM runs before you buy the hardware. Choose a GPU or Mac, and see the text stream at the predicted tokens per second, including the wait time for the first token. Compare setups side by side, and explore memory fit, 30 GPUs, Macs, and CPUs across popular models. ### Key features: - **Real-time simulation:** Text streams at the predicted tokens per second, after a realistic time to first token for your prompt length. - **Race mode:** Run two setups side by side, e.g., a 5090 against a Mac Studio. - **Memory fit:** See whether a model fits at each quantization, needs partial offload to system RAM, or won't run at all. - **30 GPUs, Macs, and CPUs × 15 popular models**, across CUDA, Metal, ROCm, Vulkan, and CPU. ### Tech stack: TypeScript ### Why it matters: Feel how fast a local LLM runs before you buy the hardware. Compare setups side by side, and explore memory fit, 30 GPUs, Macs, and CPUs across popular models.

Overview: This project is an AI-powered search tool designed to quickly summarize the content of private large unstructured documents stored in Dropbox or OneDrive. It leverages the power of large language models to provide real-time information from these documents. Key features: - Real-time AI-powered search capabilities - Integrates with Dropbox and OneDrive for seamless document access - Supports multiple languages and unstructured document types Tech stack: Python Why it matters: This tool streamlines the process of finding relevant information from private documents, making it easier for users to access and understand complex data without the need for manual searching. It enhances productivity and efficiency by providing instant insights directly from the cloud storage, which is particularly useful for businesses and individuals managing large volumes of unstructured data.

### Overview: This repository contains a collection of Jupyter Notebooks designed to provide practical examples and demonstrations of how to perform inference and fine-tuning on vision models using the 🤗 Transformers library. The notebooks cover various tasks such as object detection, depth estimation, and pose estimation, offering insights into how to effectively use the models for real-world applications. ### Key Features: - RT-DETRv2: Object Detection - DAB-DETR: Object Detection - D-Fine: Object Detection - DepthPro: Depth Estimation - PromptDepth: Depth Estimation - DistillAnyDepth: Depth Estimation - SuperGlue: Keypoint Matching ### Tech Stack: Jupyter Notebook ### Why It Matters: These notebooks serve as a valuable resource for developers and researchers looking to apply vision models from the 🤗 Transformers library to practical tasks such as object detection, depth estimation, and pose estimation. They provide a hands-on approach to understanding and implementing these models, making it easier to integrate them into existing projects or develop new applications.

Overview: A Python Telegram bot powered by Google's gemini-pro LLM API, designed to generate creative text formats based on user input. Key features: - Generate creative text formats like poems, code, scripts, musical pieces, etc. - Stream the generation process, allowing you to see the text unfold in real-time. - Reply to messages with Bard's creative output. - Easy to use with simple commands: `/start` to greet the bot and `/help` for information. - Send any text message to trigger the generation process. - Send any image with captions to generate responses based on the image (multi-modal support). - User authentication to prevent unauthorized access by setting `AUTHORIZED_USERS` in the `.env` file (optional). Tech stack: Python Why it matters: This bot offers a fun and interactive way to explore the possibilities of large language models, generating creative text formats based on user input. Repository: rabilrbl/gemini-pro-bot GitHub stars: 170

### Overview: ComfyUI-LongLook is a ComfyUI node pack that implements FreeLong spectral blending for Wan 2.2 video generation. This technology improves motion consistency within 81 frames, leading to smoother trajectories, less reversal, and better prompt adherence. It enhances the quality of video generation by ensuring consistent motion and prompt adherence, making it particularly useful for long videos. ### Key Features: - **Improved Motion Consistency**: FreeLong spectral blending dramatically reduces motion reversal and subject drift, ensuring smoother and more consistent motion within 81 frames. - **Chunked Generation**: By breaking the generation into 81-frame chunks, FreeLong ensures clean anchors and reliable continuation, making the generation of unlimited-length videos more feasible. - **Single Generation 81+ Frames**: While not always beneficial, FreeLong can sometimes help with the quality of the generation, but its primary value lies in the consistency within the native window. ### Tech Stack: The project utilizes Python for implementation and ComfyUI for integration with the ComfyUI framework. ### Why It Matters: - **Consistent Motion**: FreeLong ensures that the motion within 81 frames is more stable and predictable, leading to smoother and more coherent video sequences. - **Prompt Adherence**: By maintaining the initial motion direction, FreeLong helps in better adherence to the prompt, making the generated videos more faithful to the intended content. - **Unlimited Length**: The ability to generate unlimited-length videos by chunking the generation process makes this technology particularly valuable for long-form video content. ### Installation: To use Comfy

### Overview: This repository provides a comprehensive guide on how to augment Large Language Models (LLMs) with real-time data using Retrieval-Augmented Generation (RAG). The project includes Jupyter notebooks demonstrating the integration of LLMs with vector databases and semantic search, focusing on advanced techniques such as fine-tuning embeddings and fine-tuning with synthetic data. The notebooks are designed to be used in conjunction with the book "Quick Start Guide to LLMs," offering practical insights into enhancing LLMs with dynamic and context-aware applications. ### Key Features: - **RAG Integration with LLMs**: Demonstrates how to integrate LLMs with vector databases and semantic search. - **Fine-tuning Embeddings**: Includes notebooks on fine-tuning embeddings using custom preference data and synthetic data. - **Real-Time Data Processing**: Highlights the importance of real-time data in enhancing the capabilities of LLMs. ### Tech Stack: - Python (Jupyter Notebook) ### Why it Matters: This project is essential for developers looking to improve the performance and relevance of their LLMs by incorporating real-time data. The techniques demonstrated in the notebooks can be applied to various applications, including dynamic and context-aware apps, to provide users with more accurate and up-to-date information.

### Overview: The `awesome-mmps` repository is a collection of resources and publications focused on multimodal machine learning with physiological signals, including EEG, ECG, eye movement, EMG, and other related signals for emotion recognition and natural language processing. ### Key Features: - **EEG (Electroencephalography)**: EEG data for emotion recognition. - **ECG (Electrocardiogram)**: ECG data for physiological analysis. - **EDA (Exploratory Data Analysis)**: Techniques for analyzing physiological signals. - **Eye Movement**: Data for eye tracking and emotion recognition. - **EOG (Electrooculography)**: Data for eye movement analysis. - **EMG (Electromyography)**: Data for muscle activity analysis. - **Other**: Miscellaneous physiological signals and related data. ### Tech Stack: Python, TensorFlow, PyTorch, and other machine learning libraries. ### Why It Matters: This repository provides a comprehensive collection of resources and publications for researchers and practitioners interested in multimodal machine learning with physiological signals. It serves as a valuable resource for developing and evaluating models for emotion recognition, natural language processing, and other applications involving physiological data.

### Overview: Recent advancements in using large language models (LLMs) as agent models for interactive planning tasks have shown promising results, but they still face challenges with trial-and-error in global planning and hallucinations in local planning due to poor understanding of the real world. To address these issues, we introduce the **World Knowledge Model (WKM)**, a parametric model that integrates global prior knowledge and dynamic local knowledge to facilitate effective agent planning. Experimental results on three complex real-world simulated datasets with three state-of-the-art open-source LLMs (Mistral-7B, Gemma-7B, and Llama-3-8B) demonstrate superior performance compared to strong baselines. Additionally, we analyze that WKM can alleviate blind trial-and-error and hallucinations, enhancing the agent's understanding of the world. ### Key Features: - **World Knowledge Model (WKM)**: A parametric model that integrates global prior knowledge and dynamic local knowledge to facilitate effective agent planning. - **Task Knowledge**: Prior task knowledge provided to guide global planning. - **State Knowledge**: Dynamic state knowledge to assist local planning. - **Training and Evaluation**: Implemented using LangChain with open models via Fastchat, and evaluated on three datasets with three LLMs. ### Tech Stack: - Python ### Why It Matters: Our method can achieve superior performance compared to various strong baselines, and it effectively alleviates the issues of blind trial-and-error and hallucinations, supporting the agent's understanding of the world.

### Overview: Me LLaMA is a groundbreaking suite of open-source medical Large Language Models (LLMs), featuring the foundation models Me LLaMA 13B/70B and their chat-enhanced versions, Me LLaMA 13B-chat/70B-chat. Developed through the innovative continual pre-training and instruction tuning of LLaMA2, these models leverage a vast medical corpus, including PubMed papers, medical guidelines, and general domain corpus, to set new benchmarks on a wide array of medical reasoning tasks. ### Key Features: - **Foundation Models (Me LLaMA 13B/70B)**: These models are the core of the Me LLaMA family, designed for a wide range of medical applications. - **Chat-Enhanced Versions (Me LLaMA 13B-chat/70B-chat)**: These models are designed for conversational interactions, providing natural language responses that are contextually relevant and informative. ### Tech Stack: Python ### Why It Matters: Me LLaMA is a significant asset for medical NLP applications and research, setting new benchmarks on a wide array of medical reasoning tasks. It is designed to be used in the field of computational linguistics and medicine, ensuring ethical and appropriate use without replacing the expertise of healthcare professionals.

### Overview OpenMMReasoner is a framework designed to push the boundaries of multimodal reasoning by combining supervised fine-tuning (SFT) and reinforcement learning (RL) into a unified, transparent, and reproducible recipe. This approach aims to address the limitations of current large language models in handling multimodal data, providing a more robust and efficient learning process. ### Key Features - **SFT Training**: Constructs a cold-start dataset of 874K samples with rigorous validation steps. - **RL Training**: Utilizes a 74K-sample dataset across various domains to fine-tune and stabilize reasoning capabilities. - **Transparent and Reproducible**: The entire process is fully transparent, ensuring reproducibility and scalability. ### Tech Stack - Python ### Why It Matters By providing a clear and reproducible recipe for multimodal reasoning, OpenMMReasoner aims to democratize the research and development of large language models, making it easier for researchers and practitioners to build and evaluate models that can effectively handle multimodal data. ### Repository - **Repository**: [EvolvingLMMs-Lab/OpenMMReasoner](https://github.com/EvolvingLMMs-Lab/OpenMMReasoner) - **Stars**: 166 - **Description**: [CVPR 2026] OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe ### Installation - **SFT Training**: Construct a cold-start dataset with rigorous validation steps. - **RL Training**: Fine-t

### Overview: ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization ReNO is a novel approach designed to improve the performance of one-step text-to-image models by optimizing the initial noise based on human preferences. This method leverages gradient ascent to enhance models' capabilities at inference time, achieving significant improvements across multiple benchmarks. ### Key Features: 1. **Reward-based Noise Optimization (ReNO)**: ReNO optimizes the initial noise of one-step text-to-image models to better capture intricate details specified in complex compositional prompts. 2. **Performance on Multiple Benchmarks**: ReNO consistently outperforms current open-source models, achieving impressive results on four different one-step models across two competitive benchmarks, T2I-CompBench and GenEval. 3. **User Preference Models**: ReNO uses human preference models to guide the optimization process, ensuring the generated images align with user expectations. ### Tech Stack: Python ReNO is implemented using Python, leveraging the latest advancements in reinforcement learning and optimization techniques. ### Why It Matters: ReNO enhances the performance of one-step text-to-image models, making them more versatile and efficient at generating high-quality images. This approach not only improves the accuracy of the generated images but also makes the models more user-friendly and competitive in the field of text-to-image synthesis. Repository: [ExplainableML/ReNO](https://github.com/ExplainableML/ReNO) GitHub Stars: 167 ### Setup 1. Install necessary dependencies in `environment.yml`. 2. Manually download

Overview: Patch-as-Decodable-Token (PaDT) is an innovative approach that enables multimodal large language models (MLLMs) to directly generate both textual and visual outputs, achieving state-of-the-art performance across various visual perception and understanding tasks. Key features: - Introduces Visual Reference Tokens (VRTs) to represent visual targets directly through visual patches, enabling semantic reasoning and object-specific visual tokens prediction within the MLLM’s autoregressive generation process. - Validates PaDT across four major visual perception and understanding tasks, achieving superior performance compared to conventional character-by-character coordinate-generation MLLMs. Tech stack: Python Why it matters: PaDT represents a significant advancement in the field of multimodal language models, providing a unified paradigm for generating both text and images, thereby enhancing the capabilities of MLLMs in visual understanding tasks. Repository: Gorilla-Lab-SCUT/PaDT Primary language: Python GitHub stars: 167 GitHub description: [ICLR 2026] Official implementation of "Patch-as-Decodable-Token: Towards Unified Multi-Modal Vision Tasks in MLLMs" README excerpt: <div align='center'><h1>Patch-as-Decodable-Token: Towards Unified Multi-Modal Vision Tasks in MLLMs</h1></div> <font size=4><div align='center'>[[🔗 Released Code](https://github.com/Gorilla-Lab-SCUT/PaDT)] [[🤗 Datasets](https://huggingface.co/collections/PaDT-MLLM/padt-dataset-

### Overview: ComfyUI-TD is a custom node designed for real-time streaming of images, videos, 3D models, audio, and text from ComfyUI to TouchDesigner. It supports sending images, videos, point clouds (traditional PLY/Gaussian Splatting PLY), and audio from ComfyUI to TouchDesigner. The node is optimized based on the ComfyUI-Tooling-Nodes and must be used in conjunction with the ComfyUI2TD.tox component. ### Key features: - Real-time streaming of images, videos, point clouds, and audio from ComfyUI to TouchDesigner. - Supports multiple data types including images, videos, point clouds (traditional PLY/Gaussian Splatting PLY), and audio. - Optimized based on ComfyUI-Tooling-Nodes for better performance and compatibility. - Compatible with both cloud-based and local ComfyUI setups. ### Tech stack: Python ### Why it matters: ComfyUI-TD provides a seamless integration between ComfyUI and TouchDesigner, enabling real-time streaming of various data types, enhancing the capabilities of both applications. This tool is essential for developers and artists looking to streamline their workflows and collaborate effectively in real-time environments.

### Overview: AlumNet is an AI-powered intelligent school history exhibition and alumni management system designed for school history museums and alumni associations. It combines an immersive Three.js-powered exhibition kiosk with AI-driven natural language search (RAG), offering a museum-grade interactive experience that can be deployed in 30 minutes with Docker. ### Key Features: - **Touch-Screen Kiosk**: Sci-fi visual effects with alumni galaxies, time corridors, and vintage museums. - **AI-Powered Natural Language Search**: Intelligent answers via RAG for queries like "哪些校友在医疗领域工作？". - **Complete Admin Dashboard**: Manages alumni data, photos, messages, donations, and more. - **One-Command Docker Deployment**: Production-ready out of the box. ### Tech Stack: TypeScript ### Why It Matters: AlumNet is designed to be a versatile system that can be deployed in schools, museums, and education environments, providing a unique and engaging experience for alumni and visitors alike.

### Overview: LLM-KG4QA is a repository focused on the integration of large language models (LLMs) and knowledge graphs (KGs) for question answering. This project aims to synthesize and explore opportunities in the field of large language models and knowledge graphs for effective question answering. ### Key features: - **KGs as Background Knowledge**: Utilizes knowledge graphs to provide background information and context for questions. - **KGs as Reasoning Guideline**: Guides the reasoning process by incorporating domain-specific knowledge. - **KGs as Refiner and Filter**: Enhances the accuracy and relevance of answers by filtering out irrelevant information. ### Tech stack: JavaScript ### Why it matters: LLM-KG4QA provides a comprehensive approach to enhancing the capabilities of language models through the integration of knowledge graphs, making it a valuable resource for researchers and practitioners in the field of natural language processing and knowledge-based systems.

### Overview: This repository contains codes and data for the paper "Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMs". The dataset, BEAM, comprises 100 conversations and 2,000 validated questions, designed to evaluate the abilities of large language models in long-term memory tasks. ### Key features: - **BEAM Dataset**: A comprehensive dataset for evaluating long-term memory in language models, comprising 100 conversations and 2,000 validated questions. - **Probing Questions**: Generated for each conversation to assess ten distinct memory abilities. - **Light Framework**: A framework inspired by human cognition that equips LLMs with three complementary memory systems: episodic, working, and scratchpad memory. ### Tech stack: Python ### Why it matters: This paper introduces a novel solution to the challenges of evaluating long-term memory in language models, providing a comprehensive dataset and a framework that significantly improves model performance across various models.

### Overview: This repository is a curated collection of robotics papers focused on real-world reliability and robustness. It aims to provide insights and practical examples of how frontier robotics models can be trained and deployed in the real world, emphasizing performance and efficiency. ### Key Features: - **Real-Time EXPO-FT:** A reinforcement learning algorithm for real-time vision-language-action policies, achieving 97% success rate across 4 dynamic tasks within 10 minutes. - **VLA-Precision:** An asymmetric co-bootstrapping method for efficient real-world online RL of vision-language-action models, achieving 98.3% mean success rate across 9 tasks in 45.8 minutes. - **Towards Universal Post-Training for Robotics:** Discusses a universal post-training recipe for frontier robotics models, with a success rate of 30/30 across 6 complex manipulation tasks in 19 minutes. ### Tech Stack: - **Python, PyTorch, TensorFlow, OpenCV, ROS, Unity, Blender, C++, C#** ### Why It Matters: This repository offers practical examples of how to achieve high reliability and robustness in robotics, making it valuable for researchers and developers looking to improve the real-world performance of their models.