Projects

Overview: This project is an official implementation of the "Parallel Diffusion Model of Operator and Image for Blind Inverse Problems" (CVPR 2023) using PyTorch. The repository contains a pretrained checkpoint and detailed instructions for setting up and running the model. Key features: - Pretrained checkpoint for blind deblurring and imaging through turbulence tasks - Implementation based on the official repository of [DPS (diffusion-posterior-sampling)] - Flexible to be applied to general blind inverse problems with known functional forms Tech stack: Python Why it matters: This project showcases the potential of diffusion models in solving blind inverse problems, demonstrating their effectiveness on tasks like blind deblurring and imaging through turbulence.

Overview: ChartMimic is a benchmark designed to evaluate large multimodal models' (LMMs) cross-modal reasoning capabilities by assessing their ability to generate code for visual charts. It uses human-curated triplets of charts, instructions, and corresponding codes to evaluate the models' performance comprehensively. Key features: - 📝 4,800 human-curated triplets representing authentic chart use cases across various domains. - 🚀 4 advanced chart types and 18 regular chart types, totaling 201 subcategories. - 📈 Multi-level evaluation metrics for a thorough assessment of generated code and rendered charts. - 🚀 Evaluated by ICLR 2025. Tech stack: Python Why it matters: ChartMimic provides a comprehensive evaluation framework for LMMs, focusing on their ability to harmonize visual understanding, code generation, and cross-modal reasoning, making it a valuable tool for researchers and developers in the field of multimodal AI. Repository: ChartMimic/ChartMimic Primary language: Python GitHub stars: 135 GitHub description: [ICLR 2025] ChartMimic: Evaluating LMM’s Cross-Modal Reasoning Capability via Chart-to-Code Generation

### Overview: This repository contains a comprehensive collection of low-cost replication methods for ChatGPT, including summaries of various open-source large language models. It serves as a project card for each model, providing resources, addresses, and brief introductions. The project is designed to enhance the efficiency of searching and comparing models. ### Key Features: - Summarizes resources from open-source projects. - Lists resource addresses. - Introduces models with short paragraphs and pictures. - Tags projects for easy retrieval. - Provides models for comparison and direct use. - Tracks the latest developments in the repository. ### Tech Stack: Unknown ### Why It Matters: This project offers a valuable resource for developers and researchers interested in low-cost replication methods for large language models, particularly ChatGPT. It helps in understanding the capabilities and performance of various models, aiding in the development and fine-tuning of models for specific applications. ### GitHub Stars: 135 ### Repository: [Longyichen/Alpaca-family-library](https://github.com/Longyichen/Alpaca-family-library)

### Overview: This repository contains a survey paper titled "A Survey on Data Augmentation in Large Model Era," authored by Yue Zhou, Chenlu Guo, Xu Wang, Yi Chang, and Yuan Wu. The survey focuses on various techniques and methods used for data augmentation in the context of large models, providing valuable insights for researchers and practitioners in the field. ### Key Features: - **Image Augmentation**: Techniques like Camdiff, Diffedit, Glide, and others that enhance image quality through diffusion models. - **Text Augmentation**: Methods such as It is all about where you start and Plug-and-play diffusion features, which utilize text prompts to generate images. - **Paired Augmentation**: Techniques that combine multiple data sources to improve model performance. ### Tech Stack: The survey is based on the arXiv paper "A Survey on Data Augmentation in Large Model Era," released on 03/04/2024 and 01/27/2024. ### Why It Matters: Understanding and implementing effective data augmentation techniques is crucial for improving the performance and robustness of large language models. This survey provides a comprehensive overview of the latest methods and their applications across various domains, including natural language processing, computer vision, and audio signal processing.

### Overview: This project aims to provide a local research assistant that helps researchers find answers from a set of research papers. It uses a customized RAG pipeline and a powerful LLM, all offline and free of cost. The project is designed to be Dockerized and easy to run, with a focus on enhancing research efficiency. ### Key features: - **Local Research Assistant:** Helps researchers quickly navigate research papers. - **Customized RAG Pipeline:** Uses a customized RAG model to enhance the research assistant's performance. - **Powerful LLM:** Utilizes a high-performance language model to generate concise answers. - **Offline and Free:** The project is designed to be run locally without internet access, making it accessible to researchers worldwide. ### Tech stack: - **Python:** The project is built using Python, leveraging libraries such as LlamaIndex, Qdrant, and FastAPI. ### Why it matters: This project is crucial for researchers who need to quickly find relevant information from research papers without the need for internet access. It streamlines the research process by providing a local, offline solution to the challenge of finding answers from research papers.

### Overview: This repository contains the codes for our paper "End-to-End Full-Atom Antibody Design". The paper focuses on developing an end-to-end method for designing full-atom antibodies, aiming to improve the accuracy and efficiency of the design process. ### Key Features: - **End-to-End Antibody Design**: Implements an integrated approach for designing full-atom antibodies. - **Full-Atom Representation**: Utilizes a comprehensive representation method to capture the atomic details of antibodies. - **Scoring and Docking**: Incorporates advanced scoring and docking algorithms to evaluate the design quality. - **Data Preprocessing**: Includes tools for preprocessing antibody structures and datasets. - **Complex Structure Prediction**: Uses complex structure prediction models to enhance the design accuracy. - **Affinity Optimization**: Optimizes the affinity of the designed antibodies to improve their binding properties. ### Tech Stack: - Python ### Why It Matters: This project represents a significant advancement in the field of antibody design, offering a more accurate and efficient method for generating full-atom antibodies. The use of advanced scoring and docking algorithms, combined with comprehensive data preprocessing, results in better-designed antibodies with improved binding properties.

### Overview: Nanowakeword is a lightweight, wake word detection engine designed for training custom models with minimal effort. It seamlessly integrates into projects ranging from Raspberry Pi Zero to cloud servers, covering the entire lifecycle from development to deployment. ### Key Features: - **Customizable Models**: Train models tailored to specific use cases with minimal effort. - **Adaptive Framework**: Seamlessly integrates into projects across resource-constrained devices and cloud systems. - **Minimal Effort**: Requires minimal code to get started, making it accessible to developers of all skill levels. ### Tech Stack: - Python ### Why It Matters: Nanowakeword simplifies the process of building and deploying high-performance wake word detection models, making it an essential tool for developers looking to enhance their projects with intelligent wake word capabilities.

Overview: This repository is intended for those looking to dive deep on advanced Text-to-SQL concepts. Key features: - Text-to-SQL translation - Schema linking and joins - Properly joining tables - Overcoming core components of SQL query generation - Measuring success with Execution Accuracy (EX) and Exact Set Match Accuracy (EM) on the Spider and BIRD datasets Tech stack: Jupyter Notebook Why it matters: This repository aims to bridge the knowledge gap between non-technical users and SQL, making it easier for them to incorporate relational data sources into their processes.

Overview: MotionCraft is a unified diffusion transformer designed for crafting whole-body motion with plug-and-play multimodal controls, addressing challenges in motion generation across various tasks and modalities. Key features: - Text-to-Motion - Speech-to-Gesture - Music-to-Dance Tech stack: Python Why it matters: MotionCraft provides a unified approach to generating whole-body motion using text, speech, and music controls, enabling robust and versatile motion generation across various applications and tasks. Repository: cure-lab/MotionCraft GitHub stars: 135 GitHub description: [AAAI 2025] Official repo for paper "MotionCraft: Crafting Whole-Body Motion with Plug-and-Play Multimodal Controls"

### Overview: Graph of States (GoS) is a neuro-symbolic framework designed for abductive reasoning, specifically tailored for high-stakes domains such as medical diagnosis and distributed system failure diagnosis. GoS leverages a causal graph and state machine to ground multi-agent collaboration in structured belief states, thereby addressing the limitations of existing frameworks in handling abductive reasoning tasks. ### Key Features: - **Causal Graph ($G$):** Explicitly structures belief states by mapping causal relationships among hypotheses and evidence, mitigating evidence fabrication and context drift. - **State Machine ($S$):** Governs the reasoning trajectory, enforcing rigorous logical transitions to prevent failed backtracking and early stopping. - **Directed Search guided by reasoning focus ($h_t^*$):** Transforms aimless stochastic exploration into a convergent search by dynamically aligning reasoning focus with symbolic constraints. ### Tech Stack: Python ### Why It Matters: Graph of States (GoS) provides a robust solution for complex abductive tasks, significantly outperforming existing baselines in evaluating two real-world datasets. By addressing the limitations of existing frameworks in handling abductive reasoning tasks, GoS offers a powerful tool for high-stakes domains such as medical diagnosis and distributed system failure diagnosis.

### Overview: To keep large language models (LLMs) up-to-date and aligned with the ever-changing world knowledge, we need to refresh them without expensive retraining. This survey paper aims to explore and categorize existing methods that achieve this goal. ### Key Features: - **Methods Overview**: Categorizes methods into implicit and explicit approaches. - **Knowledge Editing (KE)**: Focuses on altering specific knowledge in pre-trained models to make new predictions without retraining. - **Meta-learning**: Methods that learn to update specific knowledge. - **Hypernetwork Editor**: Uses a hypernetwork to edit specific knowledge. - **Locate and Edit**: Directly edits specific knowledge in the model. - **Other**: Includes other methods not explicitly listed. ### Tech Stack: - Categorization and taxonomy of methods. ### Why It Matters: - Maintaining LLMs up-to-date is crucial in the rapidly evolving world of AI and language processing. ### Repository: - **Repository Name**: hyintell/awesome-refreshing-llms - **GitHub Stars**: 135 - **Description**: A curated list of papers and resources on refreshing LLMs without expensive retraining. ### Primary Language: - Unknown - **GitHub Description**: EMNLP'23 survey: a curation of awesome papers and resources on refreshing large language models (LLMs) without expensive retraining.

### Overview: This project is a Telegram bot powered by large language models (LLMs) that allows users to interact with a variety of models, including LLaMA, ExLlama, and LLMMan. It features a user-friendly interface with customizable chat templates, session management, and the ability to load characters and models dynamically. The bot supports both standalone and extension modes, making it versatile for different use cases. ### Key Features: - **Model Wrappers**: Supports multiple LLM models such as LLaMA, ExLlama, and LLMMan. - **Custom Chat Templates**: Allows users to create and apply custom chat templates. - **Separate Sessions**: Ensures that each user's conversation is isolated, preventing session data from being lost. - **Dynamic Model Loading**: Users can switch between different models and characters during conversations. - **Interactive Buttons**: Includes buttons for various actions like impersonation, continuing previous messages, and resetting history. - **Voice and Image Generation**: Enhances the chat experience with voice and image generation capabilities. - **Easy Installation**: Supports installation via GitHub repository, Google Colab, and standalone application. ### Tech Stack: - Python ### Why It Matters: This bot offers a powerful and versatile tool for interacting with LLMs, making it ideal for various applications such as chatbots, content creation, and more. The ability to switch models and characters dynamically enhances its utility and flexibility. ### Repository: - **Repository**: `innightwolfsleep/llm_telegram_bot` - **GitHub Stars**: 135 -

### Overview: This repository contains a collection of Intel tuning guides and optimization recipes tailored for software, workloads, performance analysis, monitoring tools, and hardware configurations. It offers detailed configuration recommendations, performance tuning techniques, and best practices to help unlock the full potential of Intel hardware. ### Key Features: - Software Tuning Guides: Comprehensive optimization tuning guides for various software including databases, data processing frameworks, container orchestration, and programming languages. - Workloads & Benchmarks: Industry-standard benchmark configurations and execution guides. - Performance Analysis & Monitoring Tools: Detailed guides for leveraging performance analysis and monitoring tools to identify bottlenecks and optimize workloads. - Intel Hardware Optimal Configurations: Best practices and configuration recommendations for Intel hardware, including BIOS settings, CPU tuning, memory optimization, and system-level configurations. ### Tech Stack: Shell ### Why It Matters: This repository serves as a dynamic resource for users to find the latest optimization strategies, contribute feedback, and report issues. It aims to provide a comprehensive guide for software tuning, workloads, performance analysis, and hardware configurations, ensuring optimal performance and efficiency on Intel hardware.

Overview: RagRabbit is an open-source, self-hosted AI search and LLM.txt solution for websites, powered by LlamaIndexTS and PostgreSQL. It offers a chat widget, website crawler, LLMs.txt generation, MCP server access, and easy deployment on Vercel. Key features: - Chat widget with instant search - Website crawler for indexing pages - Fully customizable LLMs.txt with TOC reorder - MCP server for Claude Desktop and Cursor IDE access - Flexible authentication and open-source deployment Tech stack: TypeScript, Next.js, Turborepo, LlamaIndexTS, PostgreSQL, pgVector Why it matters: Ideal for developers looking to integrate AI into their websites with ease, providing an open-source and secure solution for AI search and LLM.txt generation.

Overview: This project is a powerful web application that simplifies document analysis through the integration of LangChain, Streamlit, and Pinecone. It leverages OpenAI's GPT-3 for retrieval augmented generation, enabling users to perform efficient document retrieval and summarization. Key features: - Streamlit web app for an intuitive interface - Input fields for OpenAI API key and Pinecone API key - Document uploader for multiple PDF files - Document splitting for compatibility with models with token limits - Vector embeddings for text chunks - Flexible vector storage options in Pinecone or local vector stores - Interactive chat history for reference Tech stack: Python Why it matters: This project provides an efficient and user-friendly solution for document analysis, making it ideal for businesses and individuals looking to streamline their document processing workflows.

### Overview: This repository contains benchmark datasets and evaluation scripts for Multilingual Large Language Models (LLMs). The datasets cover 26 different languages and three distinct tasks: ARC, HellaSwag, and MMLU. The datasets are translated from the original ARC, HellaSwag, and MMLU datasets in English using ChatGPT. The technical paper for Okapi, which describes the datasets and evaluation results, can be found [here](https://arxiv.org/pdf/2307.16039.pdf). ### Key features: - **ARC (Advanced Reading Comprehension):** A dataset with 7,787 genuine grade-school level, multiple-choice science questions, assembled to encourage research in advanced question-answering. - **HellaSwag (HellaSwag):** A dataset for studying grounded commonsense inference. It consists of 70k multiple choice questions about grounded situations, with the correct answer being the (real) sentence for the next event. The three incorrect answers are adversarially generated and human verified. - **MMLU (Multilingual Multimodal Learning Using Language):** This dataset contains multiple choice questions derived from diverse fields of knowledge, covering subjects in the humanities, social sciences, hard sciences, and other essential areas of learning for certain individuals. ### Tech stack: Python ### Why it matters: This repository serves as a benchmark for evaluating Multilingual Large Language Models (LLMs) across 26 different languages and three distinct tasks. The datasets are essential for

### Overview: A user-friendly GUI for configuring and launching `llama.cpp` on Windows, macOS, and Linux. ### Key features: - **Install Prebuilt Releases**: Quickly install prebuilt `llama.cpp` releases for various configurations (CPU, CUDA, Vulkan, SYCL, ROCm, etc.). - **Launch Applications**: Launch `llama-server` or `llama-cli` with either a quick or full configuration. - **Chat with Streamlined Features**: Chat with streaming Markdown, Focus mode, collapsed reasoning, and optional zero-key web search. - **Benchmarking and Monitoring**: Benchmark with `llama-bench` or `llama-perplexity`, monitor live stats, and use OpenAI-compatible API snippets. - **Manage Launch Presets**: Keep two server presets on standby for quick switching, create Windows preset shortcuts, and run in-app GitHub updates. - **Custom Themes**: Choose from five themes—Tokyo and Nebula (dark), Graphite (mid-tone), Cappuccino, and Mint (light)—meeting WCAG AA contrast. ### Tech Stack: JavaScript ### Why it Matters: A lightweight, user-friendly interface for configuring and launching `llama.cpp` on Windows, macOS, and Linux, making it accessible and easy to use for developers and researchers alike.

### Overview Milvus Operator is a Kubernetes Operator designed to simplify the deployment and management of Milvus services, including milvus components and their dependencies such as etcd, pulsar, and minio. It provides a scalable and high-availability solution for Milvus clusters, making it easier to set up and manage Milvus instances in Kubernetes environments. ### Key Features - **Milvus Custom Resources**: Defines Milvus resources on top of Kubernetes Custom Resources. - **Kubernetes API Declarative Management**: Manages Milvus deployment and ensures scalability and high availability. - **Customizable Configuration**: Allows for dynamic configuration reload and health check metrics. - **Rolling Upgrade Management**: Ensures better managed rolling upgrades. - **Health Check and Diagnostics**: Provides quick suspend instances and cleanup resources after deletion. - **Native HPA Autoscaling**: Supports native Horizontal Pod Autoscaling for Milvus clusters. ### Tech Stack - Go ### Why It Matters - **Scalability and High Availability**: Ensures Milvus clusters can handle increased loads efficiently. - **Ease of Use**: Simplifies the deployment process, making it suitable for both beginners and experienced users. - **Rolling Upgrade and Health Check Metrics**: Ensures smooth and reliable upgrades with detailed metrics for monitoring. - **Native HPA Autoscaling**: Automatically scales Milvus clusters based on CPU usage, ensuring optimal resource utilization. ### Repository - **Repository**: zilliztech/milvus-operator - **GitHub Stars**: 135 - **

### Overview In this work, we propose a new task of multimodal analogical reasoning over knowledge graphs. This task is a significant advancement in the field of multimodal reasoning and knowledge graph reasoning. We provide a knowledge graph to support the task and further divide it into single and blended patterns. The relation marked by dashed arrows and the text around parentheses are only for annotation and not provided in the input. ### Key Features - **Multimodal Knowledge Representation Methods (IKRL, TransAE, RSME)** - **Multimodal Analogical Reasoning Dataset (MARS)** - **Multimodal Knowledge Graph Completion Method (MKGformer)** - **Powerpoint Presentation (ICLR2023_MKG_Analogy.pdf)** ### Tech Stack - **Programming Languages**: Python - **Frameworks**: HuggingFace, PyTorch Lightning ### Why It Matters This project addresses a critical challenge in the field of multimodal reasoning, where the ability to reason across different modalities is crucial. By providing a comprehensive solution, it opens up new possibilities for integrating and reasoning with diverse types of information, which has the potential to significantly advance various applications, such as natural language processing, computer vision, and more. ### Repository - **Repository**: zjunlp/MKG_Analogy - **GitHub Stars**: 135 - **Description**: [ICLR 2023] Multimodal Analogical Reasoning over Knowledge Graphs - **Primary Language**: Python ### Quick Links - **MKG_Analogy**: [

### Overview: This project leverages Llama3, Langchain, and ChromaDB to create a Retrieval Augmented Generation (RAG) system, enhancing the ability to answer questions about documents even when the information is not present in the training data. ### Key Features: - **RAG System**: Employs Llama3, Langchain, and ChromaDB to provide accurate answers to questions about documents. - **LLM (Large Language Model)**: Utilizes Llama3, a powerful model developed by Meta. - **Langchain**: Framework for creating applications with LLMs. - **ChromaDB**: Vector database used to index documents. - **Retrieval Augmented Generation (RAG)**: Retrieves relevant documents from the vector database before generating an answer. ### Tech Stack: - **Python** with **Transformers** library for fine-tuning Llama3. ### Why it Matters: This project successfully implemented a Retrieval Augmented Generation (RAG) solution, demonstrating its effectiveness in delivering accurate answers to questions about the EU AI Act.