Projects

### Overview: PrismQuant is an optimal null-space rotation method designed for grouped quantizers. It leverages calibrated orthogonal rotations, compact-WY representations, and groupwise asymmetric INT4 quantization to align high-energy activation directions with the constant directions of quantization groups. ### Key features: - **Optimal Null-Space Rotations:** Aligns high-energy activation directions with the constant directions of quantization groups. - **Compact-WY Representations:** Stores group scale and real affine offset in fp16 format. - **Groupwise Asymmetric INT4 Quantization:** Combines calibrated orthogonal rotations, compact-WY representations, and groupwise asymmetric INT4 quantization. ### Tech stack: Python ### Why it matters: PrismQuant provides a robust and efficient method for quantizing neural networks, ensuring optimal performance and minimal loss of information.

Overview: ClassEval is a benchmark designed to evaluate Large Language Models (LLMs) on class-level code generation, providing a comprehensive dataset of 100 class-level Python tasks with 410 methods across various domains. Key features: - 100 class-level Python coding tasks - 100 classes and 410 methods - 33.1 test cases per class - Diverse dependencies including Library Dependency, Field Dependency, Method Dependency, and Standalone - JSON format structured data accessible at [ClassEval Data](https://github.com/FudanSELab/ClassEval/blob/master/data/ClassEval_data.json) Tech stack: Python Why it matters: ClassEval offers a standardized benchmark for evaluating LLMs in class-level code generation, facilitating fair and accurate comparisons among different models.

### Overview This repository contains code for our paper [Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA](https://arxiv.org/abs/2406.17419). We propose a novel long-context benchmark, Loong, aligning with realistic scenarios through extended multi-document question answering (QA). Loong typically consists of 11 documents per test instance on average, spanning three real-world scenarios in English and Chinese: (1) Financial Reports, (2) Legal Cases, and (3) Academic Papers. Loong introduces new evaluation tasks from the perspectives of Spotlight Locating, Comparison, Clustering, and Chain of Reasoning, to facilitate a more realistic and comprehensive evaluation of long-context understanding. Loong features inputs of varying lengths and evaluation tasks of diverse difficulty, enabling fine-grained assessment of LLMs across different context lengths and task complexities. ### Key Features - **Loong Benchmark**: A novel long-context benchmark for evaluating LLMs. - **Realistic Scenarios**: Aligns with real-world scenarios such as Financial Reports, Legal Cases, and Academic Papers. - **Extended Multi-Doc QA**: Includes 11 documents per test instance on average. - **New Evaluation Tasks**: Spotlight Locating, Comparison, Clustering, and Chain of Reasoning. - **Varying Input Lengths**: Supports inputs of varying lengths (e.g., 10K-50K, 50K-100K, 100K-200

Overview: SalUn is a principled method for machine unlearning that focuses on gradient-based weight saliency, aiming to enhance the effectiveness and efficiency of unlearning in both image classification and generation tasks. It addresses limitations of existing methods by directing attention towards specific model weights rather than the entire model, thereby improving unlearning accuracy, stability, and cross-domain applicability. Key features: - SalUn is the first principled method for machine unlearning that can effectively erase the influence of forgetting data, classes, or concepts in both image classification and generation tasks. - It introduces the concept of 'weight saliency' for machine unlearning, drawing parallels with input saliency in model explanation. - SalUn improves unlearning accuracy and stability compared to exact unlearning, achieving a stability advantage in high-variance random data forgetting, such as a 0.2% gap on the CIFAR-10 dataset. - In preventing conditional diffusion models from generating harmful images, SalUn outperforms current state-of-the-art baselines like Erased Stable Diffusion and Forget-Me-Not. Tech stack: Python Why it matters: SalUn provides a principled approach to machine unlearning that addresses the limitations of existing methods by focusing on specific model weights, thereby improving unlearning accuracy, stability, and cross-domain applicability. Its ability to effectively erase the influence of forgetting data, classes, or concepts in both image classification and generation tasks makes it a significant advancement in the field of machine unlearning.

### Overview: OceanGym is a high-fidelity underwater environment designed to benchmark and evaluate autonomous agents. It simulates realistic ocean conditions with diverse scenes and challenges, providing a robust platform for evaluating agents' performance in various tasks. ### Key Features: - **Multi-Modal Perception**: Supports advanced perception techniques to analyze underwater environments. - **Water-Flow and Hydrodynamic Simulation**: Accurate simulation of water flow and hydrodynamic effects. - **Scene Customization**: Users can create, edit, and customize diverse underwater scenes. - **Task Flexibility**: Supports continuous control and multi-modal perception for comprehensive evaluation. ### Tech Stack: Python ### Why It Matters: OceanGym serves as a crucial benchmark for evaluating autonomous underwater agents, providing a realistic environment to test and improve their capabilities in various tasks such as perception and decision-making. Repository: [OceanGPT/OceanGym](https://github.com/OceanGPT/OceanGym) GitHub Stars: 154

Overview: This repository contains a curated list of papers focused on Large Language Models for User Modeling (LLM-UM), based on the survey paper "User Modeling in the Era of Large Language Models: Current Research and Future Directions" by Zhaoxuan Tan and Meng Jiang. Key features: - Papers on various approaches to LLM-UM, including common generative reasoning, recommendation as instruction following, and large language models for personality recognition. - Applications in user profiling, rating prediction, and detecting suspicious user-generated content (UGC). Tech stack: Python, Markdown, and LaTeX. Why it matters: This list provides a comprehensive overview of the current research in LLM-UM, highlighting the capabilities of LLMs in modeling, understanding, and reasoning about user behavior and preferences. It serves as a valuable resource for researchers and practitioners interested in advancing the field of user modeling through the use of LLMs.

Overview: This open-source repository provides reference code for integrating Cohere's LLMs with various datastores, enabling seamless retrieval-augmented generation (RAG) on user's own data. Key features: - Integration with popular datastores like Google Drive, Microsoft Excel, and more - Support for different types of data sources, including text, images, and more - Customizable search and retrieval capabilities - Seamless integration with Cohere's Chat API for enhanced user experience Tech stack: Python Why it matters: This repository offers a practical solution for developers and businesses looking to leverage Cohere's LLMs with their existing data, improving the accuracy and relevance of generated responses. Repository: cohere-ai/quick-start-connectors GitHub stars: 154

Overview: Langchain-ChatBI 是一种利用 Langchain 框架和本地向量库实现的对话式 BI，旨在帮助用户寻找、理解数据知识，并分析数据、洞察结果，通过自然语言对话降低数据分析门槛。 Key features: - 本地化部署，支持 ChatGLM-6B 大模型 - 可以通过 HTTP 调用百川和通义千问大模型 - 利用大模型的语言理解能力，识别用户的 BI 意图 - 解决对话式 BI 如何确保数据准确的难点 Tech stack: Python Why it matters: 该项目通过结合 Langchain 框架和本地向量库，为用户提供了一个高效、准确且易于使用的对话式 BI 解决方案，特别适用于需要快速分析和理解数据知识的场景。

Overview: odin-slides is an advanced Python tool designed to streamline the creation of PowerPoint presentations. It utilizes Large Language Models (LLMs) to summarize lengthy Word documents and generate well-organized slides, saving developers time and effort. Key features: - Smart Presentation Creation: Easily create new PowerPoint presentations or update existing ones with odin-slides' user-friendly command-line interface. - Input-Driven Presentation Generation: Leverage the power of odin-slides to transform Microsoft Word (docx) files into captivating presentations. - Seamless Integration: Seamlessly integrate odin-slides into existing projects, enhancing productivity without requiring significant changes to the codebase. Tech stack: Python Why it matters: odin-slides simplifies the process of creating PowerPoint presentations, making it easier for developers to present their ideas effectively and efficiently.

### Overview: DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learning DecAlign is a novel hierarchical cross-modal alignment framework designed to decouple multimodal representations into modality-unique (heterogeneous) and modality-common (homogeneous) components. This approach not only facilitates fine-grained alignment through prototype-guided optimal transport but also enhances semantic consistency via latent distribution matching. By mitigating distributional discrepancies while preserving modality-specific characteristics, DecAlign achieves consistent performance improvements across various multimodal benchmarks. ### Key Features: - **Hierarchical Cross-Modal Alignment**: DecAlign leverages a hierarchical structure to align multimodal representations, ensuring that each modality retains its unique characteristics while maintaining consistency with other modalities. - **Prototype-Guided Optimal Transport**: The framework uses prototypes to guide the alignment process, ensuring that the alignment is optimal and consistent. - **Latent Distribution Matching**: DecAlign matches the latent distributions of different modalities, enhancing semantic consistency and improving performance across various benchmarks. - **Discrepancy Mitigation**: The framework effectively addresses distributional discrepancies, preserving modality-specific characteristics while achieving consistent performance. ### Tech Stack: Python DecAlign is implemented using Python, leveraging the following libraries and tools: - TensorFlow - PyTorch - NumPy - Matplotlib ### Why It Matters: DecAlign provides a robust and effective solution for multimodal representation learning, particularly in scenarios where modality-specific characteristics need to be preserved while achieving consistent performance across different datasets. Its hierarchical alignment approach and prototype-guided

### Overview: Not Ai is a text editor designed to enhance the clarity and professionalism of writing, focusing on the quality of the content rather than disguising the author's identity. It is particularly useful for educational projects, LinkedIn posts, emails, factual articles, personal essays, social media, and technical documentation, especially for GitHub READMEs. ### Key Features: - **Clear and Specific Writing**: Ensures that the text is clear and specific, adhering to human writing standards. - **Voice Preservation**: Maintains the original author's voice while improving the readability and professionalism. - **Grammar and Style**: Enhances the grammar and style of the text, making it more professional and engaging. ### Tech Stack: - Python ### Why It Matters: Not Ai is designed to produce high-quality, human-readable prose that is suitable for various professional and educational contexts. It is particularly useful for projects that require clear and professional communication, such as academic papers, technical documentation, and social media posts. ### Repository: Repository: udaysharmadev/Not-Ai GitHub Stars: 151 Primary Language: Python GitHub Description: Not Ai isn’t just about bypassing AI detectors. It puts good writing first, creating clear, honest prose shaped by human writing standards. It’s perfect for college and school projects, LinkedIn/X posts, emails, factual articles, personal essays, social media posts, and technical documentation, especially GitHub READMEs. ### Install in One Command: ```bash npx skills add udaysharmadev/Not-Ai ``` Works with Claude

### Overview: This project is a Python code repository for the EMNLP 2024 paper "Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps". The code aims to detect and mitigate contextual hallucinations in large language models by analyzing the attention weights on context and newly generated tokens. ### Key Features: - **Detection Model**: A simple model that uses the ratio of attention weights on context versus newly generated tokens to detect contextual hallucinations. - **Model Training**: The model is trained on the LLaMA2 model and uses the lookback ratio features to predict hallucinations. - **Transfer Learning**: The detector can be applied to different models and tasks without retraining, demonstrating transferability. ### Tech Stack: - Python - Hugging Face Transformers ### Why It Matters: This project provides a practical solution for detecting and mitigating contextual hallucinations in large language models, which can improve the quality of generated text. The use of lookback ratios as features is effective and transferable across different models and tasks, making it a valuable contribution to the field of language model research.

Overview: This repository is a collection of study notes on **Reinforcement Learning (Human Feedback)** and **Large Language Model Reasoning**. It serves as an indexed resource for ongoing learning and includes brief summaries, personal annotations, and citations from public sources. Key features: - **RL(HF) Algorithms Overview** - **LLM Reasoning Summaries** - **Public Literature References** Tech stack: Markdown and GitHub repository Why it matters: This resource provides a structured way to access and share insights into the evolving field of reinforcement learning and language model reasoning, offering a valuable resource for developers and researchers.

### Overview: A curated list of tools, papers, and datasets for applying AI to cybersecurity tasks. This list primarily focuses on modern AI technologies like Large Language Models (LLMs), Agents, and Multi-Modal systems and their applications in security operations. ### Key features: - Tools and datasets for vulnerability assessment, threat intelligence, offensive security, and general security knowledge. - Models and datasets for specialized security scenarios, including vulnerability localization and reasoning capabilities. - Benchmarks and evaluations for vulnerability assessment, threat intelligence, and other security tasks. ### Tech stack: Python, Jupyter, and Hugging Face Transformers. ### Why it matters: This curated list provides a comprehensive overview of AI tools and datasets for cybersecurity, making it a valuable resource for researchers and practitioners looking to apply AI in security operations.

Overview: Cold Compress is a hackable, lightweight, and open-source toolkit designed for creating, extending, and benchmarking KV cache compression methods. It leverages GPT-Fast, a PyTorch-native generation codebase, and torch.compile for efficient GPU optimization. Key features: - Support for various KV cache compression methods, including attention-based evictions, sliding window attention, preservation of attention sinks, layerwise customization, and hybridization. - Easy command-line configuration and low-effort extension. - Built on top of GPT-Fast, a simple and efficient generation codebase. - Supports popular approaches like `Heavy Hitters`, `Recent Tokens`, `Global Tokens`, `Pyramid`, and `FastGen`. Tech stack: Python, PyTorch, GPT-Fast, torch.compile. Why it matters: Cold Compress simplifies the process of creating, extending, and benchmarking KV cache compression methods, making it accessible and easy to use for developers. Its lightweight and efficient implementation, combined with the flexibility of command-line configuration, makes it a valuable tool for researchers and developers looking to improve cache compression techniques.

### Overview: ChatFAQ is an open-source ecosystem designed for building AI-powered conversational solutions using RAG, agents, Finite State Machines (FSMs), and large language models (LLMs). It provides a flexible, privacy-preserving, and cost-effective framework for developers to create diverse AI chatbots and workflows. ### Key features: - **LLM-based chatbots** - **RAG-enhanced chatbots** - **Agentic workflows** - **Rule-based Finite State Machines with LLM assistance** - **Hybrid solutions combining multiple approaches** ### Tech stack: Python ### Why it matters: ChatFAQ leverages open-source technologies, ensuring flexibility, privacy, full control, and cost-effectiveness. It includes a SDK for building specialized Natural Language Generation (NLG) engines and customizable chat widgets, avoiding vendor lock-in. The platform is designed to be used with various foundational models, including integration with popular LLMs like OpenAI, Anthropic, Mistral, Gemini, and Together.

### Overview: SPM stands for **Concept Semi-Permeable Membrane**, a novel concept that allows diffusion models to selectively erase or edit specific concepts while preserving others. This solution is designed to enhance the flexibility and effectiveness of diffusion models in various applications, particularly in generating realistic images and videos. ### Key Features: - **Selective Erasure**: Prevents the generation of target concepts while preserving non-target concepts. - **Data-Free Training**: No additional text or image data required for training. - **Lightweight Implementation**: Only 0.5% of the parameters of the diffusion model, with a storage requirement of 1.7MB. - **Customizable**: Equipped with SPMs for different target concepts simultaneously. - **Model-Transferable**: Trained SPMs can be directly transferred to other models without additional tuning. ### Tech Stack: - **Python**: The primary programming language used. - **XFormers**: Accelerates the training and inference of SPMs. - **SD WebUI**: Provides an interface for generating images with SPMs. ### Why It Matters: SPM offers a significant enhancement in the ability to control and manipulate the generation of concepts in diffusion models, making it a valuable tool for applications requiring high-level control over the output. Its lightweight nature and data-free training make it particularly appealing for real-world applications where data is scarce or expensive to obtain. ### Repository: Repository: **Con6924/SPM** Stars: **153** Description: Official implementation of the paper "One-dimensional Adapter to Rule Them All

### Overview: Build your own Face App with Stable Diffusion 2.1 Base ### Key features: - **Chapter 1: Sampling but not Inference** - **Chapter 2: Stable Diffusion Dissected** - **Chapter 3: Spicing it up with CFG** - **Chapter 4: Attention, Please!** - **Chapter 5: Attention Switcheroo** - **Chapter 6: Reality Check** - **Chapter 7: FaceApp, Who?** ### Tech stack: Jupyter Notebook ### Why it matters: Learn the fundamental aspects of Stable Diffusion, including sampling, architecture, CFG, attention layers, inverse scheduler, pivotal inversion, image reconstruction, and prompt2prompt editing. Use these insights to create your own face editing app.

### Overview: KaaS is a knowledge-as-a-service platform that turns scattered notes, documents, and transcripts into a searchable, queryable personal Wiki powered by LLM-driven knowledge compilation. It compiles content through a 4-phase LLM pipeline, resulting in human-readable Markdown articles rather than a black-box vector store. ### Key Features: - **Scattered Content Compilation**: Turns notes, documents, and transcripts into a searchable Wiki. - **LLM-Driven Knowledge Compilation**: Uses a 4-phase LLM pipeline to compile content into human-readable articles. - **Queryable Wiki**: Provides a tree of Markdown articles, with rendered articles and their headings. - **MCP Access**: Offers MCP access for those looking to integrate KaaS into their existing systems. ### Tech Stack: - Python ### Why It Matters: KaaS addresses the issue of scattered knowledge by providing a structured, readable Wiki that can be easily updated and queried. This platform is particularly useful in organizations where knowledge is scattered across various sources, making it easier to maintain and update the collective wisdom of the team. ### GitHub Stars: 152 ### Repository: - **Repository**: bybit-exchange/kaas - **Primary Language**: Python - **GitHub Stars**: 152 - **GitHub Description**: Turn scattered notes, docs and transcripts into a queryable Markdown wiki — an LLM knowledge-base compiler with MCP access, no embeddings, self-hosted. ### Documentation: - **Documentation**: [https://bybit-exchange.github.io/kaas-doc/](https://bybit-ex

Overview: DM-Code-Agent 是一个本地优先、全程可审计、内核只有一个 ReAct 循环的 Python Code Agent。它具备 30 个任务的隐藏测试基准，并通过 SWE-bench 验证，所有数据都可重现，包括负结果。 Key features: - 能读写本地文件 - 跑测试用例 - 调用 lint 工具 - 使用 MCP 工具 - 生成 append-only 的会话日志 - 支持回放和诊断 - 从任意步骤分叉重跑 Tech stack: Python Why it matters: DM-Code-Agent 提供了一个能读懂、能复现、能扩展的 Python Code Agent。它在本地工作区读写文件、跑测试、调用 lint、调用 MCP 工具，并把每一步计划、工具调用和观察结果写成 append-only 的会话日志。它还具备回放和诊断功能，从任意步骤分叉重跑，确保了负结果的可重现性。