Projects

### Overview: This repository is a curated collection of tools, documents, and projects related to LLM security, focusing on adversarial attacks and defenses. It aims to provide researchers and developers with a comprehensive overview of the latest advancements in LLM security, including white-box, black-box, and backdoor attacks, as well as defense mechanisms and platform-specific security measures. ### Key Features: - **White-box attack**: Techniques for breaking into LLMs through direct access to their internal models. - **Black-box attack**: Methods to compromise LLMs without knowing their internal structure. - **Backdoor attack**: Vulnerabilities that allow attackers to inject malicious instructions into LLMs. - **Fingerprinting**: Methods to identify and exploit vulnerabilities in LLMs. - **Defense**: Strategies to protect against LLM attacks, including prompt injection and privacy attacks. - **Platform Security**: Best practices for securing LLMs in various platforms and environments. - **Survey**: Comprehensive overview of existing research and projects in LLM security. - **Benchmark**: Performance metrics and benchmarks for evaluating LLM security tools and defenses. - **Tools**: Scripts and libraries for automating LLM security testing and analysis. - **Articles**: Scholarly papers and research articles on LLM security. - **Other Awesome Projects**: Additional projects and tools related to LLM security. - **Other Useful Resources**: Additional resources for further reading and learning. ### Tech Stack: The repository uses Markdown, Git, and GitHub for version control and documentation. It leverages various programming languages and libraries for implementing security features and tools. ###

### Overview: LLM Wiki is an open-source implementation of Karpathy's LLM Wiki concept, designed to transform your scattered reading and research into a persistent, AI-maintained second brain. Capture documents, notes, and web clippings as you work, and deploy a nightly Claude Routine to autonomously synthesize those sources into a permanent knowledge base. This architecture is inspired by Karpathy's original concept, with an increased emphasis on autonomous maintenance. ### Key features: - **Connect via MCP**: Connect Claude.ai, Claude Cowork, Claude Code, or Codex (or any other MCP-compatible app). - **A Chrome extension**: Clip webpages and PDFs as you read, highlight key sections, and leave comments that Claude can see over MCP. - **Uploads**: Markdown, PowerPoint, PDFs, Word documents, and more. - **A clean Next.js web app**: Navigate your own Wikipedia — and view the underlying sources. - **Native cross-linking**: Between wiki pages, and back to the sources they came from. - **A graph viewer**: To see how your concepts and entities relate. - **Visualizations**: Charts and other visualizations, including SVGs and Mermaid diagrams. ### Tech stack: Python ### Why it matters: This architecture is heavily inspired by Karpathy's LLM Wiki concept, with an increased emphasis on autonomous maintenance. LLM Wiki transforms your scattered reading and research into a persistent, AI-maintained second brain, capturing your thoughts alongside the sources you read. This model is designed to work at

Overview: This repository is a comprehensive list of Integrated Circuits (ICs) and Intellectual Properties (IPs) related to Artificial Intelligence (AI), Machine Learning (ML), and Deep Learning (DL). Key features: - Provides a detailed list of AI-related ICs and IPs - Includes information on various manufacturers and their products - Offers insights into the latest developments in AI chip technology - Facilitates easy access to the most relevant resources and news Tech stack: PHP Why it matters: This repository serves as an essential resource for developers and researchers looking to stay updated on the latest advancements in AI chip technology. It provides a valuable reference for understanding and utilizing the most cutting-edge AI hardware and software solutions. Repository: basicmi/AI-Chip GitHub stars: 1712 GitHub description: A list of ICs and IPs for AI, Machine Learning and Deep Learning. README excerpt: <div align="center"><h1>AI Chip (ICs and IPs)</h1></div> <div align="center"><img src="https://github.com/basicmi/Deep-Learning-Processor-List/raw/master/resource/AI-chips.png"></div> <br> <div align="center">Editor <a href="https://www.linkedin.com/in/shan-tang-27342510/"><strong>S.T.</strong></a>(Linkedin)</div> <div align="center"><strong>Welcome to My Wechat Blog <a href="[https://mp.weixin.qq.com/s/axfIBbQBDhTJ2

### Overview: Real-time video and audio processing on Streamlit is achieved through the `streamlit-webrtc` package. This project allows developers to easily integrate real-time video and audio functionality into Streamlit applications, making it accessible for various use cases such as object detection, video filters, and real-time speech-to-text. ### Key Features: - Real-time video and audio streaming - Object detection - OpenCV filter - Uni-directional video streaming - Audio processing - Real-time speech-to-text - Video style transfer ### Tech Stack: - Python - Streamlit ### Why it Matters: This project simplifies the integration of real-time video and audio processing into Streamlit applications, making it easier for developers to create interactive and engaging web-based applications without the need for complex backend setups. It is particularly useful for projects requiring real-time video and audio processing, such as live streaming, video chat, and interactive tutorials.

Overview: This project focuses on enhancing the quality of photos taken on smartphones to match the standard of DSLR cameras. The solution leverages deep convolutional networks to translate ordinary smartphone photos into high-quality DSLR images, making it possible to achieve this effect regardless of the camera type. Key features: - Automatic photo quality enhancement - Generalizable to any type of digital camera - Available pre-trained models for iPhone, Blackberry, and Sony devices Tech stack: Python Why it matters: This project is significant for photographers and smartphone users who want to improve the quality of their photos without the need for expensive DSLR equipment. The ability to enhance photos from any device to a professional standard can democratize photography, making it more accessible and enjoyable for everyone.

Overview: AI Dev Tools Zoomcamp is a free, hands-on course designed for developers and technical data professionals to learn disciplined AI-assisted software development. The course focuses on using modern AI developer tools for building, testing, deploying, extending, and auditing software without losing engineering discipline. Key features include: - Comparing modern AI developer tools - Building and deploying a full-stack application - Operating with observability and agent-assisted incident response - Extending coding agents with MCP and reusable capabilities Tech stack: Python Why it matters: This course provides a practical, repeatable workflow for disciplined software development with AI-assisted tools, ensuring maintainability, tests, and reliable deployment. Repository: DataTalksClub/ai-dev-tools-zoomcamp GitHub stars: 1710 GitHub description: A free, hands-on course on using AI developer tools to build, test, deploy, extend, and audit software without losing engineering discipline. The 2026 cohort starts August 31.

### Overview: TextBrewer is a PyTorch-based knowledge distillation toolkit designed for natural language processing (NLP). It offers a range of distillation techniques from both NLP and Computer Vision (CV) fields, making it easier for users to experiment with state-of-the-art methods for model compression, performance enhancement, and reduced inference time. ### Key Features: - **Model Distillation**: TextBrewer supports various distillation techniques, including those from NLP and CV. - **Easy-to-Use Framework**: Provides a straightforward framework for distillation experiments, allowing users to quickly experiment with the latest methods. - **State-of-the-Art Methods**: Includes cutting-edge techniques like BERT-EMD and custom distillers, making it a valuable tool for researchers and practitioners in NLP. ### Tech Stack: Python ### Why it Matters: TextBrewer is essential for anyone working on NLP tasks who needs to optimize model sizes, improve inference speed, and reduce memory usage. Its flexibility and ease of use make it a go-to tool for researchers and practitioners in the field.

### Overview: Local Light Field Fusion is a novel method for practical view synthesis from sparse input images using tensor flow. It was presented at SIGGRAPH 2019 and is designed to be a real-world application for real-time rendering. ### Key features: - **Practical View Synthesis:** Generates novel views from sparse input images. - **TensorFlow Implementation:** Utilizes TensorFlow for efficient computation. - **Prescriptive Sampling Guidelines:** Provides guidelines for effective sampling to improve synthesis quality. ### Tech stack: - C++ - TensorFlow ### Why it matters: This method offers a practical solution for view synthesis from sparse input images, making it suitable for real-time applications in fields such as augmented reality, virtual reality, and computer vision.

Overview: arxiv-sanity-lite is a lightweight re-write of the popular arxiv-sanity project, designed to provide users with recommendations of similar papers based on their tags. Key features: - Tag papers of interest - Recommend new papers for each tag - Search, rank, sort, slice, and dice results - Send daily email recommendations based on tags Tech stack: Python Why it matters: arxiv-sanity-lite offers a user-friendly interface for curating tags and receiving personalized paper recommendations, making it an excellent tool for researchers and students looking to stay updated on their field of interest. Repository: karpathy/arxiv-sanity-lite Primary language: Python GitHub stars: 1708 GitHub description: arxiv-sanity lite: tag arxiv papers of interest get recommendations of similar papers in a nice UI using SVMs over tfidf feature vectors based on paper abstracts. README excerpt:

Overview: TokenFlow is an official PyTorch implementation of "TokenFlow: Consistent Diffusion Features for Consistent Video Editing" presented at ICLR 2024. It enables consistent video editing using a pre-trained text-to-image diffusion model without further training or fine-tuning. Key features: - Consistent video editing - Preserves spatial layout and dynamics - Adapts to localized and global edits - Supports semi-transparent effects like smoke, fire, and snow Tech stack: Python Why it matters: TokenFlow provides a powerful tool for consistent video editing, enhancing the quality and user control over generated content compared to state-of-the-art image models. It can be seamlessly integrated with existing text-to-image editing methods, making it a valuable addition to the video editing toolkit. Repository: omerbt/TokenFlow Primary language: Python GitHub stars: 1707

### Overview: Awesome-Jailbreak-on-LLMs is a collection of cutting-edge jailbreak methods for Large Language Models (LLMs). It includes detailed papers, codes, datasets, evaluations, and analyses. The repository welcomes contributions and is open to researchers and practitioners interested in understanding and improving the security of LLMs. ### Key features: - Papers: Comprehensive collection of research on jailbreak methods. - Codes: Implementations of various jailbreak techniques. - Datasets: Useful datasets for evaluating jailbreak methods. - Evaluations: Detailed analyses and comparisons of jailbreak effectiveness. - Contributions: Encourages contributions from the community. ### Tech stack: The repository uses Python, Jupyter notebooks, and other tools commonly used in research and development. ### Why it matters: Understanding and improving the security of LLMs is crucial for protecting sensitive information and ensuring the reliability of AI systems. This repository provides valuable insights and resources for researchers and practitioners working on LLM security.

### Overview: 本项目是一个机器学习面试题的总结，涵盖了算法工程师所需的基本概念、经典机器学习模型、特征工程方法、特征选择、线性回归、逻辑回归、线性模型和非线性模型的区别、生成式模型和判别式模型的区别等多方面内容。旨在帮助开发者系统地掌握机器学习面试中的常见问题和解决方案。 ### Key features: - 算法工程师-机器学习面试题总结 - 基本概念 - 解决机器学习问题的流程 - 损失函数的定义和选择 - 回归模型和分类模型的损失函数 - 结构误差和经验误差的区别 - 模型泛化能力的判断和提升方法 - 模型评估指标的选择和计算 - 混淆矩阵和ROC曲线的绘制 - 模型过拟合和欠拟合的识别和解决方法 - 模型选择和超参数优化 - 误差分析和模型偏差、方差的理解 - 奥卡姆剃刀定律和机器学习模型优化的启发 - 经典机器学习 - 特征工程方法 - 设计特征的流程 - 数据探索和特征选择 - 数据清洗和异常值处理 - 缺失值处理 - 归一化和离散化方法 - 线性模型和非线性模型的区别

Overview: CakeChat is a chatbot system that uses transformer-based models to express emotions in conversations. Key features: - Conditional model responses based on an arbitrary categorical variable. - Fine-tuning pre-trained models on custom data for personalized interactions. - Training from scratch or fine-tuning on existing datasets. Tech stack: Python, TensorFlow, Keras. Why it matters: CakeChat leverages state-of-the-art transformer-based models for more effective emotional dialogue, making it a promising tool for creating empathetic chatbots. Repository: lukalabs/cakechat GitHub stars: 1707 GitHub description: CakeChat: Emotional Generative Dialog System

Overview: Agent-R1 is a unified, modular framework designed for training powerful LLM agents using end-to-end reinforcement learning (RL). It focuses on agentic reinforcement learning, where every interaction is treated as a step-level MDP transition, making it distinct from single-turn RL pipelines. Key features: - Step-level MDP transition modeling - Flexible context management - Layered abstractions for agentic reinforcement learning - Support for online policy distillation (OPD) - Integration with StepPO and various recipe integrations Tech stack: Python Why it matters: Agent-R1 provides a novel approach to agentic reinforcement learning, focusing on explicit modeling of every step in the interaction process, which is crucial for training powerful LLM agents. This framework is particularly useful in scenarios where the environment and agent interactions are complex and require detailed step-by-step reasoning.

Overview: Katib is a Kubernetes-native project designed for automated machine learning, supporting hyperparameter tuning, early stopping, and neural architecture search. It is agnostic to machine learning frameworks and supports various frameworks like TensorFlow, PyTorch, and XGBoost. Key features: - Automated machine learning on Kubernetes - Supports hyperparameter tuning, early stopping, and neural architecture search - Agnostic to machine learning frameworks - Supports many ML frameworks, including TensorFlow, PyTorch, and XGBoost - Native support for Kubeflow Training Operator, Argo Workflows, and Tekton Pipelines - Integrates with Kubeflow Katib for seamless integration with Kubeflow pipelines Tech stack: Katib is built using Python and is agnostic to the underlying machine learning framework. Why it matters: Katib simplifies the process of automating machine learning experiments, making it easier for developers to experiment with different models and configurations. It supports various search algorithms and integrates seamlessly with Kubeflow, making it a valuable tool for organizations looking to improve their machine learning workflows.

### Overview: This repository is a curated collection of AI Agent papers, focusing on the latest research in the field of AI agents. It includes papers that introduce unique approaches or novel concepts, rather than aiming for comprehensiveness. The papers are categorized into four layers: capabilities, architecture, operations, and applications. ### Key Features: - **Agent Capabilities**: What an agent can do, including cognition, knowledge, action, and learning. - **Core Cognition**: Reasoning, planning, ideation, and perception. - **Knowledge & Context**: Memory, context engineering, knowledge graphs, and context perception. - **Action**: Tool use, skills, and adaptation. - **Adaptation & Self-Improvement**: Exploration, experience learning, failure attribution, self-correction, self-evolution, and agent tuning. - **Trust & Measurement**: Safety and agent evaluation. - **Other**: World models, user profiles, and prediction. - **AI Agents Architecture**: How an agent is built, including single-agent design, multi-agent systems, and runtime harness. - **Agent Design & Frameworks**: Agent design and frameworks. - **Multi-Agent Systems**: Multi-agent systems. - **Harness**: Runtime harness. - **Operations & Interaction**: How an agent is run and interacts with people. - **Observability**: Observability. - **UX**: User experience. - **Governance**: Governance. ### Tech Stack: Python ### Why it Matters: This repository highlights the latest and most innovative research in AI

### Overview: ImageReward is a comprehensive framework designed for evaluating and learning human preferences in text-to-image generation. It leverages expert comparisons to train a model that accurately reflects human preferences, surpassing existing methods in terms of accuracy and effectiveness. ### Key Features: - **ImageReward Model**: A general-purpose text-to-image human preference model trained on 137k pairs of expert comparisons, significantly outperforming CLIP, Aesthetic, and BLIP. - **Reward Feedback Learning (ReFL)**: A method for directly optimizing a text-to-image diffusion model using ImageReward, achieving a 58.4% improvement in human evaluation over an untuned version. ### Tech Stack: - Python ### Why It Matters: ImageReward provides a robust and accurate way to evaluate and learn from human preferences in text-to-image generation, making it a valuable tool for improving the quality and relevance of AI-generated images. ### Repository: - **Repository**: zai-org/ImageReward - **GitHub Stars**: 1708 - **Primary Language**: Python - **GitHub Description**: NeurIPS 2023 | ImageReward: Learning and Evaluating Human Preferences for Text-to-image Generation - **GitHub URL**: [zai-org/ImageReward](https://github.com/zai-org/ImageReward) ### Demos of ImageReward and ReFL: - **Score generated images and append to image information**: Automatically score generated images and append scores to their metadata. - **Automatically filter out images with low scores**: Automatically filter out images that receive low scores. -

### Overview: LLM Gateway is an open-source API gateway designed to manage and route Large Language Model (LLM) requests across multiple providers. It simplifies the integration of various LLMs by providing a unified API interface, enabling seamless communication and efficient management of API keys and costs. ### Key Features: - **Unified API Interface**: Compatible with OpenAI API format, facilitating easy migration. - **Usage Analytics**: Tracks token usage, response times, and costs across all LLM interactions. - **Multi-provider Support**: Connects to multiple LLM providers through a single gateway. - **Performance Monitoring**: Analyzes and compares the performance and cost-effectiveness of different models. ### Tech Stack: - TypeScript ### Why It Matters: LLM Gateway simplifies the integration and management of LLMs, making it easier to switch between providers and optimize LLM usage. Its unified interface and comprehensive analytics help developers and businesses streamline their AI workflows and reduce costs.

Overview: DingoDB is a multi-modal vector database designed by DataCanvas, offering high concurrency and ultra-low latency with MySQL-compatible SQL. It supports upserts and vector queries, integrating real-time strong consistency, relational semantics, and vector semantics. Key features: 1. Comprehensive access interface 2. Built-in data high availability 3. Fully automatic elastic data sharding 4. Scalar-vector hybrid retrieval 5. Built-in real-time index optimization 6. Cold-Hot Tiered Retrieval for Massive Datasets Tech stack: Java Why it matters: DingoDB stands out as a robust solution for modern data-driven applications, providing exceptional functionality, performance, and user-friendliness. Repository: dingodb/dingo GitHub stars: 1705

Overview: MLRun is an open-source platform designed for building and managing continuous AI applications across their lifecycle. It integrates seamlessly into development and CI/CD environments, automating data, pipelines, and applications for efficient delivery. Key features: - Supports batch and real-time data processing - Provides data lineage and versioning - Handles structured and unstructured data - Automates the development, scaling, deployment, and monitoring of AI models Tech stack: Python Why it matters: MLRun significantly reduces engineering effort, time to production, and computational resources, enabling collaboration and continuous improvements across data, ML, software, and DevOps/MLOps teams. Repository: mlrun/mlrun GitHub stars: 1704 GitHub description: MLRun is an open-source platform for quickly building and managing continuous AI applications across their lifecycle. It integrates into development and CI/CD environments and automates the delivery of production data, ML pipelines, and online applications.