Projects

### Overview: CORAL is an open-source platform designed for autonomous AI agent organizations. It supports multi-agent self-evolution, isolated workspaces, shared knowledge, and persistent state, all managed by a robust infrastructure powered by Claude Code, OpenCode, and Codex. ### Key Features: - **Multi-Agent Self-Evolution**: CORAL enables agents to evolve and improve their solutions in an isolated environment. - **Persistent Shared State**: Agents can share knowledge and state across multiple islands or workspaces. - **Autonomous Coding**: Agents can run experiments and share knowledge through a common state directory. - **Grading and Evaluation**: CORAL supports grading and evaluation of agents' work, including shared knowledge and multi-agent collaboration. ### Tech Stack: CORAL is built on Python, leveraging the capabilities of Claude Code, OpenCode, and Codex for its autoresearch and agent-driven functionality. ### Why It Matters: CORAL provides a robust, lightweight infrastructure for autonomous AI agent organizations, making it ideal for continuous learning and innovation in AI research and development. Its features enable agents to collaborate effectively, share knowledge, and improve their solutions through self-evolution, making it a valuable tool for researchers and developers alike.

Overview: ClawBench is an open-source benchmark designed to evaluate browser AI agents on daily tasks. It provides a standardized environment for testing and comparing the performance of AI-driven applications across different browsers and platforms. Key features: - Standardized benchmarking environment - Supports multiple browsers and operating systems - Tracks performance metrics for various tasks - Provides leaderboard for community benchmarking - Compatible with Hugging Face datasets Tech stack: Python Why it matters: ClawBench offers a crucial tool for developers and researchers to objectively compare the capabilities of AI-driven browser applications, ensuring fair and accurate evaluations across different environments and platforms.

### Overview: Bubble Lab is an open-core workflow engine powering the Bubble Lab platform, enabling teams to automate operations directly within Slack using AI assistant Pearl. ### Key Features: - **Workflow Execution Runtime**: Runs workflows locally or in the cloud. - **Agent and Integration Primitives**: Supports custom agents and integrations. - **Local Workflow Studio**: Allows building and editing workflows. - **Execution Tracing and Observability**: Provides detailed logs and dashboards for monitoring. - **CLI Tooling**: Facilitates command-line management of workflows. - **Exportable Workflows**: Enables workflows to be deployed anywhere. - **Self-Hosted Infrastructure**: Suitable for teams, developers, and organizations needing custom solutions. ### Tech Stack: TypeScript ### Why It Matters: This open-core engine is essential for teams using the Bubble Lab platform, developers seeking full control, organizations requiring self-hosted automation, and engineers building custom agents or integrations.

Overview: ShareGPT4Video is an official implementation of a large-scale highly descriptive video-text dataset and a general video captioner designed for various video durations, resolutions, and aspect ratios, aiming to improve text-to-video performance with high-quality video captions generated by our ShareCaptioner-Video model. Key features: - A large-scale highly descriptive video-text dataset with 40K GPT4-Vision-generated video captions and around 400K implicit video split captions. - A general video captioner for various video durations, resolutions, and aspect ratios, approaching GPT4-Vision's caption capability. - A superior large video-language model ShareGPT4Video-8B, lasting 5 hours on 8xA100 GPUs of training respectively. Tech stack: Python Why it matters: ShareGPT4Video is a significant contribution to the field of video understanding and generation, offering high-quality video captions that can enhance the accuracy and relevance of video analysis and generation tasks.

Overview: Kaldi GStreamer server is a real-time, full-duplex speech recognition server built on the Kaldi toolkit and GStreamer framework, implemented in Python. It supports full duplex communication, scalability, speech segmentation, and various advanced features like rescoring, codec support, and more. Key features: - Full duplex communication based on websockets - Very scalable: workers can be started and stopped independently - Supports speech segmentation - Supports arbitrary long speech input - Supports Kaldi's GMM and "online DNN" models - Supports rescoring with a large language model - Supports persisting acoustic model adaptation state - Supports multiple audio codecs - Supports rewriting raw recognition results using external programs Tech stack: Python Why it matters: This server is essential for real-time speech recognition applications, particularly in scenarios requiring high scalability and real-time communication. Its ability to handle large audio inputs and support advanced post-processing makes it highly valuable in various applications, including live speech recognition and language translation systems.

### Overview: TimeCraft is an innovative framework designed to generate high-quality synthetic time series for real-world applications. Leveraging diffusion models, it addresses the limitations of existing methods by enabling cross-domain generalization, text-based control, and targeted adaptation, making it a versatile tool for various domains. ### Key Features: - **Cross-Domain Generalization**: TimeCraft uses a universal latent space to encode domain-invariant features, allowing it to generate time series that align with the structure of unseen domains. - **Text-Based Control**: TimeCraft integrates a multi-agent text generation system to produce detailed time series descriptions, enhancing the controllability and interpretability of the generated data. - **Target-Aware Adaptation**: It generates samples with the aim of improving downstream model performance, rather than just mimicking the training data distribution. ### Tech Stack: Python ### Why It Matters: TimeCraft stands out by providing a robust solution for generating time series data that is both high-fidelity and domain-specific, making it invaluable in applications requiring realistic but synthetic data. Its ability to adapt to new domains and generate controlled data with rich textual descriptions makes it a powerful tool for researchers and practitioners in various fields.

Overview: TencentPretrain is a toolkit designed for pre-training and fine-tuning on various modalities, including text, vision, and audio, aiming to enhance AI technology. Key features: - Reproducibility: Tested on multiple datasets, matching original pre-training model implementations. - Model modularity: Divided into embedding, encoder, target embedding, decoder, and target parts, allowing flexible combination of modules. - Multimodal support: Compatible with text, vision, and audio modalities. - Model training: Supports CPU, single GPU, distributed training, and DeepSpeed for gigantic model training. - Model zoo: Contains pre-trained models of different properties, facilitating proper selection for downstream tasks. - SOTA results: Supports comprehensive downstream tasks and provides winning solutions for competitions. - Abundant functions: Includes feature extractor, text generation, and other pre-training-related functionalities. Tech stack: Python Why it matters: TencentPretrain is essential for researchers and developers looking to enhance AI technology by pre-training and fine-tuning on diverse modalities, making it a valuable tool for various AI applications.

Overview: Text2Room is a project that generates textured 3D meshes from text prompts using 2D text-to-image models, as presented in the ICCV 2023 paper "Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models". The repository contains the source code and includes a pretrained model for fixed depth inpainting and text-to-image models. Key features: - Generates textured 3D meshes from text prompts - Uses 2D text-to-image models for input generation - Supports generating scenes from living room templates - Outputs include mesh files, renderings, and metadata files Tech stack: Python Why it matters: This project provides a practical tool for generating detailed 3D models from text descriptions, which can be useful in various applications such as virtual reality, augmented reality, and interactive design tools. Repository: lukasHoel/text2room GitHub stars: 1093

### Overview: VoAPI 是一个全新的下一代高颜值、高性能、高扩展的智能AI大模型API聚合分发系统。它提供了一系列强大的功能，包括基础功能、多用户管理、规则引擎支持、API数据转发、多余额机制、多货币机制、每日签到、用户等级、实时RPM/TPM、渠道实时RPM/TPM、渠道单密钥实时RPM/TPM、训练场自定义多外部页面嵌入、远程模型、厂商数据同步、渠道分组支持、渠道分组固定倍率、定时倍率、渠道重试、渠道密钥错误禁用、渠道密钥错误禁用自动恢复、渠道请求熔断超时机制、渠道IP、UA规则限制、渠道全局/独立代理配置、兑换码系统、自定义菜单、第三方登录、安全过滤、API线路展示与测速、自定义SEO、主题色调与全局样式、脚本等。此外，它还支持一键第三方系统数据迁移导入、多优先级计费方式、日志分表、i18n国际化、模型多条件计费、在线支付支持、自定义多货币、自动支付汇率换算、内部换算或支付接口换算、在线自助开票、强大的工单系统、用户可独立的费率、权限定制化、高性能高扩展的动态路由、邀请好友机制、强大的第三方对象存储转存功能、自定义多前端语言、在线MCP服务分发支持、不同维度的用户行为、数据

Overview: PDFOxide is a fast, reliable, and comprehensive PDF toolkit that supports 20 languages, including Rust, Python, Go, JavaScript/TypeScript, C#, Java, Kotlin, Scala, Clojure, Ruby, PHP, C++, Objective-C, Swift, Dart, R, Julia, Zig, Elixir, and WASM. It excels in text extraction, image extraction, and Markdown conversion, achieving 0.8ms mean processing time, making it 5x faster than industry leaders and 100% pass rate on 3,830 real-world PDFs. Key features: - Rust core with bindings for 19 languages - 20 languages support: Python, Go, JS/TS, C#, Java, Kotlin, Scala, Clojure, Ruby, PHP, C++, Objective-C, Swift, Dart, R, Julia, Zig, Elixir, and WASM - 0.8ms mean processing time, 5x faster than PyMuPDF, 15x faster than pypdf - 100% pass rate on 3,830 real-world PDFs - MIT/Apache-2.0 license Tech stack: Rust Why it matters: PDFOxide is the fastest PDF library for text extraction, image extraction, and Markdown conversion, making it essential for developers looking to streamline their PDF processing workflows. Its reliability and performance make it a go-to choice for applications requiring high-speed PDF manipulation.

### Overview: MarkLLM is an open-source toolkit designed for LLM watermarking, providing a comprehensive solution for protecting large language models from unauthorized use and identification. ### Key Features: - Implementable LLM watermarking algorithms - Robust and unforgeable public verification mechanism - Compatible with various LLM models and frameworks - Easy integration into existing projects - Comprehensive documentation and examples ### Tech Stack: Python ### Why it Matters: MarkLLM simplifies the process of watermarking large language models, making it more accessible to developers and researchers. By providing a reliable and efficient solution, it helps protect intellectual property rights and ensures the integrity of AI-generated content.

Overview: AIConfig is a framework designed to simplify the creation and management of generative AI applications, making it easier to build and maintain production-grade AI models. Key features: - Version control of generative AI prompts, models, and parameters - Ability to store and iterate on AI behavior separately from application code - Streamlined workflow for rapid prototyping - Open-source framework, available on GitHub Tech stack: Python, Node.js, TypeScript Why it matters: AIConfig provides a structured way to manage AI configurations, allowing for easy version control, evaluation, monitoring, and rapid prototyping, making it a valuable tool for developers working on AI applications. Repository: lastmile-ai/aiconfig GitHub stars: 1089 GitHub description: AIConfig is a config-based framework to build generative AI applications. GitHub badge links: - Python: [](https://github.com/lastmile-ai/aiconfig/actions/workflows/main_python.yml) - Node: [](https://github.com/lastmile-ai/aiconfig/actions/workflows/main-typescript.yml) - Docs: [](https://github.com/lastmile-ai/aiconfig/actions

### Overview: Aria is an Open Multimodal Native MoE model designed for state-of-the-art performance across various multimodal and language tasks, with a long context window and high parameter efficiency. ### Key features: - **State-of-the-art performance**: Superior in video and document understanding. - **Long context window**: 64K tokens. - **High parameter efficiency**: 3.9B activated parameters per token, enabling fast inference speed and low fine-tuning cost. ### Tech stack: Jupyter Notebook ### Why it matters: Aria is a powerful tool for multimodal and language tasks, offering superior performance and efficiency, making it ideal for applications requiring high accuracy and low computational resources.

### Key features: - Evaluation of LLM hallucination - Source of LLM hallucination - Mitigation of LLM hallucination - Contact ### Tech stack: Python, Markdown, LaTeX ### Why it matters: This issue significantly undermines the reliability of LLMs in real-world scenarios.

Overview: 最近，通用领域的大语言模型（LLM），例如 ChatGPT，在遵循指令和产生类似人类响应方面取得了显著的成功，这种成功间接促进了多模态大模型的研究和发展，如通用领域的多模态大模型[MiniGPT-4](https://github.com/Vision-CAIR/MiniGPT-4)、[mPLUG-Owl](https://github.com/X-PLUG/mPLUG-Owl)、[Multimodal-GPT](https://github.com/open-mmlab/Multimodal-GPT)和[LLaVA](https://github.com/haotian-liu/LLaVA)。然而，此类多模态大模型却很少出现在医学领域的研究中，阻碍了相关研究发展。为此，我们开发了`XrayGLM`以解决上述问题。`XrayGLM`在医学影像诊断和多轮交互对话上显示出了非凡的潜力。 Key features: - 提供中文胸部X光片诊断数据集 - 在[VisualGLM-6B](https://github.com/THUDM/VisualGLM-6B)上微调训练 - 开放部分训练权重用于学术研究 - 预处理英文报告数据集，生成可用于训练的中文报告数据集 Tech stack: Python Why it matters: 通过开发`XrayGLM`，我们解决了医学多模态大模型在中文领域的研究发展滞后问题，为相关研究提供了新的研究方向和数据

### Overview: Whatlang is a Rust library designed for natural language detection, focusing on simplicity and performance. It supports over 70 languages and scripts, making it a valuable tool for developers working with international content. ### Key Features: - **Supports Multiple Languages and Scripts**: Recognizes languages and scripts like Latin, Cyrillic, and others. - **Lightweight and Fast**: Written in Rust, it is lightweight and performs well. - **Reliable Detection**: Provides reliability information to ensure accurate language detection. ### Tech Stack: - Rust ### Why It Matters: Whatlang simplifies language detection in Rust, making it easier and faster to work with international content. Its reliability and performance make it a preferred choice for developers looking to integrate language detection into their projects. ### Repository: greyblake/whatlang-rs - GitHub Stars: 1087 - Primary Language: Rust - Documentation: [https://docs.rs/whatlang](https://docs.rs/whatlang) - Supported Languages: [https://github.com/greyblake/whatlang-rs/blob/master/SUPPORTED_LANGUAGES.md](https://github.com/greyblake/whatlang-rs/blob/master/SUPPORTED_LANGUAGES.md) - Feature Toggles: [https://github.com/greyblake/whatlang-rs/blob/master/README.md#feature-toggles](https://github.com/greyblake/whatlang-rs/blob/master/README.md#feature-toggles) - Comparison with Alternatives: Whatlang offers a reliable and efficient solution for language

### Overview: This repository contains 100+ LLM interview questions and answers, designed to help ML Engineers, AI Engineers, Data Scientists, and Software Engineers prepare for modern LLM and generative AI interviews. ### Key Features: - **Interview Questions**: 100+ curated questions covering various aspects of LLMs, generative AI, and related technologies. - **Answers and Explanations**: Each question comes with clear answers and detailed explanations to ensure a comprehensive understanding. - **Book**: A comprehensive guide featuring 100+ interview questions, answers, and explanations, available for purchase with a discount code. ### Tech Stack: - **GitHub**: Repository management and version control. - **Markdown**: Used for formatting and documentation. ### Why It Matters: - **Interview Preparation**: Helps candidates prepare for interviews by providing relevant and practical questions. - **Educational Resource**: A valuable resource for developers looking to enhance their knowledge in LLM and generative AI technologies. ### Repository: KalyanKS-NLP/LLM-Interview-Questions-and-Answers-Hub - **Stars**: 1086 - **Description**: Contains 100+ LLM interview questions and answers. - **Image**:  - **Related Repositories**: - [RAG Interview Questions and Answers Hub](https://github.com/KalyanKS-NLP

Overview: Claude Code Chat is a beautiful, intuitive interface for VS Code that allows users to chat with Claude Code directly within the editor, eliminating the need for the terminal. Key features: - No terminal required - everything through the UI - Real-time streaming responses with typing indicators - One-click message copying with visual feedback - Enhanced markdown support with syntax highlighting - Auto-resizing input that grows with your content - Copy-to-clipboard for code blocks Tech stack: JavaScript Why it matters: This extension brings Claude Code's chat interface into VS Code, making AI assistance accessible, visual, and enjoyable.

### Overview: Microsoft AI for Good Lab's Biodiversity project is a comprehensive open-source platform for biodiversity monitoring and conservation. It includes models like MegaDetector, SPARROW, PytorchWildlife, and Bioacoustics, providing tools for camera-trap detection, bioacoustic analysis, species classification, and field deployment. ### Key features: - **MegaDetector**: AI model for detecting animals, people, and vehicles in camera-trap imagery. - **PytorchWildlife**: Unified platform for camera-trap, bioacoustics, and wildlife monitoring. - **Bioacoustics**: AI for biodiversity monitoring from sound classification and species identification. - **Species Classification**: Fine-tuning of camera-trap species classification classifiers for adaptability. - **Overhead Imagery Detection**: Point-based wildlife localization from aerial views. - **Sonar-Based Wildlife Detection**: Processing and feature detection in sidescan sonar imagery. ### Tech stack: Python ### Why it matters: This project serves as a central hub for Microsoft's AI for Good Lab's biodiversity research, providing a unified platform for various AI models and tools. It facilitates collaboration, code sharing, and easy extension of functionalities, making it easier for researchers and practitioners to contribute and utilize the diverse tools and models available.

Overview: This project is a PyTorch implementation of Stable Diffusion, a state-of-the-art generative model for image generation. Key features: - PyTorch-based Stable Diffusion model - Supports fine-tuning with pre-trained weights - Includes tokenizer and vocabulary files for model compatibility Tech stack: PyTorch Why it matters: This project provides a practical and efficient way to experiment with and fine-tune Stable Diffusion, leveraging the power of PyTorch for deep learning tasks.