Projects

Overview: Databend is an open-source enterprise data warehouse built in Rust, designed for large-scale analytics, vector search, and full-text search. It supports flexible agent orchestration and secure Python UDF sandboxes, making it ideal for enterprise AI workloads. Key features: - Unified architecture on S3 - Built-in analytics, vector search, and full-text search capabilities - Agent-ready with sandbox UDFs and SQL orchestration - Secure Python UDFs for agent logic - Reliable transactions and safe experimentation on production data Tech stack: Rust Why it matters: Databend offers a single engine for large-scale analytics, vector search, and full-text search, making it a comprehensive solution for enterprise AI workloads. Its agent-ready architecture supports flexible agent orchestration and secure Python UDFs, ensuring safe and reliable operations. Repository: databendlabs/databend GitHub stars: 9468

Overview: CCF-Deadlines is a platform designed to help researchers track the deadlines and other important information for CCF-ranked and worldwide conferences. Key features: - Track worldwide conference deadlines - Collaborate with researchers globally - Access a website portal, WeChat applet, and tabular portal for easy tracking - Provide a Python CLI tool for command-line integration - Offer a Raycast extension for seamless integration into various platforms - Support iCal subscription and Chrome extension for calendar integration - Implement a Model Context Protocol (MCP) server for AI clients to search conferences and retrieve historical acceptance rates Tech stack: Rust Why it matters: CCF-Deadlines simplifies the process of tracking conference deadlines and other important information for researchers, making it easier for them to stay informed and prepare for upcoming events. The platform's collaboration features and various tracking methods make it an essential tool for researchers and conference organizers alike. Repository: ccfddl/ccf-deadlines Primary language: Rust GitHub stars: 9457 GitHub description: Agenticly track ccf-ranked & worldwide conference deadlines (Website, Python Cli, Wechat Applet) README excerpt: A platform designed to help researchers track the deadlines and other important information for CCF-ranked and worldwide conferences.

### Overview: Soup is a powerful tool for fine-tuning and post-training large language models (LLMs) using YAML configurations. It simplifies the process by requiring only one command, eliminating the need for SSH or complex configuration files. ### Key Features: - **Fine-tune and Post-train LLMs in One Command**: Quickly fine-tune an 8B model on a 4 GB laptop GPU with Layer Streaming. - **No SSH Required**: Simplifies the process without the need for SSH, making it accessible even on low-spec machines. - **Lightweight CLI**: The CLI is designed to be minimalistic, requiring only a single command to fine-tune the model. ### Tech Stack: Python ### Why It Matters: Soup streamlines the LLM fine-tuning process, making it more accessible and efficient. It allows users to fine-tune large models on low-spec hardware, significantly reducing the time and computational resources required. This tool is particularly useful for developers and researchers who need to fine-tune models without the need for complex setups or extensive configuration files.

### Overview: Xberg is a powerful tool for extracting text, metadata, images, tables, and structured data from 106 formats across 140 file extensions, as well as providing code intelligence for 371 languages. It is built with Rust and leverages the Alef library for efficient and scalable processing. ### Key features: - **Text Extraction**: Extract text from various file formats. - **Metadata Extraction**: Retrieve metadata from documents. - **Image Extraction**: Extract images from PDFs and other image formats. - **Structured Data Extraction**: Extract structured data from spreadsheets and databases. - **Code Intelligence**: Provides code intelligence for 371 languages, including Python, Java, C#, and more. ### Tech stack: Rust ### Why it matters: Xberg is a highly efficient and versatile tool that simplifies the process of extracting and analyzing data from various file formats and languages. Its Rust implementation ensures fast performance and scalability, making it an essential tool for developers working across multiple languages and file types.

Overview: This repository is a curated collection of books, academic papers, online courses, videos, datasets, and toolboxes related to anomaly detection. It serves as a valuable resource for developers and researchers interested in the field of anomaly detection, including both machine learning and deep learning approaches. Key features: - Books and academic papers - Online courses and videos - Outlier datasets - Open-source and commercial libraries/toolkits Tech stack: Python Why it matters: This repository provides a comprehensive overview of the latest techniques and resources in anomaly detection, making it an essential tool for anyone looking to enhance their knowledge and skills in this field.

Overview: ParadeDB is a Postgres extension that enhances Postgres with custom indexes for full-text search, vector retrieval, and aggregations. It provides advanced search capabilities, including BM25 scoring, highlighting, filtering, aggregations, and joins, all within a single database. Key features: - Full-Text Search - Vector Search - Hybrid Search - Filtering - Aggregates (including columnar storage, bucket & metrics, and facets) - JOINs Tech stack: Rust Why it matters: ParadeDB simplifies the integration of advanced search and analytics capabilities into existing Postgres environments, making it easier to scale and improve query performance without the need for complex database migrations or additional tools.

Overview: PaperQA2 is a package designed for high-accuracy retrieval augmented generation (RAG) on various document types, including PDFs, text files, and Microsoft Office documents, with a focus on scientific literature. It aims to improve the accuracy and efficiency of answering scientific questions. Key features: - High accuracy RAG on diverse document types. - Scientific literature focus. - Supports PDFs, text files, Microsoft Office documents, and source code files. - Integrates with an LLM agent for question answering. Tech stack: Python. Why it matters: PaperQA2 is particularly useful in scientific research and academia where precise and accurate answers to scientific questions are crucial, making it ideal for tasks such as question answering, summarization, and contradiction detection. Repository: Future-House/paper-qa GitHub stars: 9401 GitHub description: High accuracy RAG for answering questions from scientific documents with citations.

### Overview: Oumi is a comprehensive platform designed to easily fine-tune, evaluate, and deploy Qwen, Gemma, or any open-agentic language model. It offers a state-of-the-art end-to-end solution for building foundation models, including support for SFT/RL, tool use, and agentic data synthesis. ### Key features: - **Fine-tuning and deployment**: Easily fine-tune models using SFT/RL, evaluate performance, and deploy models to dedicated inference endpoints. - **Tool integration**: Support for tools, environments, and agentic data synthesis. - **Multi-criteria evaluation**: Utilize RubricJudge for comprehensive evaluation. - **GRPO reinforcement learning**: Reinforcement learning capabilities for tool use and data synthesis. - **GPU support**: Tested and supported on GPUs for optimal performance. ### Tech stack: Python ### Why it matters: Oumi simplifies the process of building and deploying state-of-the-art language models, making it accessible to developers and researchers alike. Its end-to-end solution ensures high performance and reliability, making it an indispensable tool for anyone working with large language models.

Overview: FlexLLMGen is a high-throughput generation engine designed for running large language models on a single GPU, optimizing for throughput-oriented tasks. Key features: - IO-efficient offloading - Large effective batch sizes Tech stack: Python Why it matters: FlexLLMGen enables new applications of foundation models to throughput-oriented tasks on low-cost hardware, reducing latency for high-throughput processing. Repository: FMInference/FlexLLMGen Primary language: Python GitHub stars: 9345