Projects

Overview: ParadeDB is a Postgres extension that enhances Postgres with custom indexes for full-text search, vector retrieval, and aggregations. It provides advanced search capabilities, including BM25 scoring, highlighting, filtering, aggregations, and joins, all within a single database. Key features: - Full-Text Search - Vector Search - Hybrid Search - Filtering - Aggregates (including columnar storage, bucket & metrics, and facets) - JOINs Tech stack: Rust Why it matters: ParadeDB simplifies the integration of advanced search and analytics capabilities into existing Postgres environments, making it easier to scale and improve query performance without the need for complex database migrations or additional tools.

Overview: PaperQA2 is a package designed for high-accuracy retrieval augmented generation (RAG) on various document types, including PDFs, text files, and Microsoft Office documents, with a focus on scientific literature. It aims to improve the accuracy and efficiency of answering scientific questions. Key features: - High accuracy RAG on diverse document types. - Scientific literature focus. - Supports PDFs, text files, Microsoft Office documents, and source code files. - Integrates with an LLM agent for question answering. Tech stack: Python. Why it matters: PaperQA2 is particularly useful in scientific research and academia where precise and accurate answers to scientific questions are crucial, making it ideal for tasks such as question answering, summarization, and contradiction detection. Repository: Future-House/paper-qa GitHub stars: 9401 GitHub description: High accuracy RAG for answering questions from scientific documents with citations.

### Overview: Oumi is a comprehensive platform designed to easily fine-tune, evaluate, and deploy Qwen, Gemma, or any open-agentic language model. It offers a state-of-the-art end-to-end solution for building foundation models, including support for SFT/RL, tool use, and agentic data synthesis. ### Key features: - **Fine-tuning and deployment**: Easily fine-tune models using SFT/RL, evaluate performance, and deploy models to dedicated inference endpoints. - **Tool integration**: Support for tools, environments, and agentic data synthesis. - **Multi-criteria evaluation**: Utilize RubricJudge for comprehensive evaluation. - **GRPO reinforcement learning**: Reinforcement learning capabilities for tool use and data synthesis. - **GPU support**: Tested and supported on GPUs for optimal performance. ### Tech stack: Python ### Why it matters: Oumi simplifies the process of building and deploying state-of-the-art language models, making it accessible to developers and researchers alike. Its end-to-end solution ensures high performance and reliability, making it an indispensable tool for anyone working with large language models.

### Overview: Soup is a powerful tool for fine-tuning and post-training large language models (LLMs) using YAML configurations. It simplifies the process by requiring only one command, eliminating the need for SSH or complex configuration files. ### Key Features: - **Fine-tune and Post-train LLMs in One Command**: Quickly fine-tune an 8B model on a 4 GB laptop GPU with Layer Streaming. - **No SSH Required**: Simplifies the process without the need for SSH, making it accessible even on low-spec machines. - **Lightweight CLI**: The CLI is designed to be minimalistic, requiring only a single command to fine-tune the model. ### Tech Stack: Python ### Why It Matters: Soup streamlines the LLM fine-tuning process, making it more accessible and efficient. It allows users to fine-tune large models on low-spec hardware, significantly reducing the time and computational resources required. This tool is particularly useful for developers and researchers who need to fine-tune models without the need for complex setups or extensive configuration files.

Overview: FlexLLMGen is a high-throughput generation engine designed for running large language models on a single GPU, optimizing for throughput-oriented tasks. Key features: - IO-efficient offloading - Large effective batch sizes Tech stack: Python Why it matters: FlexLLMGen enables new applications of foundation models to throughput-oriented tasks on low-cost hardware, reducing latency for high-throughput processing. Repository: FMInference/FlexLLMGen Primary language: Python GitHub stars: 9345

### Overview: Mobile-Agent is a powerful GUI agent family designed for seamless mobile use experience. It offers a 2-stage training pipeline to optimize GUI tool path orchestration, ensuring optimal GUI-Tool path orchestration. ### Key Features: - **Mobile-Agent-v3.5**: Available on Alibaba Cloud Wuying Cloud Phone and Modelscope API-Inference. - **GUI-Owl-1.5**: A series of native multi-platform GUI agent foundation models (2B/4B/8B/32B/235B; Instruct & Thinking). - **GUI-Owl-32B**: A high-performance model for mobile use. - **GUI-Owl-7B**: A lightweight model for quick inference. ### Tech Stack: Python ### Why it Matters: Mobile-Agent provides a powerful and efficient solution for mobile use, ensuring optimal GUI-Tool path orchestration and achieving state-of-the-art results on various GUI benchmarks.

### Overview: Deep Lake is an AI data runtime solution that simplifies the deployment of enterprise-grade language models (LLMs) by providing a scalable, serverless database for storing and querying data. It supports various data types including embeddings, audio, text, videos, images, DICOM, PDFs, and more, enabling efficient data retrieval and training. ### Key Features: - **Multi-Cloud Support**: Upload, download, and stream datasets to/from S3, Azure, GCP, Activeloop cloud, local storage, or in-memory storage. - **Native Compression with Lazy NumPy-like Indexing**: Store images, audio, and videos in their native compression, allowing for efficient slicing, indexing, and interaction. - **Dataloaders for Popular Deep Learning Frameworks**: Built-in dataloaders for PyTorch and TensorFlow, making it easy to train models with just a few lines of code. - **Integrations with Powerful Tools**: Integrates with Langchain and LlamaIndex for vector store functionality, and supports Weights & Biases for data lineage. ### Tech Stack: C++ ### Why It Matters: Deep Lake simplifies the deployment of enterprise-grade language models by providing a scalable, serverless database for storing and querying data. It supports various data types, enabling efficient data retrieval and training, making it a valuable tool for developers and data scientists working on AI applications.

Overview: This repository is a collection of awesome generative AI applications, focusing on the landscape of Generative AI. Key features: - A collection of 3559 Generative AI applications - 80 categories of Generative AI applications - 9213 stars on GitHub - 1020 forks on GitHub - MIT license Tech stack: Unknown Why it matters: This repository provides a curated list of Generative AI applications, helping developers and enthusiasts explore and learn about the latest advancements in the field.

### Overview: ModelScope is a Model-as-a-Service platform that provides a unified interface for developers to access and utilize advanced machine learning models across various domains. ### Key Features: - Unified API for model inference, training, and evaluation. - Integration of models from different areas (CV, NLP, Speech, Multi-Modality, Scientific-computation). - Easy access to models through layered APIs, allowing customization of components. - Seamless interactions with ModelScope backend services, including Model-Hub and Dataset-Hub. - Public availability of hundreds of models, including state-of-the-art models in NLP, CV, Audio, and AI for Science. ### Tech Stack: Python ### Why It Matters: ModelScope simplifies the process of leveraging AI models in real-world applications, making it easier and more efficient for developers to integrate advanced models into their projects.

Overview: Morphic is an AI-powered search engine with a generative UI, designed to provide accurate and contextually relevant answers through a rich, interactive experience. Key features: - AI-powered search with grounded, cited answers - Generative UI — answers render rich inline components (source-credited images, grids, headings) live from a streamed JSON spec - Search modes: Quick and Adaptive - Model selector with dynamic provider detection (OpenAI, Anthropic, Google, Ollama, Vercel AI Gateway, OpenAI-compatible providers) - Multiple search providers (Tavily, SearXNG, Brave, Exa) - Chat history stored in PostgreSQL - Share search results with unique URLs - File upload support - User authentication with built-in accounts (Better Auth) or Supabase Auth - Guest mode for anonymous usage - Docker deployment ready Tech stack: TypeScript Why it matters: Morphic offers a seamless, AI-driven search experience with interactive, contextually rich results, making it a valuable tool for users seeking accurate and engaging information.

### Overview: HTML Anything is an agentic HTML editor that automates the writing and publishing of HTML documents. It uses local AI agents to generate HTML output, making it perfect for developers and designers who want to focus on their skills rather than manual HTML editing. ### Key Features: - **Local AI Agents**: Utilizes 75 composable skill templates across 9 deliverable surfaces (magazine articles, keynote decks, résumés, posters, Xiaohongshu cards, tweet cards, web prototypes, data reports, Hyperframes videos). - **Zero API Key**: No need for API keys, as it detects 9 coding-agent CLIs on your PATH. - **Quickstart**: One-click export to WeChat, X, Zhihu, or download `.html` / `.png`. - **Live Preview**: Sandboxed preview of the HTML output. - **Export Targets**: Export to WeChat, X, Zhihu, or download `.html` / `.png`. ### Tech Stack: - **HTML**: The primary output format. - **CLI**: Uses 9 coding-agent CLIs to detect and utilize on your PATH. ### Why It Matters: HTML Anything simplifies the process of creating and publishing HTML documents, making it easier for developers and designers to focus on their skills rather than manual HTML editing. Its agentic approach ensures that the output is what the reader wants, and its zero API key requirement makes it accessible and efficient.

Overview: This project is a 2.78-trillion-parameter Kimi K3 model running inference on a single CPU with 8 GB of RAM, utilizing portable C99 without BLAS, frameworks, or GPUs. Key features: - Portable C99 implementation - No BLAS, no framework, no GPU - Runs inference on a single CPU with 8 GB of RAM - Supports Linux, macOS, and Windows platforms - Inference time is measured in seconds Tech stack: C99 Why it matters: This project showcases the potential of running large models on low-resource hardware, demonstrating the feasibility of inference on CPUs with minimal resources.

### Overview: RealSense SDK 2.0 is a cross-platform library designed for RealSense depth cameras. It enables depth and color streaming, providing intrinsic and extrinsic calibration information, making it a valuable tool for developers working with depth sensing applications. ### Key Features: - Depth and color streaming capabilities. - Intrinsic and extrinsic calibration information. - Cross-platform compatibility for various operating systems. ### Tech Stack: C++ ### Why it Matters: RealSense technology is widely used in robotics, 3D scanning, and depth sensing applications, making it an essential component for developers looking to integrate advanced depth sensing into their projects. ### Repository: realsenseai/librealsense ### GitHub Stars: 9057 ### GitHub Description: RealSense SDK 2.0 is a cross-platform library for RealSense depth cameras. The SDK allows depth and color streaming, and provides intrinsic and extrinsic calibration information.

### Overview: This repository is a comprehensive collection of the world's best resources for large language models, including multimodal generation, agent development, and more. It serves as an essential guide for developers looking to leverage these models effectively. ### Key features: - Multimodal generation - Agent development - Model training - Model inference - Evaluation metrics - Data processing tools - Model architectures ### Tech stack: Python, JavaScript, and various APIs. ### Why it matters: This repository provides a curated list of resources that developers can use to enhance their skills and projects, ensuring they stay up-to-date with the latest trends and best practices in the field of large language models.

### Overview: This project provides a comprehensive guide for deploying deep-learning inference networks and deep vision primitives on NVIDIA Jetson devices using TensorRT and PyTorch. It includes examples for various tasks such as image classification, object detection, semantic segmentation, pose estimation, and action recognition. ### Key Features: - **TensorRT Optimization**: Optimized networks for inference on Jetson devices. - **PyTorch Training**: Training models using PyTorch for transfer learning. - **DNN Primitives**: Support for imageNet, detectNet, segNet, poseNet, and actionNet. - **Realtime Applications**: Examples for streaming from live camera feeds, web apps with WebRTC, and ROS/ROS2 integration. - **Docker Container**: Easy setup using Docker for Jetson environments. - **System Requirements**: Recommended system specifications for optimal performance. ### Tech Stack: - **Language**: C++ - **TensorRT**: Optimized inference library for Jetson devices. - **PyTorch**: Framework for training deep learning models. ### Why It Matters: - **Efficient Inference**: Optimized TensorRT models run faster on Jetson devices. - **Versatile Applications**: Suitable for real-time applications, web development, and robotics. - **Community Support**: Comprehensive documentation and tutorials for Jetson users. ### Repository: - **GitHub Repository**: dusty-nv/jetson-inference - **GitHub Stars**: 90,160 - **Description**: A guide for deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson. ###

### Overview: Transformer Explainer is an interactive visualization tool designed to help anyone learn how Transformer-based models like GPT work. It runs a live GPT-2 model right in your browser, allowing you to experiment with your own text and observe in real time how internal components and operations of the Transformer work together to predict the next tokens. ### Key features: - Real-time interaction with a live GPT-2 model - Experiment with your own text to see how internal components and operations work - Visual explanations of Transformer architecture and operations ### Tech stack: JavaScript ### Why it matters: Transformer Explainer provides a hands-on, interactive way to understand how Transformer-based models like GPT work, making complex concepts accessible and engaging.

Overview: Speech recognition module for Python, supporting several engines and APIs, online and offline. Key features: - Supports CMU Sphinx, Google Speech Recognition, Google Cloud Speech API, Wit.ai, Microsoft Azure Speech, Houndify API, IBM Speech to Text, Snowboy Hotword Detection, Tensorflow, Vosk API, OpenAI whisper, OpenAI Transcription API, Groq Whisper API, and Cohere Transcribe API. Tech stack: Python Why it matters: This library provides a comprehensive solution for speech recognition, making it an essential tool for developers working in various applications such as meeting transcription, voice command recognition, and more.

Overview: 本书是深度学习领域的求职指南，涵盖了数学、机器学习、深度学习、计算机视觉、自然语言处理和SLAM等方向的内容。 Key features: - 求职攻略 - 自我介绍 - 数学 - 机器学习 - 深度学习 - 强化学习 - 计算机视觉 - 传统图像处理 - 自然语言处理 - SLAM - 推荐算法 - 数据结构与算法 - 编程语言：C/C++/Python - 深度学习框架 - 面试经验 - 面试技巧 - 其它（计算机网络/Linux等） Tech stack: Python Why it matters: 本书是深度学习领域的求职指南，涵盖了数学、机器学习、深度学习、计算机视觉、自然语言处理和SLAM等方向的内容，帮助读者准备面试并提升技能。 Repository: amusi/Deep-Learning-Interview-Book GitHub stars: 8943 GitHub description: 深度学习面试宝典（含数学、机器学习、深度学习、计算机视觉、自然语言处理和SLAM等方向） GitHub URL: https://github.com/amusi/Deep-Learning-Interview-Book

Overview: Rig is a Rust library designed for building scalable and modular LLM-powered applications, featuring agentic workflows, classic agent runtime, and full compatibility with GenAI Semantic Convention. Key features: - Agentic workflows for multi-turn streaming and prompting - Classic agent runtime enabled by default - Full compatibility with GenAI Semantic Convention - Support for 20+ model providers and 10+ vector store integrations - Integration with LLMs with minimal boilerplate - Browser-WASM support for the portable core and classic runtime Tech stack: Rust Why it matters: Rig simplifies the process of integrating LLM-powered applications into projects, providing a modular and scalable solution for developers looking to leverage cutting-edge AI technology.

### Overview: BentoML is a Python library designed to simplify the process of building and deploying online serving systems for AI applications and model inference. It allows developers to easily create APIs for any AI/ML model, manage dependencies and environments, optimize CPU/GPU utilization, and ensure production readiness. ### Key Features: - **Unified Model Serving Framework**: BentoML provides a single, unified framework for building and deploying models and inference APIs. - **Easy Model Integration**: Turn any model inference script into a REST API server with minimal code and standard Python type hints. - **Docker Containers**: Simplifies environment management and dependency setup with a simple config file, generating Docker images and ensuring reproducibility. - **Performance Optimization**: Leverages built-in serving optimization features like dynamic batching, model parallelism, multi-stage pipeline, and multi-model inference-graph orchestration to maximize CPU/GPU utilization. - **Customizability**: Supports any ML framework, modality, and inference runtime, allowing developers to implement their own APIs or task queues with custom logic. - **Production Ready**: Supports local development, running, and debugging, and seamless deployment to production using Docker containers or BentoCloud. ### Tech Stack: Python ### Why It Matters: BentoML simplifies the process of building and deploying AI applications and models, making it easier for developers to integrate and optimize their models for production use. Its unified framework, Docker support, and performance optimization features make it an essential tool for anyone working on AI/ML applications.