Projects

Overview: RESTai is an AIaaS platform that provides a RESTful API for creating and consuming AI projects. It supports various LLMs and offers features like precise embeddings, prompt versioning, and live chat deployment. Key features: - RESTful API for project creation and management - Support for multiple LLMs (OpenAI, Anthropic, Ollama, etc.) - Built-in chat playground for testing projects - Detailed analytics and performance tracking Tech stack: Python Why it matters: RESTai simplifies the creation and management of AI projects, making it accessible for developers and businesses alike. Its comprehensive features and built-in tools make it a versatile solution for AIaaS solutions.

Overview: FLAVR is a fast and efficient frame interpolation technique that excels in single-shot multi-frame prediction. It uses a customized encoder-decoder architecture with spatio-temporal convolutions and channel gating to generate realistic high frame rate videos. Key features: - Fast inference times - Capable of single-shot multi-frame prediction - Uses spatio-temporal convolutions and channel gating for complex motion trajectory interpolation Tech stack: Python Why it matters: FLAVR offers a better trade-off between speed and accuracy compared to prior frame interpolation methods, making it suitable for real-time applications.

Overview: OmniNet is an official PyTorch implementation of "OmniNet: A unified architecture for multi-modal multi-task learning," authored by Subhojeet Pramanik, Priyanka Agrawal, and Aman Hussain. This repository contains the pre-trained models and scripts for training and evaluating the architecture on various tasks such as part-of-speech tagging, image captioning, visual question answering, and video activity recognition. Key features: - Pre-trained models for multiple tasks - PyTorch implementation - Supports multiple GPUs for parallel training - Compatible with various datasets Tech stack: Python Why it matters: OmniNet offers a unified architecture that can handle multiple modalities and tasks, making it highly versatile for applications requiring multi-modal multi-task learning. The pre-trained models provided by this repository enable researchers and practitioners to quickly reproduce the results demonstrated in the paper, facilitating further research and development in the field of multi-modal learning.

### Overview: Bytedesk is an open-source IM (Instant Messaging) platform designed for omnichannel customer service. It integrates AI-powered live-chat, email, ticket support, and omni-channel customer service, offering an alternative to popular tools like Slack, Zendesk, Intercom, HubSpot, Ada, Decagon, and Sierra. ### Key Features: - **AI-Powered Live-Chat**: Enhances customer interactions with AI-driven support. - **Email Integration**: Seamless integration for automated responses. - **Ticket Support**: Detailed routing strategies and performance metrics. - **Omni-Channel Support**: Multiple communication channels for better service. - **Knowledge Base**: Internal documentation and help center features. - **Agent Chat**: Facilitates communication with AI agents and knowledge bases. - **Workflow Management**: Streamlined ticket processes and form handling. - **Voice of Customer**: Feedback and survey mechanisms for continuous improvement. ### Tech Stack: - **JavaScript** - The primary programming language used for development. ### Why It Matters: Bytedesk is a comprehensive solution for businesses looking to enhance their customer service capabilities. It provides a robust platform with AI-powered support, multiple communication channels, and detailed management tools, making it an attractive alternative to established customer service tools. ### Repository: The project is hosted on GitHub at [Bytedesk/bytedesk](https://github.com/Bytedesk/bytedesk). ### GitHub Stars: The repository has been starred by 513 developers, indicating its popularity and usefulness in the community. ### Contact: For more information and support, you can reach

### Overview: DAD-3DHeads is an official repository for the paper "DAD-3DHeads: A Large-scale Dense, Accurate and Diverse Dataset for 3D Head Alignment from a Single Image" by Tetiana Martyniuk, Orest Kupyn, Yana Kurlyak, Igor Krashenyi, Jiři Matas, Viktoriia Sharmanska. The dataset is available on Hugging Face and is used for benchmarking 3D head alignment from a single image. It includes a diverse set of images and annotations for training and evaluation. ### Key features: - **Diverse Dataset**: Contains a large-scale, dense, and accurate dataset for 3D head alignment. - **Annotations**: Detailed annotations for 2D and 3D face landmarks, including model view matrix, projection matrix, and various attributes like gender, expression, age, occlusions, and pose. - **Training Code**: Utilizes Hydra for configuration management, making it easy to change training setups. - **Visualization Tools**: Provides tools to visualize ground-truth labels, including 2D and 3D face landmarks, head meshes, and pose visualizations. ### Tech stack: - **Python 3.8**: The primary programming language used for the project. ### Why it matters: The DAD-3DHeads dataset and repository are essential for researchers and practitioners in the field of 3D head alignment, providing a comprehensive resource for benchmarking and evaluating algorithms. The diverse and accurate annotations make it a valuable

### Overview: KVarN is a native vLLM KV-cache quantization backend designed for high-performance, low-accuracy workloads. It achieves 3-5x more KV-cache capacity and up to 1.3x throughput of FP16 while maintaining FP16-level accuracy, making it ideal for applications requiring long context and concurrent requests. ### Key features: - **3-5x More KV-Cache Capacity:** KVarN significantly increases the KV-cache capacity of models, allowing for longer context and more concurrent requests. - **Up to 1.3x Throughput:** It boosts the throughput of models by up to 1.3x, making it suitable for applications that require high throughput. - **FP16-Level Accuracy:** KVarN maintains FP16-level accuracy, ensuring that the model's performance is not compromised. - **Calibration-Free, One-Flag Integration:** KVarN is a native vLLM attention backend that requires only one flag to be added, making it easy to integrate into existing systems. ### Tech stack: KVarN is built on top of vLLM, leveraging its precompiled wheels for installation. It is designed to run in `float16` compute, utilizing KVarN's native KV-cache quantization backend. ### Why it matters: KVarN is specifically designed to address the limitations of traditional KV-cache quantization methods. By delivering FP16-level accuracy while maintaining throughput, KVarN allows developers to achieve the best of both worlds—longer context and higher throughput—without sacrificing accuracy

Overview: This repository is a curated list of papers and datasets related to Instruction Tuning and Following, focusing on the latest research in the field. It aims to provide a comprehensive overview of the current state of the art in instruction tuning and following, including both papers and datasets. Key features: - A wide range of papers and datasets on Instruction Tuning and Following - Papers and datasets sorted by year in descending order - Papers and datasets are sorted by relevance and importance Tech stack: Python Why it matters: This repository serves as a valuable resource for researchers, developers, and practitioners interested in instruction tuning and following. It provides a clear and organized way to access the latest research in the field, helping to accelerate progress and innovation. Repository: RenzeLou/awesome-instruction-learning GitHub stars: 512 Primary language: Python GitHub description: Papers and Datasets on Instruction Tuning and Following. ✨✨✨ GitHub description: Papers and Datasets on Instruction Tuning and Following. ✨✨✨

Overview: This project demonstrates how to perform object detection using the YOLO algorithm with OpenCV and Python. Key features: - YOLO (You Only Look Once) Object Detection - Utilizes OpenCV's `dnn` module for inference - Supports pre-trained weights from popular frameworks like Caffe, Torch, and TensorFlow Tech stack: Python, OpenCV, numpy Why it matters: This project showcases the integration of deep learning techniques with traditional computer vision tools, providing a practical example of how to implement object detection using modern deep learning frameworks. Repository: arunponnusamy/object-detection-opencv Primary language: Python GitHub stars: 512

Overview: PokerKit is an open-source Python library designed for simulating games, evaluating hands, and facilitating statistical analysis in poker. It supports a wide range of poker variants and provides a flexible architecture for custom game definitions. Key features: - Extensive poker game logic for major and minor poker variants. - High-speed hand evaluations. - Customizable game states and parameters. - Robust implementation with static type checking and extensive unit tests and doctests. Tech stack: Python Why it matters: PokerKit offers a reliable solution for poker game simulations, hand evaluations, and statistical analysis, making it valuable for developers interested in poker AI, tool creation, and online poker casino implementation. Its static type checking, extensive unit tests, and doctests ensure high reliability and accuracy.

Overview: This repository is a reference implementation of the Deep Alignment Network, a method for robust face alignment, as described in the paper "Deep Alignment Network: A convolutional neural network for robust face alignment" presented at CVPR 2017. Key features: - A Python implementation of the Deep Alignment Network - Pre-trained models available for download - Scripts for face tracking, model preparation, and experiment evaluation - TensorFlow implementations by other contributors Tech stack: Python Why it matters: This project provides a practical tool for face alignment, which is crucial in various applications such as face recognition, facial expression analysis, and more. Repository: MarekKowalski/DeepAlignmentNetwork GitHub stars: 511

Overview: This repository contains recipes for the Prodigy annotation tool, a scriptable annotation tool for text, images, and other data. Key features: - Named Entity Recognition (NER) - Fuzzy Named Entity Recognition (NER) - Manual Annotation - Customizable Recipes Tech stack: Jupyter Notebook Why it matters: These recipes provide a way to customize and extend Prodigy's functionality, allowing for more efficient and effective annotation workflows.

Overview: Robust-R1 is an official implementation of a model designed to enhance the robustness of visual understanding in large language models, addressing the limitations of limited interpretability and isolated optimization in current methods. Key features: - Robust-R1 is a model that improves the robustness of large language models. - It includes a pretrained and fine-tuned model for training and fine-tuning on a specific dataset. - The model is trained using supervised fine-tuning on the Robust-R1 dataset. Tech stack: Python Why it matters: Robust-R1 addresses the limitations of current methods by providing a more interpretable and robust solution for large language models, enhancing their ability to handle degraded visual inputs.

### Overview: XPretrain is a repository that includes recent research works in multi-modality learning, particularly focusing on pre-training methods from the Multimedia Search and Mining group of Microsoft Research. ### Key Features: - HD-VILA (CVPR 2022): High-resolution and diversified video-language pre-training model. - LF-VILA (NeurIPS 2022): Long-form video-language pre-training model. - CLIP-ViP (ICLR 2023): Adapted image-language pre-training to video-language pre-training model. - Pixel-BERT: End-to-end image and language pre-training model. - SOHO (CVPR 2021 oral): Improved end-to-end image and language pre-training model with quantized visual tokens. - VisualParsing (NeurIPS 2021): Transformer-based end-to-end image and language pre-training model. - HD-VILA-100M dataset: High-resolution and diversified video-language dataset. ### Tech Stack: Python ### Why It Matters: XPretrain serves as a valuable resource for researchers and practitioners interested in multi-modality learning and pre-training techniques, offering a comprehensive set of pre-trained models and datasets that can be used for various applications in computer vision and natural language processing.

### Overview: Willow Inference Server (WIS) is an open-source, self-hosted language inference server designed to optimize and accelerate the deployment of state-of-the-art models for speech and language tasks. It supports WebRTC, REST, and WebSocket transports, and is optimized for low-end GPUs like the Tesla P4 and GTX 1060, ensuring high performance and cost-effectiveness. ### Key Features: - **CUDA Optimized**: Supports CUDA GPUs for high performance, including the Tesla P4 and GTX 1060. - **Memory Optimization**: All Whisper models loaded simultaneously, including large-v2, medium, and base, fit within 6GB VRAM. - **ASR (Speech Recognition)**: Whisper optimized for real-time speech recognition, with minimal latency. - **TTS (Text-to-Speech)**: Customizable TTS voices for visually impaired users. - **WebRTC Integration**: Stream audio in real-time, optimizing for long-running sessions. - **REST, WebRTC, WebSocket Support**: Compatible with various transports for seamless integration. - **Customizable Whisper Models**: Supports model size, beam, and language detection/translation parameters. - **WebRTC Audio Track Management**: Optimized for high-quality audio streaming. - **Custom TTS Voices**: Create and manage custom TTS voices using small audio recordings. ### Tech Stack: Python ### Why it Matters: Willow Inference Server aims to democratize advanced language processing by enabling self-hosted inference capabilities, making it accessible and cost-effective for a wide range of applications. Its focus on low-end

Overview: FantasyPortrait is a project that enhances multi-character portrait animation using expression-augmented diffusion transformers. It leverages a multi-portrait facial expression video dataset to improve the realism and expressiveness of portrait animations. Key features: - Expression-augmented diffusion transformers for enhanced portrait animation - Multi-expr dataset for facial expression videos - Inference capabilities for both single and multi-portrait animations Tech stack: Python Why it matters: This project showcases advancements in facial expression augmentation and animation techniques, potentially revolutionizing the field of portrait animation and digital media production.

### Overview: DiffMorpher is an official code for image morphing using diffusion models, presented at CVPR 2024. It leverages the capabilities of diffusion models to create smooth transitions between images, offering a powerful tool for image editing and manipulation. ### Key features: - **Image Morphing**: DiffMorpher allows users to blend images seamlessly, creating dynamic transitions between two input images. - **Diffusion Models**: Utilizes diffusion models to generate realistic intermediate frames, ensuring a smooth and natural morphing effect. - **CUDA Support**: The code is designed to run efficiently on NVIDIA GPUs, enhancing performance and speed. ### Tech stack: Python ### Why it matters: DiffMorpher provides a practical solution for image editing tasks, making it a valuable tool for artists, designers, and anyone looking to enhance their digital creations. Its ability to create smooth transitions between images is particularly useful in fields such as art, fashion, and graphic design.

Overview: Paper-Agent 是一个面向科研人员和学生的智能论文检索与调研工具，它通过多智能体协作架构（LangGraph）帮助用户高效查找学术论文、分析文献内容，并进行论文调研。项目支持多平台集成、关键词搜索、自动分析、论文调研等功能，旨在提升学术研究的效率。 Key features: - 多来源论文检索 - 从摘要到全文的渐进式阅读 - 分层研究分析 - 证据约束下的综述写作 - 实时会话工作台 - 可视化模型配置 Tech stack: Python 3.12+, FastAPI, LangGraph, Vue 3, SQLite, Markdown, ChromaDB, SSE, OpenAI, Anthropic Why it matters: Paper-Agent 是学术调研的理想助手，适用于论文写作、学术调研、科研项目管理等多种场景，帮助用户高效完成从检索到写作的全流程任务。

Overview: 赛博医生项目——“赛博华佗”，基于多模态大模型的多功能智能体，旨在帮助用户搭建本地多模态大模型，进行疾病初诊、病历分析、专业知识问答等功能，成为他们的私人医生。该项目通过整合多模态大模型和医疗健康相关的知识图谱与知识库，实现医疗资源的跨地域传播，提升更多人的健康水平。 Key features: - 一键搭建本地多模态大模型 - 疾病初诊功能 - 病历分析功能 - 专业知识问答功能 - 多轮对话功能 - 语音输入和输出功能 - 知识图谱增强功能 - 知识库增强功能 - 联网检索增强功能 Tech stack: Python Why it matters: 赛博华佗项目通过整合多模态大模型和医疗健康相关的知识图谱与知识库，为用户提供了一站式的医疗健康解决方案，帮助他们更好地管理个人健康，提供基础的医疗建议，同时通过微调的大模型和RAG技术，将大模型的知识库扩展到任何领域，成为任意专业的专家。 Repository: ZeroTang05/cyber-doctor GitHub stars: 509 GitHub description: 赛博医生项目——“赛博华佗”，基于多模态大模型的多功能智能体，一键搭建本地多模态大模型。接入医疗健康相关的知识图谱和知识库后可以进行疾病初

### Overview: EDSL is a powerful tool for conducting computational social science and market research using AI. It allows you to design and run surveys and experiments with large numbers of AI agents and large language models, providing easy-to-analyze results and a universal remote cache for reproducibility. ### Key features: - Declarative Design: Specified question types ensure consistent results without requiring a JSON schema. - Large-scale AI Agent Simulations: Ideal for social science and market research. - Universal Remote Cache: Reproducible results with built-in analysis methods. - Secure Key Management: Access to all available models with API keys. - Easy Setup: Install with `pip install edsl` and follow the installation guide. - Coop Platform: Share workflows and survey results on a free platform for AI research. - Documentation: Comprehensive documentation available on PyPI and the official website. ### Tech stack: Python ### Why it matters: EDSL simplifies the process of conducting complex AI-based research, ensuring consistent and reproducible results. The universal remote cache and secure key management features make it accessible and scalable for research teams.

### Overview: EVF-SAM is an advanced text-prompted segment anything model designed for efficient inference and zero-shot video text-prompted capabilities. ### Key features: - **Text-Prompted Segmentation**: EVF-SAM extends SAM's capabilities by incorporating text prompts for precise segmentation, achieving high accuracy in Referring Expression Segmentation. - **Efficient Computation**: Designed for rapid inference, EVF-SAM can process images in few seconds on a T4 GPU. - **Zero-Shot Video Text-Prompted**: The model supports zero-shot video text-prompted capabilities, enabling it to perform well on video prediction using SAM-2. ### Tech stack: Python ### Why it matters: EVF-SAM offers enhanced text-prompted segmentation capabilities, making it highly effective for applications requiring precise and efficient image processing, particularly in scenarios involving text-based prompts.