Projects

### Overview: Kubeflow Trainer is a Kubernetes-native distributed AI platform designed for scalable large language model (LLM) fine-tuning and training across various frameworks, including PyTorch, MLX, HuggingFace, DeepSpeed, Megatron-LM, JAX, XGBoost, and more. It leverages MPI to efficiently manage multi-node, multi-GPU distributed jobs on high-performance computing (HPC) clusters, ensuring ultra-fast synchronization between GPUs nodes for large-scale AI training. The platform seamlessly integrates with the Cloud Native AI ecosystem, including Kueue for topology-aware scheduling, Slurm Bridge for hybrid Kubernetes and Slurm cluster scheduling, and KAI Scheduler for GPU-aware scheduling, all while utilizing existing Kubernetes-native building blocks. ### Key Features: - **MPI Support**: Kubeflow Trainer seamlessly integrates MPI, enabling efficient multi-node, multi-GPU distributed jobs on HPC clusters. - **Distributed Data Cache**: Supports distributed data caching for improved data access efficiency. - **Topology-Aware Scheduling**: Utilizes Kueue for topology-aware scheduling, ensuring optimal resource utilization. - **Multi-Cluster Job Dispatching**: Integrates with Slurm Bridge for seamless job dispatching across hybrid Kubernetes and Slurm clusters. - **GPU-Aware Scheduling**: Uses KAI Scheduler for GPU-aware scheduling, optimizing GPU resource allocation. - **Enhanced Observability**: Propagates metrics to TrainJob status for better observability. - **Support for Multiple Frameworks**: Integrates with PyTorch, MLX, HuggingFace, DeepSpeed, Megatron-LM

Overview: Claude (or any LLM) can now actually watch videos, providing scene-aware, deduplicated frames and a transcript from a URL or local file. This feature allows for a more immersive and accurate understanding of video content. Key features: - Scene-aware video viewing - Deduplicated frames for improved performance - Timestamped transcript for context Tech stack: Python Why it matters: This feature enables LLMs to truly understand and interpret video content, enhancing their ability to provide accurate and contextually relevant information.

### Overview: Model2Vec is a technique that converts any sentence transformer into a fast, static embedding model. This reduces model size by a factor of up to 50 and makes models up to 500 times faster, with a minimal performance drop. Our best model, the potion-base-32M, is the most performant static embedding model in the world. Model2Vec is widely used for tasks such as text classification, retrieval, clustering, and building RAG systems. ### Key Features: - **Fast and Lightweight**: Model2Vec reduces model size by up to 50x and speeds up models by up to 500x. - **High Performance**: Our best model, the potion-base-32M, is the most performant static embedding model in the world. - **Versatile Applications**: Can be used for text classification, retrieval, clustering, and building RAG systems. - **Easy to Use**: A lightweight base package is available for quick start, and advanced usage is detailed in the inference and distillation docs. ### Tech Stack: - Python ### Why It Matters: Model2Vec simplifies the development and deployment of static embeddings, making it easier to integrate with existing systems and reducing the computational overhead required for large-scale applications.

### Overview: Lumina-T2X is a unified framework for Text to Any Modality Generation, enabling the transformation of text into various modalities such as images, audio, and videos. This innovative approach leverages flow-based large diffusion transformers to achieve high-quality results across different dimensions of content. ### Key Features: - **Unified Framework**: Lumina-T2X integrates Text to Image, Text to Audio, and Text to Video generation. - **Flow-Based Transformers**: Utilizes flow-based large diffusion transformers to efficiently handle complex transformations. - **Versatile Applications**: Suitable for various domains including art, entertainment, and education. ### Tech Stack: - Python ### Why It Matters: Lumina-T2X represents a significant advancement in text-to-modality generation, offering unparalleled versatility and performance across multiple modalities. Its ability to seamlessly transform text into images, audio, and videos makes it a valuable tool for creators and researchers alike.

### Overview: This repository is a collection of resources for in-context learning and prompt engineering, focusing on Mastery of Large Language Models such as ChatGPT, GPT-3, and FlanT5. The latest and most up-to-date papers are included, along with a playground for experimenting with large language models and a guide on how to quickly get started with these models using LangChain. ### Key features: - 📝 Papers: Latest papers on In-Context Learning, Prompt Engineering, Agent, and Foundation Models. - 🚀 Playground: Large language models that allow prompt experimentation. - 🛠 Prompt Engineering: Techniques for leveraging large language models. - 🌍 ChatGPT Prompt: Examples of prompts that can be applied in work and daily life. - 📖 LLMs Usage Guide: Method for quickly getting started with large language models using LangChain. ### Tech stack: Jupyter Notebook ### Why it matters: This is a fresh, daily-updated resources for in-context learning and prompt engineering. As Artificial General Intelligence (AGI) is approaching, it is crucial to enhance our abilities through the use of AIGC to position ourselves at the forefront of this exciting era and strive for personal and professional greatness.

Overview: 数千篇人工智能、语言模型、自然语言处理和计算机视觉领域的顶会论文解读，每篇5分钟内快速掌握核心思想。 Key features: - 23,000+篇顶会论文解读 - 覆盖ACL、CVPR、ICLR、AAAI、NeurIPS、ICCV、ICML、ECCV等会议 - 横跨LLM推理、VLM、Agent、RLHF、RAG、AIGC、Robotics、Auto Driving等40多个研究方向 - 持续更新中 Tech stack: Python Why it matters: 简化对人工智能领域最新进展的理解，帮助开发者快速掌握关键概念。

### Overview: FreeVideo is a software solution that allows users to create videos on their existing computers without needing a dedicated video editing workstation. It utilizes the Video DeltaNet (VDN) model and runs MiniMax H3 in minimal VRAM and RAM requirements, optimizing performance for various hardware configurations. ### Key Features: - **Hardware-Adaptive Execution**: Automatically adjusts the weight format and precision based on the available hardware, optimizing performance. - **Minimal VRAM and RAM Requirements**: Runs MiniMax H3 in as little as 8 GB of VRAM and 16 GB of RAM, making it suitable for a wide range of devices. - **Community Results**: Provides detailed community results and gallery showcasing various quality levels of videos. - **AI Agents and Editing**: Enables AI agents like Claude Code or Codex to generate videos, and allows users to edit videos on a timeline. ### Tech Stack: Python ### Why It Matters: FreeVideo simplifies video creation for users by providing a lightweight and efficient solution that can run on any computer, making it accessible to a broader audience. Its ability to adapt to different hardware configurations ensures optimal performance, making it a valuable tool for both professional and personal video creators.

### Overview: This repository contains a curated list of the latest breakthroughs in AI and data science, featuring clear video explanations, in-depth articles, and code (if applicable). The papers cover a wide range of topics, including ethical aspects, biases, and promising applications in the future. ### Key Features: - **Latest Breakthroughs**: Curated list of the most interesting research papers in AI and data science. - **Clear Video Explanations**: Each paper is accompanied by a video explanation. - **In-Depth Articles**: Links to more detailed articles for a deeper understanding. - **Code**: Some papers include code for reproducibility and further exploration. ### Tech Stack: - Python - PyTorch ### Why It Matters: This repository provides a comprehensive overview of the latest advancements in AI, making it a valuable resource for developers and researchers interested in staying updated with the field's most significant developments. ### Repository: [louisfb01/Best_AI_paper_2020](https://github.com/louisfb01/Best_AI_paper_2020) ### GitHub Stars: 2244 ### Subscribe: - [Newsletter](http://eepurl.com/huGLT5) - [Twitter](https://twitter.com/Whats_AI) - [LinkedIn](https://www.linkedin.com/in/whats-ai/) ### Watch Video: [Watch the video](https://imgur.com/xzZT1ll.png) ### Support: - Sponsor this repository and use W&B for reproducible ML experiments. ### Additional Repos

### Overview: This repository collects papers that integrate knowledge graphs (KGs) and large language models (LLMs). It aims to explore how LLMs can be used to enhance the performance of knowledge graphs and vice versa. ### Key Features: - **Surveys**: A comprehensive survey of papers that explore the integration of knowledge graphs and large language models. - **Methods**: Papers detailing the methodologies for integrating knowledge graphs and large language models. - **Resources and Benchmarking**: Resources and benchmarks for evaluating the performance of knowledge graphs and large language models. ### Tech Stack: Unknown ### Why it Matters: This repository serves as a valuable resource for researchers and practitioners interested in the intersection of knowledge graphs and large language models. It provides a platform to discover and evaluate papers that aim to improve the capabilities of both knowledge graphs and LLMs.

Overview: MambaVision is an official PyTorch implementation of a hybrid Mamba-Transformer vision backbone that demonstrates strong performance in terms of top-1 accuracy and throughput. It introduces a novel mixer block to enhance global context modeling, achieving new state-of-the-art (SOTA) results. Key features: - Hybrid Mamba-Transformer backbone - Novel mixer block for global context modeling - Hierarchical architecture combining self-attention and mixer blocks - Official PyTorch implementation - Achieves new SOTA performance in top-1 accuracy and throughput Tech stack: Official PyTorch implementation Why it matters: MambaVision showcases significant advancements in vision backbone architectures, particularly in handling global context effectively, leading to improved performance metrics in image classification and other vision tasks.

Overview: Datapizza AI is a Python framework designed to build reliable Gen AI solutions without the overhead of traditional AI frameworks. It simplifies the process of deploying AI agents by providing a fast and efficient solution. Key features: - Multi-Provider Support: Supports OpenAI, Google Gemini, Anthropic, Mistral, and Azure. - Tool Integration: Built-in web search, document processing, and custom tools. - Memory Management: Persistent conversations and context awareness. Tech stack: Python. Why it matters: Datapizza AI offers a no-fluff GenAI framework that streamlines the development process, making it easier to deploy AI agents quickly and reliably. Repository: datapizza-labs/datapizza-ai GitHub stars: 2243 GitHub description: Build reliable Gen AI solutions without overhead 🍕

### Overview: AmpliGraph is a Python library designed for representation learning on knowledge graphs, providing a suite of neural machine learning models for relational learning. It facilitates the prediction of links between concepts, enabling the discovery of new knowledge, completion of large knowledge graphs, generation of standalone knowledge graph embeddings, and development of new relational models. The library is built on top of TensorFlow 2 and is open-source, available on GitHub at <https://github.com/Accenture/AmpliGraph>. It is designed to be intuitive, GPU-ready, and extensible, making it a valuable tool for researchers and practitioners in the field of knowledge graph analytics. ### Key Features: - **Intuitive APIs**: Designed to reduce the code required for learning models that predict links in knowledge graphs, with a new version API in Keras style. - **GPU-Ready**: Based on TensorFlow 2, designed to run seamlessly on both CPU and GPU devices, enhancing training efficiency. - **Extensible**: Allows users to extend knowledge graph embeddings models by extending base estimators. - **Datasets**: Helper functions to load datasets (knowledge graphs). - **Models**: Includes models like TransE, DistMult, ComplEx, HolE, and more. - **Evaluation**: Metrics and evaluation protocols to assess the predictive power of the models. - **Discovery**: High-level convenience APIs for knowledge discovery, including discovering new facts, clustering entities, and predicting near duplicates. - **Compat**: Extends compatibility with AmpliGraph 1.x, making it easier for users familiar with the older version. ###

Overview: VectorAdmin is a tool suite designed to simplify the management of vector databases, including Pinecone, Chroma, Qdrant, and Weaviate. It provides a user-friendly interface for monitoring and managing vector data. Key features: - Unified interface for managing multiple vector databases - Real-time monitoring and alerting - Easy configuration and scaling - Support for various vector database providers Tech stack: TypeScript Why it matters: VectorAdmin aims to make vector database management more accessible and efficient, reducing the complexity and overhead associated with managing multiple vector databases.

Overview: LiveAgent is a fully functional AI Agent desktop client that supports WebUI access and can be creatively customized and expanded. Key features: - Multi-model access - Local tool execution - MCP & Skills ecosystem - Remote Gateway Tech stack: TypeScript Why it matters: LiveAgent is a local-first AI agent desktop client that integrates large language model reasoning with local system tools, enabling the AI to operate your file system, run commands, and manage scheduled tasks. The Gateway provides remote access and collaboration, making it a versatile tool for developers and professionals alike.

Overview: Raster Vision is an open-source Python library and framework designed for deep learning on satellite and aerial imagery, providing built-in support for chip classification, object detection, and semantic segmentation with PyTorch. It includes utilities for geospatial deep learning workflows and supports running experiments in the cloud using AWS Batch and Sagemaker. Key features: - Built-in support for chip classification, object detection, and semantic segmentation with PyTorch - Full suite of utilities for dealing with geospatial deep learning workflows - Low-code framework for quick and repeatable configuration of machine learning pipelines - Support for running experiments in the cloud using AWS Batch and Sagemaker Tech stack: Python Why it matters: Raster Vision simplifies the process of building and deploying machine learning models on satellite and aerial imagery, making it accessible for non-experts while providing powerful tools for advanced users. Repository: azavea/raster-vision GitHub stars: 2242 GitHub description: An open source library and framework for deep learning on satellite and aerial imagery. README excerpt:  &nbsp; [](https://pypi.org/project/rastervision/) [](https://docs.rastervision.io/en/stable/?badge=stable) [![License](https://img.shields.io/badge/L

### Overview: This repository contains an implementation of an aesthetic and technical image quality model based on Google's research paper "NIMA: Neural Image Assessment". The models are trained using transfer learning with pre-trained ImageNet CNNs fine-tuned for classification tasks. ### Key features: - The repository provides pre-trained models for aesthetic and technical image quality assessment. - It includes Docker images for local CPU training and remote GPU training on AWS EC2. - The models are based on MobileNet and achieve high performance on the AVA and TID2013 datasets. ### Tech stack: Python ### Why it matters: This model is useful for automatically assessing the aesthetic and technical quality of images, which can be applied in various fields such as hotel aesthetics, image classification, and more.

Overview: NLPer-Arsenal是一个全面的NLP竞赛策略库，涵盖多种竞赛策略、任务教程、经验贴、会议时间、算力推荐和竞赛平台信息，持续更新中。 Key features: - 收录NLP竞赛策略实现 - 提供NLP各任务教程和经验贴 - 记录往期竞赛总结和当前重点赛事 - 推荐自媒体、算力资源和竞赛平台 - 跟踪记录NLP会议时间 Tech stack: Python Why it matters: 为NLP开发者提供丰富的资源和策略，帮助他们快速上手和提升竞赛成绩。 Repository: TingFree/NLPer-Arsenal GitHub stars: 2239 GitHub description: 收录NLP竞赛策略实现、各任务教程、经验贴、学习资料以及会议时间等，持续更新中。

### Overview: `textacy` is a Python library designed for performing a variety of natural language processing tasks, built on top of the high-performance spaCy library. It focuses on tasks that come before and after processing with spaCy, providing access to its core functionality and additional features for text cleaning, structured information extraction, and more. ### Key features: - Access and extend spaCy's core functionality for working with one or many documents through convenient methods and custom extensions. - Load prepared datasets with both text content and metadata, including Congressional speeches, historical literature, and Reddit comments. - Clean, normalize, and explore raw text before processing it with spaCy. - Extract structured information from processed documents, including n-grams, entities, acronyms, keyterms, and SVO triples. - Compare strings and sequences using a variety of similarity metrics. - Tokenize and vectorize documents then train, interpret, and visualize topic models. - Compute text readability and lexical diversity statistics, including Flesch-Kincaid grade level, multilingual Flesch Reading Ease, and Type-Token Ratio. ### Tech stack: Python ### Why it matters: `textacy` simplifies the process of performing NLP tasks by leveraging spaCy's powerful core functionality, making it an essential tool for developers working with text data. Its focus on tasks preceding and following spaCy's processing allows for a more comprehensive understanding of text data, enhancing the accuracy and efficiency of NLP applications.

### Overview: envd is a command-line tool designed to streamline the creation of reproducible development environments for AI/ML projects. It simplifies the process of setting up environments with complex dependencies, making it easier for developers to focus on their code without worrying about the underlying infrastructure. ### Key Features: - **Simple CLI and Language**: Integrate powerful CLI tools directly into your Python workflow, eliminating the need for new languages or DSLs. - **Isolation and OCI Image Compatibility**: Create isolated development environments using OCI images, ensuring compatibility with Docker and Kubernetes. - **Hybrid Platform Support**: Supports environments on local machines, clusters, and cloud platforms, offering flexibility and efficiency. ### Tech Stack: Go ### Why It Matters: - **Ease of Setup**: Instantly get your environment running exactly as you need with just one command. - **Isolation and Compatibility**: Provides an isolated space for training, fine-tuning, or serving, compatible with OCI images. - **Hybrid Platform Support**: Offers a versatile way to create environments on local machines, clusters, and cloud platforms. - **Build Anywhere, Faster**: Offers a wealth of advantages, including remo ### Repository: - **Repository**: tensorchord/envd - **GitHub Stars**: 2240 - **GitHub Description**: 🏕️ Reproducible development environment for humans and agents - **Primary Language**: Go - **GitHub Description**: 🏕️ Reproducible development environment for humans and agents ### Contributors: - **All Contributors**: Track contributors on GitHub

Overview: This repository contains the training code for the paper "Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose," which optimizes the OpenPose approach to achieve real-time inference on CPU with negligible accuracy drop. The code detects a skeleton to identify human poses for every person inside the image, including up to 18 keypoints. Key features: - Real-time 2D Multi-Person Pose Estimation - High accuracy on COCO 2017 Keypoint Detection validation set - Achieves 40% AP for single-scale inference without flip or post-processing Tech stack: Python Why it matters: This project is significant for developers looking for fast and accurate human pose estimation, particularly in real-time applications. GitHub stars: 2236 Repository: Daniil-Osokin/lightweight-human-pose-estimation.pytorch