Projects

### Overview: This repository contains a fast deep learning model designed to upscale low-resolution videos to high resolution at 30 frames per second. The model utilizes the SR-GAN architecture for speed optimization, achieving real-time performance on a MacBook M1 Pro GPU. ### Key Features: - Real-time video super-resolution - Upsampling from 90x160 to 360x640 (360p) in 0.01 seconds - 720p output at approximately 30 frames per second ### Tech Stack: - Python ### Why It Matters: This model enables real-time super-resolution for video upscaling, making it highly valuable for applications requiring fast and efficient video processing, such as live streaming, video editing, and content creation. ### Repository: HasnainRaz/Fast-SRGAN - GitHub Stars: 695 - Primary Language: Python - Description: A Fast Deep Learning Model to Upsample Low Resolution Videos to High Resolution at 30fps - Requirements: - Python 3.10 - Pipenv for package management - Pre-trained Model: - Generator Model on the DIV2K Dataset - 8 Residual Blocks with 64 Filters - Benchmarks: - 0.01 seconds for 90x160 to 360x640 (360p) - 0.04 seconds for 180x320 to 720

### Overview: expo-speech-recognition is a React Native library that provides Speech Recognition functionality for Expo projects, including iOS, Android, and the Web. It aims to simplify the implementation of speech recognition across different platforms by leveraging the native Speech Recognition APIs. ### Key Features: - **iOS**: Implements `SFSpeechRecognizer`. - **Android**: Implements `SpeechRecognizer`. - **Web**: Implements `SpeechRecognition`. ### Tech Stack: TypeScript. ### Why it Matters: This library simplifies the implementation of speech recognition in React Native projects, allowing developers to reuse code across web and mobile platforms without the need for multiple implementations. ### GitHub Stars: 697 ### Repository: jamsch/expo-speech-recognition

### Overview: Datumaro is a comprehensive dataset management framework and command-line interface (CLI) tool designed for building, transforming, and analyzing Computer Vision datasets. It simplifies the process of handling various dataset formats and provides features for merging, filtering, converting, and splitting datasets. ### Key Features: - **Dataset Reading, Writing, and Conversion**: Supports multiple dataset formats, including CIFAR-10/100, Cityscapes, COCO, CVAT, ImageNet, Kitti, LabelMe, LFW, MNIST, Open Images, PASCAL VOC, and TF Detection API. It allows seamless reading, writing, and conversion between these formats. - **Annotation Tool**: Provides an annotation tool for CVAT, enabling users to manage and annotate datasets efficiently. - **Dataset Building and Splitting**: Facilitates the merging of multiple datasets into a single unified dataset, filtering based on custom criteria such as removing polygons, images without annotations, occluded annotations, and small bounding boxes. It also supports splitting datasets into training, validation, and testing subsets for various tasks like classification, detection, and re-identification. ### Tech Stack: - **Python**: The primary programming language for building and using Datumaro. ### Why It Matters: Datumaro simplifies the process of managing and analyzing datasets, making it easier for developers and researchers to handle complex datasets in various formats. Its comprehensive features and robust tooling make it an essential tool for anyone involved in Computer Vision projects, from beginners to advanced practitioners.

Overview: TidyBot is a project that aims to personalize physical assistance for household robots using large language models (LLMs). The team, led by Jimmy Wu, explores how robots can learn user preferences based on a few interactions to adapt to different users' preferences. Key features: - Personalized object placement for cleaning tasks - Integration of language-based planning and perception - Use of few-shot summarization capabilities of LLMs - 91.2% accuracy on unseen objects in a benchmark dataset - Real-world application demonstrated with TidyBot Tech stack: Python Why it matters: This project leverages LLMs to enable robots to adapt to individual user preferences, enhancing the personalization and effectiveness of household cleaning assistance.

Overview: Clover is an open-source ROS-based framework designed to simplify the control of PX4-powered drones, making it accessible for educational and research purposes. Key features: - ROS-based framework for PX4 autopilots - Preconfigured image for Raspberry Pi, ready to fly - Educational programmable drone kit with Pixracer autopilot and other sensors - Documentation available at [https://clovercoex.tech](https://clovercoex.tech/) - Video compilation showcasing the drone's use in various educational and competition events Tech stack: C++, ROS, PX4 Why it matters: Clover simplifies the control and development of PX4-powered drones, making them accessible for educational and research purposes, while providing a user-friendly interface for autonomous flight control. Repository: CopterExpress/clover GitHub stars: 692 GitHub description: ROS-based framework and RPi image to control PX4-powered drones 🍀

### Overview: MOSS-Audio is an open-source audio understanding model developed by the OpenMOSS team, MOSI.AI, and Shanghai Innovation Institute. It is designed to unify speech, environmental sound, and music understanding, as well as time-aware question answering and complex reasoning. The model supports four variants: MOSS-Audio-4B-Instruct, MOSS-Audio-4B-Thinking, MOSS-Audio-8B-Instruct, and MOSS-Audio-8B-Thinking. These variants are optimized for different use cases, with the Instruct variants for direct instruction following and the Thinking variants for stronger reasoning capabilities. ### Key features: - **Unified Modeling**: MOSS-Audio models complex real-world audio, including speech, environmental sounds, and music. - **Speech Understanding**: Accurately transcribes spoken content from audio inputs, supporting both word-level and sentence-level timestamp alignment. - **Environmental Sound Understanding**: Recognizes and interprets environmental sounds. - **Music Understanding**: Captures and interprets musical elements. - **Time-Aware Question Answering**: Supports time-aware question answering to handle temporal context. - **Complex Reasoning**: Performs complex multi-step reasoning over audio inputs. ### Tech stack: - Python ### Why it matters: MOSS-Audio is a powerful tool for anyone looking to understand and analyze audio data, enabling applications in speech recognition, environmental monitoring, and music analysis. ### Repository: - Repository: OpenMOSS/MOSS-Audio - GitHub stars: 692 ### Contents: - **Introduction**: Overview

Overview: Arcjet is a runtime security platform designed to protect AI applications and agents, ensuring prompt injection detection, tool-call authorization, sensitive data redaction, bot protection, and rate limiting. Key features: - Prompt injection detection - Tool-call authorization - Sensitive data redaction - Bot protection - Rate limiting Tech stack: TypeScript Why it matters: Arcjet secures AI applications and agents by enforcing real-time security measures, ensuring prompt injection detection, tool-call authorization, sensitive data redaction, bot protection, and rate limiting, making your application more secure and reliable. Repository: arcjet/arcjet-js GitHub stars: 693

### Overview: Minke is a local-first desktop agent workspace powered by DeepSeek Harness, available on macOS, Windows, and Linux. It allows you to keep conversations, files, and terminals side by side, and access your workspace through Web, PWA, or Discord. ### Key features: - **Shared browser control**: Watch the agent work in visible browser tabs, take over at any time, and send annotated page context back to the conversation. - **One workspace**: Keep conversations, files, and terminals side by side. Organize work with split views and a bottom panel; edit files, preview documents, and review changes in place. - **Access across devices**: Reach your workspace through Web / PWA on a phone or another computer, or start tasks through WeChat, Telegram, and Discord. - **Cloud and local models**: Use cloud providers or custom endpoints, discover LM Studio and Ollama models, and optionally start local services. - **Agents and plugins**: Use DSH's presets, skills, subagents, and scheduled tasks, and extend the workspace through native plugin management. ### Tech stack: TypeScript ### Why it matters: Minke is an independent community project under active development. Packaged releases may differ from the current source.

### Overview: This repository offers two Android apps that utilize the OpenAI Whisper speech-to-text model. One app uses the TensorFlow Lite Java API for easy integration with Java developers, while the other employs the TensorFlow Lite Native API for enhanced performance. The apps include a Python script for generating TFLite models and pre-built APKs for straightforward deployment. ### Key Features: - **Whisper Java App**: An Android app using the TensorFlow Lite Java API for model inference with Whisper, suitable for Java developers integrating TensorFlow Lite. - **Whisper Native App**: An Android app utilizing the TensorFlow Lite Native API for model inference, offering enhanced performance for developers preferring native code. - **Models and Scripts**: Contains a Python script to convert Whisper models into TensorFlow Lite format and includes pre-generated TFLite models. - `generate_model.py`: Script for generating TFLite models. - `generated_model`: Directory with optimized TFLite models. ### Tech Stack: C++ ### Why it Matters: This project is valuable for developers looking to integrate speech recognition capabilities into their Android applications. The use of TensorFlow Lite APIs ensures compatibility with existing Java-based projects and provides a straightforward way to deploy the app. The inclusion of pre-built APKs and Python scripts makes it easy for developers to integrate and deploy the app without needing to write from scratch.

### Overview: SkillCorpus is an open-source infrastructure that converts scattered `SKILL.md` files into curated, retrieval-ready agent-skill corpora. It includes retrieval and evaluation tooling, making it ideal for developers looking to enhance their agent capabilities. ### Key Features: - **Scattered Files to Corpus**: Converts multiple `SKILL.md` files into a cohesive corpus. - **Quality Evaluation**: Ensures the corpus is of high quality through evaluation. - **Retrieval Tooling**: Facilitates efficient retrieval of relevant skills for agents. - **Customization**: Allows for modifying taxonomy, rules, and evaluation suites to suit specific needs. - **Self-Hosted Deployment**: Enables developers to host the retrieval models and connect their own agent hosts. ### Tech Stack: Python ### Why It Matters: SkillCorpus simplifies the process of creating and managing agent skills, making it easier for developers to integrate reliable knowledge into their agents. The open-source nature of the project also allows for community contributions and continuous improvement, ensuring that the technology remains up-to-date and effective.

Overview: Magpie is a deep learning framework designed for multi-label text classification, developed at CERN to categorize High Energy Physics abstracts and extract keywords from them. Key features: - Multi-label text classification - Word vector representation learning - Training on large labeled corpora - Predictive capabilities on unseen data - Support for both CPU and GPU training Tech stack: Python Why it matters: Magpie simplifies the process of training and deploying deep learning models for text classification tasks, making it accessible for researchers and developers in the field of high-energy physics and beyond. Repository: inspirehep/magpie Primary language: Python GitHub stars: 690 GitHub description: Deep neural network framework for multi-label text classification

### Overview: MPP-Qwen-Next 是一个基于 Qwen-LM 的多模态管道并行模型，支持图像、视频和多图对话的多轮问答功能。通过使用 RTX 3090/4090 显卡进行并行训练，该模型能够显著提升推理速度和处理能力。 ### Key features: - **图像-单轮问答**: 通过图像处理实现单轮问答功能。 - **图像-多轮对话**: 支持多轮对话，通过图像处理增强对话效果。 - **视频-对话**: 通过视频处理实现多轮对话功能。 - **多图-对话（未经过多图sft，视频sft后涌现该能力）**: 通过多图和视频处理增强对话能力。 ### Tech stack: - **Jupyter Notebook**: 用于开发和测试。 ### Why it matters: MPP-Qwen-Next 是一个强大的多模态模型，能够处理图像、视频和多图对话，极大地扩展了模型的对话能力和应用场景。通过并行训练和高效的模型并行处理，该模型在推理性能和处理能力方面表现出色，适用于各种需要多模态交互的场景。

Overview: 开源孪创是一个用于研究人和载具的影视级物理模拟器，旨在加速人和机器人的算法开发、训练和验证。 Key features: - 人机仿真 - 地面载具 - 空域载具 - 水域载具 - 开放数字资产 - VR驾驶舱 - 数据收集文档 - 空地一体 - 水下机器人 - 传感器 - 数据合成 - 交通管理器 - 多物理场仿真 - 行人导航 - Python接口 - 文档 Tech stack: C++ Why it matters: 开源孪创提供了一个全面的平台，用于研究人和载具的影视级物理模拟，加速算法开发和验证，同时提供了丰富的开放数字资产和标准规范，支持多种应用场景和高级功能。

Overview: WeCron 是一款微信上的定时提醒工具，支持多种定时任务管理功能，帮助用户实现自动化任务调度。 Key features: - 实现微信上的定时提醒功能 - 支持多种定时任务管理 - 本地运行和部署选项 Tech stack: Python, PostgreSQL, WeChat API Why it matters: WeCron 提供了一种简便的方式来管理微信上的定时任务，帮助用户实现自动化任务调度，提高工作效率。

Overview: Fabric for Deep Learning (FfDL) is a Deep Learning Platform offering TensorFlow, Caffe, PyTorch etc. as a Service on Kubernetes. Key features: - Training of Deep Learning models on distributed hardware - Open Deep Learning APIs - Running Deep Learning hosting in user's private or public cloud Tech stack: Go Why it matters: FfDL provides a unified platform for training and deploying deep learning models across various frameworks, making it accessible and efficient for both research and production environments. Repository: IBM/FfDL Primary language: Go GitHub stars: 690 GitHub description: Fabric for Deep Learning (FfDL, pronounced fiddle) is a Deep Learning Platform offering TensorFlow, Caffe, PyTorch etc. as a Service on Kubernetes README excerpt: *Read this in other languages: [中文](README-cn.md).* [](https://travis-ci.org/IBM/FfDL) # Fabric for Deep Learning (FfDL) This repository contains the core services of the *FfDL* (Fabric for Deep Learning) platform. FfDL is an operating system "fabric" for Deep Learning. It is a collaboration platform for: - Framework-independent training of Deep Learning models on distributed hardware - Open Deep Learning APIs - Running Deep Learning hosting in user's private or public cloud  To know more about the architectural details

### Overview: This repository provides a MuJoCo/Gym environment designed for robot control using Reinforcement Learning. The primary task involves predicting grasp success chances based on pixel observations. ### Key Features: - **GraspEnv**: A Gym environment specifically designed for training reinforcement learning agents to predict grasp success through visual observations. - **MJ_Controller**: A standalone class for basic robot control in MuJoCo, useful for testing and model evaluation. - **Robot Configuration**: Based on the Universal Robots UR5 and Robotiq S Model 3 Finger Gripper, with modifications to the XML file and Python bindings from `mujoco_py` and `simple_pid`. ### Tech Stack: Python ### Why It Matters: This environment is particularly useful for researchers and developers looking to apply Reinforcement Learning to robotic pick and place tasks, focusing on visual observation-based predictions.

Overview: This project allows you to chat with multiple PDFs locally, utilizing models from Huggingface and Ollama. It supports both local setup and Kaggle integration, making it accessible and versatile. Key features: - Easy to run on `Local` or `Kaggle` - Using any model from `Huggingface` and `Ollama` - Process multiple PDF inputs - Chat with multiple languages (Coming soon) - Simple UI with `Gradio` Tech stack: Python Why it matters: This project simplifies the process of integrating PDFs with chatbots, making it easier to access and utilize PDF data in conversational AI applications.

Overview: This project is designed to build LLM-powered Dart/Flutter applications, providing developers with a powerful toolset for integrating language models into their projects. Key features: - Integration with LangChain for seamless LLM functionality - Support for Dart and Flutter development environments - Easy setup and deployment options Tech stack: Dart Why it matters: This project offers a straightforward and efficient way for developers to incorporate language models into their Dart/Flutter applications, making it a valuable resource for anyone working on AI-driven projects.

Overview: Timber is a powerful tool that compiles classical machine learning models into native C99 code, significantly speeding up inference by up to 336x compared to Python. It supports XGBoost, LightGBM, scikit-learn, CatBoost, ONNX, and URDF robot descriptions. Key features: - Optimizes models for faster inference - Supports multiple ML model formats - Ships in a single `pip install` - Hardware-accelerated and safety-certified backend Tech stack: Python Why it matters: Timber simplifies the deployment of ML models, making them run efficiently and securely on various hardware and platforms, including embedded systems and ROS 2. Repository: kossisoroyce/timber

Overview: This repository implements the Pluralistic Image Completion method, which generates multiple and diverse plausible results for masked images. The model is trained using PyTorch and CUDA, and the results are shown on various datasets including CelebA, Paris, and Places2. Key features: - Generates multiple and diverse plausible results - Supports training with irregular and irregular holes - Compatible with PyTorch 1.7.0, CUDA 11.2, Python 3.6, and Ubuntu 20.04 - Provides visualization tools via visdom and dominate Tech stack: Python Why it matters: This method offers a powerful tool for image completion tasks, particularly useful in scenarios where partial images are available, such as in medical imaging or surveillance applications.