Projects

Overview: ScribeCrate is an open-source, self-hosted speech-to-text API that provides OpenAI-compatible transcription and translation capabilities using Whisper models. It supports private processing, speaker diarization, and streaming results, with support for various audio formats and output formats like JSON, SRT, and WebVTT. Key features: - OpenAI-compatible API endpoints for integration with clients - Private, local audio processing - Speaker diarization with optional extension - Streaming transcription results - Support for various Whisper models - CPU and GPU deployment options - Offline operation using pre-cached models - Built-in Docker support Tech stack: Python Why it matters: ScribeCrate offers a robust, self-hosted solution for transcribing and translating audio, making it ideal for businesses and individuals looking to integrate speech-to-text capabilities without relying on external APIs or cloud services. Repository: hwdsl2/scribecrate GitHub stars: 119 GitHub description: ScribeCrate: open-source, self-hosted Whisper speech-to-text server powered by faster-whisper. OpenAI-compatible transcription and English translation APIs, speaker diarization, JSON/SRT/VTT output, SSE streaming, offline operation with cached models, and Docker deployment on CPU (amd64/arm64) or NVIDIA CUDA (amd64).

### Overview: Reflect3r is a single-view 3D stereo reconstruction method that leverages mirror reflections to generate accurate 3D reconstructions from a single image. This approach simplifies the imaging process and enhances the compatibility with powerful feed-forward reconstruction models for robust 3D reconstruction. ### Key Features: - **Mirror Reflection Utilization**: Treats mirror reflections as auxiliary views to construct a physically valid virtual camera. - **Pixel-Domain Generation**: Directly generates virtual views from pixel-domain data while adhering to the real-world imaging process. - **Symmetric Pose Estimation**: Proposes a symmetric-aware loss to refine pose estimation, leveraging geometric symmetry introduced by mirrors. - **Dynamic Scene Support**: Extends to dynamic scenes where each frame contains a mirror reflection, enabling efficient per-frame geometry recovery. - **Customizable Synthetic Dataset**: Provides a fully customizable synthetic dataset of 16 Blender scenes with ground-truth point clouds and camera poses. ### Tech Stack: - Python ### Why It Matters: This method simplifies the imaging process, making it compatible with powerful feed-forward reconstruction models, leading to more efficient and robust 3D reconstruction.

### Overview: HEALNet is a multimodal fusion architecture designed for handling heterogeneous biomedical data at NeurIPS 2024. It learns modality-specific weights and projects them into a shared embedding, capturing both spatial and positional information. The iterative architecture allows for flexible and robust multimodal pipelines, enabling the model to handle missing modalities effectively and provide model inspection through modality-specific attention weights. ### Key Features: - **Modality-specific signal preservation**: HEALNet learns modality-specific weights and projects them into a shared embedding. - **Cross-modal interactions**: By passing a latent embedding through the fusion layers, it iteratively encodes each modality into the shared embedding, becoming the context for the next modality. - **Handling missing modalities**: The model can skip missing modalities for individual samples at train or inference time without adding much noise. - **Model inspection**: The model's modality-specific attention weights can be inspected. ### Tech Stack: - Python ### Why It Matters: - **Preserve modality-specific signal**: HEALNet learns modality-specific weights and projects them into a shared embedding, capturing both spatial and positional information. - **Cross-modal interactions**: By passing a latent embedding through the fusion layers, it iteratively encodes each modality into the shared embedding, becoming the context for the next modality. - **Handling missing modalities**: The model's iterative architecture allows skipping missing modalities for individual samples at train or inference time without adding much noise. - **Model inspection**: The model's modality-specific attention weights can be inspected.

### Overview: This repository provides a comprehensive guide on Natural Language Processing (NLP), covering various topics such as tokenization, part-of-speech tagging, machine translation, named entity recognition, classification, and sentiment analysis. It includes practical applications, libraries, and tools that will help developers enhance their NLP skills. ### Key Features: - Tokenization - Part-of-Speech Tagging (POS) - Machine Translation - Named Entity Recognition (NER) - Classification - Sentiment Analysis ### Tech Stack: Python ### Why it Matters: This guide is essential for developers looking to improve their NLP skills, whether for academic research, project development, or professional applications. It covers the foundational concepts and practical tools needed to effectively use NLP in various domains.

Overview: um is a GPT-powered CLI assistant that simplifies command-line interactions by providing shell commands in plain English. Key features: - Instantly suggests the right command without leaving the terminal - Can index your history to find commands you've run before - Instantly learns from your corrections - Simple interface for clarifying questions or providing more context - New in 0.1.6: `um recipe` for infrastructure provisioning (currently Terraform only) - New in 0.1.7: Enhanced reverse search functionality with fuzzy matching and semantic search - New in 0.1.10: Parameterless `um` for suggesting next actions based on usage patterns Tech stack: Python Why it matters: um simplifies command-line interactions by providing clear, concise commands in plain English, making it easier for developers to perform tasks efficiently and accurately. Repository: promptops/cli Primary language: Python GitHub stars: 121 GitHub description: um is a GPT-powered CLI assistant. Ask questions in plain English, get the perfect shell command. README excerpt: um is a GPT-powered CLI assistant. Ask questions in plain English, get the perfect shell command. <img src="https://github.com/promptops/cli/raw/main/media/default.png" /> # Features - find the right command without leaving the terminal - `um` can index your history to find commands you've run before - `um` instantly learns from your corrections - simple interface to clarify your question or provide more context - NEW in 0.

### Overview: This repository contains the PyTorch code for multimodal emotion recognition using pretrained Roberta and Speech-BERT models. The project is based on the INTERSPEECH 2020 paper "Jointly Fine-Tuning 'BERT-like'" Self Supervised Models to Improve Multimodal Speech Emotion Recognition". ### Key Features: - Custom dataloader for loading raw audio, face frames, and text. - Emotion prediction task implemented in Fairseq interface. - Custom architecture built on top of Fairseq, including modifications to self-attentional scripts. - Pretrained SSL models for speech features (VQ-wav2vec) and sentence features (Roberta). - Tokenized text and speech data for training. - Preprocessed datasets and labels available in the repository. ### Tech Stack: - Python ### Why It Matters: The project aims to improve multimodal speech emotion recognition by fine-tuning "BERT-like" self-supervised models on top of pretrained Roberta and Speech-BERT. This approach leverages existing state-of-the-art models and architectures to achieve better performance in emotion recognition tasks. ### GitHub Stars: 121 ### Primary Language: Python ### Repository: shamanez/BERT-like-is-All-You-Need

### Overview: SurrealDB is the ultimate cloud database for tomorrow's applications. ### Key features: - **Querying with Vike**: Use Vike for efficient querying and visualization of your data. - **Built with React and Mantine**: The documentation is crafted with React and Mantine for a modern and intuitive user experience. - **Contributing Guidelines**: Follow the guidelines to contribute to the documentation, including a list of Good First Issues. ### Tech stack: Vike, React, and Mantine. ### Why it matters: SurrealDB is designed to simplify the development of applications that require cloud-based databases, ensuring easier development, faster build times, and quicker scalability.

### Overview: This repository is a curated list of Generative AI companies, sorted by focus area and total fundraised amount. It includes companies in various categories such as horizontal application, vertical application, and enabling infrastructure. ### Key Features: - **Text, Chat & Translations**: Companies that provide AI-powered chatbots, translation services, and email assistance. - **Audio & Speech**: Platforms that use AI to transcribe audio and speech. - **Image, Visual, Design & Creative Work**: Companies that use AI for image generation, design, and creative work. - **Video**: Platforms that use AI for video creation and editing. - **3D, Simulations & XR**: Companies that use AI for 3D modeling, simulations, and virtual reality. - **General Productivity**: Tools that enhance productivity through AI. - **Search**: Companies that use AI for search and information retrieval. - **Marketing BI tools**: Platforms that use AI for marketing analytics and business intelligence. - **Code**: Companies that use AI for software development and code optimization. - **Music**: Platforms that use AI for music composition and production. ### Tech Stack: Unknown ### Why it Matters: This repository provides a comprehensive overview of Generative AI companies, focusing on their fundraising amounts and areas of focus. It helps developers and entrepreneurs identify companies that align with their needs and interests in the field of Generative AI.

### Overview: An out-of-the-box local Web UI for DeepSeek-OCR, built with FastAPI and Vue.js, providing PDF/Image uploads, progress tracking, and result visualization with bounding boxes. Easily experience the power of a top-tier OCR model. ### Key features: - 📄 PDF & 🖼️ image upload - ⏳ Progress visualization - 🗺️ Bounding-box overlay - 📑 Annotation detail view - 📜 History management - 🧩 Modular UI - 📊 Enhanced Visualization - 📝 Logs & Workflow ### Tech stack: FastAPI + Vue.js ### Why it matters: Easily experience the power of a top-tier OCR model with this out-of-the-box local Web UI for DeepSeek-OCR.

Overview: This repository contains the code for a paper that proposes using large language models to control a robot's walking motion. The LLM outputs low-level actions, allowing for real-time walking without the need for complex controllers. Key features: - Code for Prompt a Robot to Walk with Large Language Models - Uses LLM to directly output low-level actions for robot walking - Simulation runs at 10 Hz, while LLM inference is paused at 200 Hz - Supports A1 and ANYmal C robots at 10 Hz Tech stack: Python Why it matters: This approach promises real-time control of robots without the need for complex controllers, potentially simplifying robot design and improving performance. Repository: HybridRobotics/prompt2walk Primary language: Python GitHub stars: 120 GitHub description: Code for Prompt a Robot to Walk with Large Language Models https://arxiv.org/abs/2309.09969

Overview: J.A.R.V.I.S. is an Iron Man-inspired personal virtual assistant designed to perform tasks through voice commands and integrate with various IoT devices. Key features: - Voice command-based task execution - Integration with smart home devices (light bulbs, TVs, Nest, thermostats, CCTV, speakers) - Natural language processing for chatbot capabilities - Computer vision for face detection, recognition, and object detection Tech stack: Python Why it matters: J.A.R.V.I.S. aims to provide a practical, advanced personal assistant that can handle a wide range of tasks and interact with the physical world through voice commands and IoT devices, making it a valuable tool for home automation and personal assistance. Repository: KKshitiz/J.A.R.V.I.S Primary language: Python GitHub stars: 121 GitHub description: Iron man inspired Personal virtual assistant README excerpt: # J.A.R.V.I.S<br> Iron man inspired Personal virtual assistant<br> [Website](https://kkshitiz.github.io/J.A.R.V.I.S/)

Overview: SUR-adapter is a project that enhances pre-trained diffusion models with large language models to improve their textual semantic representation for text-to-image generation. This project has been accepted at the ACM MM 2023 conference. Key features: - SUR-adapter implementation - Pre-trained diffusion models enhanced with large language models - High-quality textual semantic representation for text-to-image generation Tech stack: Python Why it matters: This project leverages the strengths of large language models to enhance pre-trained diffusion models, resulting in improved semantic understanding and reasoning capabilities, which is crucial for applications requiring high-quality textual representation in text-to-image generation. Repository: Qrange-group/SUR-adapter GitHub stars: 120 GitHub description: ACM MM'23 (oral), SUR-adapter for pre-trained diffusion models can acquire the powerful semantic understanding and reasoning capabilities from large language models to build a high-quality textual semantic representation for text-to-image generation.

### Overview: This repository contains a collection of up-to-date courses, articles, and tutorials for AI engineers, focusing on various aspects of AI development and deployment. The courses cover topics such as language models, deep learning, and cloud computing, providing practical insights and hands-on experience. ### Key Features: - 🚀 LLM Bootcamp - Spring 2023: A comprehensive course on building LLM-powered apps. - 🧠 DeepLearning AI: Covers advanced topics in deep learning and AI. - 📚 Full Catalog: A group of short courses categorized by their level and duration, including partners and specific topics. - 📈 Tech Stack: The repository uses a mix of programming languages and tools for AI development. - 🤝 Why it Matters: This repository is essential for anyone looking to stay updated with the latest trends and techniques in AI, ensuring they have the necessary skills to develop and maintain AI systems effectively. ### Tech Stack: - Python - JavaScript - Docker - Kubernetes - AWS - Azure - Google Cloud ### Why it Matters: This repository is crucial for AI engineers who want to keep pace with the rapidly evolving field of AI. By regularly updating the repository with new courses, articles, and tutorials, it ensures that the content remains relevant and useful for the ever-changing landscape of AI development.

Overview: Repopack-py is a Python version of the popular npm tool repopack, designed to pack entire repositories into AI-friendly files, making it ideal for feeding codebases to Large Language Models. Key features: - Packs entire repositories into AI-friendly files - Supports feeding codebases to Large Language Models (LLMs) or other AI tools - Runs entirely on the browser, with a static webapp for the same use case - Supports customization with options like output file name, ignore patterns, and configuration files Tech stack: Python Why it matters: Repopack-py simplifies the process of preparing codebases for AI models, making it easier to integrate and utilize large codebases with AI tools. Repository: abinthomasonline/repopack-py Primary language: Python GitHub stars: 120 GitHub description: Python version of 📦 repopack (https://github.com/yamadashy/repopack) by yamadashy (https://github.com/yamadashy) README excerpt:

### Overview: CF Flux Remix 是一个使用 Cloudflare Workers 和 Remix 框架生成图像的应用程序，利用 Cloudflare 的 AI 模型生成图像，提供了一个用户友好的界面和 API 接口，支持多种图像生成模型，包括 Flux 和标准模型，适用于前端和后端开发。 ### Key features: - 使用 Cloudflare 的 AI 模型生成图像【免费】 - 支持多种图像生成模型，包括 Flux 和标准模型 - 提供 API 接口以便集成到其他应用中 - 支持提示词翻译和优化 - 一键部署 - 响应式设计，现代设计 - 图片生成不受限制(不经审查)，你懂的 ### Tech stack: TypeScript ### Why it matters: 这个项目简化了在 Cloudflare Workers 上部署图像生成应用的过程，提供了丰富的功能和易于使用的 API，使得开发者能够快速集成到其他应用中。

### Overview: This project fine-tunes a large language model to mimic the behavior of LIHKG users, creating a Cantonese post response generation language model. The model has been trained on responses from the LIHKG forum and is known for its unique linguistic style influenced by the community's culture. ### Key Features: - Fine-tuned large language model - Mimics LIHKG users' behavior and linguistic style - Can generate Cantonese post responses ### Tech Stack: Python ### Why it Matters: This model provides a unique way to understand and interact with the LIHKG community, offering insights into Cantonese language and culture through its responses.

### Overview: AutoMix is a project that automatically mixes language models to improve their performance on various tasks. It leverages self-verification and meta-verification techniques to enhance the accuracy and reliability of language models. ### Key Features: - **Self-Verification:** AutoMix uses few-shot self-verification to check the correctness of the output from smaller language models. - **Meta-Verification:** An additional layer of meta-verification is introduced to further verify the output, especially when the self-verification results are noisy. - **Colab Notebooks:** The project includes several notebooks that guide the process of running inference, self-verification, and meta-verification. ### Tech Stack: - Jupyter Notebook ### Why It Matters: AutoMix addresses the challenge of improving the performance of language models by combining the strengths of smaller and larger models, leading to better accuracy and reliability in various applications.

### Overview MaineCoon is the first real-time audio-visual autoregressive model, designed specifically for social world applications. It achieves sub-second interaction and a record-breaking frame rate of up to 47.5 FPS on a single H100 GPU, making it a game-changer for interactive social media platforms. ### Key Features - **⚡ Real-time on a single GPU:** A 22B interactive audio-visual autoregressive model capable of streaming generation and sub-second interaction, with a record-breaking frame rate of up to 47.5 FPS on a single H100. Generation cost drops well below \$0.001 per second — and keeps falling. - **🌍 A new paradigm: social world models:** MaineCoon positions and serves as the first generative core for *social world models*, a technical foundation for next-generation AI-native social platforms. - **🎓 Forcing-free streaming training:** A multi-stage training paradigm — **self-resampling**, **cross-modal representation alignment**, **domain-aware preference optimization**, and **reinforced online-policy distillation (ROPD)** — that enables native, efficient streaming audio-visual training at 22B scale. - **🧠 Agentic streaming inference:** An agentic inference framework that supports **thousand-second-scale** generation while mitigating drift through agentic cache management, chunk commitment, long-context rollout, and prompt planning. ### Tech Stack MaineCoon is built with a combination of cutting-edge technologies, including Hugging Face, TensorFlow, and PyTorch

Overview: 这个仓库实现了一个非常简易版的 DDPM (Denoising Diffusion Probabilistic Models) 模型，使用 PyTorch 实现。 Key features: - 简易版 DDPM 模型实现 - 使用 PyTorch 进行训练和推理 - 支持数据集 Stanford Cars - 提供展示数据集和前向扩散的示例代码 Tech stack: Python Why it matters: 这个简易版的 DDPM 模型简化了复杂的过程，使得学习和理解模型的工作原理变得更加容易。

Overview: OpenTPOD is an open-source tool designed for non-experts to create custom deep learning-based object detectors without writing any code. Key features: - Training data management - Seamless integration with OpenCV CVAT Labeling Tool - One-click training/fine-tuning of SSD MobileNet, Faster RCNN Inception, and Faster RCNN ResNet - One-click model export for inference with Tensorflow Serving - Extensible architecture for easy addition of new deep neural network architectures Tech stack: CSS Why it matters: OpenTPOD simplifies the process of creating custom object detectors, making it accessible to non-experts and accelerating the development of custom solutions in the field of computer vision.