Projects

### Overview: This project aims to develop a Convolutional LSTM (Conv-LSTM) network using Connectionist Temporal Classification (CTC) cost for speech recognition. The model is designed to be efficient and easy to debug, making it suitable for Kaggle competitions. ### Key Features: - **Model Architecture:** A Convolutional Residual, Backward LSTM network. - **Training Data:** Speech commands dataset with 30 short words from 1800 speakers. - **Training Method:** End-to-end training using TensorFlow. - **Evaluation Metrics:** Validation, Training, and Test accuracy for public and private leaderboard. ### Tech Stack: - **Programming Languages:** Python - **Framework:** TensorFlow ### Why It Matters: This project showcases the power of deep learning in speech recognition, demonstrating how convolutional and recurrent neural networks can be combined to achieve state-of-the-art performance in speech command detection. The use of CTC cost ensures that the model can handle sequences of characters, making it effective for tasks like speech recognition. ### Repository: - **GitHub:** huschen/kaggle_speech_recognition - **Stars:** 72 - **Description:** Conv-LSTM-CTC speech recognition network (end-to-end), written in TensorFlow. ### Installation and Usage: - **Prerequisites:** Python 3.5, TensorFlow 1.4; or Python 3.6, TensorFlow 1.7. - **Download Dataset:** Run `tools/download_dataset.sh`. - **Train Network:** Run `python train.py`. - **Make Inferences:**

Overview: Grok Imagine 2.0 API is a flat per-unit billing service that allows users to upload images and generate custom designs at a cost of $0.015 per unit. The API is compatible with OpenAI's APIMart gateway, making it accessible through a simple API key. Key features: - Upload image - Generate default design - Edit region Tech stack: Shell Why it matters: This API offers a cost-effective solution for generating custom designs, making it ideal for businesses looking to streamline their design processes without the high costs associated with traditional design software. Repository: k2c2gbd1pqdj01wo/grokimagine2.0-grok-imagine-2.0-api Primary language: Shell GitHub stars: 72

Overview: R2DM is a LiDAR data synthesis model using denoising diffusion probabilistic models (DDPMs) for the equirectangular representation. Key features: - Unconditional generation of LiDAR range/reflectance images. - Pretrained model for efficient inference. - Supports training and evaluation on the KITTI-360 dataset. Tech stack: Python. Why it matters: R2DM offers a practical solution for generating high-quality LiDAR data, which is crucial for applications in autonomous driving, robotics, and other fields that require accurate 3D point cloud data. Repository: [kazuto1011/r2dm](https://github.com/kazuto1011/r2dm) GitHub stars: 72 GitHub description: LiDAR Data Synthesis with Denoising Diffusion Probabilistic Models (ICRA 2024)

Overview: ReGenNet is an official implementation of the paper "ReGenNet: Towards Human Action-Reaction Synthesis," which was presented at CVPR 2024. This repository contains the content of the paper and includes training, evaluation codes, and pre-trained models for human action-reaction synthesis. Key features: - Training and evaluation codes - Pre-trained models for action recognition - Framework for human action-reaction synthesis Tech stack: Python Why it matters: ReGenNet provides a comprehensive solution for generating human action-reaction sequences, making it valuable for researchers and developers interested in action synthesis and imitation learning. Repository: liangxuy/ReGenNet GitHub stars: 72

### Overview: A comprehensive and open-source implementation of the Sketch-Guided Text-to-Image Diffusion model, based on the Google paper. This project provides a pretrained Latent Guidance Predictor (LGP) that can be used to generate high-quality images from sketches. ### Key Features: - **Pretrained LGP**: A pretrained Latent Guidance Predictor (LGP) trained on 4995 Imagenet images for 16 epochs. - **Dataset Requirements**: The dataset should be in the specified format, including edge maps and images in separate folders. - **Training**: The training script is provided, allowing users to customize the dataset and model parameters. - **Evaluation**: A script for evaluating the generated edge maps from the pretrained LGP. ### Tech Stack: Python ### Why It Matters: This project offers a practical tool for generating high-quality images from sketches, leveraging the capabilities of Stable Diffusion and the principles of Sketch-Guided Text-to-Image Diffusion.

Overview: Oracle VecDB Python SDK provides a Python interface to Oracle AI Database, enabling developers to integrate vector search, semantic search, and AI agents directly into their applications. Key features: - Vector search - Semantic search - RAG (Reinforcement Learning Agent) applications - Embeddings - AI agents Tech stack: Python Why it matters: Oracle AI Database's Oracle VecDB Python SDK simplifies the integration of vector search, semantic search, and AI functionalities into Python-based applications, making it easier to build advanced applications without the need for separate vector databases. Repository: oracle/vecdb-python-sdk GitHub stars: 72

### Overview: 羽人-百川 7B 是基于 baichuan-7B 的开源多模态大语言模型，旨在提供卓越的多轮对话、开放域问答、角色扮演、文本生成、文本理解、图片理解等任务表现。 ### Key features: - **多模态融合**: 羽人将 LLM 的语言模态与目前 SOTA 的 CLIP 模型进行融合，实现卓越的图片理解能力。 - **高质量 SFT 数据集**: 基于 Pleisto 自有的商业多轮对话与指令精调数据集，数据集经过多轮质检，增强数据增强。 - **商业友好**: 代码和模型权重均开源，仅需联系 [baichuan 团队](opensource@baichuan-inc.com) 获得商业使用授权。 - **兼容 ChatML**: 兼容 GPT-4 同款的 ChatML 格式，减少 Prompt Injection 风险，实现良好的 System Prompt 遵循度。 - **参数量限制**: 尚未提供更大参数量的闭源模型，但有更大参数量的模型可供考虑。 ### Tech stack: Python ### Why it matters: 羽人-百川 7B 在多轮对话、开放域问答、角色扮演、文本生成、文本理解、图片理解等任务上表现出色，为多模态应用提供了一种高效且可靠的解决方案。

Overview: This project is a React client library for the Speechly API, providing developers with a streamlined and efficient way to integrate Speechly into their applications. Key features: - Speechly API integration - TypeScript support - Simplified API usage Tech stack: TypeScript Why it matters: This library simplifies the integration of Speechly into React applications, making it easier for developers to leverage Speechly's advanced speech recognition and analysis capabilities without the need for extensive configuration or additional dependencies.

### Overview: VindexLLM is a pure Delphi, GPU-powered LLM inference engine that uses Vulkan compute shaders to run GGUF models entirely on the GPU. It performs full transformer inference without relying on Python, CUDA, or other external runtimes, requiring only vulkan-1.dll, which is typically included with modern GPU drivers. ### Key features: - **GPU-accelerated inference**: Runs all heavy computation on the GPU via Vulkan compute shaders. - **Vulkan dependency**: Uses `vulkan-1.dll` which is included with modern GPU drivers, eliminating the need for external runtime libraries. - **No Python or CUDA**: Pure compiled Delphi, starts instantly, no dependency resolution. - **Memory-mapped model loading**: GGUF files are mapped directly via `CreateFileMapping` / `MapViewOfFile`, weights are loaded into GPU memory. - **Zero install**: No separate toolkit download, no PATH configuration, no DLL hell. - **Self-contained binary**: All 37 compute shaders are compiled to SPIR-V at build time and embedded into the executable as Windows resources. - **No vendor lock-in**: Vulkan runs on any modern GPU, not tied to NVIDIA hardware. - **Persistent memory chat**: Supports multi-turn conversation with SQLite-backed persistent memory and RAG retrieval across sessions. ### Tech stack: Delphi, Vulkan, SPIR-V, GGUF, Vulkan-1.dll ### Why it matters: VindexLLM offers a zero-install, vendor-agnostic, and self-contained solution for GPU-accelerated LLM inference, eliminating the

Overview: This repository is a derivative of the official implementation of the paper "Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme" [Link](https://arxiv.org/abs/2109.13821). It builds upon their work and incorporates additional features and modifications specific to this project. Key features: - Pre-trained speaker encoder available at [Google Drive](https://drive.google.com/file/d/1Y8IO2_OqeT85P1kks9I9eeAq--S65YFb/view?usp=sharing) - Pre-trained universal HiFi-GAN vocoder available at [Google Drive](https://drive.google.com/file/d/10khlrM645pTbQ4rc2aNEYPba8RFDBkW-/view?usp=sharing) - Pre-trained voice conversion models trained on LibriTTS and VCTK available at [Google Drive](https://drive.google.com/file/d/18Xbme0CTVo58p2vOHoTQm8PBGW7oEjAy/view?usp=sharing) and [Google Drive](https://drive.google.com/file/d/12s9RPmwp9suleMkBCVetD8pub7wsDAy4/view?usp=sharing) Tech stack: Jupyter Notebook Why it matters: This repository provides a comprehensive solution for voice conversion, leveraging advanced diffusion models and pre-trained encoders to achieve high-quality conversions. The inclusion

### Overview: This repository is dedicated to exploring and showcasing advanced methods for spatial and visual reasoning in machine learning models, particularly focusing on embodied applications. It aims to enhance the ability of large language models to reason about and interact with the physical world. ### Key Features: - **Spatial Reasoning Models**: Implementations of models that can reason about spatial relationships and environments. - **Embodied Applications**: Focus on models that can reason about and interact with the physical world. - **Reinforcement Learning**: Utilizes reinforcement learning techniques to improve the reasoning capabilities of models. - **Datasets and Code**: Includes datasets and code repositories for various spatial and visual reasoning tasks. ### Tech Stack: - Python ### Why It Matters: This repository is crucial for researchers and developers interested in advancing the field of spatial and visual reasoning in AI. It provides a platform for exploring and implementing cutting-edge techniques that can enhance the understanding and interaction of models with the physical world, making significant contributions to the development of embodied AI systems.

Overview: Jarvis is an AI voice assistant designed for personal use, featuring advanced speech recognition, facial recognition, and hotword detection. It is built with Python, OpenCV, and modern web technologies, making it a reliable and user-friendly solution. Key features: - Voice Control - Face Recognition - Hotword Detection Tech stack: Python, OpenCV, SQLite, NumPy, Hugging Face, TensorFlow, Bootstrap, Git, VS Code. Why it matters: Jarvis simplifies communication and enhances security, making it a valuable tool for both personal and professional use.

### Overview: A complete AI-powered toolkit for generating, editing, and publishing books, designed to streamline the entire book creation process from concept to publication. ### Key features: - **Outline Generation**: Automatically generates detailed outlines. - **Chapter Creation**: Extends and edits chapters with quality checks. - **Cover Design**: Produces professional front and back covers using AI or ImageMagick. - **Manuscript Compilation**: Assembles a complete manuscript with title pages, Table of Contents, and other elements. - **Export Options**: Export the final book in EPUB, PDF, and other common ebook formats. - **Quality Checks**: Performs quality and plagiarism checks on chapters. - **AI Integration**: Utilizes AI providers like Gemini, Ollama, Groq, and OpenAI for enhanced content creation. ### Tech stack: Shell scripts and helper tools. ### Why it matters: This toolkit is designed to use free tools and services where possible, making it accessible and cost-effective. It aims to create professional manuscripts with AI-assisted authoring and design, ensuring high-quality output for Amazon Kindle Direct Publishing (KDP). Repository: wesleyscholl/book-generator GitHub stars: 71 GitHub description: 🤖📚 AI-powered toolkit for generating, editing, and publishing books. Create outlines, chapters, covers, and export EPUB/PDF ready for Amazon KDP. 📖➡️🌐 README excerpt: # 🤖📚 AI Book Generator A complete autonomous book creation pipeline from concept to publication - generating professional manuscripts with AI-powered

Overview: This project is a comprehensive guide for developers looking to create AI agents using Python. It provides step-by-step instructions and practical examples to help beginners and experienced developers alike. Key features: - Step-by-step video guide - Full course content - AI agent creation using Python Tech stack: Python Why it matters: This project is essential for developers who want to enhance their skills in AI development and create intelligent agents using Python, which is a widely used language in the field.

Overview: BumbleCore is an LLM training framework built from the ground up, featuring a custom BumbleBee architecture and end-to-end support for multiple open-source models across Pretraining → SFT → RLHF/DPO. Key features: - Fully Manual Training Loop - Bumblebee Model Architecture: Freely Customize Your Model - Universal Training Framework: Supporting Mainstream Open-Source Models Tech stack: Python Why it matters: BumbleCore provides maximum flexibility and control over every training detail, making it ideal for researchers and developers looking to deeply understand and customize their model training process.

### Overview: YantrikDB is a cognitive memory engine designed for persistent AI systems, providing features like temporal decay, contradiction detection, and autonomous consolidation. It is an embeddable Rust library with Python bindings, powering YantrikDB-server for HTTP gateway, MCP server, and openraft cluster. ### Key features: - **Temporal Decay**: Ensures that old information is forgotten, preventing the accumulation of near-duplicate memories. - **Contradiction Detection**: Automatically identifies and separates conflicting pieces of information. - **Autonomous Consolidation**: Helps in the automatic organization and prioritization of memories. - **Knowledge Graph**: A knowledge graph structure to enhance the retrieval of related information. - **ANN Recall via HNSW**: Efficient recall of data using the Hierarchical Navigable Small World (HNSW) algorithm. ### Tech stack: YantrikDB is built on Rust, with Python bindings for easy integration into Python projects. ### Why it matters: YantrikDB addresses the common issue of AI agents forgetting information and being unable to distinguish conflicting facts, providing a more robust and reliable memory management system for AI agents. It is designed to be embedded in other systems, making it a versatile tool for various AI applications. ### Repository: The repository is hosted on GitHub and has been starred by 71 developers, indicating its growing interest and use in the AI community. ### Primary language: The project is written in Rust, leveraging its strengths in performance and reliability. ### GitHub stars: The repository has been starred by 71 developers, indicating its growing interest and use in

### Overview: ASRDeepspeech x Sakura-ML is a deepspeech2 speech recognition model implemented in PyTorch, utilizing the Sakura-ML runtime for training and evaluation. This project supports both English and Japanese languages and achieves a CER of 34 on the JSUT test set. ### Key Features: - **Sakura-ML Runtime**: The model is trained using the Sakura-ML runtime, a clean, modular approach to deepspeech2 models. - **DeepSpeech2 Model**: Utilizes a GRU 3x512 architecture with mixed precision training and asynchronous evaluation. - **Sakura-Thread and Sakura-Process**: Runs evaluation and checkpoint writes on background threads, improving performance. - **Safe Checkpoints**: Atomic rolling checkpoints are written off the training thread, ensuring the run continues from the newest rolling checkpoint. - **Time-to-Performance Benchmark**: Measures the time to achieve a target CER of 15, showcasing the efficiency of the model. ### Tech Stack: - **Programming Language**: Python - **Dependencies**: Sakura-ML, PyTorch, CUDA 13.0, Mixed Precision, AsyncEval, AsyncCheckpoint, DDPAdapter ### Why It Matters: - **Efficient Training**: The Sakura-ML runtime and asynchronous evaluation techniques significantly reduce training time. - **Safe Checkpointing**: Atomic rolling checkpoints ensure the run continues from the latest checkpoint, improving crash safety. - **Time-to-Performance Benchmark**: Measures the time to achieve a target CER, highlighting the model's performance and efficiency. ###

Overview: Constellation Engine is a cognitive runtime designed for long-lived AI agents, providing a living star map of typed nodes, weighted edges, activation state, episodic recall, and post-turn consolidation. Key features: - Persistent knowledge graph - Activation state for attention across turns - Structured knowledge instead of opaque text chunks - Durable memory that can be inspected, edited, and migrated - External cognitive layer that survives model swaps Tech stack: JavaScript Why it matters: Constellation Engine addresses the need for long-term continuity and structured knowledge in AI agents, offering a solution that goes beyond traditional episodic memory systems, providing a more dynamic and adaptable memory layer for agents that need to accumulate history.

Overview: This project is an Android app that implements a speech recognizer using the Android native recognizer, eliminating the annoying popup and allowing continuous speech recognition without the need to keep the app listening indefinitely. Key features: - Speech recognition on Unity and Android without the annoying popup - Continuous speech recognition without the need to keep the app listening indefinitely Tech stack: Unity, Android Why it matters: This plugin simplifies speech recognition on Android without the need for the annoying popup, making it easier and more convenient for developers to integrate speech recognition into their projects. Repository: EricBatlle/UnityAndroidSpeechRecognizer GitHub stars: 71

Overview: This repository is a comprehensive solution for product cataloging using Generative AI, focusing on the Google Cloud Platform. Key features: - Product cataloging using Generative AI - Integration with Google Cloud Platform - Support for various product categories Tech stack: Jupyter Notebook Why it matters: This solution provides a scalable and efficient way to catalog products using advanced AI techniques, leveraging the power of Google Cloud for enhanced performance and reliability. Repository: GoogleCloudPlatform/genai-product-catalog Primary language: Jupyter Notebook GitHub stars: 71