Projects

Overview: OpenASR is a PyTorch-based end-to-end speech recognition system designed for minimal dependency, high performance, and modular design. Key features: 1. **Minimal Dependency**: The system does not require external software for feature extraction or decoding, relying solely on PyTorch. 2. **Good Performance**: Achieves high performance on ASHELL1 with advanced algorithms like Label Smoothing, SpecAug, LST, and CER of 6.6 on the baseline AISHELL1 test. 3. **Modular Design**: The system is divided into several modules, making it easy to extend and add features. Tech stack: Python Why it matters: OpenASR is a powerful and efficient speech recognition system that simplifies the process and reduces dependencies, making it suitable for various applications requiring high performance and ease of use. Repository: by2101/OpenASR GitHub stars: 115 GitHub description: A PyTorch-based end2end speech recognition system.

Overview: SensorLLM is a framework that aligns large language models with motion sensors for human activity recognition, achieving state-of-the-art results across various datasets and sensor types. Key features: - Aligns sensor time-series with human-intuitive, annotation-free textual trend descriptions via a QA-based framework. - Fine-grained trend-text alignment in the Sensor-Language Alignment Stage. - Handles multi-channel, multi-sensor data for downstream human activity recognition in the Task-Aware Tuning Stage. Tech stack: Python Why it matters: SensorLLM leverages motion sensors to interpret complex numerical data, enabling large language models to understand human activities and improve accuracy in various applications, such as smart home devices and physical activity monitoring. Repository: [cruiseresearchgroup/SensorLLM](https://github.com/cruiseresearchgroup/SensorLLM) GitHub stars: 115

### Overview: This TypeScript client for Pinecone.io provides a fully typed interface for interacting with the vector database, ensuring metadata is handled correctly and efficiently. ### Key Features: - **Type-Safe Metadata**: Automatically removes null metadata values, aligning with Pinecone's behavior. - **Modern Runtimes Support**: Compatible with modern fetch-based runtimes like Cloudflare Workers, Deno, and browsers. - **In-Editor Documentation**: IntelliSense and TypeScript server integration for seamless development. - **Small Package Size**: Less than 5KB gzipped, ideal for lightweight applications. - **E2E Test Coverage**: Comprehensive test suite ensuring reliability and performance. ### Tech Stack: - TypeScript ### Why It Matters: This client offers a convenient and efficient way to interact with Pinecone.io, leveraging TypeScript for type safety and modern runtimes for performance. It simplifies the process of working with vector databases while ensuring robustness and reliability. ### Repository: The repository is maintained by [rileytomasek](https://github.com/rileytomasek), with 115 GitHub stars, showcasing its growing popularity among developers. ### Install: - **ESM**: Requires converting to ESM or using dynamic `import()`. - **CommonJS**: No longer provides a CommonJS export, so ensure compatibility with your project's runtime. ### Example Usage: ```ts import { PineconeClient } from 'pinecone-client'; // Define metadata type type Metadata = { size: number, tags?: string[] | null }; // Instantiate client const pinecone =

Overview: Feros Voice Agent OS is an open-source project designed to provide an enterprise-grade infrastructure layer for voice AI, offering self-hostability, sub-second latency, and an AI-driven builder. Key features: - Self-hostable monorepo - Rust runtime for sub-second latency - AI-driven builder - Python control plane Tech stack: Rust Why it matters: Feros addresses structural problems in the current voice AI ecosystem, providing a production-ready solution with full control over data residency and compliance. Repository: ferosai/feros GitHub stars: 113

### Overview: GPTLens is a project that leverages large language models to enhance smart contract vulnerability detection, specifically for the IEEE Trust, Privacy and Security (TPS) conference 2023. This repository contains the code and datasets used in the paper "Large Language Model-Powered Smart Contract Vulnerability Detection: New Perspectives". ### Key Features: - **Auditor**: Identifies potential vulnerabilities by generating random test cases. - **Critic**: Evaluates the correctness and severity of identified vulnerabilities. - **Ranker**: Determines the overall score of vulnerabilities based on the auditor and critic outputs. ### Tech Stack: Solidity ### Why It Matters: This project provides a valuable tool for developers to identify and mitigate smart contract vulnerabilities, ensuring the security and reliability of their applications.

### Overview: DrugHIVE is a deep hierarchical variational autoencoder designed for structure-based drug design. It leverages a model trained on the PDBbind dataset to generate and optimize ligands with substructure modification, making it a powerful tool for computational drug discovery. ### Key features: - **Structure-based Drug Design**: Uses a deep hierarchical generative model to predict and generate new drug-like molecules. - **Substructure Modification**: Allows for the modification of existing molecules to create new drug candidates. - **Training and Optimization**: Includes tools for training the model on PDBbind and ZINC datasets, as well as optimizing generated molecules. ### Tech stack: Python, CUDA, PyTorch, PyTorch Lightning, RDKit, OpenBabel. ### Why it matters: DrugHIVE provides a comprehensive solution for structure-based drug design, enabling researchers to efficiently generate and optimize new drug candidates, thereby accelerating the drug discovery process.

Overview: MisterWhisper is an open-source application designed to simplify your workflow by transforming spoken words into text in real-time. Key features: - Quick voice transcription - Integration with active software - GPU acceleration (optional) - Local or remote transcription Tech stack: Java, Whisper AI model (GPU-accelerated) Why it matters: MisterWhisper is a robust multilingual transcription tool that integrates seamlessly with various software applications, making it a valuable addition to your workflow.

### Overview: A continuously-verified dataset of free-tier and trial-credit Large Language Model (LLM) APIs for developers. Every entry is dated, sourced to the provider's own documentation, and machine-readable. No hype, no dead links. ### Key Features: - **Free Tier & Trial Credits:** Access to free LLM APIs with trial credits. - **Image, Speech, Embeddings, Rerank, and OCR:** Comprehensive set of APIs including image, speech, and text-related functionalities. - **Verified & Fresh:** Entries are verified and regularly re-verified to ensure accuracy and freshness. ### Tech Stack: JavaScript ### Why It Matters: This dataset provides a reliable and up-to-date resource for developers looking to integrate free LLM APIs into their projects, ensuring they can make informed decisions without relying on outdated or unreliable sources.

Overview: LongLive-RAG is an open-source implementation of a general retrieval-augmented framework for long video generation, specifically designed to address the challenges of error accumulation and identity drift in long-term video generation tasks. Key features: - Integrates retrieval-based context into the autoregressive (AR) video generation process. - Uses past latents as extra context to maintain subject identity and visual quality over long horizons. - Does not require retraining the base generator, making it a plug-and-play solution. Tech stack: Python Why it matters: LongLive-RAG provides a novel approach to long video generation by leveraging retrieval memory, addressing key issues such as error accumulation and identity drift, and offering a more efficient and scalable solution compared to traditional methods. Repository: qiz1907/LongLive-RAG GitHub stars: 115 Primary language: Python GitHub description: Official Implementation of LongLive-RAG: A general retrieval-augmented framework for long video generation.

### Overview: This project implements a triplet loss-based model for keyword spotting, focusing on improving efficiency and performance in keyword recognition tasks. ### Key features: - Efficient representation learning for keyword spotting - Supports both triplet loss and classification models - Utilizes FAISS for fast nearest neighbor search - Compatible with NeMo and PyTorch frameworks ### Tech stack: Python ### Why it matters: This project offers a significant advancement in keyword spotting by leveraging efficient representation learning through triplet loss, making it particularly useful for real-time applications where speed and accuracy are crucial.

Overview: PII Guard is an LLM-powered tool designed to detect and manage Personally Identifiable Information (PII) in logs, supporting data privacy and GDPR compliance. Key features: - Detects identity information such as full names, usernames, email addresses, phone numbers, and more. - Recognizes sensitive categories like racial or ethnic origin, political opinions, religious beliefs, and more. - Identifies government and financial identifiers including national IDs, passport numbers, and credit cards. - Handles network and device information such as IP addresses, MAC addresses, and device metadata. - Manages vehicle information like license plates. Tech stack: TypeScript. Why it matters: This project explores how Large Language Models can detect sensitive data in logs more effectively than traditional regex-based approaches, particularly in complex and messy log data. Repository: rpgeeganage/pII-guard GitHub stars: 115 GitHub description: 🛡️ PII Guard is an LLM-powered tool that detects and manages Personally Identifiable Information (PII) in logs — designed to support data privacy and GDPR compliance.

Overview: This project evaluates and ranks the most popular Large Language Models (LLMs) based on their propensity to hallucinate, focusing on context length and open vs. closed-source LLMs. Key features: - Evaluates 22 models, including 10 closed-source and 12 open-source models. - Tests three context lengths: short, medium, and long. - Examines the impact of open-source vs. closed-source LLMs. - Uses Chain-of-Note prompting technique for enhanced performance in short-context scenarios. Tech stack: Python, Jupyter Notebook, and various LLM APIs. Why it matters: Understanding LLMs' propensity to hallucinate is crucial for developers and researchers to ensure model reliability and safety in applications like RAG systems.

### Overview: This repository is a comprehensive guide to generative AI projects and resources in Julia, focusing on projects and resources related to the Julia programming language. It aims to provide developers with the latest tools, models, and APIs for creating and utilizing generative AI projects in Julia. ### Key Features: - Generative AI Projects and Julia - Models - API SDKs - Model Providers - Cloud Services Providers - Vector Databases - General-purpose DBMS with Vector Index Support - Packages - JLL Packages - Benchmarks/Comparisons - Applications/Products - Waiting Room - Archived Projects - Tutorials/learning - Noteworthy Mentions - Local Deployments - Generative AI - Previous Generation ### Tech Stack: Julia ### Why it Matters: This guide helps developers and researchers in the Julia community to explore and utilize generative AI projects and resources, making it easier to create and deploy generative AI models in their projects.

Overview: 这是一个全面的中文RAG技术教程，从基础概念到生产部署，系统化地教授检索增强生成技术。教程包含4个模块、20章内容、17个Jupyter Notebooks、6个企业级实战案例，涵盖了LangChain、LlamaIndex、向量数据库、Agent、GraphRAG等前沿技术。 Key features: - 4个模块，20章内容 - 17个Jupyter Notebooks - 6个完整实战案例 - 89张技术图表 - 30+练习题 - 100%的中文优化 - 100%的前沿技术覆盖 - 100%的系统化学习路径 Tech stack: Jupyter Notebook Why it matters: 本教程旨在帮助开发者系统地掌握RAG技术，从基础概念到生产部署，提供全面的系统化学习路径，涵盖LangChain、LlamaIndex、向量数据库、Agent、GraphRAG等前沿技术，最终能够独立构建企业级RAG应用。 Repository: vivy-yi/rag-tutorial GitHub stars: 113

Overview: This project is a simple example demonstrating how to use the Baidu API for speech recognition with Python. Key features: - PyAudio for audio recording - Numpy for calculation - Scipy for filtering - PyLab for waveform and spectrum plotting Tech stack: Python Why it matters: This project serves as a basic example for developers to understand and implement speech recognition using the Baidu API with Python. Repository: zthxxx/python-Speech_Recognition Primary language: Python GitHub stars: 115

Overview: ElevateAI provides a .Net core 6 SDK for developers to integrate Speech-to-text, behavioral analysis, and sentiment analysis capabilities into their projects. Key features: - Upload audio files - Retrieve Interaction ID and status - Get phrase-by-phrase and punctuated transcripts - Access AI results Tech stack: C# Why it matters: ElevateAI's SDK simplifies the integration of advanced AI features into .Net projects, making it easier for developers to leverage ElevateAI's powerful tools without needing to dive into complex API documentation or programming. GitHub stars: 114

Overview: ElevateAI provides a Speech-to-text API SDK for Python developers, enabling them to integrate voice interaction analysis into their projects. Key features: - AsyncClient.py: Defines a class for concurrent processing. - Client.py: Defines a class for standard interaction processing. - ElevateAI.py: Provides functions for interacting with the ElevateAI API. Tech stack: Python Why it matters: ElevateAI's SDK simplifies the integration of speech-to-text capabilities, making it easier for developers to incorporate voice analysis into their applications, enhancing user experience and insights.

Overview: WhisperInput is an experimental offline voice input panel and keyboard with punctuation for Android powered by Whisper AI and Kõnele components. Key features: - Works as a voice keyboard, input method editor, or voice input panel. - On-device speech recognition, offline. - Auto-start, auto-stop, audio cue option. Tech stack: Java Why it matters: This project offers a practical solution for users who prefer offline voice input, providing a convenient and efficient way to interact with their devices without internet connectivity. Repository: alex-vt/WhisperInput GitHub stars: 114 GitHub description: Offline voice input panel & keyboard with punctuation for Android, experimental, powered by Whisper AI & Kõnele components.

### Overview: FreeTraj is a tuning-free method for trajectory-controllable video generation based on pre-trained video diffusion models. ### Key features: - Supports trajectory-controllable video generation. - No need for tuning parameters. - Works with pre-trained video diffusion models. ### Tech stack: Python ### Why it matters: FreeTraj simplifies the process of generating trajectory-controllable videos, making it accessible to a wider audience without the need for complex parameter tuning.

### Overview: Frido is a research project focused on complex scene image synthesis, utilizing feature pyramid diffusion techniques. It aims to generate high-quality images from text, layouts, scene graphs, and labels, with applications in various fields such as image captioning, layout-to-image synthesis, and scene graph-to-image generation. ### Key features: - **Text-to-Image**: Synthesize images from textual descriptions. - **Layout-to-Image**: Generate images from layout data. - **Scene Graph-to-Image**: Create images from scene graphs. - **Label-to-Image**: Generate images from image labels. ### Tech stack: - **Programming Language**: Jupyter Notebook - **GitHub Stars**: 114 ### Why it matters: Frido is a significant contribution to the field of complex scene image synthesis, offering innovative solutions for generating high-quality images from diverse input formats. Its applications in various fields make it a valuable tool for researchers and developers looking to enhance image generation capabilities.