Projects

Overview: TTSizer automates the process of converting raw audio and video files into high-quality Text-To-Speech datasets, making it easier for developers to create production-ready TTS datasets. Key features: - End-to-End Automation - Advanced Multi-Speaker Diarization - State-of-the-Art Models - Quality Control - Fully Configurable Tech stack: Python Why it matters: TTSizer streamlines the tedious task of creating TTS datasets, saving time and effort for developers. It ensures high-quality output with advanced models and quality control features. Repository: taresh18/TTSizer GitHub stars: 149

Overview: HiveMind Protocol is a local-first, privacy-preserving architecture for agentic RAG, designed to compete with xAI and OpenAI and push towards AGI. It is a framework for HiveMind LLM 3.0, a next-generation model. Key features: - Local-first, privacy-preserving architecture for agentic RAG - Treats memory as portable, hot-swappable artifacts called EMUs (Encapsulated Memory Units) - Next-generation model designed to compete with xAI and OpenAI - Framework for HiveMind LLM 3.0 Tech stack: TypeScript Why it matters: HiveMind Protocol offers a unique approach to privacy-preserving AI, focusing on local processing and hot-swappable memory units, making it a promising solution for privacy-conscious applications. Repository: virtuehearts/HiveMind GitHub stars: 148

Overview: Scalax is a library designed to help developers easily scale up JAX-based machine learning models by automatically handling sharding annotations for JAX's JIT compiler. Key features: - Automatically generates sharding annotations for JAX models - Supports various sharding strategies, including Fully Sharded Data Parallelism (FSDP) - Provides utilities to simplify the process of scaling up JAX code - Integrates with JAX's JIT compiler for efficient performance Tech stack: Python, JAX, PyTorch, TensorFlow Why it matters: Scalax simplifies the process of scaling up JAX-based models by automatically generating sharding annotations, making it easier for developers to parallelize their models across multiple GPUs/TPUs without worrying about the underlying pytree structure. This reduces the complexity and effort required for scaling, allowing developers to focus on the model's functionality and training efficiency.

### Overview: OneGen is a unified framework designed for fine-tuning Large Language Models (LLMs) on generation, retrieval, or hybrid tasks. By integrating generation and retrieval into a single pass, OneGen enables LLMs to perform both tasks efficiently. ### Key Features: - **Unified Generation and Retrieval**: Combines generation and retrieval into a single forward pass. - **Cross-Entropy and Contrastive Loss**: Trains the model using cross-entropy loss for generating tokens and contrastive loss for retrieval tokens. - **Efficient Inference**: Reduces inference costs by avoiding two forward passes and utilizing cached kv-parameters. ### Tech Stack: - Python ### Why It Matters: OneGen simplifies the training and inference process of LLMs by integrating generation and retrieval into a single pass, thereby reducing computational overhead and improving efficiency. ### Repository: Repository: zjunlp/OneGen GitHub Stars: 149 GitHub Description: [EMNLP 2024] OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMs.

### Overview: RecruitOps 秋招工作台是一个本地优先的招聘智能和应用运营代理，旨在帮助校园招聘过程中发现、筛选岗位、补全JD、匹配评分、辅助投递、维护投递进度、处理招聘邮件，并将笔试、面试和待办事项统一管理。 ### Key features: - **岗位发现与抓取**：从招聘聚合来源获取公司和校招入口，持续维护可抓取的官网来源。 - **岗位详情补充**：抓取岗位列表，先按标题关键词筛选，再补全需要分析的岗位详情。 - **简历解析与评分**：从简历生成技能、项目摘要和岗位关键词，按个人背景给岗位评分并解释优势与不足。 - **招聘浏览器**：在软件内打开招聘官网、保留登录状态、扫描表单并进行简历闪填。 - **投递记录与进度维护**：手动登记或从内置浏览器保存投递，维护已投递、笔试、面试、Offer 和已挂阶段。 - **官网进度复核**：只复核未挂岗位，读取官方投递记录页并保留阶段证据，不用低优先级信息覆盖已确认进度。 - **招聘邮件处理**：通过只读 IMAP 同步招聘邮件，识别投递、测评、笔试、面试和拒信。 - **日程与待办管理**：将有明确时间的招聘事项加入日程，将时间未定的事项保留为待办。 - **求职助理**

### Overview: Image Prompt Library is a local-first image prompt library designed for developers. It allows users to save, search, and organize images with their prompts, sources, and notes. The app supports optional ChatGPT and Grok connections for enhanced functionality. ### Key Features: - **Image Saving**: Users can add images and prompts to cards, giving them titles and optional notes and source links. - **Collections and Tags**: Users can create collections and tags to organize their images and prompts. - **Local Storage**: The app uses local SQLite and local image files, ensuring no need for an account. - **Optional ChatGPT and Grok Integration**: Users can generate images and suggest titles using ChatGPT and Grok, with requests running on the provider's servers. ### Tech Stack: - Python 3.10+ ### Why It Matters: Image Prompt Library simplifies the process of saving and managing images with their associated prompts, sources, and notes. It provides a visual interface for browsing and organizing images, making it easy for developers to manage their projects efficiently.

### Overview: CLIP-GEN: Language-Free Training of a Text-to-Image Generator with CLIP CLIP-GEN is a method that allows training a text-to-image generator without requiring any image or text training samples. It leverages the powerful representation capabilities of CLIP to train an image generator directly from images alone. The basic principle is that CLIP-GEN first trains a VQ-GAN to map images to a discrete space, and then trains a GPT model to map CLIP embeddings to the same discrete space. During inference, it can map text to the discrete space using the same method as the image, then decode it into an RGB image. ### Key Features: - **Text-to-Image Generation**: CLIP-GEN generates images from text inputs. - **Language-Free Training**: It trains directly from images without requiring any text or image training data. - **Discrete Space Mapping**: Maps text and images to a discrete space using CLIP embeddings. ### Tech Stack: Python ### Why it Matters: CLIP-GEN is a groundbreaking approach that opens up new possibilities for image generation by eliminating the need for extensive image and text training data. This method could significantly reduce the amount of labeled data required for training and potentially enable more efficient and effective image generation. ### Repository: HFAiLab/clip-gen - **Primary Language**: Python - **GitHub Stars**: 148 - **GitHub Description**: CLIP-GEN: Language-Free Training of a Text-to-Image Generator with CLIP - **README Excerpt**: ```

Overview: MOSS-Speech is an end-to-end speech-to-speech language model that generates spoken responses without requiring text guidance. This model directly interacts with the user through speech, overcoming limitations of text-based models and leveraging the capabilities of pretrained text language models. Key features: - True speech-to-speech modeling - Layer-splitting architecture integrating modality-specific layers on top of pretrained text language model backbones - Frozen pre-training strategy to preserve LLM abilities while extending to speech modality - State-of-the-art performance excelling in spoken question answering and speech-to-speech tasks Tech stack: Python Why it matters: MOSS-Speech offers a more natural and efficient way of speech interaction by eliminating the need for text guidance, leveraging the strengths of pretrained text models, and enabling more fluent and contextually relevant speech responses. GitHub stars: 144 Repository: OpenMOSS/MOSS-Speech

### Overview: This project investigates the effectiveness of SFT (Sequence-to-Sequence with Attention) and RL (Reinforcement Learning) in training large vision-language models. The research explores how these approaches can be used to improve the reasoning capabilities of models, particularly in the context of multimodal reasoning tasks. ### Key Features: - **SFT (Sequence-to-Sequence with Attention)**: Trained on the VLAA-Thinking-SFT-126K dataset. - **RL (Reinforcement Learning)**: Trained on the VLAA-Thinking-GRPO-25K dataset. - **VLAA-Thinking Dataset**: Contains a diverse set of multimodal data for training and evaluation. - **Examples**: Demonstrates how the model can be used to answer questions across different domains. ### Tech Stack: - Python ### Why It Matters: This project contributes to the ongoing debate on the best training methods for large vision-language models, highlighting the potential advantages of both SFT and RL in enhancing model performance. The results suggest that both approaches can achieve state-of-the-art (SOTA) performance on multimodal reasoning tasks, making it a valuable resource for researchers and practitioners in the field of AI and machine learning.

### Overview: FreeScale is a novel tuning-free method designed for higher-resolution visual generation. It leverages the advancements in diffusion models to achieve 8k image generation without the need for tuning, making it highly efficient and scalable. ### Key Features: - **Automatic Resolution Scaling**: Automatically adjusts the resolution of visual generation tasks without manual tuning. - **Cosine Scale Fusion**: Combines detail scales from different stages to maintain consistency across the entire image. - **Flexible Control for Detail**: Allows users to control the level of detail in the final image through various parameters. ### Tech Stack: - Python ### Why It Matters: FreeScale simplifies the process of generating high-resolution images, making it accessible to a broader audience without the need for complex tuning, thus reducing the learning curve and computational overhead.

Overview: NOS is a fast and flexible PyTorch inference server designed for easy deployment on any cloud or AI hardware. Key features: - 🧑‍💻 Easy-to-use: Built for PyTorch and optimized for production use without compromising on developer experience. - 🥷 Multi-modal & Multi-model: Serve multiple foundational AI models, including LLMs, diffusion, embeddings, and object detection simultaneously. - ⚙️ HW-aware Runtime: Deploy PyTorch models effortlessly on modern AI accelerators (NVIDIA GPUs, AWS Inferentia2, AMD, and even CPUs). - ☁️ Cloud-agnostic Containers: Run on any cloud (AWS, GCP, Azure, Lambda Labs, On-Prem) with our ready-to-use inference server containers. Tech stack: Python Why it matters: NOS simplifies the deployment of PyTorch models on any hardware, making it ideal for developers and researchers looking to quickly and efficiently run AI inference services.

### Overview: Edit4I is an open-source alternative to Cursor, Windsurf, Local RAG, Bring your own LLM, and Agents. It is written in Go and leverages Svelte for the frontend and Monaco for the editor component. The project aims to enhance the coding experience by providing intelligent context understanding and seamless workflow integration. ### Key Features: - **Automatic Context Generation**: Intelligently understands your codebase and provides relevant context for smarter coding assistance. - **Treesitter Based Document Symbols**: Advanced code parsing and symbol recognition for improved code navigation and understanding. - **Git Based Workflow**: Seamless integration with Git for efficient version control and collaboration. - **Terminal Execution and Debugging**: Built-in terminal support for running commands and debugging directly within the editor. - **Pinned Prompts**: AI convention-based system for consistent and customizable AI interactions. ### Tech Stack: - **Frontend**: SvelteJS - **Backend**: Go - **Backend Tools**: Wails for native desktop integration ### Why It Matters: Edit4I combines modern web technologies with powerful AI capabilities to create a high-performance editor that enhances productivity and reduces errors. It is designed to be a better alternative to existing editors, offering a more intuitive and efficient coding experience. ### GitHub Stars: 148 ### Repository: The repository for Edit4I is hosted on GitHub and is currently in the early stages of development. It is not suitable for production use but is open for contributions and community feedback.

Overview: Coco AI Server is a comprehensive enterprise search and collaboration platform designed to unify and manage various enterprise applications, data, and workflows. Key features: - Unified Search: Instantly locate anything across Google Workspace, Dropbox, Confluence, GitHub, and more. - Enhanced Collaboration: Empower team communication and innovation with centralized, easily accessible knowledge. - Gen-AI Insights: Harness AI to unlock your team’s unique expertise, delivering actionable insights and smarter decisions. - Effortless Integration: Built for simple setup, enterprise-grade security, and unmatched scalability. - Easy to Embed: Easily embed the Coco AI capacities into your website within 1 minute. Tech stack: TypeScript Why it matters: Coco AI Server is a crucial component of the Coco AI project, serving as the heart of a unified collaboration workspace, seamlessly integrating and managing enterprise applications, data, and workflows. Repository: infinilabs/coco-server Primary language: TypeScript GitHub stars: 148 GitHub description: 🥥 Coco AI Server - Search, Connect, Collaborate, AI-powered Enterprise Search, all in one space.

### Overview: This book is designed to guide readers through the creation of various forms of generative AI content, including shapes, numbers, images, text, and music, using PyTorch. It starts with foundational deep learning projects and gradually advances to more complex models like Transformers and Diffusion Models. ### Key Features: - Comprehensive project-based learning - PyTorch-based deep learning projects - Coverage of Generative Adversarial Networks (GANs), CycleGANs, and Autoencoders (AEs) - Detailed explanations and step-by-step guides ### Tech Stack: PyTorch ### Why it Matters: This book serves as a valuable resource for anyone looking to become an expert in generative AI, providing a clear, up-to-date, and comprehensive guide to creating innovative content using PyTorch and generative models.

Overview: GraspGPT is an official code implementation of GraspGPT and FoundationGrasp, leveraging semantic knowledge from a large language model for task-oriented grasping. This project is based on the paper "GraspGPT: Leveraging Semantic Knowledge from a Large Language Model for Task-Oriented Grasping" and the dataset "Language Augmented TaskGrasp (LA-TaskGrasp)". Key features: - GraspGPT implementation in PyTorch - Faster inference speed and easier training - Pre-trained models available for different classes and task splits Tech stack: PyTorch, PyTorch Lightning, CUDA 11.3, Python 3.7 Why it matters: GraspGPT aims to improve the accuracy and efficiency of task-oriented grasping by leveraging semantic knowledge from large language models, making it useful for applications requiring precise and context-aware grasping tasks. Repository: mkt1412/GraspGPT_public Primary language: Python GitHub stars: 148 GitHub description: code implementation of GraspGPT and FoundationGrasp

### Overview AXIOM is a highly efficient voice agent designed for robotics and edge environments, featuring real-time speech processing, intelligent intent classification, context-aware responses, and a clean text-to-speech output. It optimizes for low latency and minimal VRAM usage, making it suitable for devices with 4GB VRAM and GTX 1650 GPUs. ### Key Features - **Real-time Voice Interaction**: Seamless WebSocket communication for instant voice processing. - **Intelligent Intent Classification**: Secure `.safetensors` model for precise intent recognition. - **Context-Aware Responses**: Semantic RAG with 2,116+ template responses. - **3D Interactive UI**: WebGL-based carousel for visual equipment interaction. - **Multi-turn Conversation**: FIFO history management for contextual understanding. - **Clean TTS Output**: Phonetic + minimal safe correctors. ### Tech Stack - **Python 3.10+** - **FastAPI** - **Kokoro, Sherpa, SetFit** ### Why It Matters AXIOM is optimized for low latency and minimal VRAM, making it ideal for edge robotics and robotics environments. It leverages modern ML techniques and efficient inference pipelines to deliver real-time voice interaction, intelligent intent classification, and context-aware responses. The use of `.safetensors` models ensures secure and efficient model management, and the 3D interactive UI enhances user experience with visual equipment interaction. The low VRAM usage and zero-copy inference make it suitable for devices with 4GB VRAM and GTX 1650

Overview: This repository is dedicated to training a LoRA (Low-Rank Adaptation) model for the LLaMA (1 or 2) models on HuggingFace using 8-bit or 4-bit quantization. It's primarily for research purposes, focusing on LLaMA 1. Key features: - Training a LoRA model for LLaMA 1 - 8-bit or 4-bit quantization - Research only, not for commercial use Tech stack: Python Why it matters: This repository is essential for researchers and developers looking to fine-tune and adapt LLaMA models for specific tasks, leveraging advanced quantization techniques for improved performance and energy efficiency. Repository: serp-ai/LLaMA-8bit-LoRA Primary language: Python GitHub stars: 148

### Overview: LLM.js is a zero-dependency library that provides access to hundreds of Large Language Models (LLMs) in Node.js and the browser. It supports all important features for production-ready LLM apps, including chat, streaming responses, reasoning models, tools, parsers, token usage, and more. The library is compatible with various LLM providers such as OpenAI, Google, Anthropic, Groq, Ollama, and XAI, and supports dynamic model lists and cost tracking. ### Key features: - **Zero-Dependency**: No external dependencies required. - **Universal Compatibility**: Works with hundreds of LLMs. - **Chat**: Integrates message history for seamless conversations. - **Stream**: Provides instant responses with thinking and tool integration. - **Thinking**: Supports reasoning models for more intelligent responses. - **Tools**: Allows calling custom functions. - **Parsers**: Supports JSON, XML, and code block parsing. - **Token Usage**: Tracks input and output tokens on every request. - **Model List**: Provides a dynamic list of the latest models. - **Cost Usage**: Tracks the cost of every request. - **Options**: Controls temperature, max tokens, etc. - **Abort Requests**: Can be aborted mid-response. - **TypeScript**: Clean code with TypeScript. - **Tests**: Good coverage with tests. - **Node.js and Browser**: Supported on both environments. - **Zero-Dependencies**: No external libraries required. - **MIT License**: Open-source with MIT license. ### Why it matters: - **Compatibility**: Solves the compatibility

### Overview: Squeezeformer is a PyTorch implementation of the "Squeezeformer: An Efficient Transformer for Automatic Speech Recognition" paper presented at NeurIPS 2022. This model incorporates the Temporal U-Net structure to reduce the cost of multi-head attention modules on long sequences, along with a simpler block structure of feed-forward modules followed by multi-head attention or convolution modules. ### Key features: - **Squeezeformer Model**: Implements the Squeezeformer architecture for automatic speech recognition. - **Temporal U-Net Structure**: Reduces the computational cost of multi-head attention modules on long sequences. - **Simplified Feed-Forward Module**: Uses a simpler block structure for feed-forward modules, enhancing efficiency. ### Tech stack: - PyTorch - Apache License 2.0 ### Why it matters: Squeezeformer is an efficient transformer model designed specifically for automatic speech recognition, offering improved performance and reduced computational resources compared to other state-of-the-art models.

Overview: Document Chat System is a production-ready application that integrates intelligent document management with advanced AI-powered conversations. It allows users to upload and process documents in various formats, then engage in natural language interactions with semantic search capabilities and multi-provider AI support. Key features: - 🚀 Production-ready - 🔍 Semantic search with vector embeddings - 💬 Natural language Q&A - 📄 Multi-format support - 🎨 Beautiful modern UI - 🔐 Enterprise security Tech stack: TypeScript, Next.js, React, Prisma, Tailwind CSS Why it matters: This platform provides a seamless and efficient way to interact with documents, leveraging cutting-edge AI models to enhance productivity and accessibility.