Projects

Overview: ASRT语音识别系统的Python3 SDK和Demo客户端示例。 Key features: - ASRT语音识别系统调用样例 - Demo客户端实现 - Python3 SDK封装 Tech stack: Python 3.6+ Why it matters: 提供一个简洁的Python SDK和演示客户端，简化了ASRT语音识别系统的接入和开发过程。

### Overview: This repository contains code and data used in evaluating language models on the Brazilian University Admission Exams (ENEM). It includes comprehensive frameworks for evaluating GPT-4 and GPT-3.5 models on entrance exams, focusing on both textual and visual elements. The study aims to provide a realistic assessment of multimodal language models on Portuguese examinations. ### Key features: - Comprehensive framework for evaluating language models on entrance exams - Incorporates both textual and visual elements - Evaluates GPT-4 and GPT-3.5 models on ENEM 2022, 2023, and 2024 - Uses 3-shot prompts for evaluation - Textual descriptions of images from the orange ENEM exam booklet for accessibility ### Tech stack: Python ### Why it matters: This study not only reaffirms the capabilities of GPT-4 as the state of the art for handling complex multidisciplinary questions but also pioneers in offering a realistic assessment of multimodal language models on Portuguese examinations. The use of text captions transcribing visual content outperforms direct image use, suggesting that the vision model has room for improvement. Despite improvements afforded by images or captions, mathematical questions remain a challenge for these state-of-the-art models.

### Overview: OfficeBuddy is an agent that edits your Word and Excel files by interacting with Microsoft Office itself. It renders the documents through real Word and Excel, ensuring pixel-perfect accuracy and targeted repairs. The renderer is Word, not a lookalike engine, and the verification process is detailed and verifiable. ### Key Features: - **Real Interaction with Office**: OfficeBuddy edits files through real Microsoft Office applications, ensuring pixel-perfect accuracy. - **Pixel-Level Verification**: The verification process is pixel-level, with a red box indicating the exact changes made by the verifier. - **Structured Verifier**: The verification process is independent and structured, providing clear feedback on where the issue lies. ### Tech Stack: Python ### Why It Matters: OfficeBuddy is a significant improvement over traditional document editors, as it ensures that the edits are not just structural but pixel-perfect. This makes it a valuable tool for maintaining the integrity of documents, especially in professional settings where accuracy is paramount. ### Repository: richardChenzhihui/OfficeBuddy ### GitHub Stars: 53 ### GitHub Description: An agent that edits your Word and Excel files — then looks at them, through real Microsoft Office, to check its own work. It doesn't guess. It looks. ### Proof, Not Promises: OfficeBuddy is not a mockup. It renders a real `.docx` file on a real Mac, using real Word and Excel. The evidence trail is checked into `examples/harness-walkthrough/`, detailing every step of the process. ###

Overview: A web application that dynamically generates playable 'Spot the Difference' games from a single text prompt using a multimodal pipeline with Google's Gemini and Imagen models. Key features: - AI-Powered Game Creation - Dynamic Difference Generation - Client-Side Analysis - Manual Editing - Database Persistence - Interactive Gameplay - Debug Mode - Responsive Design Tech stack: TypeScript Why it matters: This project showcases the potential of AI in creating interactive and generative experiences, blending cutting-edge AI generation with classic computer vision techniques.

### Overview: This repository contains code for the CVPR'24 paper "GenHowTo: Learning to Generate Actions and State Transformations from Instructional Videos". It provides a framework for generating actions and state transformations from instructional videos, showcasing advancements in learning from videos for action generation. ### Key features: - **Code for CVPR'24 Paper**: Contains the implementation of the GenHowTo model and its evaluation. - **Model Weights**: Provides weights for generating state transformations (`GenHowTo-STATES-96h-v1`) and actions (`GenHowTo-ACTIONS-96h-v1`). - **Environment Setup**: Requires Docker to build the environment or manual installation of necessary packages. - **Prediction**: Runs the model on input images and prompts to generate corresponding state transformations and actions. ### Tech stack: Python ### Why it matters: This repository showcases advancements in learning from videos for action generation, providing a practical tool for researchers and developers interested in video-based action recognition and generation.

Overview: PNCC is an implementation of Power Normalized Cepstral Coefficients (PNCC) in Python, based on academic papers. This project provides a comprehensive feature set for analyzing audio signals. Key features: - PNCC implementation - Support for changing PNCC parameters - Installation instructions provided Tech stack: Python Why it matters: PNCC is a powerful tool for analyzing audio signals, offering advanced features for signal processing and analysis. Repository: supikiti/PNCC Primary language: Python GitHub stars: 54 GitHub description: A implementation of Power Normalized Cepstral Coefficients: PNCC README excerpt:

Overview: LLMTrace is a zero-code LLM security and observability proxy designed for OpenAI-compatible APIs, offering prompt injection detection, PII protection, and cost control with no SDK changes. Key features: - Prompt injection detection - PII scanning - Real-time monitoring and cost control Tech stack: Rust Why it matters: LLMTrace simplifies security and observability for AI and LLM applications, ensuring compliance and reducing operational overhead with minimal changes to the client-side code.

### Overview: TensorCast is a high-performance distributed tensor layer that manages model weights, KV cache, checkpoints, RL parameters, and other tensor state artifacts, enabling efficient state management across multiple nodes and processes. ### Key features: - **Tensor-native state management**: model weights, KV cache, checkpoints, RL parameters, activations, and tensor dictionaries are represented as artifacts with tensor-aware metadata, not opaque byte blobs. - **State lifecycle decoupled from workers**: tensor state can outlive the process that produced or consumed it. Serving workers can restart, scale out, or switch models by attaching to daemon-managed artifacts instead of rebuilding the same state from remote storage. - **High-performance zero-copy data path**: the Store Daemon owns local tensor memory and exposes CUDA IPC handles to clients, allowing multiple processes on the same node to share VRAM-resident tensors without an additional model load or process-local copy. - **Unified materialization pipeline**: disk loads, checkpoint restore, memory staging, network transfer, and tensor materialization are executed through a shared asynchronous data path, making DISK -> DRAM -> VRAM movement explicit, bounded, and reusable across workflows. - **Topology-aware P2P distribution**: TensorCast treats existing workers as a distributed replica pool. A node that already holds an artifact can serve downstream nodes over TCP or RDMA-capable paths, while the Global Store plans fanout with replica location, media tier, load, and network topology in mind. - **Tensor views and in-flight transformation**: consumers can request the tensor view they need, such as slices, tensor-parallel shards

Overview: This Python SDK provides a convenient interface to interact with the Yandex SpeechKit API, allowing developers to perform speech recognition and synthesis tasks. Key features: - Speech recognition and synthesis for both short and long audio files - Support for OAuth, API key, and JWT authentication - Documentation available at [Read the Docs](https://yandex-speechkit-lib-python.readthedocs.io/en/latest/index.html) Tech stack: Python, Yandex SpeechKit API Why it matters: This SDK simplifies the process of integrating Yandex SpeechKit into Python projects, making it easier to build applications that can recognize and generate speech.

### Overview: Deka is a human-in-the-loop workbench designed to align human intuition with semantic space. It converts a domain expert's intuitive query into a precise, reproducible, and scalable set of labelled results from a vector-search corpus. ### Key features: - **Probe**: Interactively tune hybrid (dense + learned-sparse) retrieval until relevance judgements converge. - **Harvest**: Treat validated FIT examples as queries and sweep the corpus for their embedding neighbourhood. - **Refine**: Distill the geometric cohort into an explicit, auditable language rubric, then judge a stratified sample with an LLM. - **Apply**: Train a low-cost classifier on the rubric-judged sample to label the full cohort at near-zero marginal cost. ### Tech stack: Python ### Why it matters: Deka is a demonstration of the harness philosophy, a general-purpose model and a busy domain expert working together to create a dependable retrieval-tuning instrument. The method generalises far beyond this particular corpus, UI, or problem.

### Overview: Ulysses is a sequence-parallel all-to-all implementation for PyTorch, optimized for high-performance distributed computing. It leverages GPU copy engines to achieve zero SM usage, resulting in significant performance gains over traditional distributed methods. ### Key features: - **No SMs.** Peer transfers are performed directly into symmetric memory addresses, eliminating the need for SMs. - **Relayout for free.** The sequence/head permutation is expressed as source and destination strides, reducing the number of kernel calls. - **Uneven shards.** Per-rank `seq_splits` and `head_splits` allow for flexible sequence padding, with even splits being the special case. ### Performance: - **8 GPUs, one node per row, same wheel and same `.so`:** - **H200:** 1.569 base, 0.860 Ulysses, 1.82× - **B200:** 1.194 base, 0.550 Ulysses, 2.17× - **B300:** 1.184 base, 0.548 Ulysses, 2.16× - **H100:** 1.616 base, 0.971 Ulysses, 1.66× ### Quick start: ```python import os import torch import torch.distributed as dist from fast_uly

**Overview:** This repository contains the TurboQuant 3-bit KV-cache quantization implementation for the LLaMA model, specifically focusing on the DualDeadline branch. This branch introduces component-staged expert streaming for offloaded mixture-of-experts inference, based on research from Marcel Butucea's paper. **Key features:** - TurboQuant 3-bit KV-cache quantization - DualDeadline component-staged expert streaming - Offloaded mixture-of-experts inference - GPU claim batch-1 expert matmuls via a persistent LRU cache - Gate/up projections needed for expert compute, down projections not consumed until arithmetic finishes - Cache hit rate of 84% for 64 experts/tensor - End-to-end performance improvement of 3-5% over monolithic approach **Tech stack:** C++ **Why it matters:** This implementation targets the regime where a discrete GPU's VRAM cannot hold the experts, providing a solution to the challenge of offloading LLaMA inference to a GPU without sacrificing performance.

### Overview: BLADE is an official PyTorch implementation of "BLADE: Block-Sparse Attention Meets Step Distillation for Efficient Video Generation." This repository separates the current training implementation from the original BLADE implementation, providing a portable example recipes and training guides for users. ### Key Features: - **Training Implementation**: Separate from the original BLADE implementation, offering a current training support. - **Attention Types**: Supports dense attention, original adaptive sparse attention, and flash_4 sparse attention. - **Models**: Models are available on Hugging Face at [GYP666/BLADE](https://huggingface.co/GYP666/BLADE). ### Tech Stack: Python ### Why It Matters: BLADE is designed to improve the efficiency of video generation by leveraging sparse attention techniques, making it suitable for applications requiring high computational efficiency while maintaining quality.

Overview: This project demonstrates how to implement speech recognition using the Web Speech API in JavaScript. Key features: - Speech recognition functionality - Integration with the Web Speech API - Code examples and explanations Tech stack: JavaScript Why it matters: This tutorial provides a practical guide for developers to understand and implement speech recognition in their JavaScript projects.

Overview: NoLLM Chat is a revolutionary AI interaction platform that enhances traditional LLM chat experiences with a versatile, visual interface. It enables users to interact with language models in creative ways, including node-based workflow creation, browser-based operation, and workflow automation. Key features: - 🎨 Visual Interface: Node-based workflow creation and management - 🔧 Browser-Based: Runs locally and free of charge with optional cloud extensions - 🔄 Workflow Automation: Create custom AI workflows tailored to your needs - 📝 Comprehensive Learning: Interactive tools for LLMs, prompt engineering, and vector databases Tech stack: TypeScript Why it matters: NoLLM Chat offers a more flexible and visual interface for AI interactions, enhancing creativity and user experience beyond basic chat. It empowers users to create custom workflows, experiment with AI components, and learn about AI technologies directly in their browser, making it accessible and easy to use. Repository: zrg-team/NoLLMChat GitHub stars: 54 GitHub description: Not-Only LLM Chat. An AI application that enhances creativity and user experience beyond just LLM chat. Noted: Seems it beta version of there is issue with DB please clear site Data in debug

Overview: This repository is dedicated to providing practical notebooks for LLM practitioners, covering various aspects of large language model engineering, including benchmarking, quantization, sharding, and inference techniques. Key features: - 1_understanding_llms_benchmarks.ipynb: Explanation of benchmarking LLMs. - 2_quantization_base.ipynb: Open Hugging Face models in 8-bit and 4-bit using BitandBytes. - 3_quantization_gptq.ipynb: Explore quantization in GPTQ format using auto-gptq. - 4_quantization_exllamav2.ipynb: Quantize a model from HF to exllamav2. - 5_sharding_and_offloading.ipynb: Sharding and offloading models to optimize resource usage. - 6_gguf_quantization_and_inference.ipynb: Quantize a model into GGUF and inferencing on OpenAI-compatible servers. - 7_gguf_split_and_load.ipynb: Split a GGUF quantized model into multiple parts for sharing. - 8_hqq_quantization.ipynb: Explore quantization using Half-Quadratic Quantization (HQQ). - 9_inference_big_model_cpu_plus_gpu.ipynb: Calculate RAM requirements and load Qwen/Qwen1.5-32B-Chat-GGUF on a system with 15GB VRAM. - a10_inference_llama3.ipynb: Demonstrate running LLama3-8B-Instruct half

### Overview: This repository contains a modified version of the Diffusion Transformer model, designed to combine the strengths of diffusion models and transformers. The model is based on the official implementation from MetaAI and has been trained on the CIFAR10 dataset for 200k steps. The code includes various techniques such as Flow Matching, Tailored SNR Samplers, ViT with registers, and Classifier-Free Guidance, aiming to enhance the sampling process and improve model performance. ### Key features: - **Diffusion Transformer Model**: A modified version of the Diffusion Transformer model from the paper "Scalable Diffusion Models with Transformers" by MetaAI. - **Flow Matching**: Uses the Flow Matching technique from the paper "Flow Matching for Generative Modeling" to sample images more efficiently. - **Tailored SNR Samplers**: Implements the Logit-Normal Sampling technique from the Research Paper "Stable Diffusion 3" to bias intermediate steps during training. - **ViT with Registers**: Adds register tokens to the Transformer Model from the paper "Vision Transformers need registers" to improve model performance. - **Classifier-Free Guidance**: Implements another forward function for classifier-free guidance for sampling. - **EMA (Exponential Moving Average)**: Uses EMA for model weights to stabilize training and improve performance. ### Tech stack: Python ### Why it matters: This repository is significant for researchers and developers interested in improving generative models, specifically in the areas of diffusion models and transformers. The techniques implemented, such as Flow Matching and Tailored SNR Samplers, offer new ways to enhance

Overview: CodeBlaze's Semantic Kernel Plugins is a collection of plugins designed to enhance the functionality of the Semantic Kernel, a powerful AI tool. Key features: - Ollama Connector - Neo4J Memory - Neo4J Cypher Plugin Tech stack: C# Why it matters: These plugins expand the capabilities of the Semantic Kernel by integrating with popular tools like Ollama, Neo4J, and Neo4J Cypher, making it more versatile and efficient for various use cases.

Overview: This repository contains the official project materials for the paper "Learned regularizations for multi-parameter elastic full waveform inversion using diffusion models" by Mohammad H. Taufik, Fu Wang, and Tariq Alkhalifah. Key features: - Reproducible notebook for the third synthetic test of the paper. - Scripts for diffusion model training, diffusion sampling, and EFWI. - Subsampled velocity models and trained diffusion model results. Tech stack: Jupyter Notebook Why it matters: This repository provides the necessary tools and materials to understand and replicate the results of the research paper, ensuring the reproducibility of the findings. Repository: DeepWave-KAUST/diffefwi GitHub stars: 53 GitHub description: Official repository for the "Learned regularizations for multi-parameter elastic full waveform inversion using diffusion models" paper. README excerpt: 

Overview: EmbeddedLLM is an API server designed for deploying LLMs on embedded devices like iGPUs, APUs, and CPUs, making it easy to integrate OpenAI-compatible servers on Windows, Linux, and macOS. Key features: - Support for multiple GPU architectures including CUDA, DirectML, IPExLLM, and OpenVINO. - Compatibility with various hardware accelerations such as x86, x64, and ARM64. - Support for models like Gemma, Llama, Mistral, Phi, and Yi. - Prebuilt binary (alpha) for Windows executable. Tech stack: Python. Why it matters: EmbeddedLLM simplifies the deployment of complex AI models on low-power embedded devices, making it ideal for edge computing and IoT applications where traditional cloud solutions are not feasible. Repository: EmbeddedLLM/embeddedllm Primary language: Python GitHub stars: 54 GitHub description: EmbeddedLLM: API server for Embedded Device Deployment. Currently support CUDA/OpenVINO/IpexLLM/DirectML/CPU GitHub stars: 54