Projects

Overview: This project is a sample chatbot built using C# and the Microsoft Agent Framework, aiming to provide a practical learning experience for developers. Key features: - Implementing a chatbot using C# and Microsoft Agent Framework - Using multiple Large Language Models (LLMs) for conversation - Deploying the chatbot using Docker and Docker Compose Tech stack: C# Why it matters: This project serves as a practical example for developers to understand and practice building chatbots using modern technologies and frameworks.

Overview: MolDiff is a tailored diffusion framework designed to generate 3D drug-like molecules with a sampling success rate exceeding 99%, showcasing significant improvements over previous diffusion models. Key features: - Addressing the Atom-Bond Inconsistency Problem - Generating 3D drug-like molecules - Sampling success rate > 99% Tech stack: Python Why it matters: MolDiff offers a more effective solution for generating 3D molecules, particularly useful for applications such as pocket-based generation and linker generation. Repository: pengxingang/MolDiff Primary language: Python GitHub stars: 98 GitHub description: MolDiff: Addressing the Atom-Bond Inconsistency Problem in 3D Molecule Diffusion Generation README excerpt: # MolDiff: Addressing the Atom-Bond Inconsistency Problem in 3D Molecule Diffusion Generation (ICML 2023)

Overview: EngAce is a cutting-edge, generative AI-powered application designed to revolutionize the way Vietnamese learn English. This app provides a personalized learning experience tailored to each user's unique needs and preferences. Key features: - Intelligent Dictionary: Offers detailed explanations, idioms, phrasal verbs, and context-specific examples. - Personalized Assignment: Generates customized practice exercises to improve English skills. - Intelligent Writing Review: Provides detailed feedback and suggestions for enhancing writing skills. - Chat with AI Expert: Engages users in conversations and offers learning tips and advice. Tech stack: TypeScript Why it matters: EngAce streamlines English learning by combining AI with comprehensive features, making it an effective tool for Vietnamese learners.

Overview: This project presents a novel method for stereo depth estimation that can predict and restore noisy depth maps for transparent and specular surfaces, specifically for robotic manipulation in tabletop environments. Key features: - The method robustly predicts transparent (bottles) and specular (basin and cups) object depths in various environments. - It utilizes diffusion models for stereo depth sensing. - The project includes example code and pre-trained weights for both stereo and RGBD cameras. Tech stack: Python, PyTorch, Hugging Face Transformers, Accelerate, and other relevant libraries. Why it matters: This method offers improved depth sensing capabilities for robotic manipulation tasks, particularly in challenging environments with transparent and specular surfaces, enhancing the accuracy and robustness of robotic manipulation systems. Repository: songlin/d3roma GitHub stars: 98 GitHub description: [CoRL'24]A diffusion model-based stereo depth estimation framework that can predict and restore noisy depth maps for transparent and specular surfaces.

### Overview: TopK is a unified retrieval engine for hybrid search, vector search, multi-vector retrieval, keyword search, and managed inference in one API, built on object storage for 10x lower cost and massive scale. ### Key Features: - **Unified Retrieval Engine**: Store and query dense vectors, sparse vectors, multi-vector embeddings, and metadata in a single collection. - **Custom Ranking**: Filter by metadata, rank with custom expressions, and run managed inference. - **Object Storage**: 10x lower cost and massive scale. ### Tech Stack: Rust ### Why It Matters: TopK simplifies hybrid search, vector search, multi-vector retrieval, keyword search, and managed inference into a single API, making it easier to manage and scale. Repository: topk-io/topk GitHub stars: 97

Overview: This project is an unofficial implementation of "Simplifying, Stabilizing & Scaling Continuous-Time Consistency Models" for the MNIST dataset. Key features: - Consistency training for MNIST - Unofficial implementation of the paper - Short and easy-to-read code - Supports JVP (Jacobian-Vector Product) for most operations - Uses BatchNorm and Flash Attention with caution - Includes scripts for training a diffusion model and a toy model Tech stack: Python Why it matters: This implementation provides a practical way to apply consistency models to MNIST, making it easier to adapt and experiment with the techniques discussed in the paper.

Overview: 构屎 GOUJI 是一个纯 C++/CUDA 的三值 mini LLM，专为低性能设备设计，不依赖训练参数，仅通过零训练参数的工具路由层实现。项目旨在展示智能不必须来自大权重，通过零训练参数的微小模型，完成任何任务。 Key features: - 三值模型 {-1, 0, +1} - 2 bit/权重，零 Python - 143 个缺陷与 29 个变异测试记录 - 8 GB 笔记本 GPU 上跑 0.22 ms/次 Tech stack: C++/CUDA Why it matters: 本项目展示了智能不必须来自大权重，通过零训练参数的微小模型，完成任何任务，展示了设备端的命令路由层。 Repository: 3145544645/gouji GitHub stars: 99 GitHub description: 构屎 GOUJI — 纯 C++/CUDA 的三值 mini LLM。2 bit/权重，零 Python；联想记忆工具路由，零训练参数；143 个缺陷与 29 个变异测试记录在案。

### Overview: `flink-jpmml` is a library designed to enable real-time machine learning predictions using Apache Flink and the PMML standard. It allows for the efficient execution of models trained on PMML-based systems, leveraging the power of Apache Flink's streaming capabilities. ### Key Features: - **Dynamic Real-Time Predictions**: Enables real-time predictions with models trained on PMML. - **Backend Independence**: Models can be evaluated independently of the backend system. - **Stream Processing**: Designed for streaming scenarios, making it suitable for real-time applications. - **Easy to Use**: Simple to integrate and deploy, requiring minimal setup. ### Tech Stack: - **Scala**: The primary language used for development. - **Apache Flink**: The streaming engine used for processing real-time data. - **PMML**: The standard format for machine learning models. ### Why It Matters: - **Efficient Model Serving**: Utilizes Flink's capabilities for efficient model serving. - **Scalability**: Designed to handle large-scale data processing. - **Backend Independence**: Models can be evaluated independently of the backend system. - **Real-Time Processing**: Ideal for applications requiring real-time predictions. ### Repository: - **GitHub Repository**: `FlinkML/flink-jpmml` - **GitHub Stars**: 97 - **GitHub Description**: A library for dynamic real-time machine learning predictions built on top of PMML models and Apache Flink streaming engine. ### Prerequisites: - **PMML Models**: Any version of PMML (3.2 or above). - **Apache F

Overview: D3R is a novel anomaly detection network designed for real-world unstable multivariate time series data. It addresses the issue of drift in non-stationary environments by dynamically decomposing and reconstructing the data. Key features: - Dynamic decomposition of long-period multivariate time series - External control of information bottleneck using noise diffusion - End-to-end training of the model Tech stack: Python Why it matters: D3R significantly outperforms existing methods, achieving a 11% average relative improvement over the previous state-of-the-art models in various real-world datasets.

### Overview Self-supervised learning for speech recognition is implemented in this project. The paper "Self-supervised Learning with Random-projection Quantizer for Speech Recognition" by the same author is used as a reference. ### Key Features - Implementation of the paper "Self-supervised Learning with Random-projection Quantizer for Speech Recognition" - Utilization of PyTorch framework for deep learning - Code follows PEP-8 guidelines for readability and maintainability ### Tech Stack Python ### Why it Matters This project demonstrates the application of self-supervised learning techniques in speech recognition, showcasing the effectiveness of the proposed method.

### Overview: The FLUX 3 Video API is a Python wrapper for Black Forest Labs' FLUX 3 video API, providing a streamlined way to generate cinematic video clips with native synchronized audio from text prompts, static images, or existing video clips. The API supports seven FLUX 3 video endpoints, including Text-to-Video, Image-to-Video, and Video Extend, with fast draft modes and a video upscaler. All endpoints are live and available on [muapi.ai](https://muapi.ai/flux-3?utm_source=github&utm_medium=readme&utm_campaign=flux-3-video-api). ### Key Features: - **Text-to-Video**: Generate a video clip from a text prompt. - **Image-to-Video**: Create a video clip from a static image. - **Start-End-to-Video**: Generate a video clip between two keyframes. - **Video Extend**: Continue an existing video clip. - **Video Upscaler**: Enhance the resolution of video output beyond its native resolution. ### Tech Stack: - Python ### Why It Matters: The FLUX 3 Video API simplifies the process of generating video content with native synchronized audio, making it easier for developers to integrate multimodal generation into their projects. With a unified model, users can leverage the same weights for generating images, videos, and audio, streamlining workflows and reducing the need for multiple API calls. This makes the FLUX 3 Video API a valuable tool for anyone looking to create engaging and synchronized video content.

Overview: mPLUG is a state-of-the-art vision-language foundation model designed for effective and efficient cross-modal understanding and generation. It introduces novel cross-modal skip-connections to address the inefficiency and linguistic signal overload issues in cross-modal alignment. Key features: - Introduces effective and efficient vision-language architecture with novel cross-modal skip-connections. - Achieves state-of-the-art results on a wide range of vision language downstream tasks. - Supports pre-trained models and datasets for VQA and image captioning tasks. Tech stack: Python Why it matters: mPLUG offers a significant improvement in both cross-modal understanding and generation, making it a valuable tool for researchers and developers in the field of vision-language models. Repository: X-PLUG/mPLUG Primary language: Python GitHub stars: 97 GitHub description: mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections. (EMNLP 2022)

Overview: This project is a Vue2 Progressive Web App that performs synchronous speech recognition using Google Cloud Speech API, integrating with a Progressive Web App (PWA) for enhanced user experience. It leverages the power of Google Cloud Speech to transcribe spoken words into text. Key features: - Synchronous speech recognition - Progressive Web App (PWA) integration - Google Cloud Speech API integration Tech stack: JavaScript, Vue2, Google Cloud Speech API, Progressive Web App (PWA) Why it matters: This project showcases the integration of speech recognition technology with modern web development practices, making it an excellent example for developers looking to create PWA applications that can perform real-time speech recognition tasks. The project's focus on PWA capabilities highlights the importance of building applications that are accessible and performant on various devices and browsers.

### Overview: Fine-tuning of Stable Diffusion models is a project that involves the fine-tuning of diffusion models, specifically focusing on the Stable Diffusion models. This project aims to optimize and improve the performance of these models through fine-tuning techniques. ### Key Features: - Fine-tuning of Stable Diffusion models using Dreambooth or Low-rank Adaptation (LoRA). - Utilizes Hugging Face's Diffusers for easy updates and compatibility. - Implements low-rank adaptation for faster and more efficient fine-tuning. - Includes data augmentation techniques such as random cropping, flipping, and resizing. - Offers flexibility with adjustable parameters and settings. - Supports text-conditioning and classifier-free guidance sampling. - Tracks training loss and prior class loss separately for better visualization. - Provides the option to generate exponentially-weighted moving average (EMA) weights for the unet. - Inference with trained models uses Diffusers pipelines. ### Tech Stack: Python ### Why It Matters: This project is crucial for developers looking to enhance the performance and capabilities of Stable Diffusion models. By fine-tuning these models, developers can achieve better results in various applications, such as image generation and style transfer, leading to more efficient and effective use of these models in their projects.

### Overview: IDiff-Face is an innovative synthetic-based face recognition approach that leverages Fizzy Identity-conditioned Diffusion Models to generate realistic identity variations for training face recognition models. This method addresses the limitations of existing synthetic datasets and achieves state-of-the-art performance on various benchmarks, including 98.00% accuracy on the LFW benchmark. ### Key Features: - **IDiff-Face**: A novel approach using conditional latent diffusion models for generating synthetic identities with realistic variations. - **Synthetic Identity Generation**: The model generates realistic identity variations for training face recognition models, addressing limitations in intra-class diversity and cross-class discrimination. - **Performance Benchmark**: Achieved 98.00% accuracy on the LFW benchmark, far surpassing recent synthetic-based face recognition solutions. - **Pretrained Model**: Available pre-trained diffusion model weights for training and evaluation. ### Tech Stack: Python ### Why it Matters: IDiff-Face addresses the limitations of synthetic datasets in face recognition, providing a promising solution for future research on synthetic-based face recognition without the need for privacy-sensitive authentic data.

Overview: FLIP is an official implementation of the paper "FLIP: Cross-domain Face Anti-spoofing with Language Guidance" presented at ICCV 2023. It aims to improve face anti-spoofing performance across different domains by utilizing language guidance to enhance the robustness of cross-domain face spoofing detection. Key features: - Integrates multimodal contrastive learning with view-based image self-supervision and cross-modal image-text similarity constraints. - Implements a language-guided approach to align visual representations with natural language semantics for better cross-domain generalization. - Utilizes pre-trained ViT models for improved FAS performance without additional bells and whistles. Tech stack: Python Why it matters: This project offers a significant advancement in face anti-spoofing research by leveraging language guidance to enhance cross-domain performance, making it a valuable contribution to the field of computer vision and artificial intelligence.

### Overview: LLMariner is an extensible open source platform designed to simplify the management of generative AI workloads on Kubernetes. Built on OpenAI-compatible APIs, LLMariner enables efficient handling of both training and inference data within your clusters. ### Key Features: - **OpenAI-Compatible APIs**: Facilitates seamless integration with powerful tools. - **Kubernetes-Based Control Plane and Worker Planes**: Simplifies the management of generative AI workloads. - **Extensible Architecture**: Supports both single-cluster and multi-cluster setups. ### Tech Stack: Go ### Why It Matters: LLMariner provides a unified platform for managing generative AI workloads, making it easier to integrate with various tools and applications. Its extensible architecture allows for efficient handling of training and inference data, making it a valuable tool for AI developers and enthusiasts alike. ### Repository: Repository: llmariner GitHub Stars: 96 GitHub Description: Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.

Overview: Log10 is a Python library designed to enhance the accuracy of language models (LLMs) by logging all API calls, including those from third-party libraries that use OpenAI's API. This tool helps in monitoring and improving the performance of LLM-powered applications. Key features: - Log all OpenAI API calls, including third-party libraries using OpenAI's API. - Provide a Python LLM library wrapper for seamless integration. - Offer a Log10 LLM abstraction for easier integration. - Support Anthropic API calls. - Handle asynchronous LLM calls using the AsyncOpenAI and AsyncAnthropic clients. Tech stack: Python Why it matters: Log10 is essential for developers looking to improve the accuracy and performance of LLM-powered applications by providing comprehensive logging and monitoring capabilities.

Overview: This is an unofficial Python SDK and API server for Meta AI, designed to generate images, chat with Llama, and manage conversations. It uses cookie-based authentication and supports browser automation for a seamless user experience. Key features: - Image generation from text prompts - Text-based chat with Meta AI (Instant + Thinking modes) - Conversation management (list, new) - Media fetching by card ID - No API key required - Deployable REST API server - Python SDK for integration in projects Tech stack: Python Why it matters: This tool simplifies interaction with Meta AI's API, making it easier to integrate Meta AI's capabilities into various projects without requiring an API key.

Overview: This Python script simulates reverberation for close-talking speech using impulse responses, providing a realistic training dataset for distant speech recognition tasks. Key features: - Efficient speech reverberation simulation - Use of impulse responses from a dataset and close-talking speech signals - Supports multiple input and output folders - Accepts a text file containing paths to original speech and impulse response files - Compatible with Python 2.7 and Ubuntu Tech stack: Python Why it matters: This script offers a practical tool for researchers and developers working on distant speech recognition, providing a realistic dataset that can be used to train models and evaluate performance in noisy environments. GitHub stars: 97