Projects

Overview: This project explores molecular geometry pretraining using the SE(3)-invariant denoising distance matching approach, focusing on pretraining with Molecule3D and fine-tuning on QM9, MD17, LBA, and LEP datasets. The GeoSSL-DDM framework is implemented and tested, providing implementations for eight pretraining baselines and GeoSSL-DDM itself. Key features: - Implementations of eight pretraining baselines and GeoSSL-DDM for molecular geometry pretraining. - Pretraining with Molecule3D and fine-tuning on QM9, MD17, LBA, and LEP datasets. - Pretraining with eight pretraining baselines and GeoSSL-DDM. - Pretraining with GeoSSL-DDM framework, including GeoSSL-RR, GeoSSL-InfoNCE, GeoSSL-EBM-NCE, and GeoSSL-DDM. Tech stack: Python, PyTorch, NetworkX, SciPy, NumPy, PyG, PyTorch Geometric, Atom3D, Biopython, ase, Rdkit, OpenMM, OpenBabel, PyTorch Geometric, PyTorch Lightning, PyTorch Lightning, PyTorch Lightning, PyTorch Lightning, PyTorch Lightning, PyTorch Lightning, PyTorch Lightning, PyTorch Lightning, PyTorch Lightning, PyTorch Lightning, PyTorch Lightning, PyTorch Lightning, PyTorch Lightning, PyTorch Lightning, PyTorch Lightning, PyTorch Lightning, Py

Overview: This repository contains code for our paper "Generalized Orderless Pooling Performs Implicit Salient Matching" published at ICCV 2017. It includes scripts for fine-tuning a pre-trained VGG16 model with our presented alpha-pooling approach. Key features: - Fine-tuning of a pre-trained VGG16 model with alpha-pooling - Visualization of decisions made by alpha-pooling approaches - Identification of parts of training images that influence the prediction of a test image - Comparison of alpha-pooling with average and bilinear pooling - Analysis of the difference in performance between average and bilinear pooling - Efficient computation of outer product using a compact bilinear layer Tech stack: Jupyter Notebook Why it matters: This repository provides a comprehensive solution for fine-grained recognition using alpha-pooling, showcasing improved performance over traditional pooling methods.

### Overview: This repository provides **PAT** (Prefix-Aware Attention), a high-performance implementation designed to optimize the decoding attention phase in transformer models. PAT identifies complex shared prefix patterns within batched sequences and schedules shared prefixes into separate CTA computations, significantly reducing KV cache reads, which is the primary bottleneck in attention computation during LLM decoding. ### Key Features: - **Prefix-Aware Attention**: PAT identifies complex shared prefix patterns within batched sequences and schedules shared prefixes into separate CTA computations. - **Resource-Efficient Multi-Tile Kernel**: This approach reduces KV cache reads, which is the primary bottleneck in attention computation during LLM decoding. - **NVIDIA A100 and H100 GPUs Support**: PAT is optimized for NVIDIA A100 and H100 GPUs. ### Tech Stack: Python ### Why It Matters: PAT is essential for accelerating LLM decoding, particularly in systems where performance and resource efficiency are critical. By reducing KV cache reads, PAT significantly improves the efficiency and performance of LLM models, making them more scalable and faster.

### Overview: This repository contains 252 curated Seedream 5.0 prompts across 9 categories, grouped by use case. The prompts are accompanied by preview images to help you see the results before running them. ### Key Features: - **Portrait & Avatar**: 28 prompts - **Product & E-commerce**: 28 prompts - **Poster & Typography**: 28 prompts - **Infographic & Diagram**: 28 prompts - **Anime & Illustration**: 28 prompts - **3D & Material**: 28 prompts - **Food & Lifestyle**: 28 prompts - **Architecture & Interior**: 28 prompts - **Nature & Landscape**: 28 prompts ### Tech Stack: The repository uses Seedream 5.0 prompts and preview images to provide a curated collection of prompts for various design and illustration purposes. ### Why it Matters: This collection of Seedream 5.0 prompts is useful for designers, illustrators, and anyone looking to enhance their work with cinematic lighting and rich color grading. The preview images help users visualize the results before applying the prompts, making it easier to choose the right effect for their projects.

Overview: HoneyBee is a comprehensive multimodal AI framework designed specifically for oncology research and clinical applications. It seamlessly integrates and processes diverse medical data types—clinical text, radiology images, pathology slides, and molecular data—through a unified, modular architecture. Built with scalability and extensibility in mind, HoneyBee empowers researchers to develop sophisticated AI models for cancer diagnosis, prognosis, and treatment planning. Key features: - Multimodal data support: clinical text, radiology (DICOM/NIFTI), pathology (WSI), and molecular data - 3-layer modular architecture: clean separation between loaders, processors, and embedding models - Clinical NLP pipeline: OCR, cancer entity extraction, temporal parsing, and medical ontology mapping - Whole Slide Image processing: tissue detection, patch extraction, stain normalization, and quality filtering - State-of-the-art embedding models: GatorTron, BioBERT, PubMedBERT, UNI, REMEDIS, RadImageNet, and more - Cross-modal integration: unified patient-level representations from multiple data modalities - Survival analysis: Cox PH, Random Survival Forest, and DeepSurv - Similar patient retrieval: find patients with matching clinical profiles - Interactive visualization: t-SNE dashboards for embedding exploration - GPU-accelerated: CuCIM backend for WSI processing with OpenSlide fallback Why it matters: HoneyBee is a cutting-edge solution for integrating and processing diverse medical data types in oncology, enabling researchers to develop sophisticated AI models for cancer diagnosis, prognosis, and treatment planning

Overview: This repository contains experiments with Hugging Face's transformers, focusing on text generation and various machine learning models. It includes models like GPT-J 6B, HuBERT, and others. Key features: - GPT-J 6B: A transformer model with a PPL of 3.99 and an ACC of 69.7%. - HuBERT: Self-supervised representation learning for speech recognition and generation. - Zeroshot: Zero-shot text classification. - NRoT: Numerical reasoning over text (NRoT) pretrained models. - ViT: Vision Transformer pre-trained on ImageNet. - BigBird: Google sparse-attention based transformer. - MSMarco: Semantic search and retrieval pre-trained on ImageNet. - Luke: RoBERTa model for named entity recognition, question answering, and entity typing. - ColBERT: Model for passage search via context-based late interaction over BERT. - AudioSeg: Pyannote audio segmentation and speaker diarization. - ASR: Automatic speech recognition. - GPT-Neo: EleutherAI's replication of the GPT-3 model. - BERT: Masked Language Modeling, Next Sentence Prediction, and Extractive Question Answering. - Summarization: Text summarization. - Translation: Multiple languages translation. - Sentiment: Sentiment analysis. - Emotions: Emotions detection. - Pokémon: Pokémon generator based on Russian RuDALL-E. Why it matters: This repository offers a collection of Hugging Face models for various text generation tasks and machine

### Overview: NarrateClaude is a hands-free voice loop for Claude Code on macOS, Linux, and Windows. It transcribes your speech on your own machine using whisper.cpp, types the text into the Claude session, and responds with your cloned voice. This app works offline and does not require an internet connection or cloud storage. ### Key features: - **Transcribes Speech**: Transcribes your spoken words using whisper.cpp. - **Types Text**: Types the transcribed text into the Claude session. - **Cloned Voice**: Responds with your cloned voice, providing a personalized interaction. - **Works Offline**: No internet or cloud storage required. ### Tech stack: Python ### Why it matters: NarrateClaude is a hands-free solution for Claude Code on macOS, Linux, and Windows, making it an essential tool for ambient computing. It allows users to interact with Claude Code without the need for an internet connection, ensuring privacy and security.

### Overview: The deepspeech-cleaner project is a comprehensive tool designed to simplify the process of collecting, cleaning, and sorting datasets for Mozilla's DeepSpeech framework. It streamlines the entire workflow, making it easier to manage and utilize datasets across multiple languages. ### Key Features: - **Multi-Language Support**: The tool supports a wide range of languages including German, Polish, Spanish, Italian, French, Ukrainian, Russian, Dutch, Portuguese, English, Hungarian, Norwegian, Romanian, Swedish, Turkish, Czech, Danish, Finnish, Estonian, Greek, Icelandic, Latvian, Lithuanian, Serbian, Slovak, Slovenian, Albanian, Bosnian, Bulgarian, Croatian, Danish, Finnish, Estonian, Greek, Icelandic, Latvian, Lithuanian, Serbian, Slovak, Slovenian, Albanian, Bosnian, Bulgarian, Czech, Danish, Finnish, Estonian, Greek, Icelandic, Latvian, Lithuanian, Romanian, Swedish, Turkish, Czech, Danish, Finnish, Estonian, Greek, Icelandic, Latvian, Lithuanian, Serbian, Slovak, Slovenian, Albanian, Bosnian, Bulgarian, Croatian, Danish, Finnish, Estonian, Greek, Icelandic, Latvian, Lithuanian, Romanian, Swedish, Turkish, Czech, Danish, Finnish, Estonian, Greek, Icelandic, Latvian, Lithuanian, Romanian, Swedish, Turkish, Czech, Danish, Finnish, Estonian, Greek, Icelandic, Latvian, Lithuanian, Romanian, Swedish, Turkish,

### Overview: This project applies LoRAs trained on Anima-2.9B to the base model Anima-2.9B, remapping the layer indices correctly to ensure seamless integration. The patch is designed to be applied at load time without workflow changes, making it ideal for developers looking to enhance their models without altering existing workflows. ### Key features: - Correctly remaps LoRA layer indices to match the expanded layout of Anima-2.9B. - Applies LoRAs trained on Anima-2.9B to Anima-2.9B without workflow changes. - No workflow nodes added, ensuring seamless integration. ### Tech stack: - Python ### Why it matters: This patch ensures that LoRAs can be seamlessly applied to a model that has been expanded from the base model, maintaining the correct layer correspondence and functionality. It resolves issues where LoRAs applied to a 28-block model would silently hit the wrong layers, leading to performance degradation or non-functional behavior.

Overview: Z-Image Turbo API, also known as `z-image-turbo`, offers a fast text-to-image service on the APIMart endpoint, priced per image. It supports a wide range of models and resolutions, with flexible parameters and real-time billing. Key features: - Supports 300+ models and resolutions - Pricing per image, with a minimum of $0.01 - OpenAI-compatible endpoint, no subscription required - Real-time billing and task tracking - Asynchronous processing, no need for bespoke integration Tech stack: Python Why it matters: Z-Image Turbo provides a cost-effective and flexible solution for generating high-quality images from text prompts, making it ideal for teams looking to integrate AI into their projects without the complexity of setting up a dedicated API. Repository: t9hid1p6ue2a/z-image-turbo-api-ko Primary language: Python GitHub stars: 47

### Overview: A Plug-and-Play Command Center for DGX Spark Fleets ### Key Features: - Real-time monitoring of GPU, CPU, memory, temperature, power, and fan usage on DGX Spark and GPU hosts. - Live monitoring of fabric switch temperatures, fans, and power usage. - Model server monitoring for vLLM, SGLang, and llama.cpp. - ComfyUI render lanes showing GPU utilization, queue depth, and memory usage. - Tokens served per model, with a daily count. - Optional extras: GPU clock caps, model server links, and Elgato key lights. - Built-in chat assistant that displays the same live data as the board. ### Tech Stack: TypeScript ### Why It Matters: This command center provides a comprehensive and easy-to-use solution for monitoring and managing DGX Spark fleets and GPU boxes, ensuring efficient resource utilization and quick access to critical information.

Overview: VectorDB is a standalone vector database built in Rust, designed for semantic search and RAG. It includes an embeddable engine, HTTP API, metadata filtering, and persistent storage. Key features: - Multiple databases, collections, explicit schemas, and partitioning - Int64 / VarChar primary keys, scalar fields, JSON, Array, and dynamic fields - Batch insert / upsert, query / get / delete, sorting, and pagination - Support for FloatVector, SparseFloatVector, BinaryVector, and various distance metrics - Multi-vector search fusion, including RRF and normalized weighted ranking - SQLite WAL with FULL synchronization, batch transactions, and crash recovery - Embeddings generated by applications, with logic, filters, and index algorithms written in Rust - Embedded Python (Alpha) package for compatibility with CPython 3.9+ Tech stack: Rust Why it matters: VectorDB offers a robust, embeddable solution for semantic search and RAG, with features that are not directly available in popular libraries like Milvus. Its Rust implementation ensures high performance and reliability, making it suitable for production environments.

Overview: This repository contains a dataset of the ICASSP 2021 MULTILINGUAL PHONETIC DATASET FOR LOW RESOURCE SPEECH RECOGNITION, focusing on the "abk" language. The full dataset can be downloaded from the release page. Key features: - Raw narrow phone annotations - Segmented and normalized transcription - Unique phoneme/phone inventory - Audio files in WAV format Tech stack: Python Why it matters: This dataset is crucial for researchers and developers working on low-resource speech recognition, providing a valuable resource for training and testing models in diverse language environments.

### Overview: Zeppelin is an open-source S3-native vector and full-text search engine designed for self-hosting. It stores all data in object storage, making it cost-effective and scalable. Zeppelin leverages IVF indexing and quantization for efficient querying, ensuring recall parity with memory-resident engines at lower cost. ### Key features: - **S3-native storage**: Data is stored in object storage, eliminating the need for a fleet of replicas. - **Stateless nodes**: Any node can serve any query, eliminating the need for rebalancing or replica fleets. - **IVF indexing**: Scale-aware IVF-Flat partitioning and Hierarchical ANN. - **Quantization**: 2-bit Extended-RaBitQ with exact f32 rerank (production default), SQ8, PQ, and f16 storage. - **BM25 full-text search**: Inverted indexes with configurable tokenization, stemming, and multi-field `rank_by` expressions. - **Late-interaction retrieval**: Multi-vector MaxSim with asynchronous semantic enrichment and RRF fusion. - **Bitmap pre-filters**: RoaringBitmap indexes for sub-millisecond attribute filtering. - **Write-ahead log**: Durable writes with compaction into indexed segments. - **Namespace forks**: Disabled-by-default copy-on-write branching; fork only, no merge. - **Strong & eventual consistency**: Choose per-query (see [Consistency semantics]). - **Security suite**: Fail-closed authorization kernel, RBAC, durable audit log, delegation tokens, and preservation holds. - **Object storage**: S3

Overview: 这份资料是针对大模型算法研究、应用开发和AI基础设施岗位的系统化面试准备资料。它包含16份文档、459道题和约36万字的内容，涵盖了大模型八股面试题、面经带复盘、大厂横向对比、专题内容、手撕代码实现等。资料包括字节11篇真实面经实录、大模型八股上下篇、RAG/Agent/推理部署/AI基础设施四大专题，以及12家大厂面试风格、高频真题、跨公司通用题的横向对比。 Key features: - 大模型八股面试题 - 面经带复盘 - 大厂横向对比 - 专题能落地 - 手撕代码实现 Tech stack: Python Why it matters: 本资料提供了全面的面试准备，涵盖了大模型算法、应用开发和基础设施的各个方面，帮助求职者在面试中展示自己的能力和工程经验。通过系统的知识体系和详细的题目设计，求职者可以更好地准备和应对面试，提高通过率。

Overview: IR3D-Bench aims to evaluate vision-language model scene understanding by recreating scenes and comparing them with ground truth, thereby enabling VLMs to estimate key attributes such as object ID, localization, color, material, and relations. This approach leverages the human-like reflection during the "recreation" process, which is expected to unlock new possibilities for improving VLMs' performance in both understanding and generation. Key features: - Evaluates vision-language model scene understanding through reconstruction and comparison with ground truth. - Enables VLMs to estimate key attributes such as object ID, localization, color, material, and relations. - Demonstrates surprising potential for human-like reflection during the "recreation" game, where VLMs update their understanding of the scene based on comparisons with original images. Tech stack: Python Why it matters: This project provides a new perspective on evaluating VLMs' spatial visual understanding, offering insights into how VLMs can be improved to better mirror human-like reasoning in understanding and generating scenes. Repository: LiuHengyu321/IR3D-Bench GitHub stars: 47

### Overview: A curated collection of proactive AI research and applications, bringing together papers, agents, models, systems, projects, products, benchmarks, and supporting infrastructure. ### Key Features: - Personal Agents - Streaming Proactivity - Domain-Specific Assistance ### Tech Stack: JavaScript ### Why it Matters: A curated collection of proactive AI research and applications, bringing together papers, agents, models, systems, projects, products, benchmarks, and supporting infrastructure.

Overview: This project is a Chinese-focused Multimodal BERT model designed for natural language processing tasks. Key features: - Chinese language support - Multimodal capabilities (text, images, etc.) - Advanced pre-training techniques Tech stack: PyTorch Why it matters: This model is crucial for applications requiring high accuracy in Chinese language understanding and processing, making it valuable for research and development in the field of natural language processing.

Overview: Hanami is a live translator that captures audio from a Windows speaker or microphone and translates it to any desired language. Key features: - Captures audio from Windows speakers and microphones - Translates spoken words to the desired language - Uses lightweight multiprocessing for efficient processing - Supports multiple languages and sources Tech stack: Python, Soundcard, SpeechRecognition, Selenium Why it matters: Hanami provides a fun and practical way to explore language translation and audio processing, even if it's not officially supported or a research topic.

Overview: ML4TSPBench is an official implementation of the ICLR 2025 paper "Unify ML4TSP: Drawing Methodological Principles for TSP and Beyond from Streamlined Design Space of Learning and Search". This repository integrates existing mainstream machine learning and search algorithms for the TSP problem, providing a pluggable and unified pipeline. Key features: - Pluggable and unified pipeline for machine learning and search algorithms - Task-agnostic skeleton and common toolkit for developing learning-based solvers - Robust support for various problem types and solution visualization tools - External accessible `ml4tsp` package with example Python code for model training and testing Tech stack: C Why it matters: This repository provides a comprehensive and unified approach to solving the Travelling Salesman Problem (TSP) and other combinatorial optimization problems, offering insights into the methodological principles of learning-based designs and the exploration of desirable principles for method designs.