Projects

Overview: GeoDiff is an implementation of a Geometric Diffusion Model designed for generating molecular conformation data, presented at ICLR 2022. It offers a framework for molecular conformation generation and includes both official and preprocessed datasets. Key features: - Geometric Diffusion Model for Molecular Conformation Generation - Official dataset GEOM - Preprocessed datasets available in Google Drive - Training instructions provided in config files - Model checkpoints and training logs saved in specified directories - Generation capabilities for entire or part of test sets - Pretrained models available in the `pretrain` branch for reproducibility Tech stack: Python Why it matters: GeoDiff provides a practical solution for generating molecular conformation data, which is crucial in drug discovery and materials science research.

Overview: CoLLiE 是一个全面的框架，用于从零开始训练大型语言模型。它集成了数据预处理、模型微调、训练监控和并行策略，旨在提高训练效率和模型质量。 Key features: - 数据并行 (DP) - 流水线并行 (PP) - 张量并行 (TP) - 零冗余优化器 (ZeRO) - 高效微调方法 (LOMO, LoRA, Flash Attention) - 设计优雅的用户界面 - 高度可定制化 Tech stack: Python Why it matters: CoLLiE 提供了一站式的解决方案，帮助用户高效地从零开始训练大型语言模型，同时保持模型质量和训练效率。 Repository: OpenMOSS/CoLLiE GitHub stars: 421 GitHub description: An efficient framework for collaborative training of large language models

Overview: This project distills pre-trained Transformers models like DistilBERT and GPT-2 for on-device inference, leveraging TensorFlow Lite with Android demo apps. It includes fine-tuned models for question answering and text generation, providing enhanced performance and reduced resource usage. Key features: - Fine-tuned DistilBERT for question answering on the SQuAD dataset. - DistilBERT and DistilGPT2 fine-tuned for on-device text generation. - Android demo apps for offline inference. Tech stack: Java Why it matters: This project simplifies the integration of Transformers models on Android devices, making them accessible for applications requiring on-device inference capabilities, such as language understanding and text generation tasks. Repository: huggingface/tflite-android-transformers Primary language: Java GitHub stars: 421 GitHub description: DistilBERT / GPT-2 for on-device inference thanks to TensorFlow Lite with Android demo apps.

Overview: PyCantonese is a Python library designed for Cantonese linguistics and natural language processing. Key features: - Accessing and searching corpus data - Parsing and conversion tools for Jyutping romanization - Parsing Cantonese text - Stop words - Word segmentation - Part-of-speech tagging Tech stack: Python Why it matters: PyCantonese provides a comprehensive solution for Cantonese linguistics and NLP tasks, making it a valuable tool for researchers, language enthusiasts, and developers working in the field of Chinese language processing.

Overview: This repository contains the code for the paper "Learning to Model the World with Language," presented at ICML 2024. Dynalang is an agent that uses language to predict the future through a multimodal world model. Key features: - Dynalang: An agent that leverages language to predict the future in a multimodal world model. - HomeGrid: An environment for training Dynalang using the HomeGrid task. - Messenger: An environment for training Dynalang using the Messenger task. - VLN: A vision-language model that predicts future states of the environment. - LangRoom: An environment that allows for simultaneous movement and talking. Tech stack: Python. Why it matters: This project demonstrates the application of language models to predict and model the future states of the world, which has potential applications in various fields such as robotics, artificial intelligence, and natural language processing.

### Overview: This repository is a curated collection of resources for Azure OpenAI, Large Language Models (LLMs), and their applications. It includes concise summaries, chronological organization, and monthly updates to keep the list relevant and accessible. ### Key Features: - Concise Summaries - Chronological Organization - Monthly Updates - Applications archived or inactive for over 12 months are listed in `applications.old.md` ### Tech Stack: Python ### Why It Matters: This repository provides a comprehensive and curated collection of resources for Azure OpenAI, Large Language Models, and their applications, focusing on concise summaries, chronological organization, and monthly updates. It is useful for developers and researchers looking to stay informed about the latest developments in the field. ### GitHub Stars: 419 ### Repository: kimtth/azure-openai-llm-notes

### Overview: This repository contains the official implementation of Relational Knowledge Distillation, a paper presented at CVPR 2019. It provides source code for experiments related to metric learning using PyTorch. ### Key features: - Source code for experiments in the metric learning domain. - PyTorch implementation of Relational Knowledge Distillation. - Supports training of a teacher embedding network using triplet loss with distance weighted sampling. - Evaluates the trained teacher embedding network. - Distills the teacher to a student embedding network. - Distills a trained model to a student network. ### Tech stack: Python, PyTorch, tqdm, h5py, scipy. ### Why it matters: This repository is essential for researchers and developers interested in metric learning and knowledge distillation, providing a practical tool for experimenting with the Relational Knowledge Distillation method.

### Overview: Chonky is a Python library designed to intelligently segment text into meaningful semantic chunks using a fine-tuned transformer model. This library is particularly useful in RAG systems and can be installed via `pip install chonky`. ### Key features: - Fine-tuned transformer model for text chunking - Supports various models including modernbert, mmbert, and distilbert - Token-based F1 scores benchmarked on datasets ### Tech stack: Python ### Why it matters: Chonky simplifies the process of text segmentation, making it easier to extract meaningful chunks of text for applications like RAG systems. Its fine-tuned model ensures better performance compared to other models, making it a valuable tool for developers working on text-based applications.

### Overview: This repository contains the code for the paper "On human motion prediction using recurrent neural networks" by Julieta Martinez, Michael J. Black, and Javier Romero. The code is based on TensorFlow and is presented at CVPR 17. It includes baselines and RNN models for predicting human motion. ### Key features: - Baselines and RNN models for predicting human motion. - Realistic long-term motion with loss computed over 1 second. - Training times for different models: 45s for Sampling-based loss, 35s for Residual (SA), 65s for Residual unsup. (MA), and 65s for Residual sup. (MA). - Training on all actions or specific actions like "walking". - Log error in Euler angles for each action to tensorboard for progress tracking. ### Tech stack: Python ### Why it matters: This repository provides a practical implementation of recurrent neural networks for motion prediction, showcasing the effectiveness of these models in predicting human motion accurately and efficiently.

**Overview:** MINE is a novel method that reconstructs the frustum of a camera from a single image, enabling the generation of novel views of a scene. This approach is based on continuous depth and NeRF for novel view synthesis, making it particularly useful for applications requiring high-resolution and diverse views. **Key features:** - Continuous Depth MPI with NeRF - Densely reconstructs the frustum of the camera - Easy to render novel views of the given scene - PyTorch implementation **Tech stack:** Python **Why it matters:** MINE offers a significant advancement in depth and view synthesis, providing a more continuous and diverse set of views compared to existing methods, making it highly valuable for applications requiring high-resolution and novel perspectives. **Repository:** vincentfung13/MINE **Primary language:** Python **GitHub stars:** 421 **GitHub description:** Code and models for our ICCV 2021 paper "MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesis"

Overview: MathPile is a diverse and high-quality math-centric corpus comprising about 9.5 billion tokens. It is significantly different from general domain-focused corpora like Pile and RedPajama, and multilingual-focused ones like ROOTS and The Stack. MathPile is unique in its math-centric approach, drawing from a wide range of sources including textbooks, arXiv, Wikipedia, ProofWiki, StackExchange, and web pages. It is the first math-centric corpus with a significant collection of high-quality textbooks (~0.19B tokens), making it a valuable resource for training foundation models in the math domain. Key features: - Math-centric: MathPile is specifically designed for the math domain. - Diversity: MathPile includes a wide range of sources, including textbooks, arXiv, Wikipedia, ProofWiki, StackExchange, and web pages. - High-quality: MathPile adheres to the principle of "less is more," focusing on high-quality data in the pre-training phase. - Data documentation: MathPile is extensively documented, including a dataset sheet and quality annotations for web-sourced documents. Tech stack: Python Why it matters: MathPile is a valuable resource for training foundation models in the math domain, providing a diverse and high-quality corpus that is significantly different from general domain-focused corpora and multilingual-focused ones.

### Overview: RecA: Reconstruction Alignment Improves Unified Multimodal Models ### Key features: - **Self-Supervised Learning**: RecA uses self-supervised learning to improve the performance of pretrained models. - **Semantic Reconstruction**: The model reconstructs images from its own visual understanding features, enhancing the model's ability to generate high-quality images. - **Unified Interface**: RecA maintains the original inference interface of the model, ensuring seamless integration with existing architectures. ### Tech stack: Python ### Why it matters: RecA provides a novel approach to improving the performance of unified multimodal models by leveraging self-supervised learning. This method enhances image generation and editing tasks, making it a valuable addition to the field of computer vision and artificial intelligence.

### Overview: CPM is a comprehensive makeup transfer framework that surpasses existing models in both light and extreme makeup styles. It includes an improved color transfer branch based on BeautyGAN and a novel pattern transfer branch. The framework is enhanced with four new datasets, including CPM-Real, CPM-Synt-1, CPM-Synt-2, and Stickers, which are real makeup styles and synthesized images with pattern segmentation masks. ### Key Features: - Improved color transfer branch - Novel pattern transfer branch - Four new datasets: CPM-Real, CPM-Synt-1, CPM-Synt-2, and Stickers - Real makeup styles and synthesized images with pattern segmentation masks ### Tech Stack: Python ### Why It Matters: CPM is a significant advancement in makeup transfer, offering a holistic approach to replicate both colors and patterns from a reference makeup style to another image. This framework is essential for enhancing the realism and accuracy of makeup transfer in various applications, including virtual makeup and augmented reality. Repository: VinAIResearch/CPM GitHub Stars: 420 GitHub Description: 💄 Lipstick ain't enough: Beyond Color-Matching for In-the-Wild Makeup Transfer (CVPR 2021)

### Overview: Parakeet-Rs is a Rust library that provides fast speech-to-text and diarization capabilities using NVIDIA's Parakeet models. It supports multiple languages and models, including English, multilingual, and streaming ASR with end-of-utterance detection. The library is optimized for performance and can run both on CPU and GPU. ### Key Features: - **ONNX Runtime**: Supports ONNX Runtime for environments that need to share a single environment. - **Multilingual Models**: Supports 25 languages with auto-detection. - **Streaming ASR**: Real-time speech recognition with end-of-utterance detection. - **Nemotron**: Cache-aware streaming ASR with punctuation and two variants for English and multilingual models. - **Cohere Transcribe**: Supports offline multilingual transcription with options for punctuation and ITN toggles. ### Tech Stack: Rust ### Why It Matters: Parakeet-Rs offers high-performance speech recognition and diarization capabilities, making it ideal for applications requiring real-time or near-real-time speech processing. Its ability to handle multiple languages and models makes it versatile for various use cases, including multilingual transcription and streaming ASR with end-of-utterance detection.

### Overview: Synthalingua is a powerful tool for real-time translation and transcription, supporting dozens of languages and integrating with various platforms like Zoom, Google Meet, and Twitch. It offers features like multilingual support, streaming and captions, and automatic filtering of repeated phrases, making it a versatile solution for both developers and translators. ### Key Features: - **Real-time translation & transcription** (stream, mic, file) - **Multilingual**: Translate between dozens of languages - **Streaming & captions**: HLS, YouTube, Twitch, and more - **Blocklist & repetition suppression**: Auto-filter repeated or unwanted phrases - **Discord & web integration**: Send results to Discord or view in browser - **Portable GUI version available** ### Tech Stack: Python ### Why it Matters: Synthalingua is a valuable tool for anyone involved in language translation and transcription, offering real-time translation and transcription capabilities that can be integrated into various platforms and applications. Repository: cyberofficial/Synthalingua GitHub stars: 419 ### Primary Language: Python ### GitHub Description: Synthalingua - Real Time Translation ### Why it Matters: Synthalingua is a powerful tool for real-time translation and transcription, supporting dozens of languages and integrating with various platforms like Zoom, Google Meet, and Twitch. It offers features like multilingual support, streaming and captions, and automatic filtering of repeated phrases, making it a versatile solution for both developers and translators. ### Installation: Synthalingua is a Python-based tool that can be installed on any system with Python

### Overview: In this series, we will guide you through the process of building a self-driving robotic car using Raspberry Pi, SunFounder PiCar-V Kit, TensorFlow, and Google's EdgeTPU Co-Processor. This project aims to bring the capabilities of advanced autonomous driving to the masses, showcasing how to create a deep-learning-based, autonomous vehicle using a budget-friendly setup. ### Key Features: - **Hardware Setup**: Raspberry Pi, SunFounder PiCar-V Kit, Google's EdgeTPU Co-Processor. - **Software Tools**: Python, OpenCV, TensorFlow. - **Project Scope**: Lane Following, Traffic Sign and People Detection, Autonomous Navigation. ### Tech Stack: - **Hardware**: Raspberry Pi 3 B+, SunFounder PiCar-V Kit, Google's Edge TPU. - **Software**: Python, OpenCV, TensorFlow. ### Why It Matters: This project is designed to democratize the technology of autonomous driving by providing a practical, low-cost solution. By following the steps outlined in this series, readers can build a functional, autonomous vehicle capable of navigating on real-world roads, paving the way for a future where self-driving cars are a reality.

### Overview: This repository is a curated list of advanced prompt optimization and tuning methods used in Large Language Models (LLMs), focusing on papers published after 2022. It aims to provide researchers and practitioners with the latest and most effective techniques for improving the performance of LLMs. ### Key features: - **LLM Optimization**: Papers discussing how to optimize the performance of LLMs through various techniques. - **Fine-tune Methods**: Techniques for fine-tuning LLMs to better fit specific tasks. - **Programming**: Guides on how to implement and optimize LLMs programmatically. - **Human Perference and Feedback**: Methods for incorporating human feedback into LLM optimization. - **Ensemble Methods**: Techniques for combining multiple LLMs to improve performance. - **Reinforcement Learning**: Algorithms that use reinforcement learning to optimize LLMs. - **Gradient-free Methods**: Techniques that do not rely on gradient information for optimization. - **In-Context Learning**: Methods that leverage in-context learning to improve LLM performance. - **Bayesian Optimization**: Techniques that use Bayesian methods for optimization. ### Tech stack: Python, Jupyter Notebooks, and various LLMs. ### Why it matters: This repository serves as a valuable resource for researchers and practitioners looking to enhance the performance of LLMs through advanced optimization techniques. It provides a comprehensive list of papers and methods, making it easier for users to stay updated with the latest advancements in the field.

Overview: This repository contains a collection of Python projects designed to help beginners and developers alike enhance their Python skills and simplify their daily tasks. Key features: - Beginner-friendly projects - Advanced scripts - Scripts for various programming tasks Tech stack: Python Why it matters: This repository serves as a comprehensive resource for Python enthusiasts, offering a wide range of projects that can be used to improve programming skills and solve common problems. Repository: king04aman/All-In-One-Python-Projects Primary language: Python GitHub stars: 420

Overview: This repository provides a guide on how to transcribe YouTube videos using OpenAI's Whisper model, focusing on the transcription process and utilizing Google Colab for the task. Key features: - Transcription of YouTube videos using OpenAI's Whisper model - Exploration of inference parameters or using the notebook as-is to store the transcript and audio in Google Drive Tech stack: Jupyter Notebook Why it matters: This notebook serves as a practical guide for developers and enthusiasts interested in speech recognition and natural language processing, demonstrating how to use the Whisper model for transcription tasks. Repository: ArthurFDLR/whisper-youtube GitHub stars: 420 GitHub description: 🔉 Youtube Videos Transcription with OpenAI's Whisper GitHub stars: 420

Overview: This repository contains the official code for CRUD-RAG, a comprehensive Chinese benchmark for evaluating Retrieval-Augmented Generation systems. It includes datasets and a tutorial on how to run experiments on the benchmark. Key features: - Datasets for evaluating RAG systems - A tutorial on how to run experiments - A vector index for text retrieval Tech stack: Python Why it matters: This benchmark is crucial for evaluating the performance of models that use retrieval to enhance language generation, providing a standardized way to measure the effectiveness of these systems. Repository: IAAR-Shanghai/CRUD_RAG GitHub stars: 418 GitHub description: CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Models README excerpt: This repository contains the official code of CRUD-RAG, a novel benchmark for evaluating the RAG systems. It includes the datasets we created for evaluating RAG systems, and a tutorial on how to run the experiments on our benchmark.