Projects

Overview: QnABot on AWS is a multi-channel, multi-language conversational interface (chatbot) designed to respond to customer questions, answers, and feedback across various channels including chat, voice, SMS, and Amazon Alexa. Key features: - Integrates with multiple channels for seamless customer interaction. - Utilizes Amazon Lex for user authentication and interaction. - Automatically saves and manages questions and answers in Amazon OpenSearch Service. - Supports text embeddings for enhanced search functionality. - Implements Amazon Bedrock for foundational model processing. Tech stack: JavaScript, AWS CloudFormation, Amazon API Gateway, Amazon S3, Amazon Cognito, Amazon Lex, Amazon DynamoDB, Amazon OpenSearch Service, Amazon API Gateway V2, Amazon Comprehend, Amazon Translate, Amazon Bedrock. Why it matters: This solution enhances customer experience by providing personalized tutorials and support, reduces call center wait times, and leverages the latest machine learning technology for engaging, human-like interactions.

**Overview:** This repository contains all the codes for Rohan Paul's YouTube channel, organized by topic. It includes fine-tuning of models like Mistral, Falcon, and Phi-1_5, as well as fine-tuning of a pre-trained transformer model. **Key features:** - Fine-tuning of models with PEFT and QLoRA - Fine-tuning of a pre-trained transformer model - Fine-tuning of DeBERTa for Amazon Review Dataset - Fine-tuning of BERT for multi-class classification on a custom dataset - Fine-tuning of a transformer model with HuggingFace Transformer - Fine-tuning of a large language model from Cerebras **Tech stack:** Jupyter Notebook **Why it matters:** This repository is essential for anyone looking to learn and practice natural language processing (NLP) techniques, as it provides a comprehensive set of codes for various NLP tasks and model fine-tuning methods.

### Overview: PixelSmile is a cutting-edge facial expression editing tool that allows for fine-grained control over facial expressions with reduced semantic entanglement and strong identity preservation. This project is designed to enhance the quality and usability of facial expression editing, making it more accessible and effective for developers and creators. ### Key Features: - **Fine-Grained Facial Expression Editing**: PixelSmile enables users to manipulate facial expressions with high precision, allowing for nuanced changes without losing the overall identity of the subject. - **Continuous Control**: Users can adjust facial expressions in real-time, providing a smooth and intuitive editing experience. - **Reduced Semantic Entanglement**: The tool maintains the integrity of the subject's identity, ensuring that the edited images remain recognizable and authentic. ### Tech Stack: - **Python**: The primary programming language used for developing the project. - **Hugging Face**: The framework and models utilized for implementing the facial expression editing capabilities. ### Why It Matters: PixelSmile represents a significant advancement in the field of facial expression editing by offering unparalleled control and identity preservation. This project is crucial for developers and creators who need to enhance their applications with advanced facial expression editing features, making it easier and more effective to create realistic and engaging content. ### Repository: Ammmob/PixelSmile - **Primary Language**: Python - **GitHub Stars**: 446 - **GitHub Description**: PixelSmile: Toward Fine-Grained Facial Expression Editing with continuous control, reduced semantic entanglement, and strong identity preservation. - **README Excerpt**: PixelSm

### Overview: Aquila2 is an open-source series of large language models proposed by BAAI, including base language models (Aquila2-7B, Aquila2-34B, Aquila2-70B-Expr) and chat models (AquilaChat2-7B, AquilaChat2-34B, AquilaChat2-70B-Expr, AquilaChat2-7B-16k, AquilaChat2-34B-16k). The repository includes pretrained models and detailed documentation for both types of models. ### Key Features: - **Model Types:** Base Language Models (Aquila2-7B, Aquila2-34B, Aquila2-70B-Expr) and Chat Models (AquilaChat2-7B, AquilaChat2-34B, AquilaChat2-70B-Expr, AquilaChat2-7B-16k, AquilaChat2-34B-16k). - **Pretrained Models:** Available for download from the BAAI ModelHub. - **Documentation:** Comprehensive guides on quickstart, finetuning, long-context understanding, and license agreement. ### Tech Stack: Python ### Why it Matters: Aquila2 and AquilaChat2 models provide powerful tools for language understanding and generation, making them valuable resources for developers and researchers in the field of natural language processing.

### Overview: This repository contains an extensive collection of papers and tutorials on large multi-modality models, parameter-efficient finetuning, vision-language pretraining, and conventional image-text matching. It serves as a preliminary guide to help developers gain insight into these advanced techniques. ### Key features: - Large Multi-Modality Model - Parameter-Efficient Finetuning - Vision-Language Pretraining - Conventional Image-Text Matching ### Tech stack: Python, Jupyter Notebook, and Markdown. ### Why it matters: This repository provides a comprehensive overview of cutting-edge research in large multi-modality models, parameter-efficient finetuning, vision-language pretraining, and conventional image-text matching. It serves as a valuable resource for developers looking to stay updated with the latest advancements in these fields.

### Overview: IGR is an innovative deep learning approach designed to learn implicit signed distance representations directly from raw point clouds, including those with or without normal data. This method optimizes the network to solve the eikonal equation with the input point cloud as the boundary condition, providing a natural and simple solution. ### Key Features: - **Implicit Geometric Regularization**: The method learns implicit signed distance representations directly from raw point clouds, offering a natural and simple solution. - **Training and Evaluation**: The code is compatible with Python 3.7 and PyTorch 1.2. It requires the following packages: numpy, pyhocon, plotly, scikit-image, and trimesh. - **Surface Reconstruction**: IGR can reconstruct surfaces from point clouds with or without normal data. - **Shape Space Learning**: The method can be used to learn shapespace from D-Faust oriented point clouds, including preprocessing and evaluation. ### Tech Stack: Python ### Why it Matters: IGR is significant because it provides a new way to learn implicit representations directly from raw point clouds, offering a natural and simple solution that can be applied to various tasks in computer vision and geometry processing.

### Overview: This repository implements the training and testing of deep regression neural networks for the paper "Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image" by Fangchang Ma and Sertac Karaman at MIT. The original implementation is provided in Torch, and the PyTorch version can be found [here](https://github.com/fangchangma/sparse-to-dense.pytorch). The repository offers the training and testing of RGB (or grayscale image) based depth prediction, sparse depth based depth prediction, and RGBd (i.e., both RGB and sparse depth) based depth prediction. ### Key features: - Training and testing of various depth prediction models. - Support for different datasets (NYU Depth V2 and KITTI). - Option to use pretrained weights for faster training. - Training scripts with multiple options for dataset, input type, loss function, and network architecture. ### Tech stack: Torch ### Why it matters: This repository provides a practical implementation of the proposed method for depth prediction from sparse depth samples and a single image, making it useful for researchers and developers working on depth estimation and 3D reconstruction tasks.

### Overview: This project is a PyTorch implementation of the YOLO9000 model, designed for object detection tasks. It includes detailed training, evaluation, and dataset preparation instructions. ### Key features: - **Model Implementation**: PyTorch-based implementation of the YOLO9000 model. - **Training**: Train your model from scratch or use pre-trained models. - **Evaluation**: Evaluate test images using either pre-trained models or your own trained models. - **Datasets**: Support for VOC2007, VOC2012, COCO2014, and COCO2017 datasets. ### Tech stack: - Python - PyTorch - OpenCV (cv2) - Tensorboard - TensorboardX (optional) ### Why it matters: This project is useful for anyone looking to implement or evaluate YOLO9000 models in their own projects, providing a comprehensive guide for training, evaluation, and dataset preparation.

### Overview: This repository is an Awesome Collection for Large Language Models (LLM) surveys, focusing on various aspects such as instruction tuning, human alignment, prompt engineering, and more. It aims to provide a comprehensive overview of the current state of LLM research and applications. ### Key Features: - **General Survey**: A survey of large language models, including instruction tuning, human alignment, prompt engineering, and more. - **Training of LLM**: Covers training methods and techniques for LLMs. - **Prompt of LLM**: Discusses chain of thought, prompt engineering, and retrieval-augmented LLMs. - **Challenge of LLM**: Addresses issues like hallucination, compression, evaluation, reasoning, explainability, fairness, graph representation, long context, factuality, knowledge, self-correction, attributions, tool usage, calibration, agent roles, vulnerabilities, efficiency, data, security, and continual learning. - **Multimodal of LLM**: Includes visual, audio, and code LLMs. - **LLM for Domain Application**: Covers applications in health, finance, education, law, mental health, robotics, and more. - **LLM for Downstream Tasks**: Includes recommendation, information retrieval, software engineering, autonomous driving, time series, detection, society, citation, text watermarking, math, environmental disciplines, information extraction, data annotation, game, and others. ### Tech Stack: Python, Markdown, and GitHub. ### Why it Matters: This repository serves as a valuable resource for researchers, practitioners, and enthusiasts interested in the latest developments

Overview: SSDNeRF is an official PyTorch implementation of the ICCV 2023 paper "Single-Stage Diffusion NeRF". It includes a unified approach for 3D generation and reconstruction, supporting single-view reconstruction on the real KITTI Cars dataset. Key features: - Support for tiled triplanes (rollout layout) - FP16 diffusion sampling - 16-bit caching - A simple GUI demo Tech stack: Python Why it matters: SSDNeRF offers a comprehensive solution for 3D generation and reconstruction, making it a valuable tool for researchers and practitioners in the field of computer vision and 3D modeling. Repository: Lakonik/SSDNeRF GitHub stars: 446 GitHub description: [ICCV 2023] Single-Stage Diffusion NeRF README excerpt: [MVEdit](https://github.com/Lakonik/MVEdit), an upgraded codebase based on SSDNeRF, supports all SSDNeRF models and configs and offers new features such as diffusers support and improved SSDNeRF GUI. The codebase is based on [torch-ngp](https://github.com/ashawkey/torch-ngp) and [MMGeneration](https://github.com/open-mmlab/mmgeneration).

### Overview: MindSQL is a Python library designed to simplify database interactions using natural language queries. It seamlessly integrates with PostgreSQL, MySQL, SQLite, Snowflake, and BigQuery, powered by advanced models like GPT-4 and Llama 2. ### Key Features: - Seamless integration with PostgreSQL, MySQL, SQLite, Snowflake, and BigQuery. - Supports ChromaDB and Faiss for context-aware responses. - Integrates with renowned databases like PostgreSQL, MySQL, and SQLite. - Utilizes large language models (LLM) like GPT-4, Llama 2, Google Gemini, and supports knowledge bases like ChromaDB and Faiss. ### Tech Stack: Python ### Why It Matters: MindSQL simplifies database interactions by enabling users to query databases using natural language, making it easier and more intuitive for developers and non-technical users alike. ### Repository: Mindinventory/MindSQL ### GitHub Stars: 446 ### GitHub Description: MindSQL is a Python RAG (Retrieval-Augmented Generation) Library designed to streamline the interaction between users and their databases using just a few lines of code. With seamless integration for renowned databases such as PostgreSQL, MySQL, and SQLite, MindSQL also extends its capabilities to major databases like Snowflake and BigQuery by extending the `IDatabase` Interface. This library utilizes large language models (LLM) like GPT-4, Llama 2, Google Gemini, and supports knowledge bases like ChromaDB and Faiss. ### Installation: ```commandline pip install mindsql

### Overview: This project aims to create a vision-based head motion capture system for VTubers, enhancing the realism of their animations through smooth and robust facial expressions using C# and Python in Unity. ### Key features: - Smooth and robust facial expressions - Realistic head motion - Compatibility with VTubers - Built with C# and Python in Unity ### Tech stack: C# and Python ### Why it matters: This project provides a practical solution for enhancing the authenticity of VTuber animations, making them more engaging and lifelike.

### Overview: TextAugment is a Python library designed to enhance the performance of short text classification tasks by augmenting textual data. It leverages global augmentation methods to generate synthetic data, making it easier and more efficient to improve model accuracy without manual effort. ### Key Features: - **Synthetic Data Generation**: TextAugment generates synthetic data to improve model performance. - **Plug-and-Play**: Seamless integration with various machine learning frameworks. - **Lightweight and Easy-to-Use**: Simple and lightweight library with minimal dependencies. - **Support for Textual Data**: Can handle textual data for various NLP applications. ### Tech Stack: Python ### Why It Matters: TextAugment offers a practical solution for improving short text classification models by enhancing model performance through synthetic data augmentation. This method reduces the need for manual data collection and preprocessing, making it an efficient and effective approach for text classification tasks.

Overview: DeepMind Lab2D is a customizable 2D platform designed for the creation of grid worlds, featuring ease of use and performance. Key features: - Customizable 2D environments - Python `dm_env` API and custom C API for agent interaction - Multiple agents support Tech stack: C++ Why it matters: Provides a flexible and efficient environment for machine learning research, making it easy to create and test various AI agents.

### Overview: This repository contains a curated list of free, high-quality university-level courses with video lectures related to the field of Computer Vision. The repository includes courses on signal processing, image and video processing, introductory and advanced computer vision, deep learning for computer vision, human vision and perception, machine learning, deep learning, and computer graphics. ### Key Features: - **Signal Processing**: Covers the fundamentals of signal and system analysis, including Fourier representations, Laplace and Z transforms, and linear systems. - **Image and Video Processing**: Includes courses on digital signal processing, image and video processing, and advanced topics such as multirate signal processing and adaptive filtering. - **Computer Vision**: Covers introductory and advanced computer vision, including deep learning techniques. - **Deep Learning**: Focuses on advanced topics in deep learning for computer vision. - **Human Vision and Perception**: Explains the science behind human vision and perception. - **Machine Learning**: Provides a comprehensive treatment of machine learning concepts. - **Deep Learning**: Offers advanced topics in deep learning for computer vision. ### Tech Stack: Python, OpenCV, TensorFlow, PyTorch, and other relevant libraries. ### Why It Matters: This repository is essential for developers interested in computer vision, signal processing, and related fields. It provides a comprehensive and practical learning experience through high-quality video lectures and practical applications.

Overview: DeepSeek-OCR-WebUI is a modern, responsive web application designed for efficient OCR (Optical Character Recognition) tasks. It supports multiple modes for document recognition and batch processing, with real-time logging for enhanced monitoring. Key features: - Multi-Mode Support - Batch Processing - Real-time Logging - Fully Responsive Interface Tech stack: TypeScript Why it matters: This application is a versatile tool for developers and professionals looking to automate OCR tasks, providing a user-friendly interface with advanced features like real-time logging and batch processing, making it ideal for various applications requiring document recognition. Repository: neosun100/DeepSeek-OCR-WebUI Primary language: TypeScript GitHub stars: 447

### Overview: slowllama is a project designed to fine-tune Llama2 and CodeLLama models, including 70B/35B, on Apple M1/M2 devices or consumer nVidia GPUs. It uses LoRA to limit parameter updates, aiming to provide finetuned models without the need for large-scale training or inference. ### Key features: - Fine-tuning Llama2 and CodeLLama models on Apple M1/M2 devices or consumer nVidia GPUs. - Uses LoRA to limit parameter updates, focusing on finetuning. - Does not use quantization, instead offloading parts of the model to SSD or main memory. - Supports both full finetuning and LoRA-based finetuning. - Uses a simple script for finetuning, allowing for easy adjustment of parameters. ### Tech stack: Python ### Why it matters: This project is experimental, but it offers a practical solution for finetuning large models without the need for large-scale training or inference, making it particularly useful for resource-constrained environments like Macbook Air or consumer nVidia GPUs. ### Repository: okuvshynov/slowllama GitHub stars: 447 GitHub description: Finetune Llama2 and CodeLLama models, including 70B/35B on Apple M1/M2 devices (for example, Macbook Air or Mac Mini) or consumer nVidia GPUs.

Overview: ClawApp 是一个基于手机浏览器的 OpenClaw AI 智能体聊天客户端，提供流式聊天、图片收发、工具调用等功能，支持离线缓存和智能重连。 Key features: - 实时流式聊天 - 图片收发 - Markdown渲染和代码高亮 - 快捷指令面板 - 工具调用实时状态显示 - 语音输入 - 多智能体支持 - 会话管理 - 主题切换 - 中英文切换 - 智能重连 - Token / Password认证 - 离线消息缓存 - PWA支持 - Android APK打包 Tech stack: JavaScript Why it matters: ClawApp 通过手机浏览器代理服务端，解决了 OpenClaw Gateway 默认只监听本机的问题，使得手机用户可以直接连接到 OpenClaw 平台，体验到更便捷的聊天功能。

### Overview: AstrBot 自主学习插件是一个用于提升AI聊天机器人对话风格、理解群组黑话、管理社交关系和好感度、以及自适应人格演化的能力的项目。它通过持续采集、学习和审查用户对话，使Bot逐渐具备自然、准确的对话风格和丰富的社交互动。 ### Key features: - **对话风格学习**: 从真实用户与Bot的对话中提取表达模式和风格审查记录。 - **群组黑话学习**: 统计高频词并结合上下文推断其含义，确保在LLM请求前注入解释。 - **人格演化审查**: 学习结果通过审查链路进行批准、拒绝、删除、回滚和自动应用策略。 - **社交关系分析**: 记录互动关系、好感度和心理状态，提供社交上下文。 - **记忆与知识图谱**: 构建记忆图、知识图谱和可视化查询入口。 - **LLM 请求注入**: 在`on_llm_request`阶段注入社交、黑话、记忆、few-shot和临时人格增量。 - **WebUI Dashboard**: 提供全量设置、审查、黑话、图谱、学习内容和日志监控。 - **多数据库支持**: 支持SQLite、MySQL、PostgreSQL，包含自动建表和轻量迁移。 ### Tech stack: Python ### Why it matters: 通过这个插件，AstrBot能够更加自然地与用户互动，理解复杂的群组文化，并与用户建立

Overview: This repository is the official source for the paper "Multimodal Garment Designer: Human-Centric Latent Diffusion Models for Fashion Image Editing" presented at ICCV 2023. The project aims to enhance the fashion design process by enabling human-centric fashion image editing using multimodal inputs. Key features: - A novel architecture based on latent diffusion models for multimodal-conditioned fashion image editing. - Extends two existing fashion datasets, Dress Code and VITON-HD, with multimodal annotations collected through a semi-automatic manner. - Provides experimental results demonstrating the effectiveness of the proposed approach in terms of realism and coherence with given multimodal inputs. Tech stack: The project uses Python, TensorFlow, and PyTorch for its development. Why it matters: This work introduces a significant advancement in fashion image editing by integrating human-centric prompts and fabric texture conditioning, making it a valuable contribution to the field of computer vision and fashion design.