Projects

### Overview: JoyHallo is a digital human model designed for Mandarin video generation, utilizing audio features from employees of JD Health International Inc. It integrates Chinese wav2vec2 for improved model performance and cross-language generation capabilities. ### Key Features: - **Audio Feature Embedding**: Uses Chinese wav2vec2 for audio feature embedding. - **Semi-Decoupled Structure**: Captures inter-feature relationships among lip, expression, and pose features. - **Improved Information Utilization Efficiency**: Enhances model performance. - **Cross-Language Generation**: Maintains strong English generation capabilities. ### Tech Stack: Python ### Why it Matters: JoyHallo demonstrates significant advancements in Mandarin video generation, showcasing the model's ability to adapt to diverse speaking styles and maintain strong cross-language generation capabilities.

### Overview: DanceNet is a dance generator project that uses Variational Autoencoder, LSTM, and Mixture Density Network to create realistic dance videos. ### Key features: - **Variational Autoencoder (VAE)**: Used for encoding and decoding dance sequences. - **Long Short-Term Memory (LSTM)**: Employed for generating new dance videos based on input images. - **Mixture Density Network (MDN)**: Utilized for modeling the distribution of dance movements. ### Tech stack: Python ### Why it matters: DanceNet offers a unique approach to generating realistic dance videos by combining advanced neural network techniques, making it a valuable tool for both researchers and developers in the field of computer vision and machine learning. Repository: jsn5/dancenet GitHub stars: 519 GitHub description: DanceNet - Dance generator using Variational Autoencoder, LSTM and Mixture Density Network. (Keras)

### Overview: `continuous-eval` is an open-source package designed for data-driven evaluation of LLM-powered applications. It provides modularized evaluation, a comprehensive metric library, and probabilistic evaluation capabilities. ### Key Features: - **Modularized Evaluation**: Measures each module in the pipeline with tailored metrics. - **Comprehensive Metric Library**: Covers various LLM use cases such as Retrieval-Augmented Generation, Code Generation, Agent Tool Use, and more. Mix and match Deterministic, Semantic, and LLM-based metrics. - **Probabilistic Evaluation**: Evaluates your pipeline with probabilistic metrics. ### Tech Stack: Python ### Why it Matters: `continuous-eval` offers a robust solution for evaluating LLM-powered applications, ensuring data-driven insights and comprehensive evaluation metrics.

Overview: This project aims to automatically generate headlines for short articles by leveraging recurrent neural networks and Keras. Key features: - Automatically generating headlines from article descriptions - Using a trained model to predict and generate headlines - Incorporating attention weights to consider context from the article Tech stack: Jupyter Notebook Why it matters: This project provides a practical application of neural networks to improve the quality of news headlines, potentially increasing readership and engagement.

Overview: This repository showcases Google Cloud's Generative AI for marketing scenarios, providing detailed, step-by-step instructions for deploying a solution that includes applications for crafting marketing materials, analyzing trends, generating content, and integrating with Google Workspace. Key features: - Crafting marketing materials like blog posts and social media content - Democratizing access to data for non-SQL users - Identifying emerging market trends - Improving search experience for internal or external content - Generating compelling email copy, website articles, and social media posts - Transferring insights and assets to Google Workspace Tech stack: Jupyter Notebook Why it matters: This repository offers a comprehensive guide for businesses looking to leverage Google Cloud's Generative AI for marketing, providing practical demonstrations and detailed explanations of how to implement these features effectively. Repository: GoogleCloudPlatform/genai-for-marketing GitHub stars: 517

### Overview: AIwaifu is an open-sourced finetunable customizable AI waifu inspired by neuro-sama, designed to empower everyone with a foundational platform for developing their own waifu. Powered by an open-source AI model, it offers a range of features such as flirtable, streamable, modifiable, and even lewd capabilities. The project is built with Python and aims to provide a fun and accessible way for users to create their own unique AI waifu. ### Key Features: - **Customizable Personality**: Tailor your AI waifu to match your preferences. - **Open-Sourced Model**: Utilizes an open-source AI model for flexibility and community collaboration. - **Flirtable and Streamable**: Engage with your AI waifu in real-time. - **Modifiable**: Easily modify and customize your AI waifu's appearance and personality. - **Finetunable**: Fine-tune the AI's responses to better suit your needs. - **LEWDABLE**: Enjoy a wide range of lewd features for a playful twist. ### Tech Stack: Python ### Why It Matters: AIwaifu is a groundbreaking project that democratizes the creation of AI waifu, making it accessible to everyone. By providing a foundational platform for customization and flexibility, it encourages creativity and community collaboration, fostering a vibrant and fun community around AI waifu development.

### Overview: Octocode is a structural code intelligence tool designed to give AI agents a deep understanding of your codebase. It integrates semantic search, knowledge graphs, and a built-in MCP server into a single Rust binary, making it easy for Claude, Cursor, and other AI clients to navigate and search through your project's structure. ### Key Features: - **Semantic Search**: Finds code by meaning, not keywords. - **Knowledge Graph**: Navigates imports, calls, and dependencies. - **Code Signatures**: View structure without reading entire files. - **LSP Precision**: Go-to-definition, find-references, and hover docs via language server. ### Tech Stack: Rust ### Why It Matters: Octocode transforms your project into a navigable knowledge graph that AI agents can search, understand, and navigate. It provides Claude, Cursor, and other AI clients with a deep understanding of your codebase, making them more effective in their tasks.

### Overview: Retrieve, Read and Link: Fast and Accurate Entity Linking and Relation Extraction on an Academic Budget (ACL 2024) ### Key Features: - **Entity Linking**: Quickly and accurately link named entities in academic papers. - **Relation Extraction**: Extract relationships between entities in a structured format. - **Budget-Friendly**: Designed for academic use with a minimal budget. ### Tech Stack: Python ### Why It Matters: This project offers a practical solution for academic researchers by providing fast and accurate entity linking and relation extraction, making it an essential tool for academic research and citation analysis.

Overview: This repository contains the implementation of E2-TTS, a model that addresses the challenges of zero-shot text-to-speech synthesis. It leverages a multistream transformer for both text and audio conditioning, differing from the original paper in its approach to conditioning. Key features: - Multistream transformer for text and audio conditioning - Conditioning done every transformer block in the E2 manner - Improvised method where text is interpolated to match the length of the audio for conditioning - Sampled from the E2TTS model for further processing Tech stack: Pytorch Why it matters: This implementation provides a practical and efficient solution to the problem of zero-shot text-to-speech synthesis, offering a significant improvement over the original paper's approach. Repository: lucidrains/e2-tts-pytorch Primary language: Python GitHub stars: 517 GitHub description: Implementation of E2-TTS, "Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS", in Pytorch README excerpt: <img src="./e2-tts.png" width="400px"></img>

Overview: Towhee Examples is a collection of Jupyter Notebook-based tutorials and examples that showcase how Towhee can be used to process various unstructured data types such as images, audio, and video. These examples demonstrate how to generate dense embeddings using Towhee, making it accessible to both beginners and large organizations. Key features: - Reverse Image Search - Image Embedding - Image Animation - Image Deduplication - Text Image Search - Q&A System Tech stack: Towhee Why it matters: Towhee Examples provide a practical way to learn and apply Towhee's capabilities to real-world problems, making it easier for developers to integrate and utilize Towhee in their projects.

Overview: Box-X is a tool box designed for efficient build and debug in Python, particularly focusing on scientific computing and computer vision. It supports both Python 2/3 and multiple platforms including Linux, macOS, and Windows. Key features: - General Python Tool - Scientific Computing and Computer Vision Tool Tech stack: Python Why it matters: Box-X provides a comprehensive solution for Python developers, making it easier to build and debug projects in scientific computing and computer vision fields, enhancing efficiency and accuracy. Repository: DIYer22/boxx Primary language: Python GitHub stars: 516 GitHub description: Tool-box for efficient build and debug in Python. Especially for Scientific Computing and Computer Vision. README excerpt: **Language :** [](https://github.com/DIYer22/boxx) [](./README_zh_cn.md) <br> <h1 align="center">Box-X</h1> <div align="center"> <strong>Hack Python and Vision</strong> </div> <div align="center"> A Tool-box for Efficient Build and Debug in Python. Especially for <strong>Scientific Computing</strong> and <strong>Computer Vision</strong>. </div> <br/> <div align="center"> <!-- Build Status --> <a href="https://travis-ci.org/DIYer22/boxx">

Overview: `ml_mdm` is a Python package designed for efficient training of high-quality text-to-image diffusion models, leveraging the Matryoshka Diffusion Models technique. This project accompanies a research paper on the topic and aims to simplify the process of training and generating images using diffusion models. Key features: - Training of high-resolution text-to-image diffusion models - Pretrained models available for download and usage - Web demo for generating images directly from the models - Installation instructions for both CPU and GPU environments Tech stack: Python Why it matters: This project provides a practical solution for generating high-quality images using diffusion models, making it accessible for researchers and developers interested in the field of image synthesis and diffusion models. Repository: apple-aiml-research/ml-mdm GitHub stars: 516

### Overview: BigCodeBench is a benchmarking tool designed to evaluate and compare the performance of code generation models. It is widely used by leading AI teams, including Zhipu AI, Alibaba Qwen, DeepSeek, Amazon AWS AI, Snowflake AI Research, Meta AI, Cohere AI, Sakana AI, and Allen Institute for Artificial Intelligence (AI2). ### Key features: - **Real-time Code Execution**: Allows users to evaluate models directly in the browser. - **Leaderboard**: Provides a leaderboard for the best-performing models. - **Public Code Execution API**: Enables public evaluation of models. - **Real-time Code Execution Session**: Offers reproducible evaluation sessions. - **1140 Software-Engineering-Oriented Programming Tasks**: Includes a wide range of tasks relevant to real-world programming. ### Tech stack: Python ### Why it matters: BigCodeBench is crucial for evaluating and benchmarking code generation models, providing valuable insights into their performance and capabilities. Its widespread use by leading AI teams underscores its importance in the field of artificial intelligence and machine learning. ### Repository: - **Repository**: bigcode-project/bigcodebench - **Stars**: 516 - **Description**: [ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI ### Primary language: - **Primary Language**: Python ### GitHub description: - [ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI

### Overview: SIREN (Simplified Implicit Neural Representations) is a Pytorch implementation of the SIREN neural network, which is designed to handle implicit representations with periodic activation functions. This project aims to provide a practical tool for researchers and developers to experiment with and apply SIREN in their projects. ### Key Features: - **SIREN Network Implementation**: A complete Pytorch implementation of the SIREN neural network. - **Implicit Neural Representations**: The network is capable of representing complex functions implicitly, making it useful for tasks where explicit representations are difficult to obtain. - **Periodic Activation Function**: The activation function used in SIREN is periodic, which can help in capturing periodic patterns in data. - **Modulation with Latent Code**: The project includes a method to condition the SIREN network with a latent code, enhancing its ability to learn from specific inputs. ### Tech Stack: - **Python**: The project is written in Python, leveraging the Pytorch framework for deep learning. ### Why It Matters: SIREN is particularly useful for tasks where explicit representations are challenging to obtain, such as in image generation, data interpolation, and other applications where implicit representations can provide a more efficient and effective solution. ### Repository: - **Repository**: lucidrains/siren-pytorch - **GitHub Stars**: 516 - **GitHub Description**: Pytorch implementation of SIREN - Implicit Neural Representations with Periodic Activation Function ### Installation: ```bash $ pip install siren-pytorch ``` ### Usage:

Overview: Agency is a Go library designed for developers to explore and utilize Large Language Models (LLMs) and other generative AI through a clean, effective, and idiomatic approach. Key features: - Pure Go - Clean code and follow clean architecture - Easy to create custom operations - Compose operations into processes with interceptors - OpenAI API bindings Tech stack: Go Why it matters: Agency simplifies the creation of autonomous AI systems, empowering users to build autonomous agents with ease and efficiency, and fills the gap in Go-native solutions for generative AI applications.

Overview: This project provides a Python interface to Stanford CoreNLP, a reference implementation for interacting with the Stanford CoreNLP server. Key features: - Python interface - Bidirectional server-client communication - Supports annotation of sentences and tokens Tech stack: Python Why it matters: This package simplifies the integration of Stanford CoreNLP into Python applications, making it easier to leverage the powerful natural language processing capabilities of Stanford CoreNLP within Python projects.

### Overview: This project is a production-ready MVP that securely chats with your documents using OpenAI's GPT models and retrieval augmented generation (RAG). It leverages the capabilities of pgvector to provide a fast and efficient chat interface. ### Key Features: - **Interactive Chat Interface:** Interact with your documentation, leveraging the capabilities of OpenAI’s GPT models and retrieval augmented generation (RAG). - **Login With 3rd Party:** Integrate one-click 3rd party login with any of our 18 auth providers and user/password. - **Document Storage:** Securely upload, store, and retrieve user uploaded documents. - **REST API:** Expose a flexible REST API that we’ll consume to build the interactive front-end. - **Row-level Security:** Secure all of your user data with production-ready row-level security. ### Tech Stack: TypeScript ### Why it Matters: This project is designed to help developers secure and efficiently chat with their documents, making it a valuable tool for anyone looking to integrate AI into their document management systems. The use of pgvector ensures fast and efficient querying, while the integration with 3rd party authentication providers makes it easy to integrate with existing systems. The production-ready row-level security ensures that user data is secure and protected. Repository: supabase-community/chatgpt-your-files Primary language: TypeScript GitHub stars: 515 GitHub description: Production-ready MVP for securely chatting with your documents using pgvector README excerpt: <img alt="pgvector to Prod in 2 hours" src="./assets/

Overview: Mergoo is a library designed to easily integrate and efficiently train multiple language models (LLMs) into a single, unified expert system. This allows for the combination of knowledge from different experts, enhancing the overall performance and capabilities of the resulting model. Key features: - Supports multiple merging methods: Mixture-of-Experts, Mixture-of-Adapters, and Layer-wise merging. - Flexible merging for each layer. - Base models supported: LLaMA, Mistral, Phi3, and BERT. - Trainers supported: Hugging Face Trainer, SFTrainer, and PEFT. - Device support: CPU, MPS, and GPU. - Training choices: Only the router of MoE layers or fully fine-tuning of the merged LLM. Tech stack: Python Why it matters: Mergoo simplifies the integration and training process of multiple LLM experts, making it easier to create highly specialized and powerful AI systems.

### Overview: Pixel-Adaptive Convolutional Neural Networks (PacNet) is a novel approach to convolutional neural networks that adapts to the local pixel context, significantly improving the performance on image classification tasks. ### Key Features: - **Adaptive Convolutional Layers**: PacNet introduces PAC layers that adaptively modify the convolutional operations based on the local pixel context, enhancing model performance. - **Layer Catalog**: The project includes five types of PAC layers: `PacConv2d`, `PacConvTranspose2d`, `PacPool2d`, `PacCRF`, and `PacCRFLoose`. - **Installation**: The project requires Python 3.5 or higher, and dependencies such as PyTorch and additional libraries are installed via pip. ### Tech Stack: Python ### Why it Matters: PacNet addresses the limitations of traditional CNNs by providing more context-aware and adaptive features, leading to improved accuracy and efficiency in image classification tasks.

Overview: Tangle is a web-based platform designed to simplify the creation and execution of Machine Learning (ML) pipelines, making it accessible to users without the need for a development environment setup. Key features: - Seamless integration with HuggingFace for easy pipeline execution - Support for various ML tasks including XGBoost training - Real-time monitoring and management of pipeline runs - Multi-tenant architecture for easy deployment and scalability - Local and cloud deployment options Tech stack: Python, Docker, HuggingFace Why it matters: Tangle simplifies the complex process of building and running ML pipelines, making it accessible to a wider audience and reducing the barrier to entry for ML enthusiasts and professionals alike.