Projects

### Overview: 이 저장소는 한빛미디어에서 출간한 "만들면서 배우는 생성 AI 2판"의 코드를 담고 있습니다. 이 책은 디지털 콘텐츠 생성을 위한 다양한 딥러닝 모델을 소개하고 있습니다. 코드는 텐서플로 2.12.0, 2.14.0에서 테스트되었습니다. ### Key features: - **生成模型介绍** (PART 1): 셍성 모델링, 딥러닝, 다층 퍼셉트론, 합성곱 신경망, 변이형 오토인코더, 생성적 적대 신경망, 자기 회귀 모델, 노멀라이징 플로 모델, 에너지 기반 모델, 확산 모델. - **生成模型应用** (PART 3): 트랜스포머, 고급 GAN, 음악 생성, 월드 모델, 멀티모달 모델, 결론. ### Tech stack: Jupyter Notebook ### Why it matters: 이 저장소는 디지털 콘텐츠 생성을 위한 다양한 딥러닝 모델을 소개하고 있습니다. 이 책을 통해 디지털 콘텐츠 생성에 대한 이해를 높이고, 다양한 딥러닝 모델을 직접 실행해보는 기회를 제공합니다.

### Overview: This project is a practical lab for exploring Apple's Core AI framework, model assets, specialization, and on-device inference. ### Key features: - **Explore** 33 pinned Apple model recipes across language, vision, audio, and diffusion. - **Run** task-specific playgrounds or inspect an arbitrary `.aimodel` through the generic Function Workbench. - **Prove and package** results with project provenance, benchmark evidence, physical-device evidence, and verified Swift integration exports. ### Tech stack: Swift ### Why it matters: This workbench provides a native SwiftUI environment for discovering, converting, inspecting, running, benchmarking, and packaging models with Apple's `CoreAI.framework`.

### Overview: Tasknet simplifies modernBERT fine-tuning and multi-task learning, making it easy to integrate with various datasets and datasets. ### Key Features: - **Easy Integration**: Tasknet seamlessly integrates with Huggingface datasets and transformers, making it straightforward to fine-tune models. - **Multi-Task Support**: Designed for multitasking, Tasknet allows you to train models on multiple tasks simultaneously. - **Customization**: Provides a flexible interface for custom task templates, allowing developers to create their own models. - **Balancing Dataset Sizes**: Automatically balances dataset sizes to ensure even training across different tasks. ### Tech Stack: - **Primary Language**: Jupyter Notebook ### Why It Matters: - Simplifies modernBERT fine-tuning and multi-task learning, making it accessible to developers. - Enables easy integration with various datasets and datasets, enhancing versatility. - Facilitates multitasking by default, reducing boilerplate code and improving efficiency. ### GitHub Stars: 66 ### Installation and Example: ```bash pip install tasknet ``` ```python import tasknet as tn from datasets import load_dataset # Example of using a classification template rte = tn.Classification( dataset=load_dataset("glue", "rte"), s1="sentence1", s2="sentence2", y="label" ) # Example of using an AutoTask template rte = tn.AutoTask("glue/rte", max_rows=5000) # Example of using a custom task template class hparams: model_name

Overview: `explainmyrepo` is a tool that converts any GitHub repository into a visually engaging explainer page, making complex projects accessible to both humans and AI assistants. Key features: - Converts any GitHub repo into a live explainer page - Provides a bespoke architecture diagram and data-flow diagram - Generates a downloadable AI knowledge pack for further exploration - Supports live examples and 5 live examples Tech stack: JavaScript Why it matters: It simplifies the understanding of complex projects, making them accessible to both humans and AI, thereby improving collaboration and knowledge sharing.

### Overview: Modality-Transferable Emotion Embeddings for Low-Resource Multimodal Emotion Recognition ### Key features: - **Modality Transferable Model**: Utilizes pre-trained word embeddings to represent emotion categories for textual data. - **Mapping Functions**: Learns two mapping functions to transfer embeddings into visual and acoustic spaces. - **Zero-Shot and Few-Shot Performance**: Achieves state-of-the-art performance on most emotion categories and outperforms existing baselines in zero-shot and few-shot scenarios for unseen emotions. ### Tech stack: - Python ### Why it matters: This model addresses the limitations of current multimodal emotion recognition models by leveraging pre-trained embeddings and modality-specific mapping functions, leading to improved performance across various modalities and low-resource scenarios.

Overview: This project is an advanced multimodal video search system that utilizes Large Language Models (LLMs) and various search techniques to provide intelligent and efficient video retrieval. The system was developed for the AIC2024 competition and accepted at the international SOICT 2024 conference. Key features: 1. Multimodal Search Capabilities: Supports text-based, image-based, and metadata-based queries for improved accuracy. 2. LLM-Powered Interaction: Integrates LLMs to handle natural language queries and deliver relevant search results. 3. User-Friendly Interface: Allows users to view results as keyframes or full video segments and interact with detailed metadata. Tech stack: Django, CLIP, Faiss, TFIDF, OpenCV, PyTorch, Transformers. Why it matters: This system aims to provide an intelligent and efficient video search solution, leveraging the power of LLMs to enhance the retrieval process and make it more user-friendly. Repository: xndien2004/LLM_Powered_Video_Search Primary language: Jupyter Notebook GitHub stars: 66 GitHub description: [SOICT 2024] LLM-Powered Video Search: A Comprehensive Multimedia Retrieval System README excerpt: ⭐ If you find this project helpful or interesting, please consider giving it a star - your support means a lot!

### Overview: Gemini WebAPI-MCP is a MCP server for Google Gemini, providing free image generation, editing, and chat functionality through browser cookies. It eliminates the need for API keys, making it accessible and cost-free. ### Key Features: - **Image Generation**: Generate images from text descriptions. - **2x Resolution**: Automatically upscales images to 2816x1536 and higher. - **Image Editing**: Edit uploaded images with prompts. - **File Analysis**: Analyze videos, images, PDFs, and documents. - **Text Chat**: Interact with Gemini via Flash, Pro, and Flash-Thinking. - **Auto-Remove Watermark**: Automatically removes the Gemini watermark using a deterministic Reverse Alpha Blending method. - **Auto-Authentication**: Authenticate using cookies from Chrome. ### Tech Stack: Python ### Why It Matters: Gemini WebAPI-MCP is a powerful tool for anyone looking to create, edit, and interact with images and text-based conversations through Google Gemini without the need for API keys, making it a versatile and accessible solution for various applications and projects.

### Overview: Dyadic Interaction Modeling for Social Behavior Generation is a pre-training strategy that jointly models speakers' and listeners' motions, enabling the generation of photorealistic videos for the listener's motion. This project was presented at ECCV 2024 and is part of the CANDOR Corpus, LM_Listener, and BIWI datasets. ### Key Features: - **Pre-training of Dyadic Interaction Model**: The model jointly learns representations that capture the dyadic context between speakers and listeners. - **Multimodal Input Processing**: The model processes inputs from the speaker and feeds them into DIM-Listener for generating realistic videos. - **Pre-trained Weights Utilization**: The pre-trained weights are utilized to enhance the model's performance on specific datasets. ### Tech Stack: - Python 3.12.3 - PyTorch 2.3.1 - CUDA 12.5 ### Why It Matters: This project introduces a novel pre-training strategy that significantly improves the generation of realistic videos for social interactions, making it valuable for applications in virtual reality, animation, and more. ### Repository: - Repository: [Boese0601/Dyadic-Interaction-Modeling](https://github.com/Boese0601/Dyadic-Interaction-Modeling) - GitHub Stars: 65 - GitHub Description: [ECCV 2024] Dyadic Interaction Modeling for Social Behavior Generation - Primary Language: Python - GitHub URL: [https://github.com/Boese060

### Overview: *Sketch-a-Sketch* is a tool that enables users to control the output of generative AI from simple sketches with just a few strokes, making it accessible for those who are not great at drawing. ### Key features: - **Controlled Diffusion**: Uses simple sketches to control the generation of images. - **Interactive Suggestions**: Provides visual feedback on the next lines to draw for better image quality. - **Winning Sketches**: Helps in creating "winning" sketches by suggesting future lines. ### Tech stack: Python ### Why it matters: *Sketch-a-Sketch* simplifies the process of controlling generative AI, making it more accessible and enjoyable for users who are not skilled at drawing.

### Overview: NovoWeave is a conceptual generative protein-design framework designed for researchers to discuss system design before committing to an implementation. The blueprint separates concerns such as backbone generation, sequence design, evaluation, and provenance into modular, testable contracts. ### Key features: - **Backbone Proposal:** A diffusion-style geometric model interface. - **Sequence Design:** A structure-conditioned transformer interface. - **Evaluation:** Confidence, geometry, and diversity metric contracts. - **Selection:** Auditable multi-objective ranking with human review gates. ### Tech stack: Python 3.10+ ### Why it matters: This framework provides a clear, modular structure for researchers to discuss system design without committing to implementation details, ensuring that discussions remain focused on the design principles rather than implementation specifics.

### Overview: This repository contains 8 tested recipes for Wan 2.7 Video Edit prompts, including examples of rendered videos and the associated costs on the APIMart route. ### Key Features: - **Model ID:** `wan2.7-videoedit` - **Cost per Image:** $0.0664 per second - **Aspect Ratios:** 16:9, 4:3, 1:1 - **Example Renders:** Product ad, Cinematic, Food ad, etc. - **Prompt Structure:** Structured in JSON format and Markdown table. ### Tech Stack: Python ### Why It Matters: This repository provides a practical guide for developers and users looking to optimize their video editing workflows using Wan 2.7 Video Edit prompts.

Overview: Seedance 1.0 Pro API (`seedance-1.0-pro`) is a fast, OpenAI-compatible API endpoint provided by ByteDance, offering a single endpoint for generating high-quality video previews and rapid previews. It supports various resolutions and pricing models, making it ideal for developers looking to integrate AI into their projects without the need for a subscription. Key features: - A single endpoint for generating high-quality video previews and rapid previews - Compatible with OpenAI, supporting 300+ models for image, video, and language generation - Billing by resolution, with a flat rate of $0.0184 per second - Supports various resolutions and pricing tiers - Assynchronous by design, allowing for easy task tracking and retries Tech stack: Python Why it matters: Seedance 1.0 Pro API is a powerful tool for developers looking to integrate AI into their projects quickly and cost-effectively. Its single endpoint and flat rate make it an ideal solution for developers who need to generate high-quality video previews without the need for a subscription. Repository: a7m-7rm3vb/doubao-seedance-1-0-pro-api-pt Primary language: Python GitHub stars: 44 GitHub description: Dreamina — Seedance 1.0 Pro API (`seedance-1.0-pro`): model id, pricing, parameters and copy-paste examples for the OpenAI-compatible APIMart endpoint.

Overview: Seedream 5.0 Flash API (`seedream-5-0-flash`) is a low-cost, OpenAI-compatible model for generating high-quality images, text, and videos. It supports batch jobs and rapid iteration, and is priced per delivered image at $0.0137. Key features: - Supports 1K, 1.5K, and 2K image generation at one per-image price. - Offers strong text rendering and multi-reference style fusion. - Runs in a single API call, making it easy to adopt in minutes. - Supports batch jobs and rapid iteration. - Pricing is per delivered image at $0.0137, with a snapshot price of $0.017143. Tech stack: Python Why it matters: Seedream 5.0 Flash API is a cost-effective alternative to higher-priced models, making it ideal for teams looking to generate high-quality images and text at a lower cost. Repository: ac0y48cgee/seedream-5-0-flash-api-ru Primary language: Python GitHub stars: 51 GitHub description: Seedream 5.0 Flash (`seedream-5-0-flash`) is a low-cost, OpenAI-compatible model for generating high-quality images, text, and videos. It supports batch jobs and rapid iteration, and is priced per delivered image at $0.0137. Seedream 5.0 Flash API is a cost-effective alternative to higher-priced models, making it ideal for teams looking to generate

### Overview: This project demonstrates the application of image classification using PyTorch, focusing on training models on a custom dataset of 43,956 images across 11 classes. The dataset includes both training and validation sets, and the project evaluates three different approaches: training from scratch, finetuning a pre-trained convolutional neural network, and using a pre-trained feature extractor. ### Key Features: - **Custom Dataset**: 43,956 images across 11 classes. - **Training Methods**: - Training from scratch - Finetuning a pre-trained convolutional neural network (VGG11, ResNet18, MobilenetV2) - Using a pre-trained feature extractor - **Models Used**: - VGG11 - ResNet18 - MobilenetV2 - **Training Details**: - VGG11: 100 epochs - ResNet18: 10 epochs - MobilenetV2: 10 epochs - Training time: VGG11 - 900 mins, ResNet18 - 42 mins, MobilenetV2 - 32 mins - Batch size: 64 - GPU: Tesla K80 ### Tech Stack: - Python - PyTorch - Torchsummary - Tensorboard ### Why It Matters: - This project showcases the effectiveness of different training methods and models in image classification

Overview: Treds is a data structure server that sorts keys in ascending order, providing fast and efficient retrieval. It is optimized for prefix searches and supports various data structures like Sorted Maps, Lists, Sets, and Hashes. Key features: - Supports Sorted Maps, Lists, Sets, and Hashes - Stores keys in sorted order for efficient retrieval - Implements a Radix Trie with Doubly Linked Lists for quick key lookup - Optimizes key creation, deletion, and updates - Supports sorted keys and values with associated scores - New commands: DELPREFIX, LNGPREFIX, and PSUBSCRIBE - Runs on a single thread with an event loop - Internally uses a modified Radix Trie with Doubly Linked Lists Tech stack: Go Why it matters: Treds is designed to handle large datasets efficiently, providing fast and accurate prefix searches, and supports various data structures for complex key-value storage. It is optimized for performance and ease of use, making it suitable for applications requiring high-speed key-value lookups and sorted data retrieval. Repository: asheshvidyut/treds GitHub stars: 65 GitHub description: Sorted Data Structure Server - Treds is a Data Structure Server which returns data in sorted order and is the fastest prefix search server. It also persists data on disk.

Overview: This repository contains curated prompts for OpenAI's GPT Image 2, designed to enhance the creation of professional-grade text, consistent characters, and pixel-perfect images. The prompts are optimized for creators and developers. Key features: - Flawless Typography - Identity Retention - Structured Design - Artistic Intent Tech stack: TypeScript Why it matters: This collection of prompts is essential for anyone looking to master OpenAI's GPT Image 2, ensuring high-quality and consistent results in their projects. Repository: bubblesslayyer-cmd/Awesome-GPT-Image-2-OpenAi GitHub stars: 65

### Overview: ShEPhERD-2 is a 3D molecular generative model that leverages interaction profiles composed of shape, electrostatics, and pharmacophores to generate diverse and contextually relevant molecular structures. This approach enables precise control over the design process through pharmacophore prioritization, substructure constraints, and the composition of multiple interaction profiles, facilitating bioisosteric fragment merging, dual-target design, selectivity engineering, and modality hopping. ### Key features: - **Interaction Profile Composition**: ShEPhERD-2 uses interaction profiles that include shape, electrostatics, and pharmacophores to generate diverse molecular structures. - **Precise Control**: Pharmacophore prioritization, substructure constraints, and composition of multiple interaction profiles provide precise control over the design process. - **Bioisosteric Fragment Merging**: Enables the merging of bioisosteric fragments, enhancing the diversity and relevance of generated molecules. - **Dual-Target Design**: Facilitates the design of molecules that address multiple targets simultaneously. - **Selectivity Engineering**: Allows for the optimization of selectivity in the design process. - **Modality Hopping**: Enables the exploration of different modes of action in the design process. ### Tech stack: Python ### Why it matters: ShEPhERD-2 represents a significant advancement in generative molecular design by providing a flexible and powerful framework for interaction-conditioned molecular generation. Its ability to generate diverse and contextually relevant structures while maintaining precise control makes it highly valuable in drug discovery and other chemical design applications.

### Overview: MatFuse simplifies material generation by leveraging diffusion models, offering fine-grained control and flexibility. ### Key features: - **Unified Approach**: MatFuse simplifies the creation of SVBRDF maps using diffusion models. - **Fine-Grained Control**: Integrates conditioning from multiple sources (color palettes, sketches, text, pictures) for detailed material synthesis. - **Map-Level Editing**: Enables editing or refining of synthesized materials after initial generation. - **Multi-Encoder Compression**: Improves reconstruction performance and enables map-level material editing through a two-fold model. ### Tech stack: Python ### Why it matters: MatFuse simplifies the process of creating high-quality materials in computer graphics, making it easier and more accessible for artists and developers.

### Overview: A versatile ComfyUI workflow designed for common use cases. Ideal for text-to-image generation, image blending, style transfer, and more. ### Key Features: - **Versatile Flux**: A highly versatile workflow. - **SDXL and SD1.5 Workflow**: Includes advanced techniques like IPadapter, ControlNet, IC light, LLM prompt generating, and more. - **ComfyUI Essentials**: Essential ComfyUI scripts. - **ComfyUI-Easy-Use**: Simplified ComfyUI setup. - **ComfyUI-Custom-Scripts**: Customizable scripts for advanced users. - **ComfyUI-GGUF**: Supports GGUF files. - **ComfyUI-IPAdapter-Plus**: Enhanced IPAdapter integration. - **ComfyUI-Anyline**: Supports Anyline files. - **ComfyUI-IC-Light**: IC light effects. - **ComfyUI-IC-Light**: IC light effects. - **ComfyUI-UltimateSDUpscale**: Upscaling for SD models. ### Tech Stack: ComfyUI ComfyUI is the tech stack used in this workflow. ### Why It Matters: This workflow provides a comprehensive set of tools for text-to-image generation, blending, style transfer, and more, making it a go-to solution for most use cases.

Overview: This project provides a Dockerfile for compiling Kaldi for Android, based on instructions from jcsilva.github.io. Key features: - Dockerfile for compiling Kaldi for Android - Instructions for installing Docker - Steps to pull or build the Docker image Tech stack: Shell Why it matters: This project streamlines the process of compiling Kaldi for Android, making it easier for developers to set up and run the application on Android devices.