Projects

Overview: This project is an implementation of the paper "Multi-Modal Sarcasm Detection in Twitter with Hierarchical Fusion Model," focusing on sarcasm detection using a hierarchical fusion model in Twitter data. Key features: - Hierarchical Fusion Model - Multi-Modal Data Integration - Twitter-Specific Preprocessing Tech stack: Python, PyTorch Why it matters: This project provides a practical solution for sarcasm detection in Twitter data, leveraging advanced machine learning techniques to improve the accuracy of sarcasm detection in social media platforms. Repository: wrk226/pytorch-multimodal_sarcasm_detection Primary language: Jupyter Notebook GitHub stars: 14 GitHub description: It is the implementation of paper "Multi-Modal Sarcasm Detection in Twitter with Hierarchical Fusion Model"

Overview: Prompt Forge 是一个用于生成 NovelAI 提示词的工具，它能将一句话翻译成符合特定风格的标签流，然后在本地运行「NAI 整理 + 提示词体检」，生成的提示词可编辑并可保存。它提供了一个纯本地、零运行时依赖的解决方案，适用于多种OpenAI接口。 Key features: - 只做 NovelAI — 正向统一 danbooru 标签流、负向统一 UC； - 生成后本地再跑一遍「NAI 整理 + 体检」； - 提供可编辑的置顶破限词、角色设定、细分标签与拼装板、设定集、历史与保存库； - 纯本地、零运行时依赖，直连任意 OpenAI 兼容接口。 Tech stack: JavaScript。 Why it matters: Prompt Forge 是一个简化了生成 NovelAI 提示词过程的工具，它能将一句话翻译成符合特定风格的标签流，然后在本地运行「NAI 整理 + 提示词体检」，生成的提示词可编辑并可保存。它提供了一个纯本地、零运行时依赖的解决方案，适用于多种OpenAI接口。 Repository: yu20000505/prompt-forge Primary language: JavaScript GitHub stars: 12 GitHub description: 一句话生成 NovelAI 出图提示词：正向 danbooru 标签流 + UC 负向，生成后本地再跑

Overview: Wan 2.7 Image API is a model id, pricing, and parameters for the OpenAI-compatible API endpoint, providing high-quality image generation at a cost of $0.0216 per image. Key features: - Supports multiple model versions for high-quality images - Offers flexible pricing with a minimum cost of $1 per task - Tracks and reports costs in real-time - Provides API key for easy integration Tech stack: Python Why it matters: This API allows developers to leverage Wan 2.7 Image model for generating high-quality images at a competitive cost, making it ideal for various applications requiring image generation without the need for a monthly subscription.

Overview: Qwen-Image 3.0 Pro API is a high-fidelity image generation service that offers 1K and 2K output resolutions at a cost of $0.0285712 per image, making it an ideal choice for developers seeking advanced image capabilities without the need for a monthly subscription. Key features: - Supports 1K and 2K output resolutions - Charges $0.0285712 per image, by resolution - Compatible with OpenAI-compatible API endpoint - Asynchronous processing, task results available via `GET /v1/tasks/{task_id}` - Pricing is based on the actual cost of the generated images Tech stack: Python Why it matters: Qwen-Image 3.0 Pro API is designed for developers who need advanced image capabilities without the overhead of a monthly subscription. It offers a single endpoint that can be used for multiple models, making it a cost-effective solution for image generation projects. GitHub stars: 12 Repository: z3phc/qwen-image-3-0-pro-api-fr

Overview: GPT Image 1.5 API is an OpenAI-compatible model serving image generation via APIMart, featuring token-based pricing and an endpoint compatible with multiple models. Key features: - Supports OpenAI's GPT Image 1.5 model - Offers token-based billing for text and image inputs - Compatible with various image resolution tiers - Supports API key authentication and direct pricing Tech stack: Python Why it matters: Provides a convenient and flexible way to integrate image generation into applications, with detailed pricing and billing information available.

### Overview: NLPCC 2024 Shared Task 10: Regulating Large Language Models focuses on developing methods to detect and mitigate undesirable outputs from large language models, ensuring the reliability and ethical standards of the generated content. ### Key features: - Multimodal Hallucination Detection for Multimodal Large Language Models - Detoxifying Large Language Models ### Tech stack: Python ### Why it matters: This task is crucial for maintaining the integrity and ethical standards of large language models, ensuring that the outputs are reliable and adhere to community guidelines and ethical standards.

Overview: A financial chatbot powered by llama-2 family of models and Retrieval-Augmented Generation (RAG) to answer finance-related questions and cite sources. Key features: - Uses llama-cpp-python for the LLM with the fastest inference on CPU. - Scrapes data from Zerodha Varsity for the RAG database. - Answers questions concisely and cites sources, improving the quality of the generated text. Tech stack: Jupyter Notebook Why it matters: This project showcases the capabilities of RAG and the potential of LLMs on CPUs, demonstrating how to create a chatbot using one's own data and reducing inference time significantly. Repository: 007prateekd/fin-edubot GitHub stars: 13 GitHub description: A financial chatbot powered by llama-2 family of models and Retrieval-Augmented Generation (RAG) to answer finance-related questions and cite sources on your CPU.

Overview: This project explores multimodal document understanding by combining vision-language models with text retrieval to answer questions over PDFs containing figures, tables, and complex layouts. Key features: Baseline retrieval, VLM evaluation pipeline, fused reranker, LLaVA answer generation. Tech stack: LLaVA-1.6 (Mistral-7B variant), LangChain, LlamaIndex, pdf2image, Tesseract, accelerate, PyTorch, transformers (HF). Why it matters: This project aims to improve the handling of technical PDFs by maintaining visual context and querying both modalities, aiming to provide better answers to questions over complex layouts. Repository: 0xzall/multimodal-doc-qa Primary language: Python GitHub stars: 13 GitHub description: Multimodal document QA: vision + retrieval over PDFs (LLaVA + LlamaIndex) Read this for the actual story: The README is mostly here so GitHub has something pretty. The honest version of what's working, what isn't, and why I made each decision lives in [RESEARCH-LOG.md].

Overview: code-llm-completion is a focused training & serving stack for fine-tuning open code LLMs into fast, IDE-grade completion models, with support for low-resource languages. Key features: - Fine-tuning of StarCoder2-7B and DeepSeek-Coder-6.7B models - Support for low-resource languages like Indonesian PHP/Laravel codebases, Lua game scripts, and in-house DSLs - Focus on low latency, high acceptance rate, and easy deployability Tech stack: Python Why it matters: This project aims to provide developer tools for the on-prem inference market, focusing on low-resource languages and fast, easy-to-deploy completion models. Repository: 0xzuky/code-llm-completion GitHub stars: 13 GitHub description: AI/ML research project on AMD ROCm

Overview: Longbow is a high-performance vector engine built on Apache Arrow, designed for modern AI/Agentic workloads. It leverages zero-copy data paths, SIMD optimizations, and advanced storage backends to deliver sub-millisecond latency. Key features: - Built on Apache Arrow for zero-copy data transfer. - Consistent hashing and gossip-based membership (SWIM protocol). - Optional `io_uring` WAL backend for high-throughput ingestion. - NUMA-aware memory allocation and SIMD vector distance calculations. - Resilient Go SDK for transparent request routing. Tech stack: Go Why it matters: Longbow is designed for high throughput and low latency, making it ideal for modern AI/Agentic workloads where performance is critical. Repository: 23skdu/longbow GitHub stars: 13 GitHub description: Apache Arrow Flight clustered vector engine for high throughput Agent memory sharing

### Overview: BDM (Bridge Diffusion Model) is a novel method designed to develop non-English language-native TTI (text-to-image) models that are compatible with the English Stable Diffusion community. This approach addresses the inherent language-related biases in existing English TTI models, ensuring a more balanced representation of diverse linguistic communities. ### Key Features: - **Language-Neutral Representation**: BDM generates images that are consistent with Chinese language semantics, making it suitable for Chinese-native users. - **Compatibility with SD Community**: The model is designed to be seamlessly integrated with the English Stable Diffusion ecosystem, ensuring compatibility and ease of use. - **Incorporation of External Translation**: By fine-tuning with Chinese-native data, BDM can assimilate Chinese-native language semantics without compromising compatibility with the English community. ### Tech Stack: Python ### Why it Matters: Developing non-English language-native TTI models is crucial because existing English models often exhibit language-related biases. By bridging these biases, BDM ensures a more balanced and diverse representation of linguistic communities, particularly those in non-English environments. This approach not only enhances the inclusivity of AI-generated content but also fosters a more collaborative and innovative ecosystem for AI development. ### Repository: 360CVGroup/Bridge_Diffusion_Model ### GitHub Stars: 13 ### Primary Language: Python ### GitHub Description: Chinese-native image generation while compatible with SD eco-system, 1st-gen, AAAI2025

### Overview: RWKV-Server 是一款高性能的大型语言模型推理服务，专为高并发场景设计。它支持动态扩缩容，兼容OpenAI API，采用CUDA优化，适用于低显存部署的大模型推理。RWKV-Server 提供了自动入队、动态容量管理和无头模式的压力测试脚本，适用于多种使用场景。 ### Key features: - **高性能推理**：支持2.9B和7.2B模型，峰值吞吐量分别为11,081 tok/s和6,373 tok/s。 - **动态扩缩容**：支持从256个任务开始，逐步减少到1个任务，以适应不同规模的部署需求。 - **兼容OpenAI API**：支持与OpenAI API的无缝集成，提供流畅的对话体验。 - **低显存部署**：适用于低显存环境，有效降低显存使用。 - **无头模式**：支持Web UI和无头模式，方便管理和监控。 ### Tech stack: Python ### Why it matters: RWKV-Server 是一个高性能、低显存、易扩展的推理服务，适用于各种需要大规模模型推理的场景，如自然语言处理、对话系统等。它的兼容性、动态扩缩容和低显存部署特性使其成为此类应用的理想选择。

### Overview: LLM Energy Lab is a web application and API designed to benchmark and compare the energy consumption, emissions, and behavior of Large Language Models (LLMs) across different inference configurations, middleware scaffolds, and post-training implementations. It provides insights into the impacts of various factors on LLM's raw behaviors and energy footprint, offering a comprehensive view of LLM's performance and environmental impact. ### Key features: - **Energy Testing Lab (primary)**: - **Per-input/output-token energy metrics**: Metrics are normalized by input or output, with Wh/1000 e-tokens. - **Live Hardware Tests (RAPL)**: Snapshot-based cumulative energy counters with start/split/end snapshots for Prefill and Decode energy. - **Estimated Benchmark Tests**: Ability to switch and create benchmarks, recalculate session with new benchmarks. - **Middleware Injections and Configuration**: Custom and Frontier System Prompts, Conversation Context, free-form injections, and temperature toggles. - **RAPL Batch Runner**: Multiple live runs using current UI query + injections with CLI logs and stop support; results table appears in Test Results. - **Real-time streaming**: Token streaming over WebSocket, supports reasoning "thinking" streams when present. - **LLM Behaviour Lab (WIP)**: - **Multi-model comparison panes with per-pane controls**. - **Streaming outputs, token counts (prompt/completion), and TPS**. ### Tech stack: Python ### Why it matters: LLM Energy Lab provides a valuable tool for developers and researchers to understand

### Overview: The Daily Paper Club is a collaborative learning community dedicated to understanding multi-modality models and their applications. Hosted at Agora every 10 PM NYC time, the club aims to foster a deep understanding of AI and mathematics through discussions on topics such as multi-modality reasoning, generation, and swarm collective intelligence. ### Key Features: - **Weekly Discussions**: Topics are covered every Thursday at 10 AM Eastern time. - **Moderation by AI**: AI-driven moderation ensures a safe and engaging environment. - **Community Engagement**: Members contribute to the community through discussions, sharing insights, and offering solutions. ### Tech Stack: The project uses Python for organizing and managing the discussions. ### Why It Matters: Understanding the intersection of various modalities in AI and mathematics will lead to breakthroughs in AI applications, making this community essential for the future of AI. ### Repository: The repository is hosted on GitHub and is open for contributions. It has received 13 GitHub stars, indicating a growing interest in the project. ### Primary Language: The primary language is not specified in the README excerpt. ### GitHub Description: The repository for the exclusive Daily Paper Club hosted at Agora every 10 PM NYC time at this discord: https://discord.gg/Gnzh6dnzyz

### Overview: ThingiPano is a comprehensive dataset containing metadata, images, and panoramic renders of 3D printing designs, designed to facilitate large-scale analysis and support design reuse in 3D printing. ### Key Features: - **Metadata**: Publicly available metadata from Thingiverse (up to 2018), including 283,873 users and 1,017,687 designs. - **Images**: 1,816,295 images associated with designs. - **Panoramic Renders**: 1,052,017 3-view panoramic 3D depth-map renders of 3D files, available for both Thingiverse and Shapenet. - **Validation Data**: Panoramic renders for Shapenet, along with Matlab source code for generating these renders. - **Compressed Sizes**: - [Thingiverse Images]: 97.5 GB - [Rendered Panoramas]: 27.6 GB - [Design Metadata]: 717 MB - [User Metadata]: 28.1 MB - [Shapenet Panoramas]: 1.34 GB ### Tech Stack: MATLAB ### Why It Matters: ThingiPano is a valuable resource for researchers and designers interested in exploring design reuse, 3D printing metadata, and large-scale analysis of 3D files. The dataset supports qualitative analysis and machine-learning applications, making it a cornerstone for advancing the field

Overview: This project fine-tunes T5, a large language model, on a dataset of varied text to improve its performance on NLP tasks such as Text-to-SQL. Key features: - Fine-tuning T5 on a large dataset of text - Pre-trained on diverse text datasets including Wikipedia, news articles, and books - Evaluated on the SPIDER dataset for Text-to-SQL tasks - Improved performance by 16% from 80% to 93% - Implemented rule-based post-processing to further improve scores Tech stack: Jupyter Notebook Why it matters: This project showcases the effectiveness of fine-tuning a large language model on a diverse dataset, achieving significant improvements in performance for Text-to-SQL tasks.