Projects

### Overview: This project involves fine-tuning a pretrained language model (LM) to generate text based on a specific novel, Moby Dick, using Python and the Flair framework. The goal is to explore the potential of using pre-trained models for generating novel content. ### Key Features: - Fine-tuning a pretrained language model (LM) for a specific task. - Using the Flair framework for language modeling. - Fine-tuning a pretrained LM on Moby Dick text. - Generating text based on the last part of the novel. ### Tech Stack: Python ### Why it Matters: This project aims to leverage pre-trained language models to generate novel content, specifically focusing on Moby Dick, a classic novel. By fine-tuning a pretrained model on Moby Dick, the project explores the potential of using pre-trained models for generating novel content, which could have applications in various fields such as creative writing, content generation, and more.

Overview: InfoVis is an open-source model designed for generating and editing slide images. It is based on a 9B parameter model that can handle various tasks such as text-to-slide generation, template-conditioned generation, multi-image grounded generation, and instruction-based slide editing. Key features: - Unified model for slide image generation and editing. - Fine-tuned from FLUX.2-klein-base-9B. - Supports input images as style references, content assets, or slide to edit. - Over 13M continual pretraining and 1.9M SFT records, and VisSlideBench benchmark with 1,527 manually screened cases. - Leads every evaluated open-source model on all eleven metrics on VisSlideBench. Tech stack: Python. Why it matters: InfoVis is a powerful tool for generating and editing slide images, making it highly relevant for professionals in the field of information visualization and presentation design. Its strong results and comprehensive benchmarking make it a valuable addition to the developer community.

Overview: The Drama Engine is a framework designed for agentic interaction with language models, written in TypeScript and focused on processing text. It is model- and provider-agnostic, making it suitable for front-end developers to work with agents directly. Key features: - Multi-agent workflows with delegation - Dynamic prompt assembly - Model- and vendor-agnostic Tech stack: TypeScript Why it matters: The Drama Engine simplifies the interaction with language models by providing a flexible and modular framework that can be easily integrated into existing projects, making it accessible to a wide range of developers. GitHub stars: 42 Repository: Write-with-LAIKA/drama-engine Primary language: TypeScript GitHub description: A Framework for Narrative Agents

### Overview: TensorFlow Serving is a powerful tool designed for deploying and serving machine learning models in production environments. This repository provides a comprehensive guide on training, saving, deploying, and interacting with TensorFlow models in a production setting. ### Key Features: - **Model Training**: Train custom CNN models for image classification using the The Simpsons Characters Dataset. - **Model Saving**: Save the trained model for future use. - **Deployment**: Deploy the model using TensorFlow Serving. - **Interactivity**: Interact with the model through a Streamlit UI for real-time monitoring and analysis. ### Tech Stack: - TensorFlow Serving - TensorFlow ### Why It Matters: TensorFlow Serving simplifies the process of deploying machine learning models, making it easier to integrate new algorithms and experiments into existing systems. It maintains the same server architecture and APIs, ensuring compatibility with existing infrastructure. ### Repository: alvarobartt/serving-tensorflow-models - Primary language: Jupyter Notebook - GitHub stars: 42 - GitHub description: Serving TensorFlow models with TensorFlow Serving - README excerpt: Guide on how to train, save, deploy, and interact with TensorFlow ML models in production environments. - Streamlit UI available at: [tensorflow-serving-streamlit](https://github.com/alvarobartt/tensorflow-serving-streamlit) - Docker deployment: Available for Ubuntu VMs, requiring Docker installation. - Requirements: - `tensorflow-model-server` - `tensorflow-serving-api` - `tensorflow==2.5.1` - `tensorflow-serving-api==2.5.

Overview: KnobGen is an official implementation of a dual-pathway framework for controlling the sophistication of artwork in sketch-based diffusion models. It seamlessly adapts to varying levels of sketch complexity and user skill, leveraging high-level semantics from both textual and sketch inputs. Key features: - Coarse-Grained Controller (CGC) module for early stages of generation - Fine-Grained Controller (FGC) module for detailed refinement - Utilizes MultiGen-20M dataset for training Tech stack: Python Why it matters: KnobGen democratizes sketch-based image generation by effectively handling a broad spectrum of sketch complexity and user drawing ability, from novice sketches to those made by seasoned artists, while maintaining the natural appearance of the image. Repository: aminK8/KnobGen Primary language: Python GitHub stars: 42 GitHub description: CVPR 2025 Workshop on CVEU

### Overview: This survey presents a comprehensive and structured synthesis of memory in Large Language Models (LLMs) and Multi-Modal LLMs (MLLMs), organizing the literature into a cohesive taxonomy comprising implicit, explicit, and agentic memory paradigms. ### Key Features: - Comprehensive literature synthesis - Taxonomy of memory paradigms - Organized into implicit, explicit, and agentic memory - Provides an analogy to human brain systems ### Tech Stack: Unknown ### Why It Matters: This survey offers a valuable lens into the evolving memory paradigms in modern AI models, particularly in LLMs and MLLMs. Understanding these mechanisms is crucial for the development and improvement of these models, ensuring they can better mimic human-like behavior and reasoning.

Overview: Jev Account Filter is a project designed to screen video-platform accounts and channels for datasets, focusing on sign language, news, piano/instrument performance, and makeup tutorials. It uses decision models and account metadata to efficiently identify relevant creators without requiring full video downloads or watching. Key features: - Account screening and channel classification with decision models - Efficiently finds sign language creators, news channels, instrument performers, and makeup tutorial creators - Fast, cheap, accurate, flexible, and promising results Tech stack: Python Why it matters: Jev Account Filter is the first project to screen video-platform accounts without downloading or watching full videos, making it a significant advancement in dataset collection and research.

### Overview: ID-Booth is an identity-consistent face generation framework that leverages diffusion models to generate in-the-wild images of consenting identities captured in a constrained environment. It uses a triplet identity loss to fine-tune Stable Diffusion for identity-consistent yet diverse image generation. The framework can augment small-scale datasets to improve their suitability for training face recognition models. ### Key features: - Identity-consistent face generation - Fine-tuning with triplet identity loss - Augmentation of small-scale datasets ### Tech stack: - Python - Jupyter Notebook - Stable Diffusion - Diffusers ### Why it matters: ID-Booth provides a practical solution for generating high-quality in-the-wild images of identities, which is crucial for various applications such as identity verification, authentication, and security. By fine-tuning Stable Diffusion with the ID-Booth framework, it achieves better consistency and diversity in the generated images compared to traditional methods.

### Overview: An implementation of the "simple diffusion" method, as published by Hoogeboom et al., focusing on high-resolution image generation. This project aims to provide a self-contained solution for training diffusion models using PyTorch. ### Key features: - **Simple Diffusion Method**: A straightforward approach to diffusion models for high-resolution images. - **Multi-GPU Support**: Utilizes HuggingFace's Accelerator for efficient multi-GPU training. - **Backbones**: Supports UNet and U-ViT architectures, with plans to include others in the future. - **Noise Schedules**: Implements a shifted cosine noise schedule with specified base noise dimension and interpolated schedules. - **Loss Weights**: Improves performance through min-SNR loss weighting. - **Architectures**: Includes UNet with variable dropout resolutions and UViT architecture. ### Tech stack: Python ### Why it matters: This project offers a practical and efficient solution for implementing diffusion models, which are crucial in generating high-resolution images. The inclusion of multi-GPU support and various noise schedules makes it a valuable resource for researchers and developers in the field of image generation and diffusion models.

Overview: minivllm 是一个旨在帮助开发者理解并实践使用 vLLM 推理引擎的开源项目。它包含100个动手练习课，以及一个可运行的迷你推理引擎，涵盖了从基础到高级的11章内容。 Key features: - 11章 100课的Jupyter notebook练习 - 一个包含500行代码的迷你推理引擎 - 8篇源码级精读的架构文档 - 12张顶会风格SVG架构图 - 参考答案和课后练习题 Tech stack: Python Why it matters: 通过实践操作，帮助开发者从零开始理解并掌握vLLM推理引擎的核心技术和架构，从而提升实际开发能力。 Repository: feiguangba/minivllm GitHub stars: 41

### Overview: This repository is dedicated to a comprehensive survey on "Anomaly Detection and Generation with Diffusion Models: A Survey," submitted to IEEE TPAMI. The survey provides a tutorial-style analysis of theoretical foundations and practical implementations of anomaly detection and generation with diffusion models across various domains, including images, videos, time series, tabular, and multimodal data. It reveals how diffusion models create a synergistic cycle where generation addresses data scarcity challenges while detection provides feedback for refined generation strategies, advancing both capabilities beyond their individual potential. ### Key Features: - Comprehensive review of theoretical foundations and practical implementations of ADGDM. - Detailed taxonomy categorizing ADGDM methods based on anomaly scoring mechanisms, conditioning strategies, and architectural designs. - Analysis of strengths and limitations of ADGDM methods. - Discussion of key challenges including scalability and computational efficiency. - Outline of promising future directions, such as efficient architectures, conditioning strategies, and integration with foundation models. ### Tech Stack: Unknown ### Why It Matters: This survey aims to guide researchers and practitioners in leveraging diffusion models for innovative anomaly detection and generation solutions across diverse applications, providing a valuable resource for advancing the field of anomaly detection and generation with diffusion models. ### Repository: Repository: fudanyliu/ADGDM GitHub Stars: 42 GitHub Description: Repository for 'Anomaly Detection and Generation with Diffusion Models: A Survey', submitted to IEEE TPAMI. GitHub URL: [https://github.com/fudanyliu/ADGDM](https://github.com/fudanyliu/ADGDM)

### Overview: This collection features 252 curated FLUX 2 prompts across nine categories, including portrait and avatar, product and e-commerce, poster and typography, infographic and diagram, anime and illustration, 3D and material, food and lifestyle, architecture and interior, and nature and landscape. Each prompt is accompanied by a preview image to help you visualize the results before running it. ### Key Features: - **9 Categories**: Nine distinct categories to choose from. - **252 Prompts**: A comprehensive collection of FLUX 2 prompts. - **Preview Images**: Generated previews to help you see the results before running the prompts. ### Tech Stack: The repository uses Markdown for its content and README for documentation. ### Why It Matters: This collection provides practical FLUX 2 prompts that work well across various applications, making it a valuable resource for developers looking to enhance their projects with photorealistic textures and composition control.

### Overview: The FDB project is an official implementation of the Fourier-constrained diffusion bridges (FDB) model for MRI reconstruction, as described in the paper "Learning fourier-constrained diffusion bridges for mri reconstruction" by Mirza et al. The model is designed to improve MRI reconstruction by leveraging Fourier constraints, enhancing the accuracy and efficiency of the reconstruction process. ### Key features: - **Model Implementation**: The project provides a PyTorch implementation of the FDB model. - **Data Types**: Supports both single-coil and multi-coil MRI data types. - **Training**: Includes scripts for single-coil and multi-coil training, with adjustable parameters such as image size, number of channels, and diffusion steps. - **Inference**: Offers sample generation capabilities for both single-coil and multi-coil data types, allowing for visualization and evaluation of the model's performance. ### Tech stack: - Python - Dependencies: `python==3.8.13`, `blobfile==2.0.2`, `h5py==3.9.0`, `imageio==2.22.1`, `mpi4py==3.1.4`, `numpy==1.24.4`, `Pillow==10.0.0`, `torch==2.0.1` ### Why it matters: The FDB model offers a significant improvement in MRI reconstruction by incorporating Fourier constraints, leading to better image quality and faster processing times compared to traditional methods. Its implementation in PyTorch makes it accessible and easy to

Overview: The Instill Console is a Next.js application built with TypeScript and TailwindCSS, designed to streamline and enhance the user experience of the Instill Core platform. Key features: - Unified, streamlined, and user-friendly interface for Instill Core components - Offers a comprehensive set of tools for creating, managing, and observing pipelines, models, and artifacts directly from the browser - Integrates seamlessly with the Instill Core repository, allowing for easy access and development on local machines Tech stack: TypeScript, Next.js, TailwindCSS Why it matters: The console provides a centralized and accessible way to interact with the Instill Core platform, making it easier for developers to manage and monitor their projects without needing to manually configure each component. Repository: instill-ai/console GitHub stars: 41 GitHub description: 📺 Instill Console for 🔮 Instill Core: https://github.com/instill-ai/instill-core

Overview: RAGdrag is a comprehensive toolkit designed to assess the security of RAG pipelines through 27 techniques across six kill chain phases, providing a detailed taxonomy of ideas and their impact. Key features: - Comprehensive security assessment for RAG pipelines - Detailed taxonomy of RAG pipeline security techniques - 27 techniques covering 6 kill chain phases - MITRE ATT&CK mapped to RAG pipeline security Tech stack: Python Why it matters: RAGdrag offers a robust solution for developers and security professionals to evaluate the security of their RAG pipelines, ensuring they meet the necessary standards and mitigating potential risks.

### Overview: This repository is a comprehensive list of AI media generation tools, covering image, video, audio, voice, 3D, enhancement, and multimodal platforms. It serves as a master guide for developers, providing up-to-date information on the latest tools and their availability. ### Key Features: - **OpenAI Image Generation**: GPT Image 2.5 Sunburst and Flare, released in September. - **Midjourney**: Midjourney V8.2 GA, native 2K, Edit Model for V8 launched on August 27, and Thinking Mode alpha on October 8. - **Microsoft MAI-Image**: MAI-Image-2.6 & 2.6-Flash GA, in Azure Microsoft Foundry. - **Play.ht**: Permanently shut down on December 31, 2025, and migrated to ElevenLabs, Fish Audio, and Cartesia. - **Haiper AI**: Permanently shut down in February 2025, with model assets sold to NetMind.AI. - **Visual Electric**: Permanently shut down in early 2026, after 90-day sunset post-Perplexity acquisition. - **Udio**: Downloads disabled across all tiers during a licensing overhaul. - **Stability AI / SD**: SD4 rumors debunked, Stable Audio 3.0 DAW plugin GA on August 18. ### Tech Stack: The repository uses a combination of primary sources for information, including OpenAI announcements, Microsoft AI

Overview: 本项目提供了一个面向开发者与初学者的交互式源码教程，通过13个纯HTML互动实验和13章中文教程，从一次`generate()`请求出发，逐步理解nano-vLLM的完整推理引擎。 Key features: - 13个纯HTML互动实验 - 13章源码级教程 - 每章配套课后习题 - 即时反馈习题 Tech stack: JavaScript Why it matters: 通过这些互动实验和教程，开发者和初学者能够深入了解nano-vLLM的推理引擎，包括调度、KV Cache、Prefix Cache等核心机制，从而更好地理解和应用这一技术。 Repository: lora-sys/nano-vllm-interactive-guide

Overview: SnapLLM is a local inference engine for multi-model serving, built on llama.cpp and stable-diffusion.cpp. It allows for the seamless switching between models without reloading their weights, optimizing performance and reducing latency. Key features: - Resident Model Selection - Reload After Eviction - Multi-Model Management - GPU/CPU Hybrid - OpenAI-Compatible API - Multi-Modal Support - KV Cache Persistence - Web UI - Desktop Application Tech stack: C++ Why it matters: SnapLLM offers an efficient and flexible solution for multi-model inference, particularly beneficial for edge AI applications and those requiring low latency.

Overview: This project is an assignment for the E6870 ASR class, focusing on developing a software solution using C programming language. It involves creating a basic application that processes audio files and transcribes speech into text. Key features: - Audio file processing - Speech-to-text transcription Tech stack: C Why it matters: This project is essential for understanding the fundamentals of audio processing and speech recognition, which are crucial skills in the field of computer science and engineering. Repository: placebokkk/e6870 Primary language: C GitHub stars: 42

Overview: This project is an AI-native cross-platform offline voice input method built with FunASR and PySide6. It is designed to be fully testable and extendable with AI agents. Key features: - Cross-platform compatibility - Low latency - GUI decoupled from the core engine - Supports multiple ASR models - Real-time device switching and crash recovery - Simplified integration with AI agents Tech stack: Python Why it matters: This project aims to provide a reliable and efficient voice input method for AI agents, ensuring seamless integration and testability with various AI tools and platforms. Repository: pofice/voice-input-method GitHub stars: 42