Projects

### Overview: This project, titled "Pyramid Discrete Diffusion for Fine 3D Large Scene Generation," explores the application of diffusion models to generate large-scale 3D scenes. The team, led by Yuheng Liu, presents a novel framework called the Pyramid Discrete Diffusion model (PDD) that uses a coarse-to-fine approach to generate high-quality outdoor scenes. The project was accepted as an Oral Presentation at ECCV 2024 and is now available on arXiv. ### Key Features: - **Pyramid Discrete Diffusion Model (PDD):** A framework that employs scale-varied diffusion models to progressively generate high-quality outdoor scenes. - **Multi-Scale Architecture:** The PDD model can be fine-tuned with different datasets, demonstrating compatibility and versatility. - **Conditional and Unconditional Generation:** The model can generate scenes both conditionally (with specific prompts) and unconditionally (without prompts). - **Real-World Data Compatibility:** The model can be trained on one dataset and fine-tuned with another, showcasing data compatibility. ### Tech Stack: - **Programming Languages:** Python - **GitHub Stars:** 140 ### Why It Matters: - The project addresses the challenge of generating large-scale 3D scenes, particularly outdoor scenes, which are complex and require comprehensive real-world datasets. - The use of diffusion models and a coarse-to-fine approach allows for the generation of high-quality scenes, demonstrating the potential of these models in 3D scene synthesis. ### Repository: - **Repository:** yuhengli

Overview: Bodhi App enables local execution of Open Source LLMs, offering both technical and non-technical user-friendly interfaces. Key features: - Built-in Chat UI - Model Management - API Token Management - Dynamic App Settings - Responsive Design - Robust Error Handling Tech stack: TypeScript, Huggingface ecosystem Why it matters: Provides a user-friendly environment for running Open Source LLMs locally, suitable for both technical and non-technical users.

Overview: NEWTON is a video generation model that addresses the specification bottleneck by introducing a physics-conditioning framework. It optimizes on-policy learning through a planner-orchestrated pipeline that includes keyframe generation, scientific computation, and prompt refinement, achieving improvements on VideoPhy-2 benchmarks. Key features: - Physics-conditioning framework - On-policy learning via Flow-GRPO - Improved accuracy from 21.4% to 29.7% on LTX-Video and 30.7% to 37.4% on Veo-3.1 Tech stack: Python Why it matters: Newton provides a solution to the specification bottleneck in video generation, ensuring that the generated videos are grounded in physical reality, leading to more accurate and physically coherent results. Repository: CUTEPKQ/NEWTON Primary language: Python GitHub stars: 139 GitHub description: NEWTON: Agentic Planning for Physically Grounded Video Generation

### Overview: DiffKG is a knowledge graph diffusion model designed for recommendation tasks. The paper, "DiffKG: Knowledge Graph Diffusion Model for Recommendation," was accepted at WSDM 2024. This project is implemented in PyTorch and focuses on integrating a generative diffusion model with a data augmentation paradigm to enhance knowledge graph representation learning. ### Key Features: - **Knowledge Graph Diffusion Model**: Proposes a novel approach to improve knowledge graph representation learning. - **Generative Diffusion Model Integration**: Combines a generative diffusion model with a data augmentation paradigm to enhance robustness. - **Collaborative Knowledge Graph Convolution**: Incorporates collaborative signals reflecting user-item interaction patterns to guide knowledge graph diffusion. ### Tech Stack: - **Programming Language**: Python - **Dependencies**: torch==1.11.0, numpy==1.23.1, scipy==1.9.1 ### Why It Matters: DiffKG offers a unique solution for recommendation tasks by leveraging knowledge graph diffusion to better align item semantics with collaborative relation modeling. This approach enhances the effectiveness of recommendation systems by incorporating user-item interaction patterns, leading to improved performance metrics such as Recall@20 and NDCG@20. ### Repository: - **Repository**: HKUDS/DiffKG - **GitHub Stars**: 139 - **GitHub Description**: PyTorch implementation for "DiffKG: Knowledge Graph Diffusion Model for Recommendation," accepted at WSDM 2024. ### Environment: - **Python Version**: 3.9

Overview: SafeGen is a project that introduces a method to mitigate the generation of sexually explicit content in text-to-image models, specifically focusing on the Stable Diffusion model. This approach is part of the ACM CCS 2024 conference and aims to contribute to the field of computer security and AI ethics. Key features: - Adjusts the self-attention layers of Stable Diffusion based on image-only triplets. - Available on Hugging Face for inference. - Released on ArXiv and available on Hugging Face for evaluation. Tech stack: Python Why it matters: This project demonstrates how to integrate AI models with ethical considerations, highlighting the importance of protecting against harmful content generation in AI applications.

Overview: 这个项目提供了一套基于科大讯飞语音听写WebAPI的浏览器端封装库，支持实时流式语音识别，能够将语音转换为文字信息，帮助机器“听懂”人类语言。 Key features: - 实时流式识别 - 轻量跨语言 - WebSocket协议支持 - 多语种/方言支持 - 安全鉴权 - NPM + CDN模块化导入 Tech stack: JavaScript Why it matters: 该库简化了语音识别的开发过程，使得开发者能够快速集成语音听写功能，提高开发效率，同时确保了系统的安全性和稳定性。

Overview: Toolio is an OpenAI-like HTTP server API implementation designed for Apple Silicon Macs, focusing on structured LLM response generation and tool calling. It supports the schema-steered structured output (3SO) method, which converts input JSON schemas into state machines to steer LLMs more effectively. Key features: - Schema-steered structured output (3SO) for better constrained LLM responses - Supports tool calling via LLMs - Primarily developed by Oori Data, offering LLMOps, data pipelines, and software engineering services around AI/LLM applications Tech stack: Python Why it matters: Toolio simplifies the process of integrating structured LLMs into applications, making it easier to achieve guaranteed structured output and efficient workflow management. GitHub stars: 139 Repository: OoriData/Toolio

### Overview: This repository is a comprehensive survey of AI alignment, providing an in-depth analysis of the field. It is based on the arxiv paper "AI Alignment: A Comprehensive Survey" by Jiaming Ji et al., which can be accessed at [arxiv.org/pdf/2310.19852.pdf](https://arxiv.org/pdf/2310.19852.pdf). ### Key Features: - Comprehensive survey of AI alignment. - Accessible through the arxiv paper. - Open platform for communication and discussion. - Email support for any questions or insights. ### Tech Stack: Python, Markdown, LaTeX. ### Why It Matters: This survey serves as a valuable resource for researchers and practitioners in the field of AI alignment, providing a detailed overview of the current state and future directions of the field.

### Overview: Generative Interfaces for Language Models aim to enable more adaptive, interactive interactions with Language Models by proactively generating user interfaces. This project investigates the effectiveness of Generative Interfaces through a series of user studies and experiments. ### Key Features: - **User Study**: Conducted by 76 participants to compare Generative Interfaces with conversational ones. - **Requirement Specification**: Parses input into a requirement specification, capturing main goals, desired features, UI components, interaction styles, and problem-solving strategies. - **Structured Representation Generation**: Generates a Structured Interface-Specific Representation based on the requirement specification. - **UI Generation**: Utilizes a component codebase and web retrieval module to generate high-quality, interactive interfaces. - **Adaptive Reward Function**: Automatically generates evaluation criteria based on user queries to improve the quality of generated interfaces. - **Iterative Refinement**: Generates multiple UI candidates and scores them using the reward function to refine the final interface. ### Tech Stack: TypeScript ### Why It Matters: Generative Interfaces offer a more intuitive and adaptive way of interacting with language models, potentially improving user satisfaction and engagement.

### Overview: This OpenAI-compatible API migration kit simplifies the process of moving workloads between different AI API gateways without the need for hard-coded credentials. It provides comprehensive checks and examples for chat completions, streaming, tool calling, JSON output, vision input, responses API, authentication, and rollback scenarios, ensuring compatibility and smooth transitions. ### Key Features: - Python, JavaScript, and cURL examples for various API migration scenarios. - A compatibility matrix and rollback switches to ensure full OpenAI compatibility. - A checklist for moving workloads between different AI API gateways. - Realistic test scenarios for chat completions, streaming, tool calling, structured output, vision input, responses API, authentication, and rollback. - A scanner that identifies migration-sensitive code paths, including streaming, tools/functions, structured output, vision, responses API, custom base URLs, and possible hard-coded keys. - A checklist that reports schema adherence and refusal behavior, and endpoint and event compatibility. ### Tech Stack: - Python, JavaScript, and cURL examples. ### Why It Matters: - **One Key, Entire Catalog**: The same `https://api.apimart.ai/v1` base URL and `Authorization` header reach the whole catalog behind one key and 300+ other image, video, and language models. - **$1 Minimum, Pay As You Go**: No subscription and no prepaid plan to size up front: top up from $1 and spend it on calls. There is no free quota to burn through first, so the price in this table is the price you pay.

### Overview: Nano Banana Pro API is a high-fidelity image generation service offered through APIMart, providing one asynchronous endpoint for 1K/2K/4K output with eleven aspect ratios, including auto. The API supports token billing and is priced at $0.02 per delivered image, making it one of the most cost-effective options on APIMart. ### Key features: - **One Key, Entire Catalog:** The same base URL and `Authorization` header reach Nano Banana Pro and 300+ other models. - **$1 Minimum, Pay As You Go:** No subscription required; pay as you go with no free quota. - **Charge Backs:** The price is reported in the response, so a spend number is read per call. - **Async by Design:** Submit, take the `task_id`, and poll `GET /v1/tasks/{id}` for results. ### Tech Stack: Python ### Why it Matters: Nano Banana Pro is the highest-fidelity Gemini image route on APIMart, providing one asynchronous endpoint, 1K/2K/4K output, and eleven aspect ratios. It supports token billing and is priced at $0.02 per delivered image, making it one of the most cost-effective options on APIMart.

### Overview: 源内(GENAI)は、デジタル庁のガバメントAI「源内」を完全ローカル環境で動かすための非公式プロジェクトです。プロジェクトはSAML認証、RAG、文字起こし、画像生成、チーム単位のナレッジをローカルで完結させています。 ### Key features: - SAML認証 - RAG（Qdrant） - 文字起こし（Whisper） - 画像生成（SD） - ノートブック（資料を集めて調べ項目に整理） - マルチテナント機能 - Markdownエディタの各種文書作成 - ローカルのSSH接続 - ローカルのアプリ表示編集 - ローカルのヒアリングシート - 申請束のオンラインフォーム - ローカルの手続き受付 - ローカルのマイ手続き - ローカルの日程調整 - ローカルの日程調整ウィザード ### Tech stack: - TypeScript - FastAPI - React - Qdrant - Keycloak - MacOS/Linux + NVIDIA GPU ### Why it matters: 本プロジェクトは、デジタル庁のガバメントAI「源内」を完全ローカル環境で動かすための非公式フォークです。プロジェクトは、

### Overview: iblai/vibe is a developer toolkit designed to quickly build AI-powered apps on the ibl.ai platform. It includes pre-built components, Claude Code skills, and a hosted backend at iblai.app, making it easy to deploy in minutes. ### Key Features: - **SSO Auth**: Seamless authentication with ibl.ai. - **AI Chat**: Full in-process agent chat surface. - **Profiles**: Profile UI and settings flows. - **Notifications**: Real-time notifications. - **Analytics**: Dashboard for project and user analytics. - **Notifications**: Customizable notifications. - **Workflow**: Streamlined workflow for project management. ### Tech Stack: - TypeScript - Next.js - Tailwind CSS - OpenAI Codex - Claude Code - Cursor - OpenCode - Copilot ### Why It Matters: iblai/vibe streamlines the development process by providing a pre-built environment and ready-to-use skills, allowing developers to focus on building their applications without worrying about the underlying infrastructure. The toolkit includes everything from authentication to analytics, making it a comprehensive solution for building AI-powered apps on the ibl.ai platform.

Overview: This repository contains experimental code for speech recognition using PyTorch and Kaldi, focusing on improving the acoustic model to produce phoneme sequences rather than end-to-end transcription. Key features: - PyTorch-based speech recognition model - Kaldi integration for advanced decoding - Customized training and testing datasets Tech stack: PyTorch, Kaldi, Python 3.6+ Why it matters: The repository aims to enhance the accuracy of acoustic models for speech recognition, offering a more refined approach compared to traditional end-to-end models. Repository: jinserk/pytorch-asr GitHub stars: 139

Overview: MychatGPT is a Python-based chat application that integrates with your existing documents, allowing you to access and utilize your files effortlessly. Key features: - Seamless integration with your documents - Real-time chat functionality - Customizable user interface Tech stack: Python Why it matters: MychatGPT simplifies access to your valuable documents, enhancing productivity and collaboration in a user-friendly manner.

### Overview: Brunnfeld is a medieval village economy simulation that runs up to 1000 LLM agents across multiple villages. The agents live in a 22-location village, each with a short background and a structured world that enforces physics, including hunger, tool degradation, seasonal yields, locked doors, expiring orders, and spoilage. The simulation is driven by supply chains, market pressure, and governance, with no behavioral instructions or economic goals. ### Key Features: - **No Behavioral Instructions**: Agents receive only a structured world state, not goals or strategies. - **Hard Supply Chains**: Wheat → mill → flour → bakery → bread; one missing link halts production. - **Real Economics**: Live order book, rolling price index, tool degradation, spoilage, seasonal yields, debt. - **Village Governance**: Pressure accumulates in the elder's perception until he calls a meeting; council votes pass laws that take immediate effect. - **Persistent Memory**: Each agent's markdown file is written by the engine and read back every turn. - **Scalable**: Generate worlds with up to 1000 agents across 5 villages via a single API call. ### Tech Stack: TypeScript ### Why It Matters: The key insight is that LLM agents behave like the people they know how to be—millers mill, bakers bake—without a physics layer, those are just words. Brunnfeld's engine enforces that only Gerda has the skill and the location. Neither alone is interesting. Together they produce a

### Overview: This repository contains a collection of Jupyter Notebooks and code related to Generative AI, Large Language Models (LLMs), Machine Learning (ML), Model-Driven Operations (MLOPS), Natural Language Processing (NLP), Computer Vision, and Graph Databases. The content is designed to support the YouTube channel "The Machine Learning Engineer," providing a comprehensive guide to various topics in the field of AI and ML. ### Key Features: - Jupyter Notebooks and Code - Generative AI, LLMs, Transformers, CV, NLP, and more - 212 videos on YouTube - Python and C# programming languages - Videos in English and Spanish - Playlists organized by topic and language - Short videos explaining complete topics - Public videos and notebooks, shorts not listed - Subscriptions and comments encouraged for support ### Tech Stack: Python and Jupyter Notebook ### Why It Matters: This repository serves as a valuable resource for anyone interested in the latest developments in AI and ML, providing practical examples and code snippets to enhance understanding and application of these technologies.

### Overview: Text2Sql.Net 是一个基于.NET平台的自然语言转SQL工具，旨在帮助开发者和数据分析师通过简单的自然语言描述快速生成数据库查询语句。项目结合了大型语言模型(LLM)和传统SQL解析技术，支持多种主流数据库。 ### Key features: - **自然语言转SQL**: 输入日常语言描述，自动生成对应的SQL查询语句。 - **多数据库支持**: 兼容SQL Server、MySQL、PostgreSQL和SQLite。 - **智能上下文理解**: 基于聊天历史理解用户查询意图。 - **向量搜索集成**: 支持基于语义的相似度搜索。 - **语法校验**: 自动检查生成的SQL语法正确性。 - **MCP协议支持**: 与IDE工具（如Cursor、Trae等）无缝集成。 - **智能问答示例系统**: 通过示例学习提升SQL生成准确性。 ### Tech stack: C# ### Why it matters: Text2Sql.Net 提供了一种高效、便捷的方式来处理复杂的数据库查询任务，特别适合需要频繁查询和生成SQL语句的开发者和数据分析师。通过结合先进的自然语言处理技术和数据库查询优化，大大提高了工作效率和准确性。

Overview: This repository contains Mongolian speech recognition models implemented using PyTorch, including QuartzNet and Jasper/QuartzNet blocks. The project is based on the DeepSpeech and NeMo libraries, and it has been trained on a proprietary Mongolian dataset with a Word Error Rate (WER) of 8%. Key features: - Implements QuartzNet and Jasper/QuartzNet blocks for speech recognition. - Uses PyTorch for training and inference. - Trained on a Mongolian Bible dataset with a WER of 8%. - Provides an online demo for quick testing. - Supports training with a single person dataset. Tech stack: Python, PyTorch. Why it matters: This project is essential for developing Mongolian language recognition systems, particularly in applications requiring speech-to-text conversion, such as translation, transcription, and voice recognition.

### Overview: AgentFM is a peer-to-peer network that turns everyday computers into a decentralized AI supercomputer. It allows you to run massive AI workloads directly across a global mesh of idle CPUs and GPUs. ### Key features: - **Desktop App**: A native mesh console for macOS and Linux, bundled with the full mesh backend, enabling easy management and task dispatch. - **Relay**: A circuit relay and DHT-based archive ledger for efficient task discovery and data storage. - **Worker**: Runs agents in a Podman sandbox, providing real-time telemetry and artifact management. ### Tech stack: JavaScript ### Why it matters: AgentFM democratizes AI by leveraging idle hardware, making it accessible and cost-effective. Its decentralized architecture ensures privacy and security, making it ideal for sensitive data processing and AI workloads.