Projects

Overview: Transcribe YouTube videos, extract topics, and answer questions interactively. Key features: - Video Processing: Automatically extract and display the transcript. - Topic Extraction: AI-powered analysis to identify and organize key topics. - Interactive Q&A: Ask questions about the video content and get AI-generated answers. - Timeline Integration: Jump to specific transcript segments or topics. Tech stack: Python with FastAPI for backend, React with Vite for frontend, and OpenAI API for AI-powered topic extraction and question answering. Why it matters: This application provides a valuable tool for users to understand and engage with YouTube videos in a more interactive way, enhancing their learning experience and making it easier to find the information they seek.

Overview: This repository demonstrates the execution of large language models (LLMs) on consumer-grade CPU hardware using packages like llamafile, with a focus on low-latency, high-throughput, and cost-effectiveness benefits. Key features: - llamafile: Demonstrates how to execute LLMs on CPUs using llamafile, a library that distributes LLMs with a single binary file. - LLaMa.cpp: Provides a lighter, more portable alternative to heavyweight frameworks, implementing the Meta’s LLaMa architecture in efficient C/C++. Tech stack: Jupyter Notebook Why it matters: This repository showcases examples of utilizing libraries and frameworks for efficient deployment of large language models on consumer-grade CPU hardware, emphasizing high-throughput and memory-efficient inference. GitHub stars: 9

Overview: FinGuru is a comprehensive news research tool that processes and analyzes news articles from URLs and PDFs using LangChain, Google embeddings, and Streamlit. It leverages FAISS for efficient retrieval and provides insights and sources based on the content of the articles. Key features: - Fetch and parse news articles from URLs or PDFs. - Split articles into manageable chunks. - Create embeddings for text using Google Embedding Model. - Store embeddings in a FAISS index for efficient retrieval. - Query the processed data to get answers and sources. Tech stack: Python 3.7+, Streamlit, LangChain, Google embeddings, FAISS. Why it matters: FinGuru simplifies the process of researching financial news, making it easier for developers to quickly find and analyze relevant information. It enhances the efficiency of financial research by providing a structured and organized approach to data retrieval and analysis.

### Overview: This is a single-purpose inference engine specifically designed for Qwen3.8-27B on one NVIDIA RTX 5090 GPU, utilizing NVFP4 weights and Gated DeltaNet projections. It is optimized for speed and context handling, delivering better performance than competing engines on the same hardware. ### Key Features: - **Speed**: Faster decode and throughput than vLLM, SGLang, and NInfer, with 1.3–2.8x single-request decode speed and 1.1–1.7x throughput of eight concurrent requests. - **Context Handling**: A 605K-token KV cache on a 32 GB card, capable of handling up to eight requests, images, and speculative decoding, with a 2.8–6.3x improvement over other engines. - **Memory Plan**: Prefills the prefix cache of the recurrent state in host memory, optimizing for long context handling. ### Tech Stack: - **C++ / CUDA**: Written from scratch, leveraging CUDA for GPU acceleration. - **Python Server**: Frontend for serving OpenAI- and Anthropic-compatible APIs. ### Why It Matters: - **Speed**: Significantly faster than competing engines, offering 1.3–2.8x single-request decode speed and 1.1–1.7x throughput of eight concurrent requests. - **Context Handling**: Capable of handling up to eight requests, images, and speculative decoding, with a 2.8–6.3x improvement over other engines. -

Overview: Speculative decoding is a technique that accelerates language model inference by using a smaller "draft" model to propose multiple tokens, which are then verified by a larger "target" model. This approach achieves significant speedups while maintaining exact output distribution equivalence. Key features: - Native Swift implementation optimized for Apple Silicon - Simple high-level API and streaming support - Compatible with any MLX-Swift LLM models - Full Swift concurrency support (async/await) - Greedy and temperature-based sampling - Comprehensive statistics and benchmarking Tech stack: Swift Why it matters: Speculative decoding is crucial for improving the efficiency of large language models (LLMs) on Apple Silicon, providing faster inference times without compromising accuracy.

### Overview: Fine tuning GPT-3 using a dataset for generating code from prompts. ### Key features: - Fine-tuning GPT-3 model using a dataset for generating code from prompts. - Supports TypeScript and JavaScript parsers. - Dataset is stored in `./dataset.csv`. - Provides command-line interface for uploading datasets and generating completions. ### Tech stack: Node.js ### Why it matters: This project allows developers to fine-tune GPT-3 for generating code from prompts, making it useful for automating repetitive tasks in JavaScript and TypeScript projects.

### Overview: This project is a MCP server specifically designed for Google Flow, allowing users to generate images and videos using their own Google account. The server utilizes Google's Veo and Nano Banana models, providing access to high-quality images and videos without the need for public APIs or SDKs. It's compatible with various MCP clients and runs on your own Google account, ensuring cost-free usage and easy integration with existing systems. ### Key Features: - **Image Generation**: Generates high-quality images using Google's Veo and Nano Banana models. - **Video Generation**: Creates videos from images or sequences, with options to upscale and combine frames. - **Google Flow Compatibility**: Utilizes Google Flow's Veo and Nano Banana models, ensuring seamless integration with the platform. - **Cost-Free**: Images cost nothing, and Google Flow models are free on a normal account. - **No Public API**: This server is the missing piece needed for Google Flow, providing a MCP server that runs on your Google account. - **Agent-Specific**: The server is designed to work with specific tools and models, making it easier for agents to use without needing to learn new commands. ### Tech Stack: - **Google Flow**: Utilizes Google's Veo and Nano Banana models. - **MCP Server**: Runs on your own Google account, making it cost-free and compatible with various MCP clients. ### Why It Matters: - **Cost-Free Images**: Google's image models are free on a normal account, eliminating the need for public APIs or SDKs. - **Direct Integration**: This server runs on your Google account

### Overview: Qwen3.6-35B-A3B is a cutting-edge model designed to deliver expert-level performance on a modest 6 GB GPU. Running at 68 tokens per second, it is 1.9x faster than the tuned llama.cpp and ik_llama.cpp, making it highly efficient and suitable for real-time applications. ### Key Features: - **Multi-Token Prediction (Default)**: This feature allows Qwen to decode multiple tokens at once, significantly improving speed and efficiency. - **FlashAttention-2 Prefill Kernel**: This kernel significantly speeds up attention operations by sharing key-value tiles, leading to 2.4-2.5x faster performance. - **Prefill Chunk 2048**: This change increases the number of tokens processed per chunk, resulting in a 21.8k-token prompt decoding time reduction to 19.8 seconds. ### Tech Stack: CUDA is the primary technology stack used to implement this model. ### Why It Matters: This model is optimized for real-time applications, making it ideal for chatbots, virtual assistants, and other interactive AI systems. Its performance is comparable to or better than state-of-the-art models while running efficiently on a modest GPU configuration.

Overview: This repository is a comprehensive collection of resources on the journey from Artificial Intelligence (AI) to Artificial General Intelligence (AGI) to Artificial Superintelligence (ASI). It includes frameworks, agents, research papers, safety & alignment, books, benchmarks, conferences, and tools for builders and researchers shaping the future of intelligence. Key features: - AGI: Artificial General Intelligence - ASI: Artificial Superintelligence - Research papers, frameworks, books, benchmarks, conferences, and tools Tech stack: HTML Why it matters: This repository provides a curated and comprehensive overview of the latest resources and tools in the field of artificial intelligence, from basic AI to advanced AGI and ASI, making it essential for anyone interested in the future of artificial intelligence and its potential impacts. Repository: nellaivijay/awesome-agi-aci-asi GitHub stars: 9 GitHub description: The most comprehensive curated collection of AGI and ASI resources -- frameworks, agents, physical AI, robotics, research papers, safety & alignment, books, benchmarks, and conferences for builders and researchers shaping the future of intelligence.

Overview: This module implements the RAPTOR retrieval strategy for the Neuron PHP AI framework, designed to handle complex, context-rich queries effectively. Key features: - RAPTOR retrieval strategy for comprehensive document coverage - Support for multi-step reasoning and thematic queries - Clustering strategy for grouping similar documents into clusters - Support for both Similarity Clustering and Gaussian Mixture Clustering Tech stack: PHP Why it matters: RAPTOR is essential for users who need to ask open-ended questions that require comprehensive coverage and context-rich answers, especially in domains involving complex topics where context is crucial. Repository: neuron-core/raptor-retrieval GitHub stars: 9

### Overview: Neural Ram is a self-hosted, open-source server that seamlessly integrates with any AI tool, allowing you to maintain continuity across different sessions, tools, and machines. It combines vector search, lexical recall, and a background dreaming cycle to consolidate and deduplicate context, ensuring that your knowledge remains accessible and relevant. ### Key Features: - **Continuity Across Tools**: Maintains context across different AI tools and sessions. - **Semantic Search and Knowledge Graph**: Provides semantic search and a deduplicated knowledge graph. - **Consolidation and Deduplication**: A background dreaming cycle consolidates and deduplicates context. - **Multi-Tenancy and OAuth 2.0**: Supports multi-tenancy and integrates with OAuth 2.0 for secure access. - **MCP (Memory Consistency Protocol)**: Connects to a remote server via MCP, allowing seamless integration with other tools. - **SQLite or PostgreSQL**: Stores and retrieves data using either SQLite or PostgreSQL. - **MIT License**: Open-source and MIT-licensed, ensuring flexibility and ease of use. ### Tech Stack: Go ### Why It Matters: Neural Ram simplifies the complex task of maintaining continuity across different AI tools and sessions, making it easier to switch between tools and machines without losing context. By providing a single, self-hosted server that integrates seamlessly with other tools, it streamlines workflows and reduces the need for manual handoffs, improving efficiency and reducing errors.

### Overview: Imagen 4.0 API (`imagen-4.0-apimart`) is a cutting-edge image generation service powered by Google's latest AI technology. It offers flexible pricing, easy integration, and a straightforward API endpoint for developers. ### Key Features: - **Key Features:** - **Model ID and Pricing:** Access Imagen 4.0 with a single model ID (`imagen-4.0-apimart`) and pay by the image delivered, with tiered pricing based on resolution. - **Billing Model:** The API is billed on a per-image basis, making it cost-effective and easy to manage. - **Model Compatibility:** Compatible with OpenAI endpoints, allowing seamless integration with existing systems. - **Pricing Model:** Prices are effective and reflect the actual cost of the generated images. - **Request Parameters:** Supports various parameters such as model, prompt, size, resolution, and more. - **Quickstart:** Provides a simple example in both Bash and Python, making it easy to get started. ### Tech Stack: Python ### Why it Matters: - **Key Features:** The ability to generate high-quality images with a single API call, combined with flexible pricing and model compatibility, makes Imagen 4.0 a valuable tool for developers looking to integrate AI into their projects without the complexity of multiple APIs. ### GitHub Stars: 9 ### Repository: oihnxor9/imagen-4-api-ja

### Overview: Seedance 1.0 Pro Fast API is a model id, pricing, and parameters-focused resource for the OpenAI-compatible APIMart endpoint. It offers a single key to access the entire catalog of models, including Seedance 1.0 Pro Fast, at a cost of $0.0079 per second. ### Key features: - **One Key, Entire Catalog**: Use the same `https://api.apimart.ai/v1` base URL and `Authorization` header to access all models. - **$1 Minimum, Pay As You Go**: No subscription required; pay only for the generated calls. - **Charge Backs**: The cost is reported in the response, so there's no guessing at the end of the month. - **Async by Design**: Submit tasks, and poll the `GET /v1/tasks/{id}` endpoint for updates. ### Tech Stack: - Python ### Why It Matters: Seedance 1.0 Pro Fast API provides a low-latency alternative to the standard Seedance 1.0 Pro route, with a per-second billing model. This makes it ideal for teams looking to optimize their API usage without the overhead of a monthly subscription.

Overview: 플럭스는 FLUX 2 Pro API를 통해 이미지 생성을 위한 모델 ID, 가격, 파라미터 및 복사 붙여넣기 예제를 제공합니다. 이 API는 OpenAI 호환 엔드포인트 하나로 월 구독 플랜이 필요하지 않습니다. Key features: - 모델 ID와 라우트 - 가격 - 파라미터 - 복사 붙여넣기 예제 Tech stack: Python Why it matters: 이 API는 OpenAI 호환 엔드포인트 하나로 월 구독 플랜이 필요하지 않으며, $1부터 충전해 호출한 만큼만 쓰면 됩니다. 또한, 비동기 설계로 제출하고 `task_id`를 받아 `GET /v1/tasks/{id}`를 폴링하면 끝입니다. 배치와 재시도는 별도 통합이 아니라 평범한 큐 작업입니다.

Overview: AI Adapter is a flexible integration layer for connecting any AI model to OpenNVR, enabling seamless support for cloud, local, and edge models through a modular architecture. It allows developers to plug in, switch, and scale AI capabilities for real-time video analytics without vendor lock-in. Key features: - Supports any AI model, including YOLO, BLIP, InsightFace, and LLMs - Enables real-time video analytics over HTTP or WebSocket - Provides governance through a governed contract, ensuring end-to-end audit, drift detection, and sovereignty enforcement - Runs on any hardware, including GPU, filesystem, and network egress - Offers per-camera fair queuing and typed error envelopes - Provides Prometheus metrics with consistent label shapes across adapters Tech stack: Python Why it matters: AI Adapter simplifies the integration of AI models into OpenNVR, making it easier to deploy and manage AI capabilities without vendor lock-in. It ensures end-to-end governance and audit, enabling secure and compliant use of AI models in various deployment scenarios. Repository: open-nvr/ai-adapter GitHub stars: 9 GitHub description: AI Adapter is a flexible integration layer for connecting any AI model to OpenNVR. It enables seamless support for cloud, local, and edge models through a modular architecture—allowing developers to plug in, switch, and scale AI capabilities for real-time video analytics without vendor lock-in. Run any AI model — YOLO, BLIP, InsightFace, LLMs

### Overview: This project is a DIY pixel art generator for Ordinal Punks, a popular collection of blockchain-based pixel art. Using text prompts, you can create a high-quality copy of the Ordinal Punks collection from scratch. ### Key Features: - **Text-Prompt Attributes**: The project allows you to generate Ordinal Punks using text prompts, including attributes such as type, hair, mouth, and clothes. - **Fixed Metadata**: The official metadata has been fixed to include correct traits for Ordinal Punks, including the correct type and associated attributes. - **Community Support**: The project includes a Discord channel where you can communicate with the creators and get help. ### Tech Stack: Ruby ### Why It Matters: This project is a practical and accessible way to create your own Ordinal Punks collection, allowing you to personalize your own unique pixel art. It's a great way to explore blockchain technology and contribute to the Ordinal Punks community. Repository: ordinalpunks/ordinalpunks.starter GitHub Stars: 9 GitHub Description: DIY (Do-It-Yourself) - Yes, You Can - Ordinal Punks, Free Quick Starter Edition - Generate Your Own Pixel Art (From Text-Prompts)

Overview: LARA is a lightweight residual-stream adapter designed for frozen language models, allowing for the addition of multiple behaviors without modifying the base model's weights. This approach enables the use of many behaviors per token, facilitating continuous learning and multimodal adaptation. Key features: - Multiple behaviors can be loaded and mixed per token. - Small adaptation artifacts (a few MB) can be combined as a Mixture of Behaviors (MoBs). - Behaviors can be scaled, pinned, or composed, with hard or soft routing. Tech stack: Jupyter Notebook Why it matters: LARA provides a simple, efficient way to add flexibility and adaptability to language models without the need for retraining, making it suitable for continuous learning and multimodal tasks.

Overview: This repository contains public OpenAPI and Protobuf files for Pinecone RESTful APIs and gRPC services. Key features: - Database API: Manages index resources and records in Pinecone databases. - Inference API: Provides access to embedding and reranking models for Pinecone. - Assistant API: Facilitates uploading documents and answering questions with Pinecone's infrastructure. Tech stack: OpenAPI and Protobuf. Why it matters: These files represent the Pinecone API services at specific points in time and are versioned according to the month and year they were released.

Overview: Products Vectro is an open-source project that provides a template for a Shopify app designed to load products data into a vector database, offering flexibility with multiple vector store and embedding model providers. Key features: - Full data ingestion cycle - Vector store providers: Pinecone, Datastax (AstraDB), and Elastic (Elasticsearch) - Embedding model providers: OpenAI, Google Generative AI, and Hugging Face Tech stack: TypeScript Why it matters: This project simplifies the process of integrating vector databases into Shopify apps, making it easier for merchants to enhance their product discovery and search functionalities.

### Overview: GLM-5.3-Flash (320B MoE) is a high-performance language model running on 8x CMP 170HX (SM80, 64 GB, PCIe Gen2 x4) with vLLM on 8x CMP 170HX. This setup includes NVFP4 weights, DFlash2 speculative decoding, and 1M context. The repository contains all patches, launch configurations, and benchmarks. ### Key features: - **Pipeline Parallelism**: 8-stage pipeline parallelism. - **NVFP4 Weights**: 181 GiB of NVFP4 weights. - **DFlash2 Speculative Decoding**: 1M context. - **LibertAIDAI/GLM-5.3-Flash-NVFP4**: ModelOpt NVFP4, routed experts only. - **DFlash2 Drafter**: 5 sliding-window layers. - **Marlin W4A16 for NVFP4 MoE**: MoE model optimized for NVFP4. - **Triton Sparse MLA / Indexer Fallbacks for SM80**: Triton fallbacks for SM80. - **FA2 for DFlash2 Drafter**: Drafting mechanism for DFlash2. ### Tech stack: - **Python**: The primary language used for development and configuration. ### Why it matters: This setup is optimized for high-performance language modeling, making it suitable for applications requiring large model sizes and high throughput. The combination of NVFP4 weights, D