Projects

Overview: Milvus Command Line Interface is a tool designed for interacting with Milvus, a distributed vector database, from the command line. It simplifies the process of managing and querying the database. Key features: - Command-line interface for Milvus - Supports creating, deleting, and managing collections, partitions, and indexes - Provides detailed collection, partition, and index descriptions - Allows for importing data into Milvus Tech stack: Python Why it matters: Milvus CLI simplifies the Milvus database management process, making it easier for developers to interact with the database from the command line, enhancing productivity and reducing the need for complex scripts or APIs.

Overview: Daisy.py is a Python platform designed to work with language model APIs, including OpenAI's GPT-3 and GPT-4. It includes classes and methods for developing modules that can be dynamically added to enhance the reasoning capabilities of large language models. Daisy incorporates "tool-forms" to utilize additional resources like web searching, context processing, and more. Key features: - Daisy voice assistant front-end - Web config tool based on Django - Dynamically loaded routes - TTS from ElevenLabs, Google Cloud TTS, and Google Translate TTS - Web scraping tool for SerpAPI - Calculator for solving solvable expression strings - Weather forecast from NOAA.gov - Conversational processing modules Tech stack: Python Why it matters: Daisy.py provides a platform for developers to create innovative applications and services using language models, enhancing the capabilities of AI through dynamic module integration and tool-forms. Repository: myrakrusemark/Daisy-openAI-chat Primary language: HTML GitHub stars: 37 GitHub description: Python platform for working with LLMs

Overview: This project is an audio-driven facial animation generator that utilizes a BiLSTM neural network for transcribing speech and a web interface to display the corresponding avatar and animation. Key features: - Audio-driven facial animation generator - BiLSTM neural network for speech transcription - Web interface for displaying avatar and animation Tech stack: Jupyter Notebook Why it matters: This project provides an interactive and visually engaging way to visualize speech through facial expressions, demonstrating the integration of machine learning and web technologies for real-time applications. Repository: nowickam/facial-animation GitHub stars: 37 GitHub description: Audio-driven facial animation generator with BiLSTM used for transcribing the speech and web interface displaying the avatar and the animation

### Overview: Ollama-compatible LLM server optimized for performance and compatibility with Ollama commands and models. ### Key features: - **Performance Boost**: Beats vLLM by 40% on single stream decode and 20% on multi stream decode, and outperforms Ollama by 2-4x. - **Custom CUDA Kernels**: Utilizes custom CUDA kernels for enhanced performance. - **MTP (Model Transfer Protocol)**: Supports Model Transfer Protocol for efficient model loading. - **Speculation**: Optimized for speculative execution for better throughput. - **3-bit Builds**: Uses 3-bit builds for reduced memory footprint and faster startup times. - **Uncensoring**: Models can be uncensored with a single command. - **Windows and Linux Support**: Runs on both Windows and Linux, optimized for NVIDIA GPUs. - **RAM Optimization**: Efficiently uses system RAM to minimize resource usage. ### Tech stack: C++ ### Why it matters: This server is designed to be a reliable and efficient alternative to Ollama, offering significant performance improvements and compatibility with Ollama's commands and models. It leverages custom CUDA kernels and speculative execution to deliver superior results, making it a valuable tool for developers and AI enthusiasts looking to enhance their model performance and compatibility.

Overview: Gemini Canvas Proxy is a free, unlimited Gemini API solution that turns a free Gemini Canvas session into a local OpenAI-compatible endpoint, allowing any OpenAI-compatible app to use the generated token as the API key. Key features: - Chat completions with tool calling, multimodal input, and image generation - Compatible with Hermes Agent, OpenClaw, LiteLLM, LangChain, the OpenAI SDK, and curl - Bearer auth, loopback-only by default - Private MessageChannel bridge — no WebSocket, immune to Chrome's Local Network Access restrictions Tech stack: Python Why it matters: This solution simplifies the integration of OpenAI-compatible apps with the Gemini API, making it accessible to a wide range of developers without the need for expensive licenses. Repository: pranrichh/gemini-canvas-proxy Primary language: Python GitHub stars: 36 GitHub description: Free unlimited Gemini API for any OpenAI-compatible app.

Overview: Second Brain is an AI-powered knowledge management system designed for local use, offering 100% local AI capabilities, including text generation, image embeddings, and advanced vision understanding without sending data to the cloud. It integrates seamlessly with Google Drive and includes features like multimodal search, knowledge graphs, and real-time analysis. Key features: - **Fully Local AI Stack**: Utilizes LLaVA, CLIP, and LLaVA models for text, image, and vision processing. - **Multimodal Search**: Searches documents, images, and ideas across text, image, and both simultaneously. - **Google Drive Integration**: Synchronizes and processes Google Drive files without storing them locally. - **Knowledge Graph**: Automatically discovers relationships and visualizes topics for better understanding. - **Complete Privacy**: Runs entirely on local models with no API keys, cloud dependencies, or tracking. Tech stack: Python, Docker, FastAPI, PostgreSQL, pgvector. Why it matters: Second Brain provides a secure, private, and cost-effective alternative to cloud-based knowledge management solutions, allowing users to manage their data locally and access it anytime, anywhere. Repository: raold/second-brain GitHub stars: 37 GitHub description: 🧠 AI Second Brain — 100% Local Knowledge Management Private, self-hosted second brain. Store, search, and synthesize documents, images, and ideas with local LLMs (LLaVA, CLIP) and PostgreSQL + pgvector.

Overview: InferMesh is a GPU-aware inference mesh designed for large-scale AI serving and training, providing distributed control, topology-aware routing, and standardized observability across heterogeneous environments. Key features: - GPU-aware routing - Network-aware routing - Pluggable strategy host for real-time GPU, runtime, and network signals Tech stack: Rust Why it matters: InferMesh addresses the three hard problems of modern AI at scale, offering improved observability, placement & routing, and cost & reliability through a two-layer architecture with three planes: Infrastructure Layer, Workload Overlay Layer, and Control Plane. It scales across clusters and regions and supports edge-to-cloud paths, optimizing for SLOs, utilization, and cost—all through one consistent API and label model. Repository: redbco/infermesh Primary language: Rust GitHub stars: 37 GitHub description: GPU-aware inference mesh for large-scale AI serving and training.

Overview: This is a simple POC of FastRTC, a framework for real-time voice conversations using Python. Key features: - FastRTC-powered real-time voice communication - Speech-to-text and text-to-speech capabilities - Integration with an OpenAI-compatible API key for language model responses - Adjustable voice-activity detection settings Tech stack: TypeScript Why it matters: This project serves as a demonstration of FastRTC's capabilities in voice communication and showcases how it can be integrated with other technologies like speech-to-text and text-to-speech. Repository: rohanprichard/fastrtc-demo Primary language: TypeScript GitHub stars: 37 GitHub description: A simple POC of FastRTC, a framework to use voice mode in python!

### Overview: DiffTAD is the first formulation of diffusion model for Temporal Action Detection (TAD) task. It introduces a novel DETR-based denoising diffusion framework that accurately predicts action proposals from random temporal proposals. ### Key features: - **First DDIM-based denoising diffusion framework for TAD task.** - **Action proposals are denoised in a DETR style framework where Transformer-Decoder is the denoiser.** - **Enhanced diffusion sampling efficiency and accuracy by introducing a novel cross-step selective conditioning during inference.** - **Denoising action proposals as queries enables faster convergence.** - **First work that solves discrete dense detection task using Transformer Decoder as denoiser.** ### Tech stack: Unknown ### Why it matters: This work presents a generative modeling perspective in Temporal Action Detection, offering a new approach to solving the discrete dense detection task using Transformer Decoder as the denoiser.

Overview: Assistant Client is a tool designed for local testing and debugging of CanvasApps, a family of Virtual Assistants called "Saluton". Key features: - Local testing and debugging of CanvasApps - Integration with the CanvasApp ecosystem - JavaScript protocol for emulating Android environment and calling native methods - Runs in a browser, eliminating the need for physical devices Tech stack: TypeScript Why it matters: It simplifies the development process by abstracting away the complexities of testing on physical devices, making it easier to develop and test without needing physical devices. Repository: sberdevices/assistant-client Primary language: TypeScript GitHub stars: 37 GitHub description: A tool for local testing and debugging of CanvasApps, a family of Virtual Assistants called "Saluton". It is implemented as a JavaScript protocol that emulates the Android environment and calls native methods. This approach does not require physical devices, allowing for the execution of a virtual assistant through a browser. README excerpt: [](https://www.npmjs.com/package/@sberdevices/assistant-client) <img src="https://user-images.githubusercontent.com/982072/97004635-0888a900-1546-11eb-8f25-283a0693608e.png" height="150px" width="150px"> Assistant Client is

Overview: This repository contains the official tools for Multi-Object Hallucination in Vision-Language Models, presented at NeurIPS 2024. It includes a dataset called ROPE, which is released to evaluate multi-object hallucination in vision-language models. Key features: - Introduction to Multi-Object Hallucination in Vision-Language Models - Authors and Affiliations - Updates and Acknowledgments - Quick Start: Cloning the repository, preparing the environment, downloading the dataset, and testing models - Issues and Contact Information Tech stack: Python Why it matters: This repository provides a comprehensive evaluation protocol for multi-object hallucination in vision-language models, offering insights into the factors contributing to hallucinations and potential solutions to mitigate them. Repository: sled-group/moh GitHub stars: 37

Overview: `beaver` is a simple, local, and embedded database designed to manage complex, modern data types without requiring a database server, built on top of SQLite. Key features: - Key-Value Dictionaries - Blob Storage - Persistent Lists - Persistent Priority Queue - Probabilistic Sketches - Document Collections - Vector Search Tech stack: Python Why it matters: `beaver` is a lightweight, local database solution that simplifies the management of complex data types, making it ideal for small, local use cases where a full-blown database server would be overkill.

### Overview: This repository contains example code and quick tutorials for solving common tasks using the VideoDB SDK. It provides a comprehensive guide for developers to build use cases with VideoDB, focusing on practical applications and real-time streaming scenarios. ### Key Features: - VideoDB Quickstart - Scene Index QuickStart - Multimodal Quickstart - Scene Level Metadata Indexing - Multicam Quickstart - Baby Crib Monitoring - Cricket Match Monitoring - Flash Flood Detection - Intrusion Detection ### Tech Stack: Python ### Why it Matters: This repository is essential for developers looking to integrate VideoDB into their AI applications, providing a clear and concise guide to building use cases with VideoDB. The tutorials are designed to be easily replicable across different programming languages, making it a valuable resource for anyone interested in leveraging VideoDB for real-time monitoring and analysis.

Overview: This repository contains the code for a paper that proposes a Multimodal Transformer model to predict in-hospital mortality using clinical notes and structured electronic health record (EHR) data. Key features: - Multimodal Transformer model for interpreting in-hospital mortality prediction - Structured EHR data processing - Unstructured clinical notes processing - Training and testing scripts for both clinical notes and multimodal models Tech stack: Python Why it matters: This model aims to enhance the interpretability of in-hospital mortality prediction by integrating structured and unstructured data, making it valuable for healthcare professionals to understand the factors contributing to patient outcomes. Repository: weimin17/Multimodal_Transformer Primary language: Python GitHub stars: 37 GitHub description: A Multimodal Transformer: Fusing Clinical Notes With Structured EHR Data for Interpretable In-Hospital Mortality Prediction

### Overview: 本项目旨在通过在BERT模型后添加Bi-LSTM进行fine-tuning的方法，对机器翻译子任务中的翻译质量进行评价。实验使用了WMT17-QE数据集，并在PyTorch框架下进行了fine-tuning。 ### Key features: - 使用BERT模型进行fine-tuning - 结合Bi-LSTM模型进行改进 - 代码参考了https://github.com/huggingface/pytorch-pretrained-BERT - 实验结果在wmt17-qe数据集上表现良好，Pearson’s相关系数达到0.6837 ### Tech stack: Python ### Why it matters: 本项目为机器翻译领域中的翻译质量评价提供了一种有效的评估方法，特别是在BERT模型基础上的fine-tuning技术的应用。

### Overview: Specspeech is a fast ASR and TTS inference solution that leverages speculative decoding and parallel processing to enhance the speed and accuracy of speech recognition and text-to-speech synthesis. This project is designed to be lightweight and efficient, making it suitable for real-time applications and edge devices. ### Key Features: - **Speculative Decoding**: The system anticipates and corrects errors in speech recognition, improving overall accuracy. - **Parallel Processing**: Audio and language processing tasks are performed in parallel, significantly reducing inference time. - **CUDA Compatibility**: Utilizes NVIDIA GPUs for optimal performance, ensuring high-speed processing. ### Tech Stack: Python ### Why it Matters: Specspeech is crucial for applications requiring real-time speech recognition and text-to-speech synthesis, such as voice assistants, call centers, and virtual assistants. By leveraging speculative decoding and parallel processing, it significantly reduces latency and improves the overall user experience. ### Repository: yaya-sy/specspeech GitHub Stars: 36 GitHub Description: Speculative Decoding for fast ASR and TTS inference

Overview: RNAGenesis is a compact RNA foundation model that addresses the challenges of sequence understanding, *de novo* RNA design, and 3D structure prediction. It achieves top results in 11 out of 13 tasks on the BEACON benchmark and outperforms wild-type scaffolds in gene knockout efficiency across CRISPR-Cas9, base editing, and prime editing platforms. Key features: - Inference-time optimization: Combining tree search and gradient-based model guidance to steer RNA design. - State-of-the-art performance: Achieves top results in 11 out of 13 tasks on the BEACON benchmark. - Versatile RNA generation: Synthesizes diverse non-coding RNAs, including natural-like aptamers and structurally optimized CRISPR sgRNAs. - Experimental validation: Designed sgRNAs outperform wild-type scaffolds in gene knockout efficiency, with up to 2× improvement across CRISPR-Cas9, base editing, and prime editing platforms. Tech stack: Jupyter Notebook Why it matters: RNAGenesis provides a powerful and versatile tool for researchers and developers in the field of RNA therapeutics, enabling the design of functional RNA molecules with improved efficiency and specificity.

### Overview: This project explores and implements optimization-based planning and learning-based control for quadrotors, focusing on motion planning and control for both individual trajectories and whole-body motion. The implementation leverages MuJoCo simulations and integrates various advanced techniques such as FIRI, GCOPTER/MINCO, GCS, BMTP, YOPO, nonlinear MPC, and PPO with differentiable MPC policies. ### Key features: - Constructing collision-free convex flight corridors with FIRI - Optimizing constrained multicopter trajectories with GCOPTER/MINCO - Planning routes and Bézier trajectories using GCS - Simulating deterministic and randomized wind disturbances - Training PPO trajectory-tracking policies and reference-free gate racing with an MLP baseline - Integrating differentiable MPC into PPO policies (ACMPC) - Independently implementing biconvex minimum-time planning (BMTP) - Adding an aerial manipulator platform for research on whole-body motion planning and control - Integrating YOPO (You Only Plan Once), an end-to-end motion planner for UAVs - Adding a MINCO-inspired whole-body motion planner for aerial manipulators - Integrating YOPO (You Only Plan Once), an end-to-end motion planner for UAVs - Implementing diffusion-based motion planning ### Tech stack: The project is built on Python, utilizing libraries and frameworks such as MuJoCo, FIRI, GCOPTER/MINCO, GCS, BMTP, YOPO, nonlinear MPC, and PPO with differentiable

### Overview: Resume Insight is a project that leverages large language models (LLMs) to analyze and understand resumes. It uses OCR, NLP, and machine learning techniques to extract, structure, and generate insightful question-answer pairs from resumes, which are then fine-tuned for better LLM performance. ### Key Features: - **Comprehensive Data Extraction**: Utilizes OCR to convert PDF resumes into text. - **Intelligent Data Structuring**: Applies NLP techniques to categorize and structure resume text into meaningful data points. - **Dynamic Dataset Creation**: Generates a rich dataset of question-answer pairs tailored for LLM training, focusing on resume insights. - **Advanced Model Fine-Tuning**: Employs cutting-edge fine-tuning strategies, including memory optimisation and parameter-efficient fine-tuning (PEFT), to enhance an LLM's capability in generating contextually relevant text. ### Tech Stack: - Python - PyTorch - Hugging Face Transformers, Datasets - OCR libraries (e.g., pytesseract, pdf2image) - Spacy for NLP tasks - bitsandbytes for memory optimisation - Ollama for generating the training dataset ### Why it Matters: Resume Insight aims to improve the accuracy and relevance of LLMs by providing them with high-quality, structured resume data. This project showcases how advanced AI techniques can be applied to enhance the performance of language models, making them more effective in analyzing and understanding human resumes. ### Repository: AlexisBalayre/UK-Resume-LLM ### GitHub Stars: 36 ###

### Overview: ChromaDB.Client is a .NET client for Chroma and Chroma Cloud, designed to simplify interaction with the Chroma search engine. It supports collections, tenants, and databases, enabling efficient querying and filtering of records. The client is compatible with Chroma versions 0.4.10 to 1.5.9 and is backed by the independent community project ChromaDotNet. ### Key features: - **Collections, Tenants, and Databases**: Manages collections, tenants, and databases for efficient data organization. - **Record Writing and Reading**: Enables writing and reading records with support for filters. - **Search API**: Provides access to the Chroma Cloud Search API, including hybrid BM25 search. - **OpenTelemetry Traces and Metrics**: Tracks and monitors OpenTelemetry traces and metrics for enhanced performance and monitoring. ### Tech stack: C# ### Why it matters: ChromaDB.Client simplifies integration with Chroma and Chroma Cloud, making it easier to implement search functionalities in .NET applications. It is compatible with various versions of Chroma and is backed by a community project, ensuring ongoing support and updates.