Moxin: The All-In-One Tool to Run LLMs on Your Personal Device – Michael Yuan, Second State
Moxin: The All-In-One Tool to Run LLMs on Your Personal Device – Michael Yuan, Second State
Moxin is a new and fast growing open-source project to run Large Language Models (LLMs) on your personal computers. It has a Rust-based GUI for rich interactions from LLM model exploration, selection, to chatting. Using the Rust in-process messaging channels, it has a backend service that runs LLM apps, and respond to user messages. The backend service is based on the WasmEdge Runtime, which enables the application to take advantage of a variety of the different GPUs or other accelerators that might be available on the device. In addition, the team has also built an LLM database that provides up-to-date meta-data and benchmark results services to all open-source LLM models on Hugging Face.
In this talk, we will showcase the Moxin app and discuss the design decisions behind it. The application offers a template and a component library for Rust developers to create their own cross-platform LLM applications. Developer will learn how to build rich UI applications, and LLM services in Rust.
by The Linux Foundation
linux foundation
