Securing LLMs in Kubernetes: Best Practices – Meenakshi Kaushik & Jayanth Srinivasa, Cisco
Securing LLMs in Kubernetes: Best Practices – Meenakshi Kaushik & Jayanth Srinivasa, Cisco
In this session, we will provide a broad overview of the various aspects of securing and operating Large Language Models (LLMs) in a live production environment. For instance, we will demonstrate how to deploy an open-source LLM, such as Mistral, on a Kubernetes (K8s) cluster. This will involve front-ending with a K8s Nginx ingress controller and securing both the LLM pod and protecting LLM API prompt communication, with the aim of preventing incidents such as sensitive data leakage.
about: Jeff Boudier builds products at Hugging Face, the #1 open platform for AI builders.
Previously Jeff was a co-founder of Stupeflix, acquired by GoPro, where he served as director of Product Management, Product Marketing, Business Development and Corporate Development
by The Linux Foundation
linux foundation
