Google releases llm-d 1.0 for distributed model serving

On Oct. 20, 2026 Google said it released llm-d 1.0 at the PyTorch Conference, an open-source Kubernetes stack for serving models across a cluster. Google said it co-founded the project with Red Hat, IBM, and others, and that version 1.0 adds multi-tenant flow control, zero-downtime prefill and decode updates, and routing that keeps an agent's context on the same replicas. Google said its own Ironwood TPU coding test cut median time to first token by more than 85 percent and raised throughput 2.4 times versus a plain Kubernetes service.