Browsing Tag
Distributed Inference
1 post
Distributed AI inference, model serving clusters, routing, caching, scaling, and accelerator orchestration.
llm-d 1.0 Brings Agent-Aware Routing to Kubernetes AI Inference
llm-d 1.0 stabilizes APIs, adds agent-aware routing and coordinates zero-downtime inference upgrades. Here is what platform teams should test before migrating.
