Kubernetes GPU Scheduling for MLOps and GPU Sharing
The default Kubernetes scheduler was built for stateless services: web servers, APIs, databases. It schedules a pod, checks that a node has enough of whatever resources were requested, and binds it. For CPU and memory, that model works fine. For GPUs, it falls apart in three specific ways. First, GPUs are treated as an opaque integer resource.