Operations | Monitoring | ITSM | DevOps | Cloud

Table of Contents A few months ago, we introduced Kubex support for the KAI Scheduler to improve GPU sharing for production inference workloads. The basic model is simple: The KAI Scheduler handles placement and GPU sharing. Kubex continuously observes usage and adjusts those allocations as demand changes. KAI provides the scheduling foundation. It lets multiple workloads share a GPU while accounting for the amount of GPU each workload requests. Kubex then closes the loop.

Table of Contents For most of Kubernetes’ history, memory management has been a blunt instrument. Cross your limit, and the kernel kills your container. There has been no equivalent to CPU throttling, no graceful backpressure, just a hard stop. With Kubernetes 1.37, that changes: Memory QoS, built on cgroups v2, graduates to Beta and is enabled by default.