Testing Kubernetes Ingress resources can be tricky, and can lead to frustration when bugs pop up in production that weren't caught during testing. This can happen for a variety of reasons, but with Ingress specifically, it often has to do with a misalignment between the data used in testing and the traffic generated in production. Tools like Postman can be a great way of generating traffic, but they have the drawback of being manually created. Not only is this unlikely to create all the needed variations for a single endpoint (different headers, different request bodies, etc.), it would be almost impossible to create all the needed variations, for all possible endpoints.

Traffic replay is quickly gaining traction as the best way to recreate production scenarios.

Organizations are starting to realize that simply writing tests to generate traffic is simply not good enough. Rather, production traffic replication is now necessary, where you record traffic from your production environment and then replay it in your development environment. To match the modern principles of this testing methodology, it makes sense to also utilize modern infrastructure, like Kubernetes. Some benefits of using Kubernetes for production traffic replication are the ability to: Additionally, load generators are ephemeral. These reasons-and a few more besides-will be covered later in this post, but let's first take a deeper look at what production traffic replication is.