Operations | Monitoring | ITSM | DevOps | Cloud

Typically, Infrastructure-as-Code or IaCs have had their own languages to learn. For example, if leveraging Terraform most likely you came across Terraform’s native syntax, HCL. Though as software engineers we might be more familiar with other languages of choice. Using a general-purpose computer language vs a provider level syntax does unlock the power of the language; anything you can do in the computer language potentially can be additional methods, calls, etc.

This article is based on a true story. The names of the company and people involved were changed to protect the innocent 🙂 . A few weeks ago, we were contacted by a pretty big e-commerce company. We can’t really share their name but, for the purpose of this story, let’s call them “KubeCorp Inc”. They reached out to us following an edge-case incident they had, which resulted in severe downtime.

Kubernetes is one of the most popular platforms for managing and deploying applications built on microservices and containers. For the public sector, deploying pure upstream Kubernetes in offline, air-gapped environments can be a big challenge. Especially when you’re dealing with strict security controls and limited bandwidth, processes, and resources in place to ramp up quickly.

Kubernetes monitoring involves tracking application performance and resource utilization across cluster components, such as pods, containers, and services. The goal is to gain visibility into the health and security of your clusters. Kubernetes provides built-in features for monitoring, including the resource metrics pipeline that tracks several metrics like node CPU and memory usage and a full metrics pipeline.

When the CTO of this growing company (freshly acquired by a billion-dollar company) contacted me, he was concerned by the ability of his team to deliver what they committed to for the current year. His main issue was 15 engineers working in the same development environment. Can you imagine developing on the same workstation? Things will get worst as they plan to quadruple their engineering team size in the next 18 months.

As a software engineer, I admit I am not the best at networking. Can’t connect to your app for some reason, one going joke is to “always blame DNS” e.g the Domain Name System. My personal DNS experience is usually editing a few records for my personal blog and connecting a few tools and that is it. Thanks to distributed systems, had to learn all about SRV records and some more DNS concepts.