Lambda is the glue that holds serverless architectures together. Before its release, most users felt it was a matter of luck as to whether AWS would let you connect a service to another. If not, you had to spin up a VM or a container to transform the events from one service in a way that your target service could handle them. Since Lambda was easier to set up, people assumed that all code they would deploy on it would run faster and cheaper than on other compute services.

As a developer, I love the versatility of Python. Over the years I have used Python for so many different use cases: game development, APIs, IoT, machine learning, and web development. It can scale tall applications in a single bound and take on any challenge faster than you can pip install flask. Something you learn very quickly in the world of app development is to build everything for scale.

Observability is a measure of how well we are able to infer the internal state of our application from its external outputs. It’s an important measure because it indirectly tells us how well we’d be able to troubleshoot problems that will inevitably arise in production. It’s been one of the hottest buzzwords in the cloud space for the last 5 years and the marketplace is swamped with observability vendors. Different tools employ different methodologies for collecting data.

When we set out to trace applications running outside of AWS Lambda, there was little doubt in our minds that building on top OpenTelemetry was by far the best course of action. There are many reasons for this, but chiefly, it is a question of coverage. At its most fundamental level, achieving coverage requires as-wide-as-possible support for technologies, and interoperability among instrumentations.

Customers expect that online services are available all the time. The truth is that outages happen to almost everyone because providing 100% service availability is challenging and costly. Creating reliable and profitable service is, amongst other things, finding the balance between application availability, costs and time to market. Faster feature delivery means less availability as constant changes to production may cause issues and introduce bugs.

As a developer, Python for me is a heavy-lifting and versatile language. I’ve used it for building APIs, internet of things projects, file and data conversions, machine learning and (of course) web development. Like with any modern, commonly used language, the functionality behind the application is only as good as the infrastructure that it is deployed onto.

It’s no secret that application containerization has revolutionized the digital world as we know it by providing a transient gateway into elastic infrastructure that can scale and grow as needed. Where traditional virtualization was all about creating a single homogenous entity, containers are self-contained units of software, able to run in just about any environment, making them extremely portable.

You’ve probably seen Rush Hour, a logic puzzle where you have to slide cars and trucks out of the way to steer the red car towards the exit. In real life, when your customers are responsible for tracking hundreds or thousands of data points from dozens of valuable, mission-critical sensors, you’re tracking engine speed, network signal level, distance from the RF, and more—and not just through traffic but across continents.

The combination of SNS to SQS to Lambda is a common sight in serverless applications on AWS. Perhaps triggered by messages from an API function. This architecture is great for improving UX by offloading slow, asynchronous tasks so the API can stay responsive. It presents an interesting challenge for observability, however. Because observability tools are not able to trace invocations through this combination end-to-end. In X-Ray, for example, the trace would stop at SNS.

Serverless backends have different monitoring challenges when compared with traditional applications, mostly due to the distributed and proprietary nature of serverless. Making monitoring and debugging efficient for serverless requires a unique set of tools and techniques. In this article, we’ll discuss the challenges of debugging serverless backends and how to utilize third party tools to improve the monitoring process.