This simple FastAPI service will help you find data in a data lake Data lakes provide a myriad of benefits. They are data agnostic and don’t require you to define a schema upfront. However, without a proper structure, it may be challenging to find the data that you need. In this article, we’ll address this problem by creating a FastAPI abstraction allowing us to query the AWS Glue metadata catalog.

At first glance, there may seem to be little not to love about the ELK Stack. It’s open source. It’s free (if you set it up and manage it yourself, at least). It’s a widely used solution with a thriving ecosystem surrounding it. But if you’ve ever actually built and managed an ELK stack environment, you have probably found that the theory doesn’t match the reality. The ELK stack is full of hidden costs, and it often fails to deliver real value over the long term.

In my last post we looked at the structure of AWS IAM policies and looked at an example of a policy that was too broad. Let's look at a few more examples to explore how broad permissions can lead to security concerns. By far the most common form of broad permissions occurs when policies are scoped to a service but not to specific actions.

Choosing the right service plan is crucial when you are using Amazon Web Services (AWS). With 170 distinct services, ranging from compute to storage to networking and content delivery — each offered at different price points — the process requires careful consideration to make the right choice for your business. By default, AWS services are available on-demand and you pay a monthly bill for services used.

Stackery can be used to create a new CloudFormation template or to quickly visualize an existing one. Code is automatically generated as you simply drag-and-drop resources on a graphical grid. The experience is much more intuitive than previous generation tools like AWS CloudFormation Designer. Stackery visualizes resources the way a human would perceive them, grouping related resources together.

For the first time, a software load balancer exceeds 2-million RPS on a single Arm instance. A few weeks ago, while I was working on an HAProxy issue related to thread locking contention, I found myself running some tests on a server with an 8-core, 16-thread Intel Xeon W2145 processor that we have in our lab. Although my intention wasn’t to benchmark the proxy, I observed HAProxy reach 1.03 million HTTP requests per second.

If you’re worried that switching to serverless infrastructure is too expensive for your business, you’re not alone. Total spending on cloud services will top $284 billion by 2024. The good news is there are many ways to track and lower your serverless operation costs without slowing down your business. Lambda and how can it help your business? Find out more by reading these Lambda frequently asked questions.

As your infrastructure grows, getting a handle on all your AWS resources can be overwhelming. While that’s probably an understatement, help could be right around the corner. We’ll cover a few CloudFormation visualizer tools that can help, but let’s level set first. AWS CloudFormation is an established Infrastructure-as-Code solution that allows you to define, provision, organize, manage and update your AWS resources from a text-file template.