One of the most common concerns when moving to the cloud is cost. Given that cloud allows you to turn IT costs from CAPEX (long-term investments ex. in hardware equipment and software licenses) into OPEX (day-to-day operating expenses), it’s crucial to choose the right service and estimate it properly. In this article, we’ll look at the common pitfalls and discuss how you can avoid them to truly benefit from the cloud’s elasticity.
It shouldn’t be a surprise that Lambda configuration error is one of the most common error messages, and we all know AWS error messages aren’t known for being especially detailed. Oftentimes you will come across other vague error messages like “encoding not enabled,” or “stream is failing,” and depending on the context, this could mean your services could be completely down.
In the 21st century, it’s quite easy to manipulate machines and computers. Our worries are no longer if something is doable, but if something can be perfected. Therefore, we mostly search for new ideas and ways to make our work impeccable. For example, if you’re using a particular software and you realize that the software is excellent, but it could be better in some ways that would allow you to work even faster, you’ll explore the alternatives.
With the volume, velocity, and variety of today’s data, we have all started to acknowledge that there is no one-size-fits-all database for all data needs. Instead, many companies shifted towards choosing the right data store for a specific use case or project. The distribution of data across different data stores brought the challenge of consolidating data for analytics.
When you want to stay ahead and on top of things in a fast-moving industry, machine learning (ML) is surely one of the trending solutions. Today, innovative companies already have leading Machine Learning tools well-integrated into their processes. In comparison, your start could seem dreadfully slow. Or maybe you just don’t have the time or resources to invest in running your own Machine Learning training infrastructure.
SQL is great, but sometimes you may need something else. By and large, the prevalent type of data that data engineers deal with on a regular basis is relational. Tables in a data warehouse, transactional data in Online Transactional Processing (OLTP) databases — they can all be queried and accessed using SQL. But does it mean that NoSQL is irrelevant for data engineering?
When using services created by other people, it’s often neither obvious what they mean, let alone how to fix them. One of these error messages you might see when using Amazon API Gateway is “encoding not enabled”. The first question here is, what kind of encoding does this error message refer to? The first thought might go into the video or audio encoding direction and lead to a dead-end since you probably didn’t send any audio or video files.
One of the serverless best practices is one-purpose functions. You should keep your Lambda functions small and solve exactly one use-case. This way, you can optimize them better and keep potential security problems contained. But creating many small functions can get overwhelming quickly. Even small projects can end up with more than 20 Lambda functions.
