While platform engineering is not a new concept, it has been rapidly gaining popularity. It is estimated that by 2026, 80% of major software engineering organizations are expected to form platform engineering teams.

One of the reasons why platform engineering is getting so much attention is that it helps to automate deployment while allowing developers to remain in a productive state and avoid high cognitive loads. In this article, I will explore how packaging applications using open-source Cloud Native Buildpacks can contribute to 3 critical pillars of a successful platform engineering strategy: developer productivity, security, and standardization.

Looking at current containerizing strategy

Once developers are done developing their application, they need to containerize it so it can be moved along the CI/CD pipeline and deployed. Stackoverflow's latest Developer survey found that Docker is the developer's most used tool, so writing a Dockerfile is likely the next step. Writing a basic Dockerfile is easy, but writing an exhaustive Dockerfile involves tailoring it for the application and stack-specific attributes, organizing instructions efficiently for optimal caching, and ensuring security.

When faced with this task, developers often copy a Dockerfile from another project, with potentially a different stack and requirements that do not fit their project. And it's hard to blame them for doing that. A simple web search on how to create a good Dockerfile will return an overwhelming number of strategies to follow. Writing a Dockerfile is often a cause of frustration for developers whose goal is to build features rather than building flawless metadata for generating container images; it's essential to simplify their tasks for improved efficiency.

According to a recent survey by Microsoft, the cognitive load – a term that represents the mental effort needed for a developer to finish a task – plays a vital role in boosting productivity. The study highlights that developers who find their tools intuitive and easy to use feel they are 50% more innovative than their counterparts. By eliminating the need for developers to write Dockerfiles, they can redirect their focus toward building applications.

Boost developer productivity

Developers can use open-source Cloud Native Buildpacks to automate the process of transforming application source code into a runnable container image. A Buildpack is a set of executables that inspects source code and creates a plan to build the application. This concept has demonstrated its effectiveness for over a decade, starting with Heroku and then used by platforms like Cloud Foundry, Google App Engine, GitLab, CircleCI, and more.

Using the pack cli, just one command is needed to create a container image from an application source code. For example, for a Python application, you could just run this command at the root of your application source code:

pack build my-python-app --builder paketobuildpacks/builder:base

Let's explore a bit what is happening behind the scenes. Let's start with the specified builder in the command. A builder is an OCI image containing an ordered combination of buildpacks, a build-time base image, a runtime base image and a lifecycle binary.

• The build-time base image provides the base environment for the builder – such as an Ubuntu image.

• The runtime base image provides the base environment required to run the application.

• The lifecycle binary orchestrates buildpack execution, and then assembles the resulting artifacts into an OCI image.

As for buildpacks, they have two tasks to do, the detect and build phases.

• Detect step: the buildpack determines if it is needed or not. It does this by checking for files that are specific to a stack. For example, a Python buildpack may look for a setup.py or requirements.txt file. While a Java buildpack might look for a pom.xml file or a build.gradle.

• Build step: the buildpack sets build-time and run-time environment variables, download dependencies, source code compilation (if needed), and configures the application entry point and any startup scripts.

Cover security and compliance basics

Now that we have a good understanding of what a buildpack is and how to create a container image using it, let's jump into how it can help with security as part of platform engineering strategy. Container images are vital to an engineering platform IDP and application security will highly depend on how well these images are secured.

Reproducible Builds

Buildpacks generate reproducible builds of images. That means if given the same input for application compilation, the exact same output as a previous build will be produced. Reproducible builds ensure that the artifact was generated from a trusted source. This is a robust way to prevent unauthorized applications from being added to containers in the software delivery process. This helps companies to meet the requirements for level 4 of Supply Chain Levels for Software Artifacts (SLSA), which, on top of requiring a two-person review of all changes and a hermetic, recommends also having a reproducible build process.

SBOM

In addition, buildpacks can generate Software Bill of Materials (SBOMs) that can be used to secure software. SBOMs give a detailed list of all components and dependencies used in software applications. SBOMs ensure that only up-to-date and secure components are used by comparing the SBOM with known security issues.

Using the pack cli command below will generate an SBOM for a Python application named "my-python-app".

pack sbom download my-python-app

Rely on the community

Security is an endless task, and for organizations that don't have a dedicated team for this task, it can be hard to cover the basics. For example, despite the strong recommendation against the practice, nearly three-quarters of workloads are permitted to run with root access.

Here the wider buildpack community can help these companies by providing standardized build and run images (typically LTS versions). Providers like Google, Heroku, and Paketo Buildpacks offer production-ready buildpack images built with best practices and security protocols already implemented for various stacks. Companies can benefit from the very sharp expertise from these communities at no cost.

More specifically, when it comes to security, community-led Paketo Buildpacks commits to weekly rebuilds to keep packages up-to-date and promises updates for critical CVEs within 48 hours. And if this isn't quick enough for you, these open-source images are easily customizable and can be promptly patched.

Standardization

A key element of platform engineering involves establishing standardized operations to ensure consistency, efficiency, and reliability across the organization's infrastructure. So, ideally, no matter what language your application is made of, the process and tools should be the same.

With the rise of microservices, especially for large companies, the need to support multiple stacks is increasingly common. This can drastically increase the complexity of providing a uniform experience.

By working with any stack and programming language, Buildpacks can be a building block of an IDP. Each buildpack provider (Heroku, Google, and Paketo) will provide a list of supported stacks. For example, Paketo Buildpacks supports many frameworks and stacks, such as Java, .NET Core, Node.js, Go, PHP, Ruby, and more. In addition to using images from multiple providers depending on the needs, companies can also create custom buildpacks if they cannot find what fits them.

Conclusion

To implement a platform engineering strategy, companies need to assemble a number of different tools to create a smooth developer experience. Open-source tools are often required because interoperability is a must when assembling the different components together. Cloud Native Buildpacks fit the bill and can help an engineering organization on multiple fronts.

Author Bio: Sylvain Kalache

Sylvain Kalache is an entrepreneur and software engineer who has worked in the tech industry for over a decade. He co-founded Holberton, an education company that trains the next generation of developers at scale. Sylvain started his career as an SRE, maintaining infrastructure for companies like LinkedIn and Slideshare.