The industrial automation landscape is evolving rapidly, driven by a fundamental shift in how software solutions are developed and deployed. In the past, many factories had in-house IT expertise to build and maintain their basic software stacks, the present reality is quite different. Small-scale factories often lack the IT know-how to manage their increasingly complex digital infrastructure, and they often rely on contractors to set up and maintain their systems.

Containers are a lightweight, portable, and consistent way to package applications and their dependencies. Containers provide an isolated environment, ensuring an application runs reliably across different environments. Enterprises and tech-savvy individuals are using container technologies because of their benefits. However, container orchestration tools have become necessary to manage clusters with the rise in container usage.

A lot of people, like myself, believe automotive is the most innovative sector today, especially when it comes to software. We are living a critical moment in automotive, where evolution is being pushed onto the market. This rapid software shift is posing challenges that are complicating developers’ progress. Most of these challenges are hardware-related. Gaining access to target development hardware has become an impossible task, the recent global microchip shortage did not help.

Storage and container management systems are almost polar opposites of each other. One deals with permanently storing, and protecting data for as long as it’s needed. The other automatically manages highly dynamic workloads, scaling resources up and down as required. More organisations are taking a container-first approach to application deployment and management, but the underlying challenge of safely and securely storing data still remains the same.

Today, Canonical announced the release of Charmed Spark – an advanced solution for Apache Spark® that provides everything users need to run Apache Spark on Kubernetes. Apache Spark is suitable for use in diverse data processing applications including predictive analytics, data warehousing, machine learning data preparation and extract-transform-load (ETL).

We’re super excited to announce that we have shipped the first release of our solution for big data – Charmed Spark. Charmed Spark packages a supported distribution of Apache Spark and optimises it for deployment to Kubernetes, which is where most of the industry is moving these days.

Open source projects are a testament to the possibilities of collective action. From small libraries to large-scale systems, these projects rely on the volunteer efforts of communities to evolve, improve, and sustain. The principles behind successful open source projects resonate deeply with the divide-and-conquer strategy, a universal approach that has proven effective across multiple disciplines.

The last interim release before an LTS (Long Term Supported release for those new to the Ubuntu terminology) is a particularly exciting time. This is the release where the team aims to land as many major changes as possible to ensure that the community has the chance to take them for a spin and provide feedback for further refinement ahead of Ubuntu 24.04 LTS. These features span the entire Ubuntu Desktop stack, from the user interface, to software management, to core security and architectural changes.