Should you run big data workloads on Kubernetes? Not so long ago, most people would have said certainly not. Among the reasons you might have heard: Although early adoption of Kubernetes was dominated by stateless services rather than data-centric applications, more and more teams responsible for big data platforms are now looking to adopt Kubernetes, especially those looking to deploy and operate big data workloads in the cloud.
Continuous integration and deployment (CI/CD) processes are becoming the standard for many organizations operating in the cloud—including many Spot by NetApp customers—with Jenkins as a core tool for building, testing and deploying code through CI/CD pipelines. Spot has had an integration with Jenkins since our early days, allowing our users to configure Elastigroups managed by Jenkins and deploy jobs running on spot instances.
AWS’s annual re:Invent wrapped up this week, and although we didn’t get to see customers, colleagues and, friends in Las Vegas this year, it was still an eventful three weeks. Spot by NetApp sponsored the virtual conference and hosted a number of activities including our AWS JAM session on Kubernetes and infrastructure scaling, an expert session on cloud compute optimization, and an interview with Spot by NetApp’s CEO and founder, Amiram Shachar.
As in years past, re:Invent is where AWS announces many new features, services and enhancements. This year is no exception and here at Spot by NetApp we are proud of our product and engineering groups for delivering immediate and full support in relevant areas such as the new GP3 EBS volume and new EC2 instance types – D3, D3en, R5b and M5zn.
With over 1 million customers and over 14 years of experience working with them, Amazon Web Services has some real insight into best practices and technologies for setting up and running secure, well governed cloud deployments. The recently launched AWS Control Tower puts this knowledge to use providing companies with a quick and simple way to securely set up complex, multi-account environments for rapid, yet fully controlled growth and innovation.
Cloud cost-efficiency is a crucial ingredient for ensuring business success. However, uncoordinated attempts at reducing costs can create even greater inefficiencies and outright cloud waste. In this post we’ll focus on one example of this – using EC2 spot instances when AWS Savings Plans have already been purchased – and how Spot by NetApp intelligently utilizes all available cloud compute pricing models to eliminate cloud waste and achieve dramatic cost reduction.
Every application has its unique compute infrastructure requirements. This is why container orchestrators such as Kubernetes provide application engineers with the use of taints and affinities to control precisely where pods run.
