Operations | Monitoring | ITSM | DevOps | Cloud

Spot

Ready to run! Get Started with Spark on Kubernetes

The Apache Spark and Kubernetes integration was recently officially declared Generally Available and Production Ready, generating a lot of interest from the community. More and more companies choose to run their big data workloads on Kubernetes to benefit from containerization and a standard cloud-native ecosystem.

Spot Ocean now supports Kubernetes pod topology spread constraints

As a premium autoscaler for containers and Kubernetes applications, Spot Ocean automatically and continuously executes scaling actions based on the requests and specified constraints of pods and specific containers. This container-driver autoscaling approach is core to how Ocean leverages and optimizes the compute infrastructure required to run containers in the cloud.

Ocean explained: Ocean controller deepdive

As a managed data plane service for containerized applications, Spot Ocean provides a severless experience for running containers in the cloud. Ocean integrates with the control plane of your choice, and handles key areas of infrastructure management, from provisioning compute and autoscaling, to pricing optimization and right-sizing. A core component of Ocean’s architecture is the Ocean controller, which is how Ocean and your Kubernetes cluster integrate and interact.

Cluster Roll feature enhancements now available

Spot by NetApp’s Ocean includes a powerful feature called “cluster roll.” This feature simplifies applying changes to Kubernetes worker nodes. Typical changes include applying a new image, modifying or adding user data, and updating security groups. A cluster roll applies these changes without having to disable the Ocean autoscaler. It also removes the need for you to manually attach new nodes or remove replaced nodes from the cluster.

Improve Apache Spark performance with the S3 magic committer

Most Apache Spark users overlook the choice of an S3 committer (a protocol used by Spark when writing output results to S3), because it is quite complex and documentation about it is scarce. This choice has a major impact on performance whenever you write data to S3. On average, a large portion of Spark jobs are spent writing to S3, so choosing the right S3 committer is important for AWS Spark users.

Ocean Insights now available for Google Cloud

As companies move more applications into the cloud, and package them into containers, environments become more complex with limited visibility. While infrastructure is abstracted away as much of it is delivered by the hyperscalers, this creates an opaqueness that makes it hard to control costs and understand resource utilization. As a result, many companies are experiencing high cloud bills and lots of cloud waste.

Ocean explained: container-driven autoscaling with Kubernetes

Whether you’re using a managed Kubernetes service like AWS EKS, GCP GKE or Azure AKS, or self-managing a DIY cluster deployed with open source tools like kops and Kubespray, the underlying hardware can vary from container to container. Each container requires specific resources (CPU/ memory/GPU/network/disk) and as long as the underlying infrastructure can provide those resources, the container will be able to execute its business logic.

Accelerate incident analysis by incorporating Ocean logs in any pipeline

Spot Ocean delivers container-driven autoscaling to continuously monitor and optimize your cloud environment. Positioned at a busy crossroads in the application deployment pipeline, Ocean has a critical role when shipping new containers. Given the highly dynamic nature of Kubernetes environments, events happen constantly and take shape as logs in Ocean. These can help you understand the chain of events in different scaling scenarios, from debugging cluster issues to incident analysis.

AWS re:Invent 2021: Scale the Kubernetes-first, cloud-native model

The widespread adoption of Kubernetes has made it table stakes for the modern cloud native stack. Software is now being purpose-built for Kubernetes, and as companies enter this new phase of their cloud journey, they are looking to scale. Building upon years of experience with Kubernetes, Spot by NetApp is continuously innovating to bring new ways to achieve this goal. From container-optimized, serverless infrastructure to Kubernetes-native continuous delivery, Spot is connecting the dots to vastly improve the efficiency and manageability of Kubernetes applications and environments.