Operations | Monitoring | ITSM | DevOps | Cloud

At Pepperdata we have been analyzing many thousands of Spark jobs on many different clusters, on-prem and cloud production clusters running Spark across a variety of industries and applications, and even workload types. In this presentation, Alex and I are going to cover a very brief intro to Spark, and we'll discuss some of the common issues we have seen and the symptoms of those issues and also how you can address and overcome some issues like that without having to think too hard.

Capacity Optimizer is an automated solution that continuously improves the performance of analytics clusters without manual intervention or application tuning.

In 2019, the CTO of a large global bank realized a problem: Their data continued to grow, costs for their Hadoop cluster rapidly escalated, and these costs started eating into their annual IT budget. Moving off of Hadoop, or “lift-and-shift” was out of the question. They needed a way to cap their cost and growth without impacting their ability to remain market competitive.

Have you changed the way you use big data in your business? Understanding the rapid pace of data usage across your organization and planning for the future of big data is a key skill. Sometimes we all need a little insight.