In this blog series we are examining the Five Myths of Apache Spark Optimization. So far we’ve looked at Myth 1: Observability and Monitoring and Myth 2: Cluster Autoscaling. Stay tuned for the entire series! The third myth addresses another common assumption of many Spark users: Choosing the right instances will eliminate waste in a cluster.

In this blog series we’ll be examining the Five Myths of Apache Spark Optimization. (Stay tuned for the entire series!) If you’ve missed Myth #1, check it out here. The second myth examines another common assumption of many Spark practitioners: Cluster Autoscaling stops applications from wasting resources.

In this blog series we’ll be examining the Five Myths of Apache Spark Optimization. (Stay tuned for the entire series!) The first myth examines a common assumption of many Spark users: Observing and monitoring your Spark environment means you’ll be able to find the wasteful apps and tune them.