Performance in Apache Spark: Benchmark 9 Different Techniques

Comparison of different approaches for array processing in Spark 3.1

David Vrba
Towards Data Science
12 min readMar 9, 2021

--

Photo by Kolleen Gladden on Unsplash

In Apache Spark, it is quite common that the same transformation can be achieved in different ways. This is also a consequence of the continuous development of Spark because new techniques and functions are available in new releases.

--

--

Senior ML Engineer at Sociabakers and Apache Spark trainer and consultant. I lecture Spark trainings, workshops and give public talks related to Spark.