Data Transformation Using the Window Functions in PySpark

Demonstrated with a use case

Photo by Philip Myrtorp on Unsplash

Background

I work as an actuary in an insurance company. For various purposes we (securely) collect and store data for our policyholders in a data warehouse. One example is the claims payments data, for which large scale data transformations are required to obtain useful information for…