
0. Recap
In the first part of this colorful journey, we explored some interesting chromatic examples picked up from famous movies, trying to guess the reasons behind directors’ choices and scratching the surface of what can be done through R scripting.
In the second part, ** we put more emphasis on technical details introducing the Chroma**R toolkit, an R package that renders all the chromatic visualizations previously shown in part I.
Exploring chromatic storytelling in movies with R: the ChromaR package
Instead of focusing on sequences of colors and chromatic patterns across entire movies, the cornerstone of this third part will be the visual inspection of a single frame. Specifically, our main objective will be the automatic generation of proper color palettes from any input frame.

First things first, let’s start with a few bits of color theory.
1. Color Quantization
Extracting color palettes is all about color quantization. In computer graphics, color quantization is a process that reduces the number of distinct colors used in an image.

This technique is often used to display images with many colors on devices that can only manage a limited number of colors, due to memory limitations or stylistic choices. In these cases, finding the most representative set of color is crucial to reconstruct the original picture with the highest fidelity possible.
The color quantization process is can be divided into two steps:
- Choose a color space (e.g. sRGB, HSL, CIELAB, etc.) to represent colors mathematically as tuples of numbers. Different color spaces imply different strategies to organize colors. CIELAB, for instance, is a color space specifically designed to encompass all colors the average human can see, while sRGB is designed for monitors and printers.
-
Choose a quantization strategy. The easiest way to group points in a 3-dimensional space is the uniform quantization, where each axis is treated independently and divided into k equal size segments, defining k³ uniform regions of the space. Of course, this is also the weakest strategy as pixels/points are not uniformly distributed across the color space. Most standard techniques, instead, treat color quantization as a clustering problem, where each pixel is seen as an n-dimensional point of the color space and assigned to a group of similar pixels according to some metric (usually Euclidean distance). A valid alternative to Clustering algorithms is the median cut, which gets palettes by recursively cutting each set of data at the median point along the longest dimension.

2. Palette Extraction Implementation in ChromaR
For the ChromaR package, the choice has been to implement palette extraction through the K-Means clustering algorithm, using the Lab color space, in which Euclidean distance is more consistent with a perceptual difference.
Color Space and Metrics
In this color space, the nonlinear relations for L*, a*, and b* are intended to mimic the nonlinear response of the eye.

Different metrics have been proposed to quantify the difference between two Colors fitting human perception. Here we used the first and simplest one, Delta E 76, which is a standard Euclidean distance.

Several iterations improved this formula over the years, coming up with very accurate (and complex) metrics like Delta E 2000.
K-Means clustering algorithm
K-means is a clustering algorithm that minimizes the within-cluster sum of squared distances between the centroid and the other points of the cluster (variance is used as a measure of cluster scatter).
The number of clusters k represents the desired size of our color palette, where colors are determined by the RGB/Lab value of each centroid.
To keep it simple, the distance metric chosen for ChromaR is our Delta E 76 a.k.a. Euclidean distance.

Just like any other unsupervised Machine Learning algorithm, K-Means ends up marking each pixel of our frame with a label. The k distinct values this label can assume will generate our color palette.
Going one step further, we can use these labels to reconstruct the input image with fewer colors.

K-Means drawbacks and limitations
Despite its simplicity, K-Means can still produce competitive results. However, it doesn’t lack drawbacks and limitations.
K-Means is highly dependent on three things:
- Number of clusters: k is an input parameter, and an inappropriate choice of it may yield poor results
- Distance metric: results can be influenced by the way we decide to measure differences between points
- Initial points: the algorithm does not guarantee convergence to the global optimum, which means that the quality of the results may depend on the initial conditions
To mitigate the effect of these drawbacks, ChromaR implements the K++ algorithm, which limits the fluctuations caused by the choice of the initial values, and adopts a color space based on human perception.

Palette generation with ChromaR
All the palettes shown in this article have been generated using the ChromaR package running the following script:
Please note that the chromaR package is still a beta and many improvements must be performed in order to make it 100% operational. Therefore, please report any bug or issues you’re having!
Conclusion
The automatic palette extraction tool is indeed another useful arrow in the quiver for our chromatic explorations, but can hardly challenge the levels of human evaluation. In fact, human-designed color palettes typically show some high-level structure (e.g. color gradients, hues grouped together, high contrast between colors, etc.) that is completely absent in automatically created palettes.

However, there is still big room for improvement on this front. Generative Adversarial Networks (GANs), for instance, would probably help in generating palettes that are visually more plausible, while more complex metrics like Delta E 2000 may unlock human-like perceptions in clustering.
Even without our warm, magical human touch, though, we can still achieve pretty impressive results!
Tommaso Buonocore – Author – Towards Data Science | LinkedIn