Context – it’s everything. After we sort, organize, and interpret our data, we make descriptive visualizations to tell the story of the data. When we’re using the plots to understand the data ourselves, it’s all well and good – we understand the context of our plot. However, if our goal is to inform others and drive decision making in a broader audience, we want plots that don’t just share the data, but clearly drive a conclusion. Your audience simply won’t know the data as well as you, so it’s up to you to guide their thought processes. This usually entails plot elements like trend-lines or color coding, but even then you’re assuming a lot from your audience – is an upward trend a good or bad thing? How will they know?
Here, we’re going to describe how to create custom matplotlib cmap objects (color maps) to create "danger-zone" plots. These create plots that are instantly understood by technical and non-technical audiences alike to show when some data is trending in a way that’s "bad" for the organization. Compare the two plots below – without knowing anything about what this data is, you’d know from the plot on the right that there’s something "bad" about months 2 and 3, just by the addition of some custom background color gradients.
We also don’t always control how our data is shared. What may start as a presentation to internal and external customers (where you can explain the data trends and insights) may turn into your figures being sent by email without deeper context. If the plot can explain itself and indicate conclusions one should draw, it makes it less mentally taxing on those seeing it for the first time. It also helps ensure that others draw the same conclusions that you did.
What makes a colormap (cmap)?
Before we go about making custom cmaps, let’s dive into the basics of how they work. The built-in Matplotlib cmaps are color gradient groups that fit into one of four categories: sequential, diverging, cyclic, and qualitative. There‘s a solid visual repository for your options in matplotlib’s documentation here, but here’s a quick taste:
When we fetch a cmap object, python generates a 256-element list of colors used to define the colormap. Each of the colors is internally represented as a 4-element array of Red-Green-Blue-Alpha (RGBA), where alpha is best thought of as the "transparency" of the color. So, for example, if we pull the 'Reds' cmap object, and investigate the 100th color (out of 256), we see this:
Of course, none of the pre-made cmaps are going to perfectly fit with the specific colors and transition points we want to make a proper ‘danger plot’. But with this kind of direct access to a cmap’s core elements, re-writing rows to get what we want becomes and interesting proposition. This is certainly one way to do it (which I include in my github notes), but it’s harder than it needs to be, especially when trying to manually blend between colors.
As a more automated path, we’ll use LineasrSegmentedColormap() from matplotlib. But before we get there, we’ll need a lightweight function to view the colormaps we create before we apply them to our plots:
Building Custom CMAPs
The LinearSegmentedColormap() function allows us to give a list of colors, with a list of "nodes". The "nodes" refers from points on the colormap (scaled from 0 to 1) where the listed colors reside – outside of that, it blends with it’s neighbors. So, in the example below, the 256-element cmap has element 1(node 0.0) as pure green (rgba= 0,1,0,1), element 128 (0.5) as pure yellow (1,1,0,1) and element 256(1.0) as pure red (1,0,0,1).
Ultimately, we’re going to design a function where we tailor where colors transition. By adding more nodes we can effectively do this. If two neighboring nodes are the same color, then the space between them will automatically be the same color. Take a look at the following example, where we add some repetitive colors:
Ah! Much tighter transitions! In this case, we transition from green to yellow between 40% and 50%, and from yellow to red between 60%-70%. Great! This in itself could be used to manually create the cmaps needed for your own custom-shaded danger plots. However, when we’re dealing with our data, we’re more familiar with the value that we want the cmap to change color, instead of the % of the total axis length. Let’s functionalize this for our plots.
Two-Color and Three-Color Custom CMAPs
Using this same technique, we’ll build 2-color blends and 3-color blends. These functions both take in the colors we want to use – keep in mind that LinearSegmentedColormap() is flexible enough to take in colors as names (‘Red’), rgba ([1,0,0,1]), or even hex (#ff0000). In fact, here’s a simple color-picker tool to help you get just the color you want. These functions also take in the value for the transition between colors, the width of that transition, as well as the plot axis limits (min and max) so that the transition values can be scaled appropriately.
In the function call, we pretend we have a plot with a min of 0 and max of 10, with the transition happening right at 5, with a width of 2. Sure enough, the transition occurs halfway through the 256-element generated cmap.
Expanding this concept to a 3-color function, we do much of the same, just with a little more complexity.
Applying to Plots
First, let’s generate some dummy data to use for our plots. Here we have a dataframe with the number of patients seen in a hospital per month:
To add any gradient cmap to the background of a matplotlib plot, we use imshow(), which displays data as an image. The most challenging variable to pass is the first one ("X"). This maps a location on the plot to an element in our cmap. Since we want to step across our custom cmap linearly, we’ll use np.linspace() to create a linear array of 256 elements between 0 and 1. We’ll also want to pass in the extent parameter to let imshow() know the limits of the plot that we’re stretching our cmap across. Finally, note that ylim() gives an output list of (y min, y max), so we use bracket notation to access each as the limits when calling our two_color_cmap() function.
Applying to our data and setting an arbitrary transition point, we get this:
Or, if want to create a ‘warning’ region before we get into a red ‘danger’ zone, we can use our three-color function to add in some yellow:
Awesome! Now our audience’s eyes are drawn to Month 2, where there were an unacceptably high number of patients.
Horizontal Bar Charts
These custom cmaps also support horizontal plots. We only have to make a few tweaks to our plot code. The most important is how we change the "X" variable in imshow(). In this case, we remove the transpose .T which turns grad from a set of 256 1-element arrays to a single 256-element array. Investigate the grad variable with print(grad) to understand the subtle differences on how we make this work. Finally, note that np.linspace() now goes from 0->1. If we ever wanted to flip how our cmap is displayed, we can turn this back into 1->0 like we used previously.
Applying to Line Plots
There are many situations where we want to highlight an acceptable ‘range’ for our data. In such cases, both high and low values indicate problems, but in the middle there is a happy medium. For these situations line plots make a more telling visual. Using the same data and the same three_color_cmap() function we made, we can make a line plots with our custom cmap.
On the left is one of our standard ‘blended’ color maps, but our custom cmap functions work perfectly well for zero-transition cmaps. The zero-transition cmap on the right would be more useful if the values for failure are very specifically known and there is no "wiggle-room". This is particularly useful for statistical process control (SPC) plots where known upper and lower control limits exist.
Conclusions
Besides being visually appealing, Danger Plots add context to our plots. They enable our data to speak for itself and can instantly empower anyone to identify data points representing when something has gone wrong. With the framework of these cmap-generating functions, the sky is the limit – make your own and join as many colors as you want! As you’ve seen, once our setup functions in place,they’re simple to implement, so you’re not burning calories generating the perfect plot.
As usual, the entire code walk-through notebook can be snagged from my github. Please follow me if you found this useful! Cheers, and happy coding out there.
