The world’s leading publication for data science, AI, and ML professionals.

DON’T underestimate these Python dunder methods!

There are tons of great articles, books and videos about Object Oriented Programming with Python, many of which deal with dunder (also…

Photo by Rhett Wesley on Unsplash
Photo by Rhett Wesley on Unsplash

There are tons of great articles, books and videos about Object Oriented Programming with Python, many of which deal with dunder (also called magic) methods. I’ll provide some links to resources I have found useful for myself at the end of this article.

Bear with me, though.

I’ll make my point now: The following list of dunder methods seemed quite irrelevant to me before I had a glimpse of just how useful they can be.

source: https://docs.python.org/3/reference/datamodel.html
source: https://docs.python.org/3/reference/datamodel.html

These are known as rich comparison methods. Now, in hindsight, they may seem a little redundant or even useless, given that python itself provides functions for comparing objects just as well.

The thing is, when you’re building a class, you may want it behave in a certain way. Now, behavior is different from functionality in many ways.

Let’s see how it may apply to a Data Science problem.


Suppose you are testing different supervised learning techniques for a classification problem. For simplicity, I’ll assume you are familiar with these terms .

Generally, you would have a set of raw features to be used as input variables for training the algorithm. However, important transformations such as filling out missing values, standardizing variables and so forth will almost certainly take place.

As in scikit-learn’s pipelines, you could build different transformers for, say, Logistic Regression.

Notice that, even though the steps are identical and carry the same name, it is not possible to say that they are equal with regular comparison in Python.

>>> from sklearn.preprocessing import StandardScaler, MinMaxScaler
>>> from sklearn.impute import SimpleImputer
>>> from sklearn.pipeline import Pipeline
>>>
>>> mean_transformer_1 = Pipeline(steps=[
...     ('imputer', SimpleImputer(strategy='mean')),
...     ('scaler', StandardScaler())])
>>> 
>>> mean_transformer_2 = Pipeline(steps=[
...     ('imputer', SimpleImputer(strategy='mean')),
...     ('scaler', StandardScaler())])
>>> 
>>> mean_transformer_1==mean_transformer_2
False

And there you go! Comparing models before and after you apply different pipelines can be quite troublesome.

One way to compare them, is to compare each step’s classes to see if they are the same:

>>> class_1 = mean_transformer_1.steps[0][1].__class__
>>> class_2 = mean_transformer_2.steps[0][1].__class__
>>> class_1 == class_2
True

Now, besides ugly and verbose, this can be quite lengthy if you have a bigger pipeline. With all honesty, hardly ever do pipelines this small go to production in real life.

CAVEAT: for the sake of attention span, I kept the comparison of classes only. However, the arguments passed such as 'mean' should also be compared if you really want to get to the bottom of comparison.

And as Raymond‘s famous quote states: there has to be a better way! And there is.

Let’s make a class to compare those:

>>> class SmartCompare:
...     def __init__(self, pipeline):
...         self.pipeline = pipeline
...
...     def __eq__(self, other):
...         return [m1.__class__ for m1 in self.pipeline]==[m2.__class__ for m2 in other.pipeline]
... 
>>> mean_transformer_1_c = SmartCompare(mean_transformer_1)
>>> mean_transformer_2_c = SmartCompare(mean_transformer_2)
>>> mean_transformer_1_c==mean_transformer_2_c
True

Of course, this is not sooooo much better but it makes a point!

You could also inherit the Pipeline class instead of making it an attribute to your class. Or even use other methods for comparison.

In any case, dunder methods go beyond regular comparison methods and make your code more readable and pleasant.

Thanks for you patience!!! Critique is welcome, feel free to drop a comment.


Some useful resources:

Enriching Your Python Classes With Dunder (Magic, Special) Methods – dbader.org

Welcome to Python, Meet the Dunders


Related Articles