Magic/Dunder Methods in Python
Magic or dunder methods are built-in methods automatically available to the user upon the substantiation of a class. Magic methods are usually surrounded by double underscores on either side of the method name. Everyone knows the most basic magic method, __init__. It is the way that we can define the initialisation behaviour of an object. There are however, many more magic methods. To find the directory of magic methods available to you, simply run the built-in Python function dir() on the name of your class.
Here, the directory of attributes, including the magic methods are output as a list when the dir() function is run on two built-in classes, dict and list.
To help users understand magic methods better, I have created a short example-led guide detailing how the standard default behaviour of a built-in magic method can be manipulated for a selection of useful magic methods.
The examples presented throughout however, come with a warning. If you modify the behaviour of a magic method, and you package your code up into a module intended to be used by other programmers, they may reasonably expect default behaviour for the magic methods. My advice therefore, is to make it explicitly clear to an audience if any method’s default behaviour has been overridden.
The aim for this tutorial is to provide some short snippet examples, illustrating how magic methods can selectively customised. The examples presented are deliberately short to help showcase the inner workings.
Restaurants class and Magic Methods
1. add(self, other)
To begin, I will create a contrived example based on restaurants. In this example I have a class where the user can pass the weekday earnings as as list to the init constructor. The list passed gets set as an attribute in the object.
Lets propose we have two restaurants in 2 cities and we would like to know the combined earnings for each day of the week, Monday through to Friday. We can easily implement this behaviour in our class by overriding the magic method, __add__(self, other).
In the magic add method, we perform a list comprehension, where we zip two lists together, iterate over the now zipped lists, and sum the zipped elements to produce a single numeric integer for each day of the week. The list comprehension is assigned to the variable, retval. Finally, the method can return a Restaurants object, with this new list (retval) passed.
In order to get the output we desire, we must now implement another magic method, the__str__(self) method. This method, takes the list attribute and returns it. When the str method is called, which is what happens when we print an object, the new list will be shown, as illustrated below in the code snippet.
To provide insight, when we use the plus operator, the call to the add method is made__add__(self, other). Here, both self and other represent objects of the Restaurants class. This is best illustrated in the example below. We can call the add method explicitly as shown on line 4, but that seems less, well, ‘magical’.
2. iadd(self, other)
The iadd method method can also be implemented in the Restaurants class. To demonstrate how iadd could be used, lets propose we have 2 restaurants in the same city and we would like a single numeric integer for the takings of those 2 restaurants.
We can override the iadd method. We can use the built-in Python sum function to sum the entries for week 1 in both restaurants, and save the result in the variable named retval, which can then be returned.
Now, when we call the assignment operation, += on our objects, this call will resolve to the method call, iadd. For this example, there may be simpler ways to get the answer requested, but I wanted to demonstrate the workings of the iadd method.
3. getitem(self, key)
We can also customise the indexing in our Restaurants class. Lets propose we would like to have the dual situation where the user can index a weekday’s takings using either standard indexing of the list we passed upon object creation, or choose to index the weekday evenings by typing in the name of the weekday to retrieve the takings on that day.
To do this, we can define custom behaviour for the getitem method. This method is invoked when a container is indexed. In the getitem method, a dictionary is defined, with the days of the week as keys, and the values as the corresponding numeric integer.
When getitem is called, which happens when we index our object as shown, if the day of the week passed in, matches any key from weekday_dict, we retrieve the value from the weekday key provided. This returns a numeric integer, which can be used to index our original list of takings.
As shown, we can index, wk1_city_1 with ‘Monday’ and we get back the value of 10. We can also choose to preserve the original indexing method by including the else statement shown. In this way, we have introduced greater flexibility into our class, through multiple ways of indexing.
4. ge(self, other) and le(self, other)
We can also implement the greater than or equal to, and less than or equal to magic methods into our class design. Lets propose we would like to determine which restaurant in city 1 or city 2 performed better in terms of takings. We can do this by implementing the following methods which return booleans:
__ge__(self, other)
Defines behaviour for the greater-than-or-equal-to operator, >=.
__le__(self, other)
Defines behaviour for the less-than-or-equal-to operator, <=.
When we use these operator, under the hood they resolve to the corresponding, ge/le magic method call.
Now, we can use the greater than, or less than or equal to operators on our objects. As shown, wk1_city_1 returned a boolean False when the operator >=was invoked against wk1_city_2.
The equality operator ==, __eq__(self, other), non-equality operator, !=, __ne__(self, other), and the less than <, __lt__(self, other), and greater than >,__gt__(self, other), operators work in the exact same way as the examples shown here.
5. setattr(self, key, value)
Custom behaviour can be implemented when attributes are set in the object. In the contrived restaurant example we have been using throughout this tutorial, the magic methods we have incorporated into our class, rely on a list being passed and set as an attribute in the object. In addition, we would like all the elements within the list to be numeric integers. These two conditions can be defined, in the magic setattr method.
First, we check whether the argument passed to the value parameter, in the setattr method is a list using the built-in isinstance method. If the user passes in any other object, a TypeError exception will be raised, with a useful message informing the user how the class is intended to be used. Next, we can iterate through the list, making sure each element is in fact an integer or float. This makes sense in the context of measuring weekday earnings for the restaurant.
If both conditionals are satisfied, we can set the attribute in the objects dictionary using the syntax:
self.dict[key] = value
As shown, if an element, such as 20 is passed as the string ’20’, a valueError exception is raised and the user is reminded what the expected input should be.
Be careful how you call setattr. In the first example shown below, the object.attribute syntax will call the method you are already in. In this case, an endless recursive loop will be invoked.
6. getattr(self, name)
Custom behaviour can be invoked when a user attempts to access an attribute that does not exist (either at all or yet). This can be useful for catching and redirecting common misspellings, giving warnings about using deprecated attributes.
This can be achieved via the magic method, getattr. In the example, when the user calls the correctly named attribute on the object, the object attribute’s value is returned.
If the attribute they call on the object does not exist, the custom getattr method can raise an AttributeError exception, informing the user, which attributes can be accessed.
7. repr(self)
The repr method defines behaviour for when repr is called on an instance of your class. The major difference between the str and repr magic methods are the intended audience. repr is intended to produce output that is mostly machine-readable, for example it could be valid Python code, whereas str output is intended to be human-readable.
Without custom behaviour defined for repr, when we print the object, the console output informs the user that the object is a Restaurants object, and gives the address in memory with the hex code. This can be structured more informatively by customising repr. The class name can be returned, followed by the list. When printed, as shown below, we now have valid Python code, equivalent to when we create an instance of the Restaurants class.
Summary
Magic methods can extend the functionality of our class, and in some cases preserve existing default behaviour. This approach was adopted when custom behaviour was implemented for indexing a container using getitem.
One caveat when using magic methods, is that the user may expect default behaviour, so you must explain this via error messages or a readme doc. Magic methods do have the benefit, when used appropriately, to be intuitive. The add method for example, gave output the user might expect. In addition, the user does not need to learn new method names.
This tutorial served as a primer to demonstrate how to use magic methods. The complete code for these examples can be found here. Enjoy using magic methods in your class design!
