Using first-principles thinking to learn to code better to process our data. Understanding what problems the optional object solves, breaking it down, and building it back up
Overview
Starting from C++17, C++ Standard Library offers std::optional which is a class template that manages optional contained value. Optional type or sometimes also called Maybe type represents an encapsulation of an optional value.
In this post, I will cover the problems that std::optional solves, describe the requirements to solve them, and finally build a simple version of the optional class to demonstrate how it works under the hood.
Problems optional object solves
In some scenarios in our code, we will face situations where we want to return an optional object from a function or accept optional parameters into our functions. For example, in the following code, we want to return a data object that is only available if some conditions are met.
Another example is when we want to set data, but sometimes they are not available and we want the function to check and decide.
Without using a class template like std::optional to wrap Data object, prior to C++17 we can use the following techniques.
Wrap our data in a struct/class with a flag
We can create the following struct, OptionalData:
And this is how we invoke this function:
Not only this technique adds overhead in size (bool + padding), but also time overhead to construct Data object which when it is not valid is unnecessary.
Use pointer to wrap our Data
To avoid having to construct Data object when we don’t need to, we can use smart pointers such as std::unique_ptr to wrap our Data. When conditions are not met we can return nullptr.
And at the caller side, this is how it looks like:
This technique is much better compared to the previous one, but the code isn’t very expressive because using pointer for this purpose doesn’t explain the intention of the function very well.
Return multiple values
Since C++11, we can return multiple values using std::tuple, or in this case we want to return two values, we can use std::pair. This is probably the most common technique to use.
At the caller side, this is how it looks like:
Semantically, this technique is the same as our first technique where we wrap our Data object in a struct. So it suffers from the same problem and additionally the programmer has to remember whether to use "first" or "second".
Passing in pointer and returning boolean status
We can also use a C-like technique where we pass a pointer and return a boolean.
At the caller side, this is how it looks like:
Not only that we use raw pointer here that might be considered unsafe, but also it is less expressive though it works and Data object needs to be constructed, always.
Problems
We can now list the problems with various techniques above, they are as follows.
- They don’t convey the intention clearly. Our intention is not getting a pointer, or a pair of objects, but we want to optionally get Data object when it’s available
- Some of them unnecessarily construct Data object
- For the C-like technique, it is unsafe especially if the pointer is allocated from free store / heap
How do we solve those issues?
From the problems listed above, these are the requirements that we have.
- The optional object has to contain the real object
- The contained object should be within the optional object (not dynamically allocated)
- The optional object may be empty
- The contained object may be set later
- The contained object may be destroyed before the optional object
Writing simple optional object
We build our optional object to understand how std::optional works. We omit most of the interfaces and focus on the main things which are to store the object.
The first requirement is to contain any object and has a flag to check whether it is empty or not.
But this is the same as our first technique above, just that now it is a class template. We want the optional object not to construct the contained object by default. One way to achieve it is to use std::aligned_storage to reserve the space for the contained object.
What happens here is that the contained type is not stored, but the space necessary to store it is reserved when we construct optional object.
When we construct an empty optional object, we only initialize those two. To initialize the contained object we use placement new, that is using new operator to construct an object in an existing place.
Another approach is to use anonymous union, which may be a better and cleaner approach.
The next requirement is that we may want to destroy the contained object before destroying the optional object, we add reset() function for that purpose.
We may also want to set the contained object later, such as in the example below:
We can implement copy assignment and move assignment operators for this purpose.
We can implement move assignment operator in a similar way. Data object is first implicitly converted to optional type by invoking optional(const Data& t) constuctor followed by the invocation of copy assignment operator.
The rest of the interfaces
To get the contained object, we can easily implement value() function that returns the contained object if it is present and throws exception otherwise.
The rest of interfaces can be implemented easily, not all shown in this post because it can become too long. You can see this page to see other interfaces.
Other improvements to the existing std::optional
There are further improvements that can be made to the existing std::optional. In some scenarios we may want to invoke other functions based on the status of our optional object.
For this, according to this page, C++23 will add the following:
-
and_then(f) If the returned optional object contains another object, callable ‘f’ is executed which can return any type, otherwise return empty optional object.
-
transform(f) If the returned optional object contains another object, callable ‘f’ is executed which returns optional type, otherwise return empty optional object.
- or_else(f) If the returned optional object is empty, callable ‘f’ is executed which returns optional type, otherwise return optional object.
They are called monadic functions whose job is to abstract away boilerplate code, we can modify our code above to in a simpler and more readable form.
Similar Functionalities in Python
You may have realized that in Python we can achieve optional object easily just by returning None.
In C++, we have to use std::optional in C++ standard library to achieve this.
Summary
Key takeaways are:
- We use std::optional to make our code more expressive
- std::optional contains the object within itself, depending on where it is stored (stack/data/heap)
- std::optional makes a copy of the contained object
- Monadic functions will be added in C++23 to improve the abstraction in our code by removing the needs of writing boilerplate code