When developing artificial intelligence algorithms of varying complexity, it is sometimes necessary to perform specific operations on the available data. In this context, we will work with Tensorflow 2. X and in particular with the Keras API. This working environment provides a wide range of mathematical operations, data processing and layers, but it may be necessary to perform specific and original operations. In another article, we have already talked about custom layers and how these tools allow us to create completely original models suitable for our application. However, we don’t always need to define a real original layer, especially when the operations to be carried out are simple and immediate. We are thinking, for example, of pre-processing operations, data resizing, post-processing…
Keep in mind that you might always utilize only TensorFlow operations in your Custom Layers and Lambda Layers to get the best performance from your algorithm and benefit from TensorFlow’s graph features.
In this article
We will introduce the concepts of the lambda function, its uses in the Python language, its potential and typical applications. We will see how to create a lambda layer in Tensorflow from a theoretical point of view and then in the application case. I remind you to follow my Medium profile to support my work and that you can find all the code and illustrations of its use in my YouTube channel and GitHub repo.
Python Lambda Function: the anonymous function
If you are familiar with Python programming, you’ll probably know the def statement for the definition of function objects. However, there’s another expression for the creation of this kind of statement: the lambda. This name could frighten at a first look (the name itself derives from the lambda calculus, a kind of symbolic logic), but in Python, it’s just a keyword that allows you to introduce quick expressions as a shortcut.
Just like the def statement, this expression creates a function that you’d call later, but in this case, it returns the function, without assigning it to a name in the actual scope. For this reason, we usually refer to lambda functions as anonymous functions.
The main application of the lambda function is related to the inline definition of a function, without assigning a name to the function itself in the actual scope.

The application of the lambda statement is quite simple: we use the lambda word followed by one or more arguments.
lambda x, y, z: f(x, y, z)
where x, y, z are the lambda function arguments and f is the desired expression, using the arguments themselves. We can find a strong relationship with the def statement. However, let’s focus on the dissimilarities, useful in some situations:
- The lambda function is an expression, not a statement: the def statement assigns the new function to the name in the header, the lambda function returns it as a result. Remember: lambda returns a value (a new function).
- The body of a lambda function is an expression, not a block of statements: the lambda expression body (f) can be thought of as the return statement in a standard function with the def statement. For this reason, the lambda function is also less general because its body is limited to a single expression, without useful logic statements (if, for…).
We could note that def and lambda do mainly the same job. The following example shows you how to generate the same _dummycalculator function with both strategies. Defaults work on lambda arguments just like in a def.

The lambda function follows the scope lookup rule like a function inside another one, introducing a local scope that sees names in enclosing functions, in the module and obviously the built-in scope.
Why should I use a lambda function?
Summing up: lambdas functions are a function shorthand. You just have to embed a function definition in the code that uses it. However, remember that they’re an absolutely optional solution, but can become a powerful solution for embedding small slices of code.
With a lambda function is possible to generate another interesting Python code structure, the jump tables: lists/dictionaries of actions that will be performed when/where required by the user. Let’s check the following example:

This simple example allows me to introduce another concept: lambda functions could be used also where def statements are illegal syntactically. The presented code shows a list of callable functions embedded in a list literal. I can’t do so with def statements: they’re statements, not expressions.

You can enhance this solution by generating a dictionary where each of the nested lambdas functions generates a function that you’d call later, creating a general multiway branching tool.

For the sake of completeness, let’s remind another important aspect: code proximity. If your function has to be used in a single context, the creation of a lambda might be a good solution. At the same time, def statements require assigning a name to the new functions in the actual scope, but these names may clash with the other ones.
Lambdas are not a limit
As I stated before, the body of a lambda function has to be a single expression, not a block of code. This looks like a severe limit on the complexity of the expression itself. In reality, by exploiting some of the possibilities that the Python syntax provides, it is possible to create much more complex logical expressions, even if they are contained in a single expression. We can even create loops and iterations with a quick single line lambda, let’s check the following code:

Lambda functions and Python mantra
As stated in the Zen of Python, by Tim Peters:
Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts.
For this reason, lambda functions are usually limited to simple expressions and small pieces of inline code. For a larger logic, a def statement has to be preferred.
Lambda function in Tensorflow
In Tensorflow, a Lambda layer "Wraps arbitrary expressions as a Layer object".
The
Lambda
layer exists so that arbitrary expressions can be used as aLayer
when constructingSequential
and Functional API models.Lambda
layers are best suited for simple operations or quick experimentation. For more advanced use cases, follow this guide for subclassing[tf.keras.layers.Layer](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Layer)
.
Keep in mind: Lamba layers have some important (de)serialization limitations.
The main reason to subclass
[tf.keras.layers.Layer](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Layer)
instead of using aLambda
layer is saving and inspecting a Model.Lambda
layers are saved by serializing the Python bytecode, which is fundamentally non-portable. They should only be loaded in the same environment where they were saved. Subclassed layers can be saved in a more portable way by overriding theirget_config
method. Models that rely on subclassed Layers are also often easier to visualize and reason about.
The code shown here concerns a brief analysis of the famous fashion-mnist dataset. It is a problem of image processing and classification for clothes images. These are 28×28 black and white images. The goal is to develop an algorithm capable of classifying them automatically, with the highest possible accuracy.

For this task, I created two very simple models: a fully connected Dense net and a Convolutional Classifier. The aim of this tutorial is the application of Lambda layers in an AI workflow, so let’s check their application.
Data normalisation is an important tool for maximising the effectiveness of our model training. In general, Python and Tensorflow provide us with various tools for image standardisation and processing, but in this case, I decided to normalise the data using the Lambda layer. This layer has been instantiated with a python lambda function as the first argument and has been calculated in the output of the previous one, following the syntax of the Keras Functional API.
In this second application case, we can see how the practicality of the lambda layer has been exploited to perform a simple operation of expanding the dimensionality of the data to make it compatible with the subsequent two-dimensional convolutional layers.

Conclusions
We recalled the key concepts of lambda functions in Python language, saw how to implement the analogue layer in Tensorflow to perform some convenient operations in our workflow and saw a useful application example on a famous multiclass classification dataset.
Until next time, Marco
3 Minutes Machine Learning
3-minutes machine learning is a series of tutorials, videos and articles related to the world of AI, Deep Learning, and Data Science. You can find the complete videos collection on YouTube. The GitHub repository contains all the Google Colab notebooks shown in the articles and videos. I hope these contents will be useful or simply of interest to you. Any feedback is welcome.
Check out the other episodes: