The world’s leading publication for data science, AI, and ML professionals.

Deploying AWS Lamba Function & Layer with Serverless Framework

A tutorial on installing dependencies on an AWS Lambda Layer and extending them from an AWS Lambda Function developed and deployed through…

Figure 1. AWS Lambda, API Gateway, Serverless Framework | Image by author
Figure 1. AWS Lambda, API Gateway, Serverless Framework | Image by author

Serverless environments have emerged as a disruptive alternative for the deployment of applications in cloud instances without the need to configure, manage, operate and maintain a traditional server.

In this tutorial, we will see an example about how to develop, configure, integrate and deploy a Lambda Function [1] that extends the functionalities of a Lambda Layer [2] through the Serverless Framework [3].

This blog will be divided as follows:

  • AWS Lambda Function & Layer
  • Serverless Framework
  • What we are going to do
  • Installing Serverless Framework
  • Deploying AWS Lambda Layer
  • Deploying AWS Lambda Function

If you’re already familiar with AWS Lambda and the Serverless Framework, feel free to skip the next two sections and go straight to the examples.

Let’s get our cup of coffee and enjoy ☕️!

AWS Lambda Function & Layer

AWS Lambda is a computing service that allows you to run code without the need to configure and manage a server. To use AWS Lambda, you just need to package your code as well as the dependencies you will need (as a .zip file or a docker image) and organize them in a Lambda function.

In Figure 2 we see a representative description of the organization of the elements within a Lambda function.

Figure 2. Lambda functions | Image by author
Figure 2. Lambda functions | Image by author

As we can see in the figure above, each Lambda function is an isolated environment, that is, by nature they cannot share or extend functionality directly with another Lambda function. And this is one of the reasons why Lambda Layer emerged.

Lambda Layer is an AWS Lambda extension that works directly with Lambda functions. The aim of Lambda Layer is to contain applications, requirements, dependencies, and even custom runtime components that can be extended from a Lambda Function.

In figure 3 we see a description of the integration of a Lambda layer with a pair of Lambda functions.

Figure 3. Lambda layer & functions | Image by author
Figure 3. Lambda layer & functions | Image by author

As we can see from the previous figure, the implementation of a Lambda layer favors the organization of components that can be shared and extended through Lambda functions. Also, the use of Lambda layer helps on reducing the size of Lambda functions because it would not be necessary to pack dependencies multiple times (as seen in Figure 2).

However, in practice, defining and configuring one or more Lambda functions, as well as integrating with one or more Lambda layers, can be a laborious and tedious process, since AWS Lambda is commonly integrated with different services such as Amazon API Gateway, Amazon S3, Amazon SQS, Amazon SNS and many more. This is where Serverless Framework takes center stage.

Serverless Framework

Serverless Framework is a tool that facilitates the definition and configuration of AWS Lambda as well as its integration with other AWS services. In a nutshell, Serverless Framework streamlines your AWS Lambda development, integration, and Deployment process.

Serverless is a cloud-agnostic framework, that is, it allows the development and deployment of services through the definition of a YAML file and the use of a CLI, to different cloud providers such as AWS, Azure, Google Cloud, Knative, and more.

Once we are clear on the relationship between AWS Lambda Function & Layer as well as the relevance of Serverless Framework in the development and deployment of cloud services, let’s go straight to the tutorial.

Don’t forget to fill up your cup of coffee ☕️!

What we are going to do

We are going to create a Lambda function and a Lambda Layer. The Lambda layer will contain only the numpy library. The lambda function will extend the numpy library from the Lambda layer and implement it in its main method. The entire development and deployment process will be through the Serverless Framework.

Once the context is defined, let’s go for it!

Installing Serverless Framework

To install Serverless, we first need to have node.js. To install node.js on macOS:

$ brew install node

For installing node.js on Ubuntu:

$ sudo apt install nodejs
$ sudo apt install npm

Or if you prefer, you can go directly to the official node site to download it. Once node.js is installed and the npm manager is enabled, we proceed to install Serverless:

$ npm install -g serverless

If you have any problems with the installation through the npm manager, you can follow the official Serverless installation guide for installing the standalone binary.

Since we are going to develop and deploy to AWS in this example, you need to add the credentials of an AWS account, as follows:

$ serverless config credentials --provider aws 
                                --key <access_key> 
                                --secret <secret_key>

Once Serverless is installed and the credentials configured, we are ready to start with the fun part, let’s go for it!

Deploying AWS Lambda Layer

First, let’s create a template for our Lambda layer with the serverless CLI, which we will call lambda-layer.

(venv) $ serverless create --template aws-python3 
                           --name layer-numpy 
                           --path layer-numpy

The above command will generate a directory called lambda-layer which will contain the handler.py and Serverless.yml files.

(venv) $ tree
.
└── layer-numpy
    ├── handler.py
    └── serverless.yml

In essence, the handler.py file contains the code that will run on AWS Lambda. The serverless.yml file contains the definition of the Lambda function or layer as well as all the necessary configurations for the deployment of the service.

Nervertheless, since in this case we want to deploy a Lambda layer that only contains a library, the handler.py file is not necessary. On the contrary, the file we require is requirements.txt which will contain the library that Lambda layer will host.

So, removing the handler.py file and adding the requirements.txt file, our directory looks like this:

(venv) $ tree
.
└── layer-numpy
    ├── requirements.txt
    └── serverless.yml

where the requirements.txt file only contains a reference to the numpy library:

(venv) $ cat requirements.txt 
numpy==1.21.0

To deploy the content of the requirements.txt file via serverless, we are going to require the serverless-python-requirements plugin which allows serverless to access the content of the requirements.txt file and prepare it for deployment.

To install the serverless-python-requirements plugin we type:

(venv) $ sls plugin install -n serverless-python-requirements

When installing the plugin, the serverless.yml file is updated by adding the definition of the installed plugin. The default content of our serverless.yml is like the one shown in the following code snippet.

As we can see, in line 10 we have the definition for a Lambda function created by default, however for the Lambda layer deployment we do not need it. On the other hand, in line 14 we have the reference to the plugin installed in the previous step.

Our serverless.yml file, after the modifications for the deployment of our Lambda layer, is as follows:

As we can see, from line 13 the definition of our Lambda layer begins, which is of type pythonRequirements whose content will be dockerized . Our Lambda layer is called python-numpy and is compatible with python 3.8 runtime.

It is important to mention that, since we enable the dockerization of the requirements, it is necessary to have docker installed on our local machine from where we will perform the deployment.

Great, once we have the requirements.txt file, customized the serverless.yml file and installed the plugin , we are ready to deploy. For the deployment, we only type:

(venv) $ sls deploy

And that’s it! our Lambda layer should be visible from our AWS console as shown in the following figure:

Figure 2. Lambda layer "python-numpy" in AWS console | Image by author
Figure 2. Lambda layer "python-numpy" in AWS console | Image by author

Once our Lambda layer is deployed, it is necessary to identify the associated ARN since it will be necessary to reference from the definition of our Lambda function.

Great, we have deployed our Lambda layer. Now let’s see how to deploy a Lambda function that extends dependencies of the Lambda layer.

Let’s go for it!

Deploying AWS Lambda Function

First, we are going to generate an aws-python3 template for our Lambda function which we will name test-function.

(venv) $ serverless create --template aws-python3 
                           --name test-function 
                           --path test-function

The above command will generate a directory called test-function which contains two files, handler.py and serverless.yml. The handler.py file will contain the code that AWS Lambda will run. The serverless.yml file will contain the definition of the deployment.

(venv) $ tree
.
└── test-function
    ├── handler.py
    └── serverless.yml

First, let’s modify the handler.py file. Since we assume that we will extend the numpy library from the Lambda layer created in the previous section, all we have to do here is import numpy and declare some basic functionality. Finally, our handler.py file would look like this:

Now we will update the serverless.yml file. Basically, we will define 3 main things: the Lambda function, the Lambda layer extension and an API Gateway method (which we will use to make requests to our Lambda function). Finally, the serverless.yml file looks like this:

In line 12 we define the name of our function ( hello ), in line 13 we define the method that will be executed from the handler.py file. In line 15 we define the extension to Lambda layer (this is where we use the ARN of our Lambda layer created in the previous section) and finally in line 19 we define a GET method to make requests to our Lambda function.

And that’s it, we are ready to deploy!

(venv) $ sls deploy

Once our Lambda function is deployed, we could already see it in our AWS console. In the following figure we see the characteristics of our AWS Lambda function deployed from the console.

Figure 3. Lambda function "test-function" in AWS console | Image by author
Figure 3. Lambda function "test-function" in AWS console | Image by author

As we can see, our Lambda function has a Lambda layer attached and it has an API Gateway as a trigger.

Finally, to test our Lambda function, we use the serverless CLI as follows:

(venv) $ sls invoke -f hello 
{
    "statusCode": 200,
    "body": ""Random number: 17. Numpy version: 1.21.0""
}

And that’s it! our AWS Lambda function extends the numpy library from a Lambda layer and the entire development and deployment process has been through the Serverless Framework.

References

[1] AWS Lambda Function

[2] AWS Lambda Layer

[3] Serverless Framework


Related Articles