How to Create an LDAP Sandbox for Pipeline Testing

Use Docker containers, either locally or in the cloud

Published in

Towards Data Science

7 min readNov 17, 2022

Many companies use LDAP, or Lightweight Directory Access Protocol, to store information about employees and to allow employees to access organizational resources depending on what their individual roles are. Microsoft’s Active Directory (AD) is the best-known database that uses this protocol.

For a recent data engineering project, a client needed a pipeline created that would batch-add new employees to its on-premise AD and assign these new employees roles, usernames, and other attributes based on various unique factors. While working on the project, I needed some way to test my work in progress. But the client couldn’t allow access to the AD server because of security reasons, and even if they could, the server was in active use and not a good platform for testing.

To create a testing sandbox for this pipeline, I used:

Docker and a cloud service, in my case Azure. Instead of a cloud service, I could also use a tunneling service to my local machine.
An interface to interact with the server
A pipeline integration platform, in my case SnapLogic

Here are my steps to create the sandbox, with a couple of different options shown.

Create the LDAP Server from a Docker Image

OpenLDAP is an open-source implementation of LDAP, and I’m running it using the osixia/openldap docker image. (Check out the links for documentation.)

Option 1: Run it as an Azure container instance

I first need an Azure account and to have the Azure CLI installed on my computer. Then, I run the following in the terminal:

When the container is created, it appears in the Azure portal in the resource group’s container instances. Notice in the right column how Azure has assigned the container a public IP address.

Option 2: Run the container on your local machine

You might not have a subscription to Azure or another cloud service, or you just may prefer to run the container locally. With docker installed on my Mac, I create a docker container with an equivalent command to the one for Azure above.

LDAP servers use port 389 so that needs to be included in the -p flags. Port 636 is sometimes used as well.

Connect an interface to the LDAP server

Option 1: Spin up an interface in a docker container

According to its main page, “phpLDAPadmin is a web-based LDAP client. It provides easy, anywhere-accessible, multi-language administration for your LDAP server.” And like with the OpenLDAP server, this client exists as a docker image; it’s found at osixia/phpldapadmin.

With this image, I’m going to create the container locally, but I’ll connect it to the Azure instance of the server. Note how in an environment variable, I include the IP address created by Azure for the LDAP server container.

The client container uses port 443, and the image documentation says to map it to port 6443. When it’s up and running, I can go to localhost:6443 to access the client. I’m presented with a login page.

I use the environment variables I defined when creating the server on Azure for the login credentials. Note, however, the login username isn’t simply admin. I need the admin’s distinguished name, which in this case is cn=admin,dc=example,dc=com, and in an environment variable, the password I defined as pass_word.

Once I login, I can create sample user accounts, organizational units, and other objects.

Option 2: Create a tunnel to your localhost

If I chose to do Option 2 above where I created the container on my local machine, I could connect the local server to the local client interface container, but remember that eventually I want to connect the LDAP server to a data integration platform, and that platform is in the cloud. To get that connection from the cloud to a server on my local machine to work, I would have to figure out how to bypass the firewall, deal with possibly changing IP addresses, and other headaches.

It easier to create a tunnel, and a service called ngrok allows me to do just that. When I sign up, I get one free active tunnel, and once the ngrok agent is installed on my computer, I can run:

ngrok tcp 389

That tells ngrok to create a TCP connection to port 389, which is mapped on the localhost to the server container. (You need to use the TCP protocol instead of HTTP, because LDAP uses TCP.) Ngrok will generate a page like this one in the terminal.

Notice the Forwarding line, where a URL is forwarded to my localhost:389. That URL will be used in the next step.

Another line to notice is that the web interface is at http://127.0.0.1:4040. Ngrok allows you to monitor the tunnel at that address, but for it to work, you need to also publish port 4040 when creating the container.

Once again, I need an interface to interact with the LDAP server. I could use phpLDAPadmin again, but this time to show another possibility, I’ll use the Apache Directory Studio LDAP browser, which can be dowloaded for free for Windows, Mac, and Linux.

When I have it installed, I can go to the connections pane and create a new connection. In the Network Parameter panel, I enter the URL created by ngrok and its associated ngrok port number (not port 389).

And in the Authentication panel, I’ll enter the same distinguished name and password as in the first option above. When the connection is authenticated, I can start adding entries to the server.

Connect the pipeline to the LDAP server

As mentioned before, I’m using SnapLogic for my pipeline. In order to interact with the server, I have to add my account with the correct credentials in the platform.

If I’m doing Option 1 with the server running on a container on Azure, I need to add the URL created by Azure along with the standard LDAP port 389 and the admin’s distinguished name and password.

If I’m doing Option 2 with the server running locally on my machine, I need to add the ngrok tunnel’s URL, its port, and, again, the admin’s distinguished name and password.

And that should do it. I provided a couple of different options. Whichever one I choose, my sandbox is ready to go.

Bonus: Add a Custom Schema

OpenLDAP comes with standard object classes, but if you need object classes with custom attributes, you’ll have to add your own schema.

In this case, I’ll be showing how to do it with the local server.

To begin, I create a text file with the extension of .schema. Here, I’m saving it as cs.schema. Here are the contents of this file:

In this file, I’m creating only two new attributes, ‘sAMAccountType’ and ‘myadditionnalAttr’ in a new object class called ‘personnel’ which is based on inetOrgPerson. The first attribute is an existing one for LDAP and the second one is a made-up one for this schema. For more info on LDAP schema structure, attributes, and codes, check out other documentation such as the LDAP wiki.

Here’s the docker command I use to to create a new container, which is similar to the one above with a few modifications:

With the --volume flag, I’m mounting the schema file into a particular directory in the container. Also, for this to work right, it turns out I need to use the image’s --copy-service command.

When the container is up and running, again I can create a tunnel to it using ngrok and continue following the Option 2 steps above.