Recently, I have been creating data pipeline scripts using Python in Jupyter Notebooks for storing and analyzing stock market data. I have been posting these scripts in my Github repository to share with the public through a series of previous articles on Medium.
These scripts rely on pulling raw data from APIs provided by my data partner, Intrinio, and then pushing the data to my data warehouse on AWS. These resources require login credentials, which I do not want to share with the public for obvious reasons. So how can I store these credentials in a separate file, hidden on my local drive, and automatically import them into a Python script without exposing the name or the location of my credentials file?
Here is the simplest solution that I found. First, create a credentials file in a text or JSON format. I prefer JSON because it is easy to read and parse in Python. A JSON format is really just a Python dictionary file using double quotes, not single quotes, around the key and value names. This is what it would look like:
{
"data_api_username":"johnbsmith",
"data_api_key":"8675309JENNYjenny",
"aws_access_key":"there'sAladyWHO'SsureALLthatGLITTERSisGOLD",
"aws_secret_key":"ANDshe'sBUYINGaSTAIRWAYtoHEAVEN"
}Then give it a name you would remember. Most people use something like "credentials.json". But you could call it anything, even "myfavoritesongs.json", if you wanted to hide it from a potential hacker.
Normally, you would use the Python Open function to grab this file. But then you have to expose the file path in your code. Instead, you could use a GUI package included in Python called Tkinter. Within Tkinter, the FileDialog module allows you to open a window to select a file.
Here is the code you could use if you are running Jupyter from your local machine:
Or if you are running your Jupyter notebook from Google Colab, the process is slightly different, but just as easy. Here is the code you could use for that platform:
This module will conveniently produce a "Choose File" button right inside the Colab notebook cell, like this:
With both of these techniques, you can keep your credentials file, and the specific path to get to it, totally hidden. This would allow you to share your Jupyter notebooks without having to worry about giving away your secret keys.
Happy coding!
Bryant Sheehy has 15+ years of experience in financial data sales and business development, and is now transitioning to data engineering and analytics. You can follow him on Medium or contact him on LinkedIn.
