It’s been more than a year since I have been getting my hands dirty doing deep learning. Libraries like PyTorch and Keras make it easier to implement a deep model and provide standard implementations of a lot of deep architectures like CNN, LSTM, soft max, embeddings, etc. If we can visualize or propose a deep learning-based solution to a problem, then given the dataset, it becomes quite easy to do a proof of concept whether the given claim makes sense. Doing deep learning can be fun and tiresome at the same time because of lack of good quality GPU leading to slow training, is unpredictable sometimes and hard to debug, time-consuming, and dependent on the scale of the dataset. Based on my experience, I have found the following libraries to be quite handy to perform the experiments efficiently.

Hydra – It provides a configuration file for the entire experiment. We can have different parameters to be set. It can be very helpful when we want to share our code with someone else or run the experiments on a different machine. It gives us the flexibility to set the desired configurations like learning rate, model hidden layer sizes, epochs, data set name, etc. without exposing someone to make changes to the actual code.
Pipreqs – This proves quite useful when we want to port our code to a different machine and install all the dependencies. It helps us in creating a list of python dependencies along with the versions that our current working code is using and save them in a file that can be easily installed anywhere else.
>> pipreqs
INFO: Successfully saved requirements file in D:blogsrequirements.txt
To install these dependencies successfully in a different machine, copy the requirements.txt file and run
>>pip install -r requirements.txt
Getopt— It can be used to pass the parameters using command line arguments. It provides us the flexibility to set the mandatory parameters, default values of parameters as well. argparse library is another popular alternative.
Run the following command and pass the arguments as command line arguments.
>>python getopt_demo.py -r demo_experiment -d ./ -t CNN -l 0.4
Tensorboard – It is used for real-time plots of measuring the progress of our experiments by logging and plotting metrics like training loss, training accuracy. The Tensor board logs are part of the Summarywriter object which keeps track of certain variables like training loss, validation loss, training accuracy, validation accuracy, etc. with epochs and plots it beautifully for visualization. This comes in handy when we want to visualize our results in real-time or at a later point in time.
To see the real time progress run in cmd
>> tensorboard --logdir=runs
Serving TensorBoard on localhost; to expose to the network, use a proxy or pass --bind_all
TensorBoard 2.4.1 at http://localhost:6006/ (Press CTRL+C to quit)
Tqdm – As a beginner, I used to manually write down the training time for one epoch, and it was really cumbersome. Tqdm when used with a loop(here we use with a loop over a torch.utils.data.DataLoader object) gives a good picture of time per gradient step or epoch which can help us to set our logging frequency of different results or saving the model or get an idea to set the validation intervals.
Loguru -It provides the functionality of a logger to log configuration, experiment name, and other training-related data. This proves to be quite helpful when we do multiple experiments and want to distinguish the results of different experiments. Thus, if we log the actual configuration as well as the results, then it’s easier to map the appropriate setting to the outputs that we got.
H5py —It can be used to store all the intermediate loss values in a dictionary mapped to appropriate key which can be loaded to be reused as a python code, as an appropriate object in future for reuse.
Pickle – It can be used to save and load the python classes or PyTorch models for reuse. e.g. In an experiment involving text preprocessing, we need to store the vocabulary of the training dataset and the corresponding embeddings as well as the word2index and index2word dictionary. We can pickle the objects and load it in future to save the time for preprocessing.
These libraries save a lot of time and help us in keeping the implementations clean and make results of experiments more presentable. Let’s make Deep Learning fun and easy.