Identifying Candlestick Patterns using Deep Learning

Training a neural network on candlestick charts and then using it to identify patterns on it

Some of the heatmaps generated using the method described below (Image by author).

Recently I completed a very interesting project wherein I attempted to identify candlestick patterns on charts of Large Cap Indian stocks using Deep learning, and the results were very interesting! This blog is a tutorial so that you can try out this fascinating “Experiment” on your own.

I am calling this an “Experiment” to emphasize that it is one and to make it very clear that you should not take any financial decisions based on the content given here!

Briefly, we will be executing the following steps (using python) :

• Collecting the daily historical stock data for multiple stocks using the python library nsepy.
• Converting the data into Candlestick charts.
• Labelling the Candlestick charts as having an “Up” or “Down” movement based on percentage movement (of close price) in a particular direction.
• Training a Deep Learning Model on the data.
• Getting the areas of interest (based on the values of activations)and visualizing them using heat maps.

You can find the GitHub repo containing the Jupyter Notebooks for this project here. We will be requiring the following libraries as we move ahead :

• nsepy
• pandas
• numpy
• matplotlib
• fastai

I will be explaining the code piece by piece. So let’s get started!

In the above piece of code we have defined functions to obtain the historical stock data and to plot Candlestick charts for it. A brief overview :

• The function “obtain_data” takes the ticker symbol and the start and end dates as input and gives out a Pandas Dataframe containing the stock data.
• The function “plot_candles” (which was originally written by Daniel Treiman to which I have made minor changes) takes in the Pandas Dataframe and outputs a Candlestick chart.

Sorry for the long output but I wanted it to be so, to give you an idea of how the code is working.

In the above section we are completing the following steps :

• Obtaining the historic stock data (for the past two years) using the “obtain_data” function.
• Determine whether a particular chart should be classified into “Up” or “Down” based on movement in the upcoming five days.
• Create a Candlestick chart for a time period of 20 days.
• Save the Candlestick chart to the respective folder (Up/Down).
• Repeat for a number of different stocks.

We are now done with data gathering; let’s start training the model!

In the above piece of code we are setting up the data such that we can use it train a deep learning model. We are completing the following tasks :

• Setting the batch size for training the model.
• Splitting the data into training and validation datasets.
• Applying augmentations on the data (to reduce overfitting).

Now let’s start training the model !

We trained a deep learning model (having an accuracy of nearly 62 % on a validation set) above. A more detailed overview of what we did is as follows:

• Used transfer learning to train a pre-trained neural network on our charts using an appropriate learning rate.
• Increased the size of our images (for the earlier training cycles it was 224 by 224 and then we increased it to 352 by 352)
• Used transfer learning again to train the previous network on the new larger images using an appropriate learning rate.

And we got an accuracy of nearly 62 % on our validation dataset !

Now we will be creating heatmaps highlighting areas which were of interest to the neural net using the activations. Let’s do so !

We completed the following steps in the above section :

• Created a list of all the charts to be analyzed (the ones which were correctly predicted with a high degree of certainty).
• Got a particular image from the above list and transferred it to the GPU.
• Used hooks to hook into the model during the forward pass to obtain the activations.
• Used the activations to create a heatmap using matplotlib.
• Saved the heatmap.

And Voila ! We managed to train a neural network and use it to identify patterns in a candlestick chart.

I would like to emphasize this again that please don’t use any of the content given here to take any financial decisions!

Hope you enjoyed reading this blog,
Thanks for reading it till the end!

P.S.
Please feel free to connect with me on LinkedIn for any questions or suggestions.

P.P.S.
I have generated a lot heatmaps of candlestick charts, feel free to contact me if you would like to have a look at them.