The world’s leading publication for data science, AI, and ML professionals.

What would happen if you gave genetic algorithms the ability to trade?

A different approach to stock trading with machine learning

Future Generative Chatbot
Photo by Arseny Togulev on Unsplash

Most applications of machine learning, or at least most of the content on medium, consists of training a machine-learning algorithm to predict the future price or future direction of a stock. This has proved to be ineffective, as stock data is not suitable for time series prediction methods. This is because there is no correlation between data points. This would make it so that any weights learned by the neural network would just be statistical noise within the data.

This made me think that maybe the problem could be with the scope of the solution: Maybe direct prediction based on historical data would not be successful, as it was not directly linked. A direct connection between predictions and stock trading could come up with different results. I started to theorize on ways to apply this concept.

The idea that I came up with would be a setup similar to that of reinforcement learning: There would be an agent, powered by a neural network, that would interact with an environment. The environment would act as a brokerage for past stock data.

In order to combat most of the problems that came with time-series regression, I used a genetic algorithm instead. this would mean that the neural network would never actually be trained on the data, thus ruling out possibilities of "lagging behind" the actual data, or overfitting on trading data. This is at the expense of efficiency, as genetic algorithms take a long time to converge if they even manage to do so.

The program that I created consists of three parts:

  1. Building the Agent
  2. Creating the Environment
  3. Constructing the Genetic Algorithm

Before the actual program, here are all the libraries that I used for this project:

Building the Agent:

The agent’s functionalities are very clear. It should be able to create impactful interactions with the environment and must store its fitness (i.e. profit made).

As you can see, it is very simple. It other than the init function, it only has two other functions. This is as most of the functionalities needed for the agent to interact with the environment is contained within the keras model.

The clone_model function 3is necessary so that every single agent generated has different weights, while keeping the same model architecture.

This is the model architecture that I used for this project. It is exceedingly simple, as this project is just a proof of concept. Additionally, making it more complicated makes the computation time exponentially larger. You can control the computation time by looking at the number of parameters when calling model.summary.

A better model for this project could consist of 1-dimensional convolutional layers, as they would be more effective in feature selection from the data.

Creating the Environment:

The environment comes hand in hand with the agent. It should be able deal with the agent’s requests and should be able to efficiently send data back and forth. It should also be able to calculate the fitness of the agent.

However, the environment is more complex than the agent, because the environment must stay clear from any functions that could make the agent prone to overfitting, or just lagging behind the actual stock data.

This is the init function of the environment. It contains all the variables that all other functions need to function. The start date and end date variables are used to control the time where the data can be drawn from. This can be used to compare results of the model over different time periods, to better analyze results.

This setup function can setup and reset the environment. It initializes all variables to their original values, and accesses the data. With the get_data function, the agent can access the data and use that data to make predictions.

Notice that the adjusted close price and volume columns are dropped. Adjacent close price is not very useful for the neural network, and volume values are too large. This is a problem as standard scaling should not be used on data that is not split, as it would give insight into future data points.

These two functions are they key functions that the agent uses to interact with the environment. The close_position function is actually a sub-function of the open-position function. This is as the model’s output is directly fed into the open-position function. Only if the output maps to [0,0,1], then is the close_position function called.

The open-position function returns the price of one share of the asset, while he close-position function returns the profit (or loss) made after closing the position. These values can then be used to calculate the total profit made by the agent.

Constructing Genetic Algorithm:

The genetic algorithm was used as another means to prevent the issues that arise from using classification and regression. This is as there is no such thing as overfitting, as there are no labels to the data. You could try to implement other non-supervised algorithms, that could improve the speed of the program.

Firstly, we take our agent code and add it to the genetic algorithm.

This script creates a defined number of agents, with a defined model architecture.

This is the main section of the genetic algorithm that was changed to fit this problem. This script basically sets up the environment, and allows the agent to interact with the model at every timestep. This process can be ran multiple times, based on the variable len_episodes. This is so that we have a more comprehensive look at the fitness of each agent.

The value that will be set as the fitness will be the profit of the model, over the set number of timesteps.

This function selects the top 20% of agents, based on their fitness. All other agents are discarded. Try changing the percentage of agents that are selected: the tradeoff is between computational speed and convergence. The lower the cutoff point, the higher the computational speed. However, a higher cutoff point could increase the chance of convergence.

This selection function has been modified to work with keras models. The original crossover function only worked with my custom-made neural networks. This function is quite complex. To understand why this is so, you have to understand how the crossover function works:

Two random "parent" agents from the top 20% of agents are chosen. The weights of both of these agents are flattened. A random splitting point is found. The first parent’s weights are extracted until that point, and the second’s parents weights after that point are concatenated onto the first parent’s weights.

The problem is that the weights must be flattened. This becomes difficult when the weights within the keras model created nested lists with different shapes. Therefore, I combined list comprehension, numpy’s flatten function, as well as a separate unflatten function that I wrote.

This function is the mutation function. For every agent, there is a small 10% percent chance that a random weight will be changed to a random value. This allows for a genetic algorithm to slowly but surely get out of local minima.

As all of this is contained an execution function, this part of the program runs all the functions defined earlier. It returns the first agent of the list. This agent would, in theory, be the best agent created by the genetic algorithm. The loss from every generation has also been returned so that the loss can be plotted, to see if the genetic algorithm is making progress.

This function runs the genetic algorithm. I ended up changing the model to a convolutional network, but it really doesn’t make a massive difference.

Results:

The computational cost for this program was so high that I needed to use a Google Colab GPU to run this script. I found that the results were quite inconsistent, mainly because the model was too simple. I tried to increase the model’s complexity but the computational cost was just too high.

I think that the best way to combat this would be to improve each of the individual parts, to make it faster to run.

Thank you for reading this article!

Note from Towards Data Science‘s editors: While we allow independent authors to publish articles in accordance with our rules and guidelines, we do not endorse each author’s contribution. You should not rely on an author’s works without seeking professional advice. See our Reader Terms for details.

My links:

If you want to see more of my content, click this link.


Related Articles