The world’s leading publication for data science, AI, and ML professionals.

Getting Familiar with Keras

Two Neural Network Regression Problems

Photo by Ketut Subiyanto on Pexels
Photo by Ketut Subiyanto on Pexels

Neural networks are computing systems loosely inspired by brain connectivity. In short, neural networks make a series of transformations to input, the results of which are used as features during learning. Keras is an open-source library in python that enables easy experimentation with neural networks. Keras provides many of the building blocks of neural networks including objective functions, optimizers, activation functions, various types of layers, and a host of additional tools.

In this post, we will build three regression neural network models using the Keras library.

Let’s get started!

Predicting the Value of a Car

In the first example, we will be predicting the potential value of a car sale based on customer attributes. These attributes include information like age, income, and gender. The data can be found here.

To start, let’s import pandas and read the data into a data frame:

import pandas as pd 
df = pd.read_csv("cars.csv")

Let’s print the first five rows:

print(df.head())

Next, we will define our input and output variables. We will use ‘age’, ‘gender’, ‘miles’, ‘debt’, and ‘income’ to predict ‘sales:

import numpy as np
X = np.array(df[['age', 'gender', 'miles', 'debt', 'income']])
y = np.array(df['sales'])

We now need to split our data for training and testing. We will import the ‘train_test_split’ method from ‘sklearn’:

from sklearn.model_selection import train_test_split

We then define our test set as a random sample (20% of our data):

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state = 42)

Next, we need to reshape our array of labels:

y_train = np.reshape(y_train, (-1,1))

Now we can define our model. First, we will need to import a few packages:

from tensorflow.python.keras.models import Sequential
from tensorflow.python.keras.layers import Dense

Next, we define a sequential model object:

model = Sequential()

Let’s build a simple neural network with an input layer, a hidden layer, and an output layer. We will also be using the ‘ReLu’ activation function in the input and hidden layers. For the output layer, we will use a linear activation function. The input and hidden layers will have 32 neurons:

model.add(Dense(32, input_dim=5, kernel_initializer='normal', activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(1, activation='linear'))

Next, we compile the model. We will use the ‘adam’ optimizer and mean squared error loss function:

model.compile(loss='mse', optimizer='adam', metrics=['mse','mae'])

Finally, we fit our model. Let’s fit with 100 epochs and a batch size of 10:

model.fit(X_train, y_train, epochs=100, batch_size=10)

We see that both mean squared error and mean absolute error are both pretty low compared to the sale magnitudes which is good.

Now, let’s make predictions on the test set:

y_pred = model.predict(X_test)

Next, we can visualize our results. Let’s import matplotlib and seaborn and display a scatter plot of true values vs. predictions:

import matplotlib.pyplot as plt 
plt.scatter(y_test, y_pred)
plt.xlabel('True Values')
plt.ylabel('Predictions')

This is a decent performance for a first pass. We can do even better with further hyper-parameter tuning. To further improve performance, I encourage you to play around with the number of layers, neurons, epochs and the batch size. You can also try some other optimizers, instead of ‘adam’, like ‘rmsprop’, or ‘sgd’. You can also try transforming the data using normalization, standardization or min/max scaling.

Predicting Fuel Efficiency

Now, let’s move on to another problem. In this example, we will be predicting fuel efficiency measured in miles per gallon. We will be using information like horsepower, weight, and the number of cylinders as inputs into our predictive model. The data can be found here.

Let’s import the data and print the first five rows:

df = pd.read_csv("auto-mpg.csv")
print(df.head())

Next we will define our input and output. We will use ‘Cylinders’, ‘Displacement’, ‘Horsepower’, ‘Weight’, ‘Acceleration’, ‘Model Year’, and ‘Origin’ to predict miles per gallon (MPG):

X = np.array(df[['Cylinders','Displacement','Horsepower','Weight',
                'Acceleration', 'Model Year', 'Origin']])
y = np.array(df['MPG'])

We now need to split our data for training and testing. We will define our test set as a random sample of our data:

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state = 42)

Next, we need to reshape our array of labels:

y_train=np.reshape(y_train, (-1,1))

Now let’s define our model. Let’s start with a neural network with input and hidden layers with 64 neurons. The input dimension is 7, the optimizer is ‘adam’ and the loss function is ‘mse’. We will also use 1000 epochs and a batch size of 10 :

model = Sequential()
model.add(Dense(64, input_dim=7, kernel_initializer='normal', activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(1, activation='linear'))
model.compile(loss='mse', optimizer='adam', metrics=['mse','mae'], validation_split = 0.2)
model.fit(X_train, y_train, epochs=1000, batch_size=10)

Now, let’s visualize our results:

y_pred = model.predict(X_test)
plt.scatter(y_test, y_pred)
plt.xlabel('True Values')
plt.ylabel('Predictions')

We can see performance is pretty good. I encourage you to try to improve performance or to try to build neural network models for other prediction problems. For example, you may be interested in using neural networks to predict wildfire size using US wildfire data.

To summarize, in this post we walked through the model building process for two regression problems: predicting the value of a car and predicting fuel efficiency. We discussed how to initialize sequential model objects, add layers, add neurons, specify optimizers, specify epochs and specify batch sizes. We also went over model validation using true vs. predicted scatterplots. I hope you found this post useful. The code from this post is available on GitHub. Thank you for reading and happy Machine Learning!


Related Articles