Why Data should be Normalized before Training a Neural Network

And Why Tanh Generally Performs Better Than Sigmoid

Photo by Clint Adair on Unsplash

Among the best practices for training a Neural Network is to normalize your data to obtain a mean close to 0. Normalizing the data generally speeds up learning and leads to faster convergence. Also, the (logistic) sigmoid function is hardly ever used anymore as an…