Activation Functions in Neural Networks: What You May Not Know

How to identify the best (and worst) activation functions

Photo by JJ Ying on Unsplash

When I first started learning ML, I was told that activations are used to mimic the “neuron activations” in the brain, hence the name neural networks. It was not until much later that I learned about the finer intricacies of this building block.