Thoughts and Theory
The Dying ReLU Problem, Clearly Explained
Keep your neural network alive by understanding the downsides of ReLU
Published in
6 min readMar 30, 2021
Contents
(1) What is ReLU and what are its advantages?
(2) What’s the Dying ReLU problem?
(3) What causes the Dying ReLU problem?
(4) How to solve the Dying ReLU problem?