Courage to Learn ML: Tackling Vanishing and Exploding Gradients (Part 2)

A Comprehensive Survey on Activation Functions, Weights Initialization, Batch Normalization, and Their Applications in PyTorch

Amy Ma
Towards Data Science
37 min readMay 3, 2024

--

Welcome back to a new chapter of “Courage to Learn ML.” For those new to this series, this series aims to make these complex topics accessible and engaging, much like a casual conversation between a mentor and a learner, inspired by the writing style of “The Courage to Be Disliked,” with a specific…

--

--

👩‍💻 Data Scientist & Machine Learning Enthusiast 🚀 | 🐾 Cat Lover & New Mom 🍼 | Video Game Adventurer 🎮 | Here to make ML fun and accessible! 🌟