The world’s leading publication for data science, AI, and ML professionals.
Stochastic Gradient Descent approximates Bayesian sampling
![Figure 5: Our main result. (a) compares _Pᵦ(f|S) with Pₒₚₜ(f|S). (b) shows how Pᵦ(f|S) varies over many orders of magnitude - there is a clear inductive bias towards simple functions in Pᵦ(f|S). (c) shows that simpler functions generalise better. (d) shows Pᵦ(f|S) from (a). (e) and (f) use different hyperparameters to (a) to show that this effect is not hyperparameter-dependent. From [1]._](https://towardsdatascience.com/wp-content/uploads/2020/12/0A5xjaZcfJFseBK1z.png)
