Deep Learning beyond 2019

Progress in (slow) conscious task solving?

This figure is synthesized from recent talks by Yoshua Bengio (NeurIPS 2019 talk), Yann LeCun and Leon Bottou. Acronym IID in figure expands to Independent and Identically Distributed random variables; OOD expands to Out Of Distribution

TL;DR

1. Self-supervised learning — learning by predicting input
2. Leverage power of compositionality in distributed representations
3. Drop IID(Independent and Identically Distributed random variables) assumption
4. Approaches for self-supervised representation learning