XLNet: Autoregressive Pre-Training for Language Understanding

Understanding Transformer-Based Self-Supervised Architectures

Photo by Tim Mossholder on Unsplash

State of the art Language Models like BERT, OpenAI GPT have been stellar in Natural Language Processing in recent times. These models are based on the Transformer architecture, which has driven RNN-based and Convolution-based models out of the business.