Too powerful NLP model (GPT-2)

What is Generative Pre-Training

Photo by Edward Ma on Unsplash

OpenAI released generative pre-training model (GPT) which achieved the state-of-the-art result in many NLP task in 2018. GPT is leveraged transformer to perform both unsupervised learning and supervised learning to learn text representation for NLP downstream tasks.