Devising tests to measure GPT-3's knowledge of the basic sciences

Could students learn from OpenAI’s newest language model and use it as a 24/7 consultant? Could students use it to cheat in exams? Can GPT-3 assist basic research? Does it tell when it doesn’t know?

Introduction

Generative Pre-trained Transformers (GPTs) are deep-learned autoregressive language models trained on a large corpus of text that, given an input…