Understanding the Open Pre-Trained Transformers (OPT) Library

Towards transparency and inclusion in language modeling…

Depiction of a decoder-only language modeling architecture (created by author)

Recently, Meta AI published “OPT: Open Pre-Trained Transformer Language Models” [1] and an associated code repository with the intent of open-sourcing high-performing large language models (LLMs) to the public. In particular, OPT provides an entire suite of LLMs…