Learning Transformers Code First: Part 1 — The Setup

A 4 Part Exploration of Transformers Using nanoGPT as a Starting Point

Photo by Josh Riemer on Unsplash

I don’t know about you, but sometime looking at code is easier than reading papers. When I was working on AdventureGPT, I started by reading the source code to BabyAGI, an implementation of the ReAct paper in around 600 lines of python.