Opinion, ARTIFICIAL INTELLIGENCE | NEWS
Until now, if we wanted to communicate with a computer, we had to learn its language. We had to adapt to it. From now on, computers will adapt to us.
OpenAI has done it again. In July last year, they released GPT-3, the first of its kind. This AI system showed a mastery of language like no other before. Mimicking Shakespeare’s poetic genius, writing a rap song about Harry Potter in the style of Lil Wayne, or writing productivity articles are just a few of GPT-3’s capabilities.
GPT-3 was, at the time, the largest neural network ever created. It became clear for OpenAI – and the rest of the world, given how promptly others followed their example – that large pre-trained language models were AI’s answer to the secrets of human language.
However, GPT-3 had some abilities not even OpenAI’s researchers had thought of. Sharif Shameem was one of the first to notice GPT-3’s coding skills. He successfully made the system build a generator that wrote code for him. The world was on the verge of one of the most important AI revolutions of our time.
Last month, OpenAI – in partnership with Microsoft and GitHub – presented GitHub Copilot, an AI pair programmer intended to aid developers in their boring day-to-day tasks. Reading documentation? Copilot does it for you. Writing unit tests? Copilot does it for you. The system acts as a super-powerful autocomplete. And it’s fueled by OpenAI’s latest star: Codex.
Codex, GPT-3’s younger brother, is another language model. But, instead of being a jack of all trades like GPT-3, Codex is a master. A master of coding. It’s fluent in dozens of programming languages and can understand natural language and interpret it to create unambiguous commands that a computer can understand.
Yesterday, OpenAI released a new version of Codex through a new API (we can access it by signing on a waitlist). They’ve also created a new challenge in which people will code against – and with Codex (you can register here). On top of it, Greg Brockman and Ilya Sutskever, two high-profile OpenAI researchers, conducted a live demo to show Codex’s capabilities and give a hint of its current potential – and what awaits in the future.
In this article, I’ll review what OpenAI revealed in the demo with impressive examples of Codex’s skillset. I’ll finish with my thoughts on the implications of this technology and how it could reshape the future.
Why Codex isn’t just another GPT-3
GPT-3 was a pioneer. When it came out, most people had never heard about language models, the Transformer architecture, unsupervised learning, or pre-training techniques. It showed the world what language AI systems were capable of. It became an AI celebrity, pretty much like IBM Watson, or AlphaZero before it.
But it wouldn’t be fair to write about Codex in the shadow of its older brother. Codex is, in and of itself, an AI superstar. It isn’t just a quantitative improvement from GPT-3. The way Codex could impact our world is closer to how the iPhone did in 2007 than to how AlphaZero did(n’t) in 2017. Our way of living is radically different from the pre-iPhone era and Codex promises to change the world in an arguably comparable way.
One of the main drawbacks of GPT-3 was its lack of memory. The context window – the amount of previous information it can access when you interact with it – was small and made the system commit errors. It’d diverge on its theses in long essays, enter endless loops, or make incongruent arguments. Codex solves this problem with an increased memory store – from 4KB to 14KB, more than 3x GPT-3’s size. This allows it to grasp a better, wider understanding of the task at hand.
GPT-3, as impressive as it is, lives within a virtual world, forever confined to the boundaries of an unperceivable reality of ones and zeroes. GPT-3 can influence the world only in an indirect way, "through the mind of the reader," as OpenAI’s CTO Greg Brockman says. Codex, in contrast, can directly affect the world by creating commands from English prompts to make a computer do something. Codex can take actions that have (I hope beneficial) consequences in the real world.
"Proficient in more than a dozen programming languages, Codex can now interpret simple commands in natural language and execute them on the user’s behalf – making it possible to build a natural language interface to existing applications."
3 examples of what Codex can do – The highlights
After watching the demo live I can only say Codex seems impressive. Not because it can write code in several languages or because it does it fairly well. But because its inferences of the English prompts could be qualified as deep, nuanced, creative, and precise – although we should never forget it doesn’t understand language in the same sense we do.
Watching Codex interpret natural language commands made me drop my jaw quite a few times. Ambiguous orders, informal language, or even metaphors. Codex overcame those obstacles to achieve its goals with a more than adequate JavaScript. It simply takes an English prompt – similar to a comment in a traditional program – as input and then it writes the code and sends it to the computer which displays the output on the screen.
Here are the curated highlights of three examples (from the demo and the blog post) and my impressions.
1. Hello world
They wanted to showcase the basics, so no other way to start the demo than with "hello world." The inputs are in bold, and the outputs are in regular font.
Say Hello World.
Hello World
Now say it with feeling!
Hello World!!!
Even louder please.
HELLO WORLD!!!
The first thing we notice is that the model can self-reference. It can store in memory (within the same session) previous commands and use them adequately. It understands that "it" refers to "Hello World," which requires a certain understanding of syntax and context. The last prompt is even more complex as it has some ellipsis. A standalone "even louder please" command means nothing. In practice, it’s as if we were having a conversation with Codex that it can remember and use in the future.
Then, they wanted to "broadcast that message to the world" so they asked Codex to create a web page and then a server to host that page. They asked the viewers to subscribe at openai.com/c to take part in the demo so Codex could send us a message with "Hello World." And, to make the mail amusing, they added the price of Bitcoin too. To send this email they used the Mailchimp API – which Codex learned to use just by reading the two middle sentences.
Look up the current Bitcoin price, and save it to a variable.
Here's a new method you can use.
codex_chimp.send(subject, body)-> sends Mailchimp email blast
Now send everyone an email, telling them (a) hello world and (b) the current Bitcoin.
A few seconds later I got this email. Amazing.

2. Creating a game
For this example, they decided to write a larger, more complex program. Instead of testing Codex with one-line commands, they created a game: The main character is a person that has to dodge a falling boulder by moving left and right on the screen. First, they imported the image of a person, adapted the size to fit in the screen, and located it at the bottom of the screen.
They wanted the person to move right and left with the keys so they prompted Codex: "Now make it controllable with the left and right arrow keys." How can Codex interpret this command correctly? What does "make it controllable" mean? It’s a very ambiguous way of expressing "move X pixels to the left/right when the left/right key is pressed."

Still, Codex managed to interpret the instruction and generated code to move the image 10 pixels left and right when the keys were pressed. Tackling nuanced orders like this one is one of the most remarkable capabilities of Codex. How to write JavaScript code is a skill not many people have, but anyone knows what it means to move the person left and right with the keys. Codex closes the gap between our desires – that we often are unable to express precisely even in natural language – and our expectations.
Also, the system has to hold a notion of what’s going on to follow the instructions. We can use the screen as an aid to grasping what’s going on, but the model only has access to the previous code.
Then, they included an image of a boulder, that the person has to dodge. However, as it was too big, they ask Codex to "make it small." The system has to not only understand the meaning of "small," but to have a decent sense of the relative nature of this adjective. An ant is small to us, but not to an atom. A house is small to the Earth, but not to us. Yet, Codex converted the boulder to a reasonable size for the task at hand.


Finally, after adjusting the size of the boulder, they wanted it to fall from the sky and appear at a random location after hitting the bottom of the screen. They asked Codex to "set its position to the top of the screen, at a random horizontal location. Now have it fall from the sky, and wrap around." Which sent the image of the boulder downwards until it reached the bottom and then appeared again at the top.
Notice that the sentence "fall from the sky" is a metaphor. There’s no sky and there’s no actual falling. Codex had to understand that "fall" meant moving the image in a specific direction (down) continually. And it had to understand that "sky" was referring to the top of the screen.
3. Talking to your computer
To further emphasize Codex’s potential, they decided to use a real-world example. At the end of the day, most people don’t know how to program or don’t even care that an AI system could help them do it. But, as I said at the beginning, the promises of Codex aren’t constrained to the AI/tech world. Codex – and its future successors – will change how we interact with computers.
For this example, they used a JavaScript Microsoft Word API to allow Codex to make changes into Word documents. To top it off, they included a speech recognition system.
Picture this: You talk to your computer, the speech system recognizes your voice and converts it into text. The text is sent to Codex – which already knows how to use the Microsoft API – it transforms the command into JavaScript code and does what you intended to the document.


The easiness with which Codex learns how to leverage APIs is another great strength. The programming world is getting "API-ified," so these systems will become more useful as time progresses.
"It will take a long time for the technology to get good enough, but eventually we may forget that [we] used computers any other way. As more things become API-ified, AI systems that write code can easily make a lot of things happen in the world." – Sam Altman
Just imagine what Codex could do in a programmable world. Since the beginning of computer science, programming languages have become increasingly high-level. Codex is the missing link between human and computer language. We will be able to express a thought and the computer will understand what we mean without uncertainty. Anyone will be able to communicate with computers in a way that feels natural to us.
Codex is a primitive version of J.A.R.V.I.S.
"What you see here is a taste of the future. As the neural network gets really good at turning instructions to correct API codes, it’ll become possible to do more and more sophisticated things with your software just by telling it what to do." — Ilya Sutskever
Thoughts on Codex
Improving human-computer communication
From punch cards to Python, the history of human-computer communication has never favored us. We’ve always had to sacrifice our commodities and adapt to machines. For this reason, people closely involved in computer communication jobs have been considered high-skill, high-value workers. Even today, when programming is considered a must-know skill, most people don’t know how to create programs to make computers do things.
However, this is likely to change in the years to come thanks to systems like Codex. They may not replace programmers, but will for sure be a catalyst to empower laypeople to enter the world of programming and develop healthy relationships with our friends, the computers.
High interpretability
One fair criticism of neural networks is their lack of accountability and interpretability. These systems do amazing things but also fail catastrophically. Failing itself isn’t the problem. The main issue is that neural networks are black boxes. Whatever goes inside, stays inside. there’s no way for us to take a peek and get a better sense of what’s going on. Debugging is extremely hard.
Here Codex contrasts with GPT-3. Both systems are trained as black boxes, but at test time, Codex lets us see the code it writes from our prompts. We don’t just get the output from the computer, but also what Codex has interpreted. It allows us to check the code, delete it, or make the system rewrite it if necessary. This point is key for developers. No professional programmer would use a code-generating system whose code remains hidden.
Removing the boring side of programming
Greg Brockman talked about the two sides of programming. First, you need to understand the problem you want to solve and how to divide it into smaller pieces. Then, you need to take those pieces and map them to existing code. As Brockman wrote in the blog post, "The latter activity is probably the least fun part of programming (and the highest barrier to entry), and it’s where OpenAI Codex excels most."
Codex isn’t meant to remove programmers from their jobs, but to reduce the boring tasks they have to do and let them focus on those aspects of their job that require a higher level of cognitive effort. Adapting a problem to syntax that a computer can understand is boring. Facing a seemingly unsolvable problem, understanding it little by little, and making small advances towards a solution is what makes programming fun.
"[Codex ] is really fun to use, and brings back the early joy of programming for me." – Sam Altman
A radical change in our day-to-day lives
Sam Altman has said this technology is in diapers. But it’s the first hint of what will come and it seems plausible that Codex and its successors will transversely change how we live life. One thing is to revolutionize AI. Another, very different thing is to revolutionize the world to the point that the little things we individually do on any given day, change.
Codex isn’t just something that’s happening in AI that most of the world won’t even notice. It, if successful, will eventually modify how we all interact with computers. The same way the iPhone revolutionized how we interact with our phones.
Subscribe to my free weekly newsletter Minds of Tomorrow for more content, news, opinions, and insights on Artificial Intelligence!
You can also support my work directly and get unlimited access by becoming a Medium member here! 🙂