PODCAST

Are we *under-hyping* AI?

Sam Bowman on AGI, its potential and its safety risks

Editor’s note: The TDS Podcast is hosted by Jeremie Harris, who is the co-founder of Mercurius, an AI safety startup. Every week, Jeremie chats with researchers and business leaders at the forefront of the field to unpack the most pressing questions around data science, machine learning, and AI.

Google the phrase “AI over-hyped”, and you’ll find literally dozens of articles from the likes of Forbes, Wired, and Scientific American, all arguing that “AI isn’t really as impressive at it seems from the outside,” and “we still have a long way to go before we come up with *true* AI, don’t you know.”

Amusingly, despite the universality of the “AI is over-hyped” narrative, the statement that “We haven’t made as much progress in AI as you might think™️” is often framed as somehow being an edgy, contrarian thing to believe.

All that pressure not to over-hype AI research really gets to people — researchers included. And they adjust their behaviour accordingly: they over-hedge their claims, cite outdated and since-resolved failure modes of AI systems, and generally avoid drawing straight lines between points that clearly show AI progress exploding across the board. All, presumably, to avoid being perceived as AI over-hypers.

Why does this matter? Well for one, under-hyping AI allows us to stay asleep — to delay answering many of the fundamental societal questions that come up when widespread automation of labour is on the table. But perhaps more importantly, it reduces the perceived urgency of addressing critical problems in AI safety and AI alignment.

Yes, we need to be careful that we’re not over-hyping AI. “AI startups” that don’t use AI are a problem. Predictions that artificial general intelligence is almost certainly a year away are a problem. Confidently prophesying major breakthroughs over short timescales absolutely does harm the credibility of the field.

But at the same time, we can’t let ourselves be so cautious that we’re not accurately communicating the true extent of AI’s progress and potential. So what’s the right balance?

That’s where Sam Bowman comes in. Sam is a professor at NYU, where he does research on AI and language modeling. But most important for today’s purposes, he’s the author of a paper titled, “When combating AI hype, proceed with caution,” in which he explores a trend he calls under-claiming — a common practice among researchers that consists of under-stating the extent of current AI capabilities, and over-emphasizing failure modes in ways that can be (unintentionally) deceptive.

Sam joined me to talk about under-claiming and what it means for AI progress on this episode of the Towards Data Science podcast. Here were some of my favourite take-homes from the conversation:

• Sam points out that researchers will often point to outdated papers that explored the failure modes of old AI systems — failure modes whose persistence in modern systems has not been demonstrated. As an example, he points to Jia and Liang (2017), which highlighted surprising limitations of then-current language models on reading comprehension tasks. Jia and Liang is heavily cited to this day by papers that imply that current AI systems have similar points of failure. But 2017 is an eternity ago in AI terms, and critically, it precedes the entire era of transformers and foundation models that began to revolutionize language modeling starting in 2018.
• Another common argument used to downplay AI capabilities is the claim that, “AI systems may be able to make good predictions, but they don’t really understand anything, you see. Not like, really understand.” This argument is generally made without defining what “understanding” is supposed to mean in the first place, nor does it explain why statistical inference when performed by the human brain qualifies as “understanding”, whereas a very similar process, carried out by machines (and often with superhuman performance) does not.
• A third trend Sam lumps into the “under-claiming” category is adversarial data collection. This happens when researchers develop benchmarks designed to probe at the weaknesses of new models. This often involves finding examples of embarrassing model failures — like the misclassification of what to humans seem like easy-to-categorize sentences. Adversarial data collection can make highly capable systems appear “dumb” by emphasizing points of failure at the expense of missing the big picture, qualitative performance explosion that we’ve seen in scaled models.
• Sam also shares ideas for researchers who want to combat the under-claiming trend in a responsible way (and without over-compensating!).

You can follow Sam on Twitter here, or me here.

Chapters:

• 0:00 Intro
• 2:15 Overview of the paper
• 8:50 Disappointing systems
• 13:05 Potential double standard
• 19:00 Moving away from multi-modality
• 23:50 Overall implications
• 28:15 Pressure to publish or perish
• 32:00 Announcement discrepancies
• 36:15 Policy angle
• 41:00 Recommendations
• 47:20 Wrap-up