The world’s leading publication for data science, AI, and ML professionals.

An OpenAI Model Learns to Summarize Books

What's the harm in these machine learning models?

Photo by Seongho Jang on Unsplash
Photo by Seongho Jang on Unsplash

Many large tech companies are competing to develop general-purpose Artificial Intelligence (AI) – allowing their model to approach and solve just about any problem we give them, no matter how time-consuming or challenging. This is referred to as the alignment problem.

To test out and scale a potential solution, the OpenAI team recently trained an artificial intelligence model to recursively summarize books. Using natural language processing through GPT-3 can get you the gist of a book of any length.

According to the OpenAI team, the model "achieves a 6/7 rating (similar to the average human-written summary) from humans who have read the book 5% of the time and a 5/7 rating 15% of the time."

Using recursive task decomposition, each long text is broken down into smaller and smaller pieces. These small pieces or chapters are then summarized and then those chapter summaries can themselves be summarized. People can then provide feedback on these reviews to further improve the model.

AI-powered summaries are a small test case for many future applications, where supervised machine learning can scale to many difficult tasks.

But there are clearly some limitations as the models are only able to achieve human-level accuracy 5% of the time. The decomposition method may also miss important connections between sections of a book. While the source code and training set will not be made public (ironic, given the Open AI moniker).


Where’s the Harm

According to VentureBeat, Google, Microsoft, and Facebook are all working on similar tools to deliver text summaries to users. While some individuals use it to collaborate and co-write novels with AI, it may also lead to more news aggregation. Google, Facebook, and a select few other companies are now the gatekeepers and arbiters for the rest of the internet.

Devaluation of News and Knowledge

To earn ad revenue, news companies need to fit their articles to an algorithm, ensuring that people will click. To save on costs, companies are already using automated systems to gather and organize content, or suggest headlines. This is part of journalist Franklin Foer’s argument in his book World Without Mind: The Existential Threat of Big Tech:

"Magazines and newspapers used to think of themselves as something coherent – an issue, an edition, an institution. Not as the publisher of dozens of discrete pieces to be trafficked each day on Facebook, Twitter, and Google."

Even reputable outlets need to generate clickable stories to survive and gain traction on social media. News has been shifting to a digital model, with younger people more likely to get news online, through Google or social media. In a recent survey, 42% of Americans aged 18–29 responded that they most often get their news from social media.

There are legitimate risks that these AI algorithms could become the preferred way to summarize a lot of the day’s news. After all, these models could be trained to include a clickable title and SEO keywords. While it may also free up time for journalists to work on more exciting projects, that require more direct oversight and investigation.

After all, many underlying natural language processing models are biased, and implementing these summarizing algorithms at scale could have unintended harms. There is also some fear that automating the news-gathering process could further harm the press and the value of journalism.

Exclusively Licensing Powerful Tools

There’s also the issue that Microsoft has an exclusive license for GPT-3, monopolizing what could have been an open-source tool. When a large company achieves such a monopoly, other industries become reliant on it.

Take the example of Amazon, which rolled out eBooks for a uniform price of $9.99. However, Hachette Book Group entered a contract dispute because they didn’t want to arbitrarily reduce the cost of all their eBooks. In response, Amazon prevented customers from pre-ordering any Hachette books, tanking a lot of the profits.

Amazon could leverage this incredible power because they’d effectively squashed all the competition, leaving many publishers reliant on them for selling their products. Microsoft will be able to exert similar power through its exclusive license and what it might deem appropriate use for the algorithm.

The Issue with GPT-3

Many researchers have brought up legitimate criticisms against natural language processing algorithms like GTP-3, citing their proclivity towards racism and sexism. Any general AI models, even those used for summarizing books and information, need to be carefully audited before they’re put into use.

"I’m worried about groupthink, insularity, and arrogance in the AI community," ex-Google researcher Dr. Timnit Gebru wrote.

Learning to summarize text from a select set of books is a small, directed goal. But time and time again, unleashing similar language processing models to the internet at large has proven problematic. Not only is GPT-3 prone to racism and sexism, but researchers also found these models become more toxic as they grow.

Books, documents and anything else fed into GPT-3 won’t be neutrally coded. Many people in charge of evaluating the algorithms for accuracy will also add in their own errors and biases. While many tech companies have departments to deal with just this problem, they aren’t perfect.

Dr. Timnit Gebru worked for Google, researching the ethics of AI and its biases. Her work focused on large-language models which included GPT-3 and Google’s own algorithms. However, a group of executives wanted her to retract the paper after it went through internal review processes and was submitted to a conference.

She was unceremoniously fired after Google executives deemed the research unacceptable, citing that it was too bleak. How do we trust that companies will act against their own financial interests, and implement limitations and expensive fixes for their Technology? We can see that anti-vaccine rhetoric and misinformation are still rife on Facebook and Twitter.


Just because we can do something with AI, doesn’t mean we shouldn’t think about the potential consequences beforehand. All this is much easier to do before the models are implemented and used in our day-to-day lives. Providing exclusive access to one company for this technology sets a harmful precedent.

As we’ve seen over and over, large tech companies aren’t the best at ethical oversight, especially if it harms their financial interests. While the technology itself is incredible, I worry that these algorithms may continue on their current problematic path.


Hi! I’m a former grad student and journalist with multiple scientific publications studying neuroscience and the microbiome. If you want to read more stories on Medium, you can subscribe with a membership [here](http://linktr.ee/simonspichak). You can find my work on Medium, Massive Science, Futurism, and Being Patient. If you’re interested in my Writing, you can read more here or sign up on my substack for a bi-weekly roundup of my stories.


Related Articles