What does an AI system think about Austria?
Some experiments with the GTP2 Language Model
Natural Language Processing (NLP), the ability of a computer program to understand human language in this way, is an application area of artificial intelligence (AI). Language models that are created (“trained”) with large amounts of text are an important basis for NLP. The texts usually originate from articles published on the Internet and therefore reflect the opinions of the authors.
Do modern language models, which use a huge amount of such “training texts”, represent the general opinion?
In this article, I would like to investigate this by looking at a language model regarding opinions about Austria.
The model “GPT2” from “Open AI” is used, a powerful system for English, which was created with 40GB of text data. I use the largest available variant where 1.5 billion parameters were optimized.
The implementation of Max Woolf (“gpt2-simple“) serves as a basis for the experiments on “Google Colaboratory”.
If you want to learn more about GPT2 you will find an introduction here:
We start with the installation and import of the required packages in Google Colab.
Then we download the model.
We can now use the language model to continue a given text. By choosing a suitable starting text, we try to find out what the language model “thinks” about Austria.
By setting the parameters to determine how long the text should be (length), how many different continuations we want (samples), and how “creative” the model should be (temperature) we let the model generate new texts.
Let us look at some examples of initial texts and continuations:
„The most popular person in Austria is … ”
- The most popular person in Austria is probably the head of the Social Democratic Party, Gabriel. His party won the most votes in the parliamentary elections in October 2016
- The most popular person in Austria is still the queen, but there are rising numbers of people who prefer …
- The most popular person in Austria is almost always the owner of the most luxurious car, the same car which is the envy of the entire world.
The system does not seem to be very sure about the most popular people in Austria. Here, our small country is confused with its big neighbors Germany (“SPD boss Gabriel”) and England (“Queen”) in Europe. For an “American language model” this was to be expected.
The third answer is not only humorous but perhaps also apt, even if not only specifically in Austria.
“The most beautiful place in Austria is … ”
- The most beautiful place in Austria is in the valley near Vienna. At the top of the valley is the village of Strausbach.
- The most beautiful place in Austria is near Vienna and this beautiful landmark is about to get more beautiful and gaudy as it is set to be sold to Russian developer STX.
- The most beautiful place in Austria is the Biennale Artistic Villa near Vienna, it’s decorated in the old country style.
Here the focus is strongly on Vienna and its surroundings.
“In the history of Austria, there have often been … ”
- In the history of Austria, there have often been such examples. The people of a country, living in an age when such a problem was virtually unknown, protested against the national socialism
- In the history of Austria, there have often been problems in connection with the Chinese.
- In the history of Austria, there have often been very few neighbours who lived and worked side by side in peace and good neighbourliness.
Some historical knowledge seems to be available. But the fact that there were often problems in relations with China seems a bit strange.
“The most popular sport in Austria is … ”
- The most popular sport in Austria is still football, and in Vienna, it is the most popular sport too.
- The most popular sport in Austria is running. A number of outdoor leagues are held each year and there are several high school-based running clubs.
- The most popular sport in Austria is football, but other forms of sports have also become popular.
Football as the most popular sport is certainly true, but it could also be a fluke, as this is true for many countries. The importance of running in Austria is misjudged. The fact that there is nothing about winter sports, which are so important for us, is somewhat surprising.
“The climate in Austria is … ”
- The climate in Austria is warm and moderate. The average high temperature in July is 11°C and the lowest is -2°C, with the highest temperatures occurring in December.
- The climate in Austria is unlike anywhere else in Europe. Every summer, snow covers some of the country’s highest mountains, and the country experiences the warmest summers on Earth.
- The climate in Austria is certainly very warm and this is a good reason to have a love affair with St. Bernardin.
The weather is consistently considered warm, which is not inaccurate in the context of climate change. However, the temperature figures do not match.
All in all, it is noticeable that the language model is capable of writing syntactically correct sentences that do not appear to be completely random nonsense. But the factual knowledge of the model, which is the result of a large number of unstructured texts, is rather shaky.
If you would like to experiment with your own startup texts, you can find the entire Google Colab document at:
