|LLM|CREATIVITY|AI|HALLUCINATION|

Creativity requires the courage to let go of certainties. – Erich Fromm
Creativity involves breaking out of established patterns in order to look at things in a different way. – Edward de Bono
Creativity is considered along with the ability to reason a uniquely human ability. The arrival of Large Language Models ([LLMs](https://en.wikipedia.org/wiki/Large_language_model)) and their ability to mimic human skills has begun to undermine this view. There has been much discussion about whether LLMs are capable of reasoning, but less about the creativity of these models.
Quantifying reasoning is easier (evaluation is conducted on problem-solving benchmark datasets), but quantifying creativity is more complex. Nevertheless, creativity is one of the activities that makes us human: writing books and screenplays, generating poetry and works of art, making groundbreaking discoveries, or devising theories all require creativity.
The Savant Syndrome: Is Pattern Recognition Equivalent to Intelligence?
Although model creativity is a less explored topic, it is no less important. Therefore, in this article, we will focus on three main questions:
- How creative are models?
- On what does model creativity depend?
- Can hallucinations help increase model creativity?
Artificial Intelligence is transforming our world, shaping how we live and work. Understanding how it works and its implications has never been more crucial. If you’re looking for simple, clear explanations of complex AI topics, you’re in the right place. Hit Follow or subscribe for free to stay updated with my latest stories and insights.
Are LLMs creative?
Creativity is the ability to form novel and valuable ideas or works using one’s imagination. Products of Creativity may be intangible (e.g. an idea, scientific theory, literary work, musical composition, or joke), or a physical object (e.g. an invention, dish or meal, piece of jewelry, costume, a painting). – source: Wikipedia
In general, there is no agreement on what creativity is or about a unique definition. Most authors agree that creativity is the production of something new, original, and useful. This something new can be a product or an idea. More formally, Margaret Boden defines creativity as: "the ability to come up with ideas or artifacts that are new, surprising and valuable" [1]
Defining value is an easier task. The LLM’s production of code is valuable when it performs its function properly. But is it new and surprising?
For an object, novelty refers to the dissimilarity between a manufactured artifact and others in its class. This is a problematic definition because it could result from a simple modification of existing objects or a new recombination. Thus, to be truly creative an object must be not only new (different from what previously existed) but also valuable (have some form of utility) and surprising (not be a simple variation or recombination). Since the output of an LLM is a text, what does it mean for a text to be creative?
To maintain consistency with the definition given above, we could define a creative text as a surprising elaboration that is not a simple variation or reworking of previous texts. LLMs are trained on a wide corpus of texts, and given an instruction they can generate a text in seconds. To be innovative the output text must be different from what is seen during pretraining (novelty) but also different from a simple variation (surprising). Since the decoding of an LLM has a stochastic component, an LLM will accidentally insert variations into the generated text.
Defining a text as surprising is complex, but it is the focal point for defining the creativity of an LLM. Boden [2] defines three types of creativity with respect to surprise:
- Combinatorial creativity. Finding unfamiliar combinations of familiar ideas. For example, combining two genres that have not been combined previously.
- Exploratory creativity. Finding new, unexplored solutions inside the current style of thinking. An example might be using an established narrative style, but introducing a unique twist within its confines (such as telling a classic love story from an unexpected point of view).
- Transformational creativity. Change the current style of thinking. Inventing a new way of presenting text, such as a novella written only of footnotes or an innovative chronological order (For example, the research work on new structures and patterns conducted by OuLiPo).
The autoregressive nature of the models should not lead to generating anything surprising. Even if it has a stochastic component, the model follows the given distribution with which it was trained (the pretraining texts) [3]. On the other hand, if instructed an LLM can generate poetry about mathematical theorems, write a theorem in the style of Hemingway, and other examples that seem to meet the definition of surprise. In fact, on close inspection, they come across as trivial and generally repetitive [4].
Recently, an article [5] took this issue further by trying to quantify the linguistic creativity of a text by reconstructing it from existing text snippets on the web. Human creativity is influenced by what we learn, but an author’s original text cannot be just attributed to some sources. If every text generated by an LLM can be mapped to other texts, it is overwhelming evidence of a lack of creativity.

They showed that humans exhibit a higher level of creativity than LLMs in all tasks (based on unique word and sentence combinations). For the authors, that little unquantifiable creativity comes from stochastic processes or the fact that we do not know the entire pretraining dataset.

In conclusion, at present LLMs do not show creativity. According to some authors [6–7], creativity is not only what is achieved but also how. In other words, creativity is a process that requires: motivation, perception, learning, thinking, and communication [8]. Creativity is based on knowing and finding information, but also on transcending the status quo. This last step requires going beyond the imitation game (their auto-regressive nature), to explore and challenge the current view. These last steps require self-awareness, a purpose, self-assessment, and a model of the world. Some studies try to push toward these aspects, but at the same time, it means we are far from true creativity.
What conditions are important for LLM creativity?
As we said above, LLMs are incapable of true creativity, but that does not mean we can’t improve the text quality they generate. We can describe three strategies with which to influence the output of a pre-trained LLM:
- Acting on the hyper-parameters of an LLM.
- Conducting additional training for an LLM.
- Prompting strategy.
The first strategy is basically to alter the temperature of a model [9]:
Temperature controls the uncertainty or randomness in the generation process, leading to more diverse outcomes by balancing probabilities for candidate words. – source
Increasing the degree of randomness does not mean getting true creativity. Adjusting the temperature affects how confident or exploratory the model is when selecting its next token (word, phrase, etc.). At low temperatures (e.g., 0.1–0.5), the model generates deterministic, focused, and predictable outputs (i.e., it selects the most likely tokens and thus regurgitates more closely what it saw during training).
With low temperatures, the model is repetitive, unoriginal, and sounds robotic but is more factually correct. With high temperatures (e.g., 0.7–1.5), the model generates more diverse and unpredictable text (during decoding it samples lower-probability tokens). Generally, choosing a high temperature is used for creative text or generating novel ideas. At a temperature higher than 2, the model generates chaotic and non-sensical text.
This study [9] analyzed what happens when a model is asked to generate a story using the same prompt but varying the temperature. As the temperature increases, they become more semantically diverse.

The increase in semantic difference does not mean either a difference in content or an increase in creativity. With the temperature increase, these stories may appear to be slightly more creative but lose consistency.

In general, temperature does not allow the LLM to leverage different regions of the embedding space, but it does enable some novelty when generating limited samples (as is the case for any real-world application). We observe a weak positive correlation between temperature and novelty, and unsurprisingly, a negative correlation between temperature and coherence. Suggesting a tradeoff between novelty and coherence. – souce
The authors [9] suggest that temperature does not allow one model to go beyond the training data distribution but leads the model to merely stochastically generate novel variations. This comes at a price; the output generated loses consistency.
The second strategy means exploiting post-training strategies such as instruction tuning or alignment to human preference. However, techniques such as RLHF or DPO, reduce variety and thus often have the opposite effect [10]. The third strategy means altering the instructions contained in the prompt and does not affect the ability to go beyond data distribution. Prompt engineering makes the model use better its acquired knowledge.
Hallucinations as a creativity phenomenon
In the previous sections, we have seen how LLMs are not inherently creative, and there are not many alternatives to overcome this limitation.
Classically, hallucinations have been seen as a problem to be solved, but some authors [11–12] suggest that they can be looked at from another perspective: as creative phenomena.
Although hallucinations are not invariably detrimental, in certain instances, they can stimulate a model’s creativity. – source
Hallucination is a factually incorrect output. We could also see a hallucination as an unexpected element that might be of interest in creative writing. Or even might be useful for fields where factuality is required instead (such as the field of scientific research). In this paper [12] they note that "hallucinations" have historically led to scientific discoveries. For example, heliocentrism was considered a factual error, and proposing heliocentrism to solve the retrograde motion of the planets could be considered at the time as a kind of hallucination. Similarly, stochastic events led to revolutionary discoveries such as penicillin.

Research on human creativity indicates that creative thinking involves both the activation of the left prefrontal cortex (implicated in imaginative thinking) and the hippocampus (region important for memory) [13]. In other words, human creativity would be related to recombining learned information with an imaginative element. In LLMs, hallucination could, therefore, introduce an imaginative element to the information recalled from the model.
An example, in this study [14] shows that hallucinations can be used in a scientific field such as drug discovery. The authors provide prompts that contain a molecule description and then ask the LLM to classify it by a certain chemical property. They provide either the prompt with a hallucinated description (or a description that does not contain them or a baseline). Paradoxically, adding a prompt with hallucinations seems to improve the performance of the model in the next task.

The Cybernetic Neuroscientist: Smarter Than Experts?
AI Planning or Serendipity? Where Do the Best Research Ideas Come From?
Parting thoughts
LLMs are not creative, and their reasoning ability is debated. This does not make them useless, but these limitations should be considered. Especially now that LLMs and AI agents will be used to accomplish real-world tasks, the lack of reasoning and creativity are serious limitations. Some interesting studies propose the use of agents to help researchers in drug discovery and chemistry [15–17]. Lack of creativity is a limitation to automating complex tasks or the entire research process. Agents, however, can be useful tools and be used to automate many tasks that do not require creativity.
If you have found this interesting:
You can look for my other articles, and you can also connect or reach me on LinkedIn. Check this repository containing weekly updated ML & AI news. I am open to collaborations and projects and you can reach me on LinkedIn. You can also subscribe for free to get notified when I publish a new story.
Here is the link to my GitHub repository, where I am collecting code and many resources related to Machine Learning, artificial intelligence, and more.
GitHub – SalvatoreRa/tutorial: Tutorials on machine learning, artificial intelligence, data science…
or you may be interested in one of my recent articles:
Can I Really Trust You, ChatGPT? Bridging AI Confidence and Human Understanding
Beyond Text: Navigating Toward a Multimodal RAG Horizon
The Dream Machine: Decoding Why LLMs Hallucinate Reality
You Cache Only Once: Cache-Augmented Generation (CAG) Instead Of RAG
Reference
Here is the list of the principal references I consulted to write this article, only the first name for an article is cited.
- Franceschelli, 2023, On the Creativity of Large Language Models, link
- Boden, 2003, The Creative Mind, link
- M. Shanahan, 2022, Talking about large language models, link
- Hoel, 2022, The banality of ChatGPT, link
- Lu, 2024, AI as Humanity’s Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text, link
- Gaut, 2003, Creativity and imagination, link
- Floridi, 2020, GPT-3: Its Nature, Scope, Limits, and Consequences, link
- Rhodes, 1961, An Analysis of Creativity, link
- Peeperkorn, 2024, Is Temperature the Creativity Parameter of Large Language Models? link
- Kirk, 2024, Understanding the Effects of RLHF on LLM Generalisation and Diversity, link
- Wang, 2024, LightHouse: A Survey of AGI Hallucination, link
- Jiang, 2024, A Survey on Large Language Model Hallucination via a Creativity Perspective, link
- Benedek, 2014, To create or to recall? neural mechanisms underlying the generation of creative new ideas, link
- Yuan, 2025, Hallucinations Can Improve Large Language Models in Drug Discovery, link
- Swanson, 2024, The Virtual Lab: AI Agents Design New SARS-CoV-2 Nanobodies with Experimental Validation, link
- Kudiabor, 2024, Virtual lab powered by ‘AI scientists’ super-charges biomedical research, link
- Caldas Ramos, 2024, A Review of Large Language Models and Autonomous Agents in Chemistry, link