From The Odyssey to Agatha Christie’s crime novels, humans have created countless captivating stories that have filled the bookshelves of generation after generation. But a study suggests that artificial intelligence may now be starting to compete with human writers.
A study published in the journal Judgment and Decision Making found that researchers asked 1,682 adults to read six short stories, three written by humans and three generated by ChatGPT. Each AI-written story focused on a theme similar to that of one of the human-written works.
The researchers told participants whether the stories they were reading had been written by humans or AI, but this information was not always accurate. The participants were then asked to rate the stories’ appeal, reading experience, and overall quality.
The results showed that participants who read AI-generated stories found them more engaging and rated their quality higher than participants who read human-written stories. However, when participants were told that a story had been written by a human, they gave it higher ratings.
Dr. Deena Skolnick Weisberg of Villanova University in Pennsylvania, the study’s lead author, said: “AI systems are already capable of writing short stories, and people consider these stories to be at least as good as human-written works, and possibly even better. We should rethink what this tells us about AI’s capabilities.”
However, Weisberg is also a creator herself. She said this did not mean that writing should be handed over to AI. She noted that novels created by technology companies using AI would likely differ from works written by human authors.
“We may need to make room for novels created by AI, as well as novels co-created by AI and humans, but that does not mean the human creative process is no longer worth appreciating,” she said. “People write in order to express themselves creatively, to challenge themselves, and to understand and process their experiences. The fact that AI can generate stories resembling those created by humans does not change any of that.”
The study also found that people’s attitudes toward AI affected their evaluations. Participants with more positive attitudes toward AI gave higher ratings after being told that a story had been created by ChatGPT than participants with more negative attitudes toward AI.
The researchers then conducted two additional experiments involving a total of 905 adults. Participants were shown two of six short stories. The two stories shared the same theme, but only one had been written by a human. Participants had to determine which story had been written by a human and which had been generated by AI.
In one of the experiments, about 40% of participants guessed correctly, which was even lower than the accuracy expected from random guessing; in the other experiment, the accuracy rate was 52%.
The researchers wrote: “There was no general evidence that participants were able to distinguish human-written stories from AI-generated stories.”
The researchers also found that participants’ self-reported experience reading or writing fiction was not clearly related to how accurately they identified the authors of the stories. However, people who were more familiar with AI systems had higher accuracy rates.
Weisberg said: “AI writing has a particular style, and through practice we may gradually learn to recognize that style, just as we can learn to recognize the distinctive writing styles of different human authors.”
Weisberg believes that people’s preference for stories labeled “written by a human” may be related to their pursuit of authenticity. In terms of the actual reading experience, however, people may prefer AI-generated stories because they are easier to read and understand.
Luke Kennard, a professor of film and creative writing at the University of Birmingham in the UK who was not involved in the study, stressed that AI remains a highly controversial topic.
“Because we are human, we tend to anthropomorphize AI, imagining it as some kind of glowing robot with a benevolent and neutral face,” he said. “But in reality, it is merely predictive code operating on a vast stolen database of writing, running in enormous, expensive, and destructive hyperscale data centers that have a catastrophic impact on surrounding communities.”
Kennard also stressed that AI is not an ordinary “tool” like a spell checker or thesaurus, but an existential threat.
“Of course, with the right prompt and some adjustment and polishing, it can certainly produce a coherent, even pleasing poem. After all, its training data comes from thousands of years of human creative work; if it could not do that, it would be embarrassing,” he said. “I think the real question we should be asking is: Is any of this worth it? The answer is absolutely—and without question—no.”