Traditionally-crafted news articles are more comprehensible than articles produced with automation. This was the finding of a Ludwig Maximilian University of Munich (LMU) study that was recently published in the journal Journalism: Theory, Practice, and Criticism.
The research team at the Department of Media and Communication (IfKW) surveyed more than 3,000 online news consumers in the UK. Each of the respondents rated one of 24 texts, half of which had been produced with the help of automation and half of which had been manually written by journalists.
“Overall, readers found the 12 automated articles significantly less comprehensible,” says lead author Sina Thäsler-Kordonouri. This was despite the fact that the AI-generated articles had been sub-edited by journalists prior to publication.
Worse handling of numbers and word choice
According to the survey, one of the reasons for reader dissatisfaction was the word choice used in the AI texts. Readers complained that the AI-produced articles contained too many inappropriate, difficult, or unusual words and phrases. Furthermore, readers were significantly less satisfied with the way the automated articles handled numbers and data.
The deficiencies readers perceived in the automated articles’ handling of numbers and word choice partly explain why they were harder to understand, the researchers say. However, readers were equally satisfied with the automated and manually-written articles in terms of the character of the writing and its narrative structure and flow.
More human sub-editing required
Professor Neil Thurman, who led the project, suggests that “when creating and/or sub-editing automated news articles, journalists and technologists should aim to reduce the quantity of numbers, better explain words that readers are unlikely to understand, and increase the amount of language that helps readers picture what the story is about.”
This study is the first to investigate both the relative comprehensibility of manual and automated news articles and explore why a difference exists.
“Our results indicate the importance not only of maintaining human involvement in the automated production of data-driven news content, but of refining it,” says Thäsler-Kordonouri.
More information:
Sina Thäsler-Kordonouri et al, Too many numbers and worse word choice: Why readers find data-driven news articles produced with automation harder to understand, Journalism (2024). DOI: 10.1177/14648849241262204
Provided by
Ludwig Maximilian University of Munich
Citation:
AI-generated news is harder to understand, study shows (2024, October 24)
