It can’t beat us. Yet. But an artificial intelligence (AI) program developed by scientists at IBM is so good at debating against humans that it may just be a matter of time before we can’t compete.
Project Debater, an autonomous debating system that’s been in development for several years, is capable of arguing against humans in a meaningful way, taking a side in a debate on a chosen topic, and making a strong case for why its viewpoint is the right one.
The AI, described in a new study published in Nature, isn’t yet at the point where it can trump human argumentative logic, but a public demonstration of its abilities in 2019 showed just how far its artificial way of reasoning has come.
In a showcase debate against expert debating champion Harish Natarajan, Project Debater revealed that it was able to form an argument on a complex topic (whether preschool should be subsidized for families), and present a case in support of the argument.
(IntelligenceSquared Debates/YouTube)
Within natural language processing, the ability to do this involves what’s called argument mining, in which an AI will parse a huge amount of disparate information, looking to link together relevant sections.
In the case of Project Debater, it was fed an archive of some 400 million news articles, from which it can now compose opening statements, rebuttals, and closing summations on a range of approximately 100 debate topics.
While the system still needs refinement – and isn’t yet at the point where it can match a human debating expert – the majority of observers rating transcripts of Project Debater’s arguments scored it highly, ranking the AI’s performance as decent in the debates.
That’s quite an achievement, given the technological challenges of argument mining for this kind of purpose were considered all but impossible even for state-of-the-art AI just a decade ago.
“Since then, a combination of technical advances in AI and increasing maturity in the engineering of argument technology, coupled with intense commercial demand, has led to rapid expansion of the field,” explains argument technology researcher Chris Reed from the University of Dundee in the UK, who wasn’t involved in the study, in a commentary on the research.
“More than 50 laboratories worldwide are working on the problem, including teams at all the large software corporations.”
Of course, in recent years we’ve seen AIs surpass human capabilities many times over in the world of game-playing, but argument mining is in many ways a more complex endeavor, given the massive amounts of information that need to be analyzed and then linked together.
“Debating represents a primary cognitive activity of the human mind, requiring the simultaneous application of a wide arsenal of language understanding and language generation capabilities, many of which have only been partially studied from a computational perspective (as separate tasks), and certainly not in a holistic manner,” the researchers, led by principal investigator Noam Slonim explain in their study.
“Therefore, an autonomous debating system seems to lie beyond the reach of previous language research endeavors.”
While Project Debater represents a significant step forward, the field is not yet at the point where AIs can perform this feat of reasoning better than humans can.
The team says debating still effectively lies outside the ‘comfort zone’ of AI systems, and that novel paradigms will need to be developed before we see machines besting people’s inherent argumentative skill in complex and ambiguous topics.
The findings are reported in Nature.
