The Real-World Implications of IBM Project Debater Extend Beyond Debate
Champion debater Harish Natarajan took on IBM Project Debater at Think 2019 in San Francisco
Image by Visually Attractive for IBM
By Jim Utsler08/01/2019
It’s not easy taking part in a formal debate. People spend years honing the necessary skills to become informed, well-reasoned, persuasive, orderly and dynamic—all at the same time.
Given that this discipline requires a level head—and not the yelling, finger pointing and histrionics people now consider typical “debate” elements—it’s hardly surprising that computers have entered the debate conversation.
And not as topics, but as actual participants.
This wouldn’t have been possible, though, without recent rapid advances in artificial intelligence (AI), as supported by IBM Research and its approach to integrating varied disciplines, including text-based decision making, to solve particular problems.
In the case of Project Debater, for example, researchers were tasked with creating a system that could go head to head with champion debaters such as Harish Natarajan, who holds the world record for most debate competition victories and was the 2016 World Debating Championships Grand Finalist and the 2012 European Debate Champion.
To that end, IBM researchers refined earlier work in this area to create a system that could gather relevant debate-topic facts and opinions, form them into structured arguments, and then use precise language in a clear and persuasive way, all within a defined format that included a four-minute opening statement, a four-minute rebuttal and a two-minute summary.
Behind the Podium
A live public demonstration of Project Debater’s abilities took place at Think 2019 in San Francisco, with the system supporting the position that, “We should subsidize preschool.”
Its opening argument supported the resolution by citing relevant studies and figures, with the argument grounded on the premise that subsidizing preschool isn’t simply a matter of finance, but also a moral and political imperative to support society’s most vulnerable members.
These conclusions were founded on Project Debater’s knowledge base that included around 10 billion sentences from a variety of newspapers and journals. During the debate, the system searched these short pieces of text for information related to its position. This type of “understanding” requires a keen comprehension of human language, its oft-odd nuances and very precise statement presentation, something humans can have trouble with.
The Think debate results were tabulated via a real-time online poll. Prior to the debate, 79% of participants agreed that preschools should be subsidized, while 13% disagreed (8% were undecided). Following the debate, 62% of poll participants agreed that preschools should be subsidized, while 30% disagreed.
Although Natarajan was declared the winner because he persuaded 17% of the audience to his position, it should be noted that the system had little upside because a majority of poll participants were already in favor of the subsidizing position. Despite this “loss,” 55% of the participants indicated that Project Debater had increased their understanding of the topic, compared to 23% who said the same and 22% for champion debater Harish.
A similar outcome happened after the AI technology’s first small live public debate in June 2018. As Dr. Ranit Aharonov, manager, Project Debater, and Dr. Noam Slonim, IBM Distinguished Engineer, explain ed in a blog post, “Project Debater was tested against the 2016 Israeli national debate champion, Noa Ovadia. Following closing summaries from both sides, a snap poll showed that a majority of audience members surveyed thought Project Debater enriched their knowledge more than its human counterpart, a thrilling outcome for our research team.”
A Future Envisioned
In every debate scenario, Project Debater removes redundant argumentative texts, selects the strongest remaining claims and evidence, and arranges those by theme, creating the base of the narrative to support or contest a motion.
Project Debater then has to listen and comprehend the opponent’s response and generate a rebuttal, the most complex part of a debate for both humans and machines. Project Debater applies many techniques to do this, including anticipating and identifying the opponent’s arguments, and subsequently responding with claims and evidence that counter those arguments. The real-world implications of the science behind Project Debater’s text-based decision support capabilities are seemingly endless.
“Though the format and challenge of debate have allowed us to shape Project Debater’s capabilities,” Aharonov and Slonim wrote, “we envision a future for the technology beyond the podium. It could be used, for instance, to promote more civil debates in online comment forums or by an attorney preparing for a trial where it could review legal precedents and test the strengths and weaknesses of a case using a mock legal debate. In the financial services industry, Debater could identify financial facts that either support or undermine an investment strategy. Or it could be applied as a voice interaction layer for various complex customer experiences, or even to improve critical thinking and critical writing skills of youngsters.”
Dr. Noam Slonim, principal investigator, Project Debater, IBM Research-Haifa and Dr. Ranit Aharonov, manager, Project Debater, IBM Research-Haifa.
Speech by Crowd
IBM is also taking the core AI of Project Debater and developing it as new cloud service called Speech by Crowd. Speech by Crowd uses AI to collect free-text arguments from large audiences on debatable topics and automatically construct persuasive viewpoints to support or contest the topic. After the AI analyzes the arguments for their polarity, strength and relevance to the topic, the results are displayed on this indicator scale.
Slonim recently demonstrated Speech by Crowd at the United Nation’s AI for Good Summit. The team collected 1,755 arguments from attendees and from the web on the topic “Social media does more harm than good” and then presented the results to the nearly 900 attendees who commented about how the technology could eliminate some of the delays and bureaucracy in political organizations.
Future Practical Use
Based on demonstrations such as those at Think and the United Nations, the one thing that isn’t up for debate is the future practical use of Project Debater.
Jim Utsler, IBM Systems magazine senior writer, has been writing for IBM since the mid-1990s.