Enhancing Fake News Detection with Large Language Models Through Multi-agent Debates

Large language models (LLMs) are showing dramatic progress in terms of language generation and in reasoning tasks. Existing works on fake news detection mostly focus on fine-tuning small language models such as BERT. One downside of fine-tuning is that it requires a lot of data which might not always be available. With the prevalent spread of fake news and misinformation, alternative ways are needed especially in cases where there is lack of enough training data. In this paper, we propose using multi-agent debate strategies to enhance fake news detection by leveraging the capabilities of LLMs. We introduce two approaches: a uniform prompt multi-agent debate and a diverse prompt multi-agent debate where each LLM agent adopts distinct roles such as fact-checker, journalist, or data scientist. These methods are benchmarked against single LLM evaluations to assess the impact of collaborative reasoning. Our experiments on the PolitiFact and GossipCop datasets reveal that the multi-agent debate methods outperform single LLM assessments. Notably, the diverse persona debate approach achieves the highest performance, demonstrating the value of incorporating different perspectives in reasoning. These results suggest that multi-agent debates can effectively harness the strengths of single LLMs to improve the reliability of fake news detection systems.