Verifying ambiguous claims with a reasoner-translator framework

Fact-checking is a task that verifies the claim's veracity based on evidence. During fact-checking, the ambiguity in natural language often poses a significant challenge, which leads to disagreement among fact-checkers even when presented with the same evidence (e.g., simultaneously determining the veracity label as Support and Neutral). Tracing where and how ambiguity occurs is crucial for resolving these disagreements, yet this area remains unexplored. In response, we introduce the entailment tree, a novel tree-structured representation that traces ambiguity in the fact-checking process. Specifically, we represent the evidence-based entailment reasoning process as a tree with premise and conclusion nodes, with ambiguity captured through “forked nodes”. A shared conclusion highlights the ambiguous point in forked nodes, while diverging premise branches reflect different interpretations. To automate the construction of entailment trees, we propose a Reasoner-Translator framework based on Large Language Models (LLMs). This framework operates in two stages: a Reasoner conducts Chain-of-Thought (CoT) reasoning, and a Translator translates the reasoning into the corresponding entailment tree. We evaluate our framework on the AmbiFC dataset. Compared to baseline methods that directly produce trees, the Reasoner-Translator framework achieves better reasoning performance and generates higher-quality trees. Additionally, we present a linguistic analysis of forked nodes in the case study to benefit further research.