VecDB@VLDB 2026 submission
An Input-Regime Audit of Conflict Detection for Retrieval-Augmented Generation
Dipankar Sarkar
Abstract
An audit of three conflict detectors for vector-database-backed RAG pipelines (a DeBERTa-MNLI cross-encoder, an LLM-as-judge baseline, and the same judge question-conditioned) across curated probes with bootstrap CIs, a 10-model LLM sweep, a prompt-robustness ablation, long-passage realism, an answer-bearing scoping study and external validation on 100 human-labelled FEVER pairs. The detector trade-off is input-distribution dependent; the findings are synthesised into a five-row input-regime ladder.
Vector-database-backed retrieval-augmented generation (RAG) pipelines increasingly add a conflict-detection stage that flags contradictions among retrieved passages before generation. Two directions dominate practice: a pairwise NLI cross-encoder over retrieved passages, and a query-conditioned formulation that lets the NLI label depend on the question being answered. Practitioners must choose between them with little empirical guidance.
This paper audits three detectors — the classic DeBERTa-MNLI cross-encoder, an LLM-as-judge baseline, and the same LLM judge question-conditioned — on curated probes (n=63 with bootstrap confidence intervals), a 10-model LLM sweep, a prompt-robustness ablation, a long-passage realism step, an answer-bearing scoping study, and external validation on 100 human-labelled FEVER pairs.
The cross-encoder is competitive on its native short claim/evidence distribution but unusable on long multi-claim passages; a same-name “sense trap” fools pairwise judging unless the judge is conditioned on the question; and pairwise detection is ill-posed on multi-hop fragments. The findings are synthesised into a five-row input-regime ladder recommending a detector per retrieval context. Code and result artefacts are released.
For a practitioner-level walkthrough of how to tell conflict patterns apart in your own pipeline, see Diagnosing Conflicts in RAG Pipelines.
Frequently Asked Questions
What does the Input-Regime Audit study?
It audits conflict detection, the stage in a RAG pipeline that flags contradictions among retrieved passages before generation. Three detectors are compared: a pairwise DeBERTa-MNLI cross-encoder, an LLM-as-judge baseline, and the same LLM judge conditioned on the question being answered.
What are the main findings?
Five findings: (1) LLM judges keep perfect precision on labelled FEVER pairs and flag far fewer top-k retrieved non-gold passages than the cross-encoder (0.09 vs 0.43 flag rate). (2) A same-name 'sense-trap' false positive fools both the cross-encoder and pairwise LLM judging; only question-conditioning closes the gap (precision 0.68 to 1.00 on curated probes, replicated for 8 of 10 LLM families). (3) The cross-encoder versus LLM trade-off depends on the input distribution: FEVER F1 0.85 on short claim/evidence pairs but 0.18 on long multi-claim passages. (4) The small-model regression under question-conditioning is a capability limit, not a format limit. (5) Pairwise conflict detection is ill-posed on multi-hop evidence fragments and must be scoped to answer-bearing units.
What is the input-regime ladder?
A five-row summary that recommends a detector for each retrieval context a vector-database practitioner is likely to encounter, derived from the experiments above. Its recommendations are bounded by the probes, models and datasets used in the paper.
Where are the paper and artefacts?
The compiled paper, its source, the JSON result artefacts for each experiment and a minimal runnable Python slice are in github.com/sarkar-dipankar/ebrag-vecdb-2026-paper (MIT licence). The paper is not on arXiv.