CoExVQA
Department of Informatics, University of Oslo, Oslo, Norway
*Corresponding authors: kjetiki@ifi.uio.no, changkyc@uio.no
Abstract
Document Visual Question Answering (DocVQA) requires vision–language models to reason not only about what information in a document is relevant to a question, but also where the answer is grounded on the page. Despite strong predictive performance, existing DocVQA systems entangle these two aspects and operate largely as black boxes, offering limited means to verify how predictions depend on visual evidence. We propose CoExVQA, a self-explainable DocVQA framework that enforces a grounded reasoning process through a chain-of-explanation design. The model first identifies question-relevant evidence, then explicitly localizes the answer region, and finally decodes the answer exclusively from the grounded region. By making both evidence selection and spatial grounding intrinsic to prediction, CoExVQA enables direct inspection and verification of the reasoning process across modalities. Empirical results show that restricting decoding to grounded evidence yields competitive performance while providing transparent and verifiable predictions.
Method Overview
Example Prediction
Results
CoExVQA achieves state-of-the-art performance among self-explainable methods on PFL-DocVQA, and reduces the gap to non-explainable baselines by 50% — while providing explicit, verifiable grounding at every prediction.
Comparison on DocVQA
| Model | Explainability | DocVQA | Answer Localization | |||
|---|---|---|---|---|---|---|
| ACC ↑ | ANLS ↑ | IoUmean ↑ | Coveragemean ↑ | ARmean | ||
| Pix2Struct | ✗ | — | 0.77 | — | — | — |
| Donut | ✗ | — | 0.68 | — | — | — |
| DocVXQA | ✓ | 0.38 | 0.54 | — | — | — |
| Ours (CoExVQA) | ||||||
| Mask Variant | ✓ | 0.10 | 0.24 | 0.16 | 0.28 | 2.13 |
| Crop Variant | ✓ | 0.34 | 0.43 | 0.06 | 0.37 | 19.53 |
Table 1. Comparison to DocVQA baselines. Explainability indicates whether the method provides an explicit, human-interpretable rationale alongside the prediction. Entries marked '—' indicate that the corresponding metric is not reported in the cited baseline paper or is not applicable. For each metric, the best explainable method is highlighted in bold.
Comparison on PFL-DocVQA
| Model | Explainability | PFL-DocVQA | Answer Localization | |||
|---|---|---|---|---|---|---|
| ACC ↑ | ANLS ↑ | IoUmean ↑ | Coveragemean ↑ | ARmean | ||
| Pix2Struct | ✗ | 0.80 | 0.92 | — | — | — |
| DocVXQA | ✓ | 0.43 | 0.66 | — | — | — |
| Ours (CoExVQA) | ||||||
| Mask Variant | ✓ | 0.35 | 0.63 | 0.43 | 0.69 | 2.85 |
| Crop Variant | ✓ | 0.61 | 0.78 | 0.47 | 0.76 | 4.92 |
Table 2. Comparison to PFL-DocVQA baselines. The Pix2Struct performance is not reported in the original Pix2Struct paper, but taken from the DocVXQA paper. For each metric, the best explainable method is highlighted in bold.
Citation
Loading…