A survey on out-of-distribution evaluation of neural NLP models

Abstract

Adversarial robustness, domain generalization, and dataset biases are three active lines of research contributing to out-of-distribution (OOD) evaluation on neural NLP models. However, a comprehensive, integrated discussion of the three research lines is still lacking in the literature. In this survey, we:

1. Compare the three lines of research under a unifying definition. 2. Summarize the data-generating processes and evaluation protocols for each line of research. 3. Emphasize the challenges and opportunities for future work.