Towards Optimization and Model Selection for Domain Generalization: A Mixup-guided Solution. (arXiv:2209.00652v1 [cs.LG])

The distribution shifts between training and test data typically undermine
the performance of deep learning models. In recent years, lots of work pays
attention to domain generalization (DG) where distribution shift exists and
target data are unseen. Despite the progress in algorithm design, two
foundational factors have long been ignored: 1) the optimization for
regularization-based objectives (e.g., distribution alignment), and 2) the
model selection for DG since no knowledge about the target domain can be
utilized. In this paper, we propose Mixup guided optimization and selection
techniques for domain generalization. For optimization, we utilize an adapted
Mixup to generate an out-of-distribution dataset that can guide the preference
direction and optimize with Pareto optimization. For model selection, we
generate a validation dataset with a closer distance to the target
distribution, and thereby it can better represent the target data. We also
present some theoretical insights behind our proposals. Comprehensive
experiments on one visual classification benchmark and three time-series
benchmarks demonstrate that our model optimization and selection techniques can
largely improve the performance of existing domain generalization algorithms
and even achieve new state-of-the-art results.