A Metacognitive Approach to Out-of-Distribution Detection for Segmentation. (arXiv:2311.07578v1 [cs.CV])

Despite outstanding semantic scene segmentation in closed-worlds, deep neural
networks segment novel instances poorly, which is required for autonomous
agents acting in an open world. To improve out-of-distribution (OOD) detection
for segmentation, we introduce a metacognitive approach in the form of a
lightweight module that leverages entropy measures, segmentation predictions,
and spatial context to characterize the segmentation model’s uncertainty and
detect pixel-wise OOD data in real-time. Additionally, our approach
incorporates a novel method of generating synthetic OOD data in context with
in-distribution data, which we use to fine-tune existing segmentation models
with maximum entropy training. This further improves the metacognitive module’s
performance without requiring access to OOD data while enabling compatibility
with established pre-trained models. Our resulting approach can reliably detect
OOD instances in a scene, as shown by state-of-the-art performance on OOD
detection for semantic segmentation benchmarks.