1Northwestern Polytechnical University
2Australian National University
Most of the existing salient object detection (SOD) models focus on improving the overall model performance, without explicitly explaining the discrepancy between the training and testing distributions. In this paper, we investigate a particular type of epistemic uncertainty, namely distributional uncertainty, for salient object detection. Specifically, for the first time, we explore the existing class-aware distribution gap exploration techniques, i.e. long-tail learning, single-model uncertainty modeling and test-time strategies, and adapt them to model the distributional uncertainty for our class-agnostic task. We define test sample that is dissimilar to the training dataset as being “out-of-distribution” (OOD) samples. Different from the conventional OOD definition, where OOD samples are those not belonging to the closed-world training categories, OOD samples for SOD are those break the basic priors of saliency, i.e. center prior, color contrast prior, compactness prior and etc., indicating OOD as being “continuous” instead of being discrete for our task. We’ve carried out extensive experimental results to verify effectiveness of existing distribution gap modeling techniques for SOD, and conclude that both train-time single-model uncertainty estimation techniques and weight-regularization solutions that preventing model activation from drifting too much are promising directions for modeling distributional uncertainty for SOD.
Visualization of different types of uncertainty, where aleatoric uncertainty \(p(y|x^\star,\theta)\) is caused by the inherent randomness of the data, model uncertainty \(p(\theta|D)\) happens when there exists low-density region, leading to multiple solutions within this region, and distributional uncertainty \(p(x^\star|D)\) occurs when the test sample \(x^\star\) fails to fit in the model based on the training dataset $D$.
“OOD” samples for salient object detection. Different from the class-aware tasks, OOD for saliency detection is continuous, which can be defined as attributes that break the basic saliency priors, i.e. center prior, contrast prior, compactness prior, etc. We aim to explore distributional uncertainty estimation for saliency detection.
Performance of classic distribution bias modeling strategies.
Visual comparison of MCDropout with DeepEnsemble. The first column shows the input image and segmentation GT, and the other columns show the generated segmentation predictions and uncertainty maps. Compared to other models, the DeepEnsemble method integrates the information of multiple decoders, making it more accurate in distributional uncertainty modeling.
Performance of long-tail learning based distribution bias modeling strategies.
Performance of single model uncertainty modeling methods.
Qualitative comparison of single-model uncertainty modeling strategies, i.e. ReAct and TCP, which both indicate more correct predictions and reliable uncertainty maps.
Performance of test-time strategies.
Visual comparison of test-time strategies, i.e. test-time training (CoTTA) and test-time augmentation (CTTA). It shows that test-time augmentation strategy can help explore low-density regions and calibrate incorrect predictions.
The distribution of AUROC metric on DUTS testing dateset, where x-axis is the AUROC measure, y-axis is the number of samples. The white line with the black number on the left indicates the mean AUROC of each method. The short blueline on the left represents the value of the lower 5% percentile.
@inproceedings{tian2023modeling,
title={Modeling the Distributional Uncertainty for Salient Object Detection Models},
author={Tian, Xinyu and Zhang, Jing and Xiang, Mochu and Dai, Yuchao},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={19660--19670},
year={2023}
}