Proactive Pseudo-Intervention: Pre-informed Contrastive Learning For Interpretable Vision Models
Deep neural networks excel at comprehending complex visual signals, delivering on par or even superior performance to that of human experts. However, ad-hoc visual explanations of model decisions often reveal an alarming level of reliance on exploiting non-causal visual cues that strongly correlate with the target label in training data. As such, deep neural nets suffer compromised generalization to novel inputs collected from different sources, and the reverse engineering of their decision rules offers limited interpretability. To overcome these limitations, we present a novel contrastive learning strategy called Proactive Pseudo-Intervention (PPI) that leverages proactive interventions to guard against image features with no causal relevance. We also devise a novel pre-informed salience mapping module to identify key image pixels to intervene and show it greatly facilitates model interpretability. To demonstrate the utility of our proposals, we benchmark it on both standard natural images and challenging medical image datasets. PPI-enhanced models consistently deliver superior performance relative to competing solutions, especially on out-of-domain predictions and data integration from heterogeneous sources. Further, saliency maps of models that are trained in our PPI framework are more succinct and meaningful.
