Marlis Ontivero-Ortega, Luca Faes, Jesus M. Cortes, Daniele Marinazzo, and Sebastiano Stramaglia. Assessing high-order effects in feature importance via predictability decomposition. Physical Review 111: L033301, 2025. [pdf]
Abstract
Building on recent advances in describing redundancy and synergy in multivariate interactions among random
variables, we propose an approach to quantify cooperative effects in feature importance, a key technique in
explainable artificial intelligence. Specifically, we introduce an adaptive version of the widely used metric Leave
One Covariate Out (LOCO), designed to disentangle high-order effects involving a particular input feature in
regression problems. LOCO measures the reduction in prediction error when the feature of interest is added to
the set of features used in regression. Unlike the standard approach that computes LOCO using all available
features, our method identifies the subsets of features that maximize and minimize LOCO. This results in a
decomposition of LOCO into a two-body component and higher-order components (redundant and synergistic),
while also identifying the features that contribute to these high-order effects in conjunction with the driving
feature.We demonstrate the effectiveness of the proposed method in a benchmark dataset related to wine quality
and to proton versus pion discrimination using simulated detector measurements generated by GEANT.
