How to predict if you do not know what the driver is? "Conditional Feature Importance (CFI; Strobl et al. (2008)) is a classical variable importance measure that accounts for the relationship between the studied feature and the others." "We want to avoid making strong assumptions about the relationship between inputs and outputs, as it can be complex. However, we assume that the relationship among the input covariates is simple, typically because they originate from the same generative process (e.g. a measurement device). Then, it can be much more efficient and accurate to study the relationship between the j-th covariate, Xj, given the rest, X−j, rather than directly regressing y on X−j. This approach is also advantageous in settings where labeling data is expensive, as there are typically more available samples for the input pair (X−j, Xj) than for the pair (X−j, y)." Angel Reyero-Lobo, Pierre Neuvial, Bertrand Thirion (2026). https://arxiv.org/abs/2501.17520 #stats #statistics #probabilities #machineLearning #featureSelection