Comment on "Quantifying the informational value of classification images": A miscomputation of the infoVal metric.
Mathias SchmitzMarine RougierVincent YzerbytPublished in: Behavior research methods (2020)
Brinkman et al. (2019) recently introduced an innovative metric-infoVal-to assess the informational value of classification images (CIs) relative to a random distribution. Although this measure constitutes a valuable tool to distinguish random from nonrandom CIs, we identified two noteworthy discrepancies between the mathematical formalization of the infoVal metric and the authors' computation. Specifically, the computation was based on the one norm instead of the Euclidean norm, and the k constant was omitted in the denominator of the ratio that produces infoVal. Accordingly, the simulations and experimental results reported by Brinkman et al. do not build on the correct infoVal computation but on a biased index. Importantly, this discrepancy in the computation affects the statistical power and Type I and error rate of the metric. Here we clarify the nature of the discrepancies in the computation and run Brinkman et al.'s Simulation 1 anew with the correct values, to illustrate their consequences. Overall, we found that relying on the miscomputed infoVal metric can lead to misguided conclusions, and we urge researchers to use the correct values.