Method Calibration or Data Fitting?

We investigate by explicit parameter optimization to what extent basis sets of polarized double-ζ quality can introduce compensating errors in five different density functional methods. It is shown that minor changes in the contraction coefficients of the valence functions in the basis sets can have a significant impact and allow different density functional methods to achieve very similar performances. This holds for nuclear magnetic shielding constants and for isomerization energies, barrier heights, and noncovalent interactions. It is furthermore shown that errors due to neglect of vibrational and solvent effects can be absorbed in the combined method and basis set errors. These findings hold for data sets consisting of 50-150 data points. This raises the question of whether the common practice of identifying combinations of density functional methods and basis sets that have a good performance against a selected set of reference data should be considered as data fitting in the combined parameter space spanned by the method and basis set.