Accurate predictions of machine learning interatomic potentials (MLIPs) rely on training datasets covering relevant thermodynamic conditions and phases. Using water as an example, we show that insufficiently sampled datasets of MLIPs lead to wrong predictions of densities, melting temperatures, and nuclear quantum effects. In contrast, MLIPs trained on comprehensive datasets with the same electronic structure method yield consistent property predictions agnostic of MLIP architecture, with densities in quantitative agreement with reference calculations.