Reliable uncertainty quantification (UQ) remains a major challenge for machine learning interatomic potentials (MLIPs). Here, we benchmark UQ strategies for message passing neural networks such as MACE and Gaussian process (GP)-based interatomic potentials using coupled-cluster reference data for argon as a test case. We develop a hierarchical GP potential combining physics-informed priors with Bayesian propagation of hyperparameter uncertainty that retains useful discrimination of prediction errors while remaining conservative rather than overconfident, with a probabilistic structure that allows the sources of predictive uncertainty to be explicitly dissected. These results expose important distinctions between accuracy, calibration, sharpness, and error discrimination and provide practical design principles for MLIP UQ in uncertainty-critical applications.