Accurate estimation of predictive uncertainty is essential for
optimal decision making. However, recent works have shown that
current neural networks tend to be miscalibrated, sparking
interest in different approaches to calibration. In this paper,
we conduct a large-scale empirical study of the probabilistic
calibration of neural networks on 57 tabular regression
datasets. We consider recalibration, conformal and
regularization approaches, and investigate the trade-offs they
induce on calibration and sharpness of the predictions. Based on
kernel density estimation, we design new differentiable
recalibration and regularization methods, yielding new insights
into the performance of these approaches. Furthermore, we find
conditions under which recalibration and conformal prediction
are equivalent. Our study is fully reproducible and implemented
in a common code base for fair comparison.