SPEAKER: Ola Rønning, postdoc at ITU
ABSTRACT: SLAM and odometry systems emit a pose belief, but the standard pipeline (APE/RMSE via evo) collapses it to a mean and treats the reference as flawless. SMFEval lifts both sides to distributions, the estimator Q and the observer P, and asks two questions. First, how does Q score against P? Proper scoring rules (CRPS, energy, log, interval) give single numbers that reward calibration and sharpness together, enabling principled ranking. Second, is Q calibrated against P? The probability integral transform should be uniform under correct calibration; a Kolmogorov Smirnov test makes this checkable. From a SQUARE belief file (a distributional analogue of TUM) and a reference trajectory, SMFEval produces an automated report with sync, alignment, scores, per axis calibration verdicts, and a recommendation.