Hierarchical Decision Ensembles- An inferential framework for uncertain Human-AI collaboration in forensic examinations. (arXiv:2111.01131v1 [cs.HC])

Forensic examination of evidence like firearms and toolmarks, traditionally
involves a visual and therefore subjective assessment of similarity of two
questioned items. Statistical models are used to overcome this subjectivity and
allow specification of error rates. These models are generally quite complex
and produce abstract results at different levels of the analysis. Presenting
such metrics and complicated results to examiners is challenging, as examiners
generally do not have substantial statistical training to accurately interpret
results. This creates distrust in statistical modelling and lowers the rate of
acceptance of more objective measures that the discipline at large is striving
for. We present an inferential framework for assessing the model and its
output. The framework is designed to calibrate trust in forensic experts by
bridging the gap between domain specific knowledge and predictive model
results, allowing forensic examiners to validate the claims of the predictive
model while critically assessing results.