A0747
Title: Accounting for covariates in forensic error rate assessment and evidence interpretation
Authors: Larry Tang - University of Central Florida (United States) [presenting]
Abstract: These error rates reported by recent forensic black-box studies are mainly obtained by pooling all the decisions from examiners or computer algorithms with same-source or different-source pairs. These measures report the average error rates across a population of examiners for evidence sources. It would be ideal to account for covariates such as 1) source subjects' covariate information, including their demographics and/or source images' attributes and quality, and 2) examiners' covariate information, such as their training background and demographics. Appropriately accounting for covariates in error rate assessment and evidence interpretation requires sophisticated statistical analyses with modern statistical concepts and methods. The NIJ-funded work is presented on the ROC regression framework for error rate quantification by allowing covariates specific to source subjects and examiners. Statistical techniques are discussed by fitting ROC regression in order to relate covariates to error rates quantified by the receiver operating characteristic (ROC) curve. The resulting covariate-specific ROC curves in face recognition, handwriting, and latent print databases will model the relationship between covariates and decision scores and give the error rates for specific values of covariates. An R-Shiny app is also presented to facilitate the implementation of the developed methods for black-box studies.
