A1228
Title: A metric for the performance assessment of ordinal-scaled response variable models
Authors: Javier Espinosa-Brito - University of Santiago of Chile (Chile) [presenting]
Leonardo Videla-Munoz - University of Santiago of Chile (Chile)
Abstract: Ordinal variables occupy a unique position between nominal and interval scales, carrying an inherent order yet lacking defined inter-category distances. Many models for ordinal response variables do not provide a predicted category as a direct result, such as some artificial neural networks, the proportional odds cumulative logit model, the continuation ratio model, and others. Instead, each unit of analysis delivers an estimated probability distribution (EPD) over the $k$ ordered categories rather than a single estimated label. Conventional evaluation collapses these EPDs into a single predicted category for each subject based on some criteria, such as choosing the category with the highest estimated probability. Then, these estimated labels are compared with observed outcomes via confusion-matrix measures, thereby discarding much of the probabilistic information. A new metric is proposed that evaluates models outcomes directly from their EPDs while taking into account the ordinality of the response. Simulation studies and an observational application demonstrate its robustness and advantages over existing, indirectly derived alternatives.
