B0654
Title: Projecting the performance of polytomous item response models onto a common scale with the InterModel Vigorish
Authors: Klint Kanopka - New York University (United States) [presenting]
Benjamin Domingue - Stanford University (United States)
Abstract: There exists a wide range of item response models for ordered categorical polytomous responses, including the graded response model and generalized partial credit model. Structurally, these models contain different assumptions about the item response process and are not merely exchangeable transformations of each other. In applied settings, decisions about which model to apply benefit from tools that help to quantify the impact of the decision. The methods typically used to quantify goodness of fit often work only under limited assumptions or are only able to adjudicate between different models applied to the same data. This has the downside of creating highly contextualized knowledge about the relationships between models and data. The InterModel Vigorish (IMV) is extended, a method for quantifying increases in predictive accuracy along a common scale for dichotomous outcomes, to polytomous item response models by dichotomizing the categorical responses in two ways. The first looks at correctly predicting whether or not a response is above or below a threshold and the second looks at predicting the correct response conditional on being in one of two categories. Applications and simulations are used to describe the different underlying structures in response data that each of these two approaches is sensitive to and how to interpret them together to aid operational model selection.
