Effects of Response Format and Number of Indicators on Results of Item Response Theory Models and Multivariate Techniques

Abstract

The determination of a standard test length that gives optimal statistical information about latent constructs with their dimensionality is a task that remains for investigation. In this regard, several datasets have been simulated under varied conditions of response scales, sets of items and dimensionality using the R package mirt. The data are examined using Item Response Theory (IRT) and compared with Factor Analysis output. The results show that IRT model fitness improves with increasing scale-points and higher dimensionality and generally with increasing set of variables. It is observed that on dichotomous scale in particular, higher number of items are suitable for measuring unidimensionality on lower point scale but could be misconstrued for bi-dimensionality. A higher measurement scale is found generally suitable to capture the appropriate underlying dimensionality. For both techniques, adequate model fitness is achieved with indicators not exceeding 40 in number.