Artboard 2 copy 35Artboard 64 copy 13Artboard 2 copy 19Artboard 2 copy 31Artboard 64 copy 18Artboard 64 copy 10Artboard 64 copy 11Artboard 64 copy 15Artboard 64 copy 12Artboard 64 copy 13Artboard 64 copy 14Artboard 2 copy 34Artboard 64 copy 19Artboard 64 copy 16MinusArtboard 2 copy 44Artboard 2 copy 38Artboard 2 copy 36PlusArtboard 64 copy 17Artboard 2 copy 43Artboard 2 copy 45Artboard 2 copy 46Artboard 64 copy 16Artboard 64 copy 18Artboard 64 copy 19Artboard 64 copy 17

State of the psychometric methods: Comments on the ISOQOL SIG psychometric papers

Tidsskriftartikel - 2019


BACKGROUND: Psychometric analyses of patient reported outcomes typically use either classical test theory (CTT), item response theory (IRT), or Rasch measurement theory (RTM). The three papers from the ISOQOL Psychometrics SIG examined the same data set using the tree different approaches. By comparing the results from these papers, the current paper aims to examine the extent to which conclusions about the validity and reliability of a PRO tool depends on the selected psychometric approach.

MAIN TEXT: Regarding the basic statistical model, IRT and RTM are relatively similar but differ notably from CTT. However, modern applications of CTT diminish these differences. In analyses of item discrimination, CTT and IRT gave very similar results, while RTM requires equal discrimination and therefore suggested exclusion of items deviating too much from this requirement. Thus, fewer items fitted the Rasch model. In analyses of item thresholds (difficulty), IRT and RMT provided fairly similar results. Item thresholds are typically not evaluated in CTT. Analyses of local dependence showed only moderate agreement between methods, partly due to different thresholds for important local dependence. Analyses of differential item function (DIF) showed good agreement between IRT and RMT. Agreement might be further improved by adjusting the thresholds for important DIF. Analyses of measurement precision across the score range showed high agreement between IRT and RMT methods. CTT assumes constant measurement precision throughout the score range and thus gave different results. Category orderings were examined in RMT analyses by checking for reversed thresholds. However, this approach is controversial within the RMT society. The same issue can be examined by the nominal categories IRT model.

CONCLUSIONS: While there are well-known differences between CTT, IRT and RMT, the comparison between three actual analyses revealed a great deal of agreement between the results from the methods. If the undogmatic attitude of the three current papers is maintained, the field will be well served.


Bjorner JB. State of the psychometric methods: Comments on the ISOQOL SIG psychometric papers. Journal of patient-reported outcomes 2019;3(1):49.
doi: 10.1186/s41687-019-0134-1

Gå til Tidsskriftartikel