Inter-rater agreement and reliability of outcome measurement instruments and staging systems used in hidradenitis suppurativa.

The very wide intervals for limits of agreement imply that substantial changes are needed in clinical research in order to rule out measurement error. The results illustrate a difficulty, even for experienced HS experts, to agree on the type and number of lesions when evaluating disease severity. The apparent caveats call for global efforts, such as the HIdradenitis SuppuraTiva cORe outcomes set International Collaboration (HISTORIC) to reach consensus on how best to measure physical signs of HS reliably in randomized trials. What's already known about this topic? Without valid and reliable instruments to measure outcomes, researchers and clinicians lack the necessary benchmarks to assess primary and secondary end points of interventional trials properly. Hidradenitis suppurativa (HS) is a chronic inflammatory skin disease. Several outcome measure instruments exist for HS, but their validation is generally incomplete or of relatively low methodological quality. What does this study add? Using a prospective completely balanced design this study examined inter-rater reliability with HS-experienced dermatologists as the rater population of interest. The study did not find very good reliability for any included instrument or lesion counts. This study illustrates the difficulty in finding agreement on the type and number of HS lesions, even among experts. The results question whether physical signs are best measured by a traditional physician lesion count instrument. What are the clinical implications of this work? For staging, Hurley staging and physician global visual analogue scale proved to be acceptable instruments in terms of inter-rater reliability. For the instruments designed to measure changes in health status, our study illustrates how difficult it is, even for experts, to measure the physical signs of HS using a simple rater counting. Consequently, other assessment methods of physicals signs, such as ultrasound evaluation, require consideration.