Login / Signup

ONEST (Observers Needed to Evaluate Subjective Tests) Analysis of Stromal Tumour-Infiltrating Lymphocytes (sTILs) in Breast Cancer and Its Limitations.

Bálint CserniDarren KilmartinMark O'LoughlinXavier AndreuZsuzsanna Bagó-HorváthSimonetta BianchiEwa ChmielikPaulo FigueiredoGiuseppe FlorisMaria Pia FoschiniAnikó KovacsPäivi HeikkiläJanina KulkaAnne-Vibeke LaenkholmInta Liepniece-KareleCaterina MarchioElena ProvenzanoPeter RegitnigAngelika ReinerAleš RyškaAnna SapinoElisabeth Specht StovgaardCecily QuinnVasiliki ZolotaMark WebberSharon A GlynnRita BoriErika CsörgőOrsolya Oláh-NémethTamás PancsaAnita SejbenIstván SejbenAndrás VörösTamás ZomboriTibor NyáriGrace CallagyGábor Cserni
Published in: Cancers (2023)
Tumour-infiltrating lymphocytes (TILs) reflect antitumour immunity. Their evaluation of histopathology specimens is influenced by several factors and is subject to issues of reproducibility. ONEST (Observers Needed to Evaluate Subjective Tests) helps in determining the number of observers that would be sufficient for the reliable estimation of inter-observer agreement of TIL categorisation. This has not been explored previously in relation to TILs. ONEST analyses, using an open-source software developed by the first author, were performed on TIL quantification in breast cancers taken from two previous studies. These were one reproducibility study involving 49 breast cancers, 23 in the first circulation and 14 pathologists in the second circulation, and one study involving 100 cases and 9 pathologists. In addition to the estimates of the number of observers required, other factors influencing the results of ONEST were examined. The analyses reveal that between six and nine observers (range 2-11) are most commonly needed to give a robust estimate of reproducibility. In addition, the number and experience of observers, the distribution of values around or away from the extremes, and outliers in the classification also influence the results. Due to the simplicity and the potentially relevant information it may give, we propose ONEST to be a part of new reproducibility analyses.
Keyphrases
  • machine learning
  • peripheral blood
  • healthcare
  • single cell
  • social media
  • fine needle aspiration