ONEST (Observers Needed to Evaluate Subjective Tests) Analysis of Stromal Tumour-Infiltrating Lymphocytes (sTILs) in Breast Cancer and Its Limitations
Abstract
Simple Summary Tumour-infiltrating lymphocytes (TILs) reflect the host's response against tumours. TILs have a strong prognostic effect in the so-called triple-negative (oestrogen receptor, progesterone receptor, and human epidermal growth factor receptor-2 negative) subset of breast cancers and predict a better response when primary systemic (neoadjuvant) treatment is administered.
Although they are easy to assess, their quantitative assessment is subject to some inter-observer variation. ONEST (Observers Needed to Evaluate Subjective Tests) is a new way of analysing inter-observer variability and helps in estimating the number of observers required for a more reliable estimation of this phenomenon.
This aspect of reproducibility for TILs has not been explored previously. Our analysis suggests that between six and nine pathologists can give a good approximation of inter-observer agreement in TIL assessments.
Tumour-infiltrating lymphocytes (TILs) reflect antitumour immunity. Their evaluation of histopathology specimens is influenced by several factors and is subject to issues of reproducibility.
ONEST (Observers Needed to Evaluate Subjective Tests) helps in determining the number of observers that would be sufficient for the reliable estimation of inter-observer agreement of TIL categorisation. This has not been explored previously in relation to TILs.
ONEST analyses, using an open-source software developed by the first author, were performed on TIL quantification in breast cancers taken from two previous studies. These were one reproducibility study involving 49 breast cancers, 23 in the first circulation and 14 pathologists in the second circulation, and one study involving 100 cases and 9 pathologists.
In addition to the estimates of the number of observers required, other factors influencing the results of ONEST were examined. The analyses reveal that between six and nine observers (range 2-11) are most commonly needed to give a robust estimate of reproducibility.
In addition, the number and experience of observers, the distribution of values around or away from the extremes, and outliers in the classification also influence the results. Due to the simplicity and the potentially relevant information it may give, we propose ONEST to be a part of new reproducibility analyses.