Reproducibility of Malignant Pleural Mesothelioma Histopathologic Subtyping: Methodologic Issue

Our plain-English summary of Reproducibility of Malignant Pleural Mesothelioma Histopathologic Subtyping: Methodologic Issue, by M. Naderi, S. Sabour (Archives of Pathology & Laboratory Medicine, 2018).

The study

• Authors: M. Naderi, S. Sabour
• Journal: Archives of Pathology & Laboratory Medicine
• Published: 2018;142(11):1311
• Type of article: Letter, Comment
• Cited by: 6 articles, as counted by NIH iCite
• Read it: PubMed · Publisher

Summary of the article

This letter to the editor commented on the statistical methods of a study by Brcic and colleagues on how reproducible the histologic subtyping of malignant pleural mesothelioma (MPM) is. That study had selected one hematoxylin-eosin-stained slide for each of 200 patients with MPM. Three pathologists independently classified the slides using the World Health Organization classification. The study reported agreement using the Cohen kappa statistic. Overall interobserver agreement was fair (kappa 0.36) and rose to substantial (kappa 0.63) in the second round. The letter writers argued that no kappa value is internationally accepted as a sign of good agreement. They said kappa has two weaknesses: it depends on the prevalence in each category, and it depends on the number of categories. To illustrate this, they gave hypothetical tables for two pathologists. In both tables, 90% of ratings agreed and 10% disagreed, yet the kappa values differed: 0.44 (moderate) when one category made up 90% of ratings and 0.80 (very good) when the two categories were equal. They suggested using weighted or Fleiss kappa, especially with more than two observers. They stated that the original conclusion, that moderate to substantial agreement in MPM typing and subtyping can be achieved, should be supported by addressing these statistical and methodologic issues.

PubMed has no abstract for this study, and the article doesn't print one. We wrote this summary from the full article with the help of AI. It is not the authors' wording.

In plain English

This was a letter to the editor, not a new study. The authors commented on a 2018 paper by Brcic and colleagues that measured how consistently pathologists sorted malignant pleural mesothelioma into histologic subtypes. In that paper, 3 pathologists reviewed one slide from each of 200 patients. The letter's authors argued that the agreement statistic used, kappa, has two weaknesses. First, it depends on how common each category is. In a hypothetical table of their own, two pathologists agreed 90% of the time in two situations, yet kappa was 0.44 (moderate) in one and 0.80 (very good) in the other. Second, kappa depends on the number of categories. They suggested weighted or Fleiss kappa and said Brcic's conclusion that moderate to substantial agreement can be achieved should be supported by these methodological points. In a reply, the original authors said their article contained an error: they had actually used Fleiss kappa, not Cohen kappa. They also said that for symmetrical tables Fleiss kappa equals Cohen kappa and so shares its dependence on prevalence. They called this an 'alleged weakness' and explained that it arises from correcting for chance agreement.

We wrote this plain-English summary from the full article with the help of AI, and a second AI model checked each sentence against the article. It is not the authors' wording; the abstract and the article are their own account.

What the study did and found

• Design: Letter to the editor (methodological commentary on a published study), followed by the beginning of a reply from the original study's authors; no new data, only a hypothetical example table
• Who was studied: No patients were studied by the letter's authors. The letter discusses the study by Brcic and colleagues, in which one slide was selected for each of 200 patients with malignant pleural mesothelioma and reviewed independently by 3 pathologists.
• Main results:
  • The letter's authors stated that the kappa value depends on the prevalence in each category.
  • In the letter authors' own hypothetical table, agreement was 90% and disagreement 10% in both situations, but kappa was 0.44 (moderate) in situation (a) and 0.80 (very good) in situation (b).
  • The letter's authors stated that kappa also depends on the number of categories.
  • The letter's authors suggested using weighted or Fleiss kappa, especially with more than 2 observers.
  • In their reply, the original study's authors corrected their Materials and Methods section: they had used Fleiss kappa, not Cohen kappa as stated.
  • In their reply, the original study's authors explained that kappa's dependence on prevalence arises from correction for chance agreement.
  • In their reply, the original study's authors stated that for symmetrical tables Fleiss kappa is the same as Cohen kappa, so it is affected by the same dependence on prevalence, which they called an 'alleged weakness'.
• The authors' conclusion: The letter's authors concluded that Brcic and colleagues' conclusion, that moderate to substantial agreement in typing and subtyping of malignant pleural mesothelioma can be achieved, should be supported by the statistical and methodological issues they raised.

Topics

The National Library of Medicine indexes this study under subjects our site covers: Lung cancer, Mesothelioma.

Medical terms in this study, explained in our Lexicon: Pleural Mesothelioma.

Related studies

• Reproducibility of Malignant Pleural Mesothelioma Histopathologic Subtyping — L. Brcic, G. Vlacic, et al. (2018)
• Reproducibility of histological subtyping of malignant pleural mesothelioma — L. Brčić, M. Jakopović, et al. (2014)
• Interobserver Agreement in Histopathological Subtyping of Malignant Pleural Mesotheliomas — M. Mlika, F. Mezni (2021)
• New Insights on Diagnostic Reproducibility of Biphasic Mesotheliomas: A Multi-Institutional Evaluation by the International Mesothelioma Panel From the MESOPATH Reference Center — F. Galateau Salle, N. Le Stang, et al. (2018)
• The histopathological diagnosis of malignant mesothelioma v. pulmonary adenocarcinoma: reproducibility of the histopathological diagnosis — B.G. Skov, A.F. Lauritzen, et al. (1994)
• Accuracy and reproducibility of pleural effusion cytology — A. Fassina, U. Fedeli, et al. (2008)

Browse more research in Archives of Pathology & Laboratory Medicine, or return to medical research.