Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images

Faisal Mahmood, Daniel Borders, Richard J. Chen, Gregory N. Mckay, Kevan J. Salimian, Alexander Baras, Nicholas J. Durr

Research output: Contribution to journalArticlepeer-review

22 Scopus citations


Nuclei segmentation is a fundamental task for various computational pathology applications including nuclei morphology analysis, cell type classification, and cancer grading. Deep learning has emerged as a powerful approach to segmenting nuclei but the accuracy of convolutional neural networks (CNNs) depends on the volume and the quality of labeled histopathology data for training. In particular, conventional CNN-based approaches lack structured prediction capabilities, which are required to distinguish overlapping and clumped nuclei. Here, we present an approach to nuclei segmentation that overcomes these challenges by utilizing a conditional generative adversarial network (cGAN) trainedwith syntheticand real data. We generate a large dataset of H&E training images with perfect nuclei segmentation labels using an unpaired GAN framework. This synthetic data along with real histopathology data from six different organs are used to train a conditional GAN with spectral normalization and gradient penalty for nucleisegmentation. This adversarial regression framework enforces higher-order spacial-consistency when compared to conventional CNN models. We demonstrate that this nuclei segmentation approach generalizes across different organs, sites, patients and disease states, and outperforms conventional approaches, especially in isolating individual and overlapping nuclei.

Original languageEnglish (US)
Pages (from-to)3257-3267
Number of pages11
JournalIEEE transactions on medical imaging
Issue number11
StatePublished - Nov 2020


  • Nuclei segmentation
  • adversarial training
  • computational pathology
  • deep learning
  • histopathology segmentation
  • synthetic data
  • synthetic pathology data

ASJC Scopus subject areas

  • Software
  • Radiological and Ultrasound Technology
  • Electrical and Electronic Engineering
  • Computer Science Applications


Dive into the research topics of 'Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images'. Together they form a unique fingerprint.

Cite this