Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images

The Cancer Genome Atlas Research Network

Research output: Contribution to journalArticlepeer-review

217 Scopus citations


Beyond sample curation and basic pathologic characterization, the digitized H&E-stained images of TCGA samples remain underutilized. To highlight this resource, we present mappings of tumor-infiltrating lymphocytes (TILs) based on H&E images from 13 TCGA tumor types. These TIL maps are derived through computational staining using a convolutional neural network trained to classify patches of images. Affinity propagation revealed local spatial structure in TIL patterns and correlation with overall survival. TIL map structural patterns were grouped using standard histopathological parameters. These patterns are enriched in particular T cell subpopulations derived from molecular measures. TIL densities and spatial structure were differentially enriched among tumor types, immune subtypes, and tumor molecular subtypes, implying that spatial infiltrate state could reflect particular tumor cell aberration states. Obtaining spatial lymphocytic patterns linked to the rich genomic characterization of TCGA samples demonstrates one use for the TCGA image archives with insights into the tumor-immune microenvironment. Tumor-infiltrating lymphocytes (TILs) were identified from standard pathology cancer images by a deep-learning-derived “computational stain” developed by Saltz et al. They processed 5,202 digital images from 13 cancer types. Resulting TIL maps were correlated with TCGA molecular data, relating TIL content to survival, tumor subtypes, and immune profiles.

Original languageEnglish (US)
Pages (from-to)181-193.e7
JournalCell Reports
Issue number1
StatePublished - Apr 3 2018


  • artificial intelligence
  • bioinformatics
  • computer vision
  • deep learning
  • digital pathology
  • immuno-oncology
  • lymphocytes
  • machine learning
  • tumor microenvironment
  • tumor-infiltrating lymphocytes

ASJC Scopus subject areas

  • General Biochemistry, Genetics and Molecular Biology


Dive into the research topics of 'Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images'. Together they form a unique fingerprint.

Cite this