Cell segmentation in imaging-based spatial transcriptomics.
Viktor PetukhovRosalind J XuRuslan A SoldatovPaolo CadinuKonstantin KhodosevichJeffrey R MoffittPeter V KharchenkoPublished in: Nature biotechnology (2021)
Single-molecule spatial transcriptomics protocols based on in situ sequencing or multiplexed RNA fluorescent hybridization can reveal detailed tissue organization. However, distinguishing the boundaries of individual cells in such data is challenging and can hamper downstream analysis. Current methods generally approximate cells positions using nuclei stains. We describe a segmentation method, Baysor, that optimizes two-dimensional (2D) or three-dimensional (3D) cell boundaries considering joint likelihood of transcriptional composition and cell morphology. While Baysor can take into account segmentation based on co-stains, it can also perform segmentation based on the detected transcripts alone. To evaluate performance, we extend multiplexed error-robust fluorescence in situ hybridization (MERFISH) to incorporate immunostaining of cell boundaries. Using this and other benchmarks, we show that Baysor segmentation can, in some cases, nearly double the number of cells compared to existing tools while reducing segmentation artifacts. We demonstrate that Baysor performs well on data acquired using five different protocols, making it a useful general tool for analysis of imaging-based spatial transcriptomics.
Keyphrases
- single cell
- deep learning
- single molecule
- convolutional neural network
- induced apoptosis
- cell cycle arrest
- cell therapy
- high resolution
- magnetic resonance imaging
- gene expression
- magnetic resonance
- living cells
- computed tomography
- photodynamic therapy
- machine learning
- transcription factor
- big data
- electronic health record
- mesenchymal stem cells
- cell death
- atomic force microscopy
- fluorescence imaging