Pretraining a foundation model for generalizable fluorescence microscopy-based image restoration.
Chenxi MaWeimin TanRuian HeBo YanPublished in: Nature methods (2024)
Fluorescence microscopy-based image restoration has received widespread attention in the life sciences and has led to significant progress, benefiting from deep learning technology. However, most current task-specific methods have limited generalizability to different fluorescence microscopy-based image restoration problems. Here, we seek to improve generalizability and explore the potential of applying a pretrained foundation model to fluorescence microscopy-based image restoration. We provide a universal fluorescence microscopy-based image restoration (UniFMIR) model to address different restoration problems, and show that UniFMIR offers higher image restoration precision, better generalization and increased versatility. Demonstrations on five tasks and 14 datasets covering a wide range of microscopy imaging modalities and biological samples demonstrate that the pretrained UniFMIR can effectively transfer knowledge to a specific situation via fine-tuning, uncover clear nanoscale biomolecular structures and facilitate high-quality imaging. This work has the potential to inspire and trigger new research highlights for fluorescence microscopy-based image restoration.
Keyphrases
- single molecule
- deep learning
- high resolution
- atomic force microscopy
- high speed
- optical coherence tomography
- high throughput
- mental health
- artificial intelligence
- healthcare
- convolutional neural network
- energy transfer
- machine learning
- mass spectrometry
- single cell
- risk assessment
- fluorescence imaging
- rna seq
- electron transfer