Comprehensive Summary
Morano et. al. proposed MIRAGE, the first multimodal retinal image analysis system, trained to segment and analyze both optical coherence tomography (OCT) and scanning laser ophthalmoscopy (SLO) retinal images. The model was trained on both OCT and SLO images simultaneously, and outperformed previous unimodal foundation models and models that use different analysis models for each imaging mode. Morano et. al. compared the quality of MIRAGE’s generalization and image segmentation abilities to current clinically used FMs with both generalized and specific applications. They found that MIRAGE performance was generally superior in both aspects, with a slight disadvantage in specific applications where the previously developed model was curated for a highly specific function in mind. Overall, it was found that MIRAGE’s multimodal training using two types of images greatly enhanced image segmentation quality and diagnosis of general, well-known retinal conditions. Future work remains to expand the training dataset of MIRAGE to different data centers, include 3D OCT images, and to improve its specificity for segmenting more complex images.
Outcomes and Implications
MIRAGE serves as the first multimodal foundation model for retinal image analysis that has promising potential utility in research and clinical settings through evaluation in clinical applications. Additionally, Morano et. al.’s thorough assessment of the capability of different types of FMs on OCT images serves as a reference point for future progress tracking of FM development. With an implementation of a wider range of training data, MIRAGE may become one of the highest quality retinal image analysis models, which could greatly improve workflow in clinical practice.