The research team led by Professor Scott Yuan Wu, Assistant Professor of Department of Biomedical Engineering at The Chinese University of Hong Kong (CUHK), in collaboration with Professor Wang Ningli’s team at Beijing Tongren Hospital, has developed an artificial intelligence (AI) ophthalmic imaging foundation model called VisionFM. This model enables the prediction of the presence of tumors from retinal images for the first time, representing a breakthrough in ophthalmic disease diagnosis. The research foundings have been published in NEJM AI, a publication of the New England Journal of Medicine Group.
Vision impairment and its associated major eye diseases, such as cataracts, age-related macular degeneration and glaucoma, have become a pressing global health problem. It is estimated that by 2050, about 474 million people may suffer from moderate to severe vision impairment. The situation is particularly acute in low-income countries where there is a shortage of specialised ophthalmologists. Despite the rapid advancements of AI in automated ophthalmic diagnosis, there are still significant limitations to existing models. They often rely on vast amounts of labelled data, which is relatively time-consuming and costly to collect. In addition, many existing models target on a single or limited number of eye diseases and utilise only one imaging modality, such as fundus photographs, preventing their broader application in clinical diagnosis.
To address these challenges, the CUHK and Beijing Tongren Hospital research teams developed VisionFM, a groundbreaking AI ophthalmic imaging foundation model. VisionFM was pre-trained on the world’s largest ophthalmic data cohort containing 3.4 million images from eight different ophthalmic modalities, covering a wide range of ophthalmic diseases, imaging modalities and devices, as well as clinical scenarios. This innovative model has been tested for multiple applications, including ophthalmic disease diagnosis, progression prediction, systemic biomarker prediction through ocular imaging, intracranial tumor prediction, and lesion, vessel, layer segmentation.
VisionFM outperforms existing models in ophthalmic disease diagnosis, achieving diagnostic accuracy comparable to an ophthalmologist with 4-8 years of clinical experience. Additionally, VisionFM demonstrates remarkable few-shot learning capabilities in diagnosis and anatomical segmentation, enabling it to adapt to imaging modalities and devices not encountered in the pre-training phase, or require only a small number of gold-standard samples for rapid fine-tuning. The model also reveals for the first time the association between intracranial tumors and retinal images, enabling the prediction of tumors directly from low-cost retinal images. This holds great potential for the early detection in community and primary care. The model has been deployed to diagnose common eye diseases in Henan Province.
Furthermore, the research result indicates that high-quality synthetic ophthalmic data, which passed the Turing test, can significantly enhance the pre-training of foundation models like VisionFM, further improving their efficacy. With its open-source codebase and model, it is poised to address the global challenges and improve patient outcomes through advanced AI technology.
The full research paper can be found at: https://ai.nejm.org/doi/full/10.1056/AIoa2300221
About NEJM AI and the research article
About NEJM AI
NEJM AI is a new journal launched by the New England Journal of Medicine (NEJM) in 2024, focusing on groundbreaking research in artificial intelligence in the medical field. As the newest member of the most influential family of journals in the medical community, NEJM AI upholds NEJM's rigorous academic standards and is dedicated to publishing AI innovations that can truly transform medical practice. The launch of this journal marks the recognition of the importance of artificial intelligence in the field of medicine by the mainstream medical community.
Research Article
“ Development and Validation of a Multimodal Multitask Vision Foundation Model for Generalist Ophthalmic Artificial Intelligence” by Professor Scott Yuan Wu’s team at CUHK in collaboration with Professor Wang Ningli’s team at Beijing Tongren Hospital
Authors：Qiu Jianing†, Wu Jian†, Wei Hao†, Shi Peilun, Zhang Minqing, Sun Yunyun, Li Lin, Liu Hanruo, Liu Hongyi, Hou Simeng, Zhao Yuyang, Shi Xuehui, Xian Junfang, Qu Xiaoxia, Zhu Sirui, Pan Lijie, Chen Xiaoniao, Zhang Xiaojia, Jiang Shuai, Wang Kebing, Yang Chenlong, Chen Mingqiang, Fan Sujie, Hu Jianhua, Lv Aiguo, Miao Hui, Guo Li, Zhang Shujun, Pei Cheng, Fan Xiaojuan, Lei Jianqin, Wei Ting, Duan Junguo, Liu Chun, Xia Xiaobo, Xiong Siqi, Li Junhong, Lam Kyle, Lo Benny, Tham Yih-chung, Wong Tien-yin, Wang Ningli ∗, Yuan Wu ∗
† Co-first；∗ Co-corresponding
