CUHK and Beijing Tongren Hospital develop the new foundational AI model to transform ophthalmic care worldwide

Date: 
2025-01-10
Thumbnail: 
Body: 

The research team led by Professor Scott Yuan Wu, Assistant Professor of Department of Biomedical Engineering at The Chinese University of Hong Kong (CUHK), in collaboration with Professor Wang Ningli’s team at Beijing Tongren Hospital, has developed an artificial intelligence (AI) ophthalmic imaging foundation model called VisionFM. This model enables the prediction of the presence of tumors from retinal images for the first time, representing a breakthrough in ophthalmic disease diagnosis. The research foundings have been published in NEJM AI, a publication of the New England Journal of Medicine Group.

Vision impairment and its associated major eye diseases, such as cataracts, age-related macular degeneration and glaucoma, have become a pressing global health problem. It is estimated that by 2050, about 474 million people may suffer from moderate to severe vision impairment. The situation is particularly acute in low-income countries where there is a shortage of specialised ophthalmologists. Despite the rapid advancements of AI in automated ophthalmic diagnosis, there are still significant limitations to existing models. They often rely on vast amounts of labelled data, which is relatively time-consuming and costly to collect. In addition, many existing models target on a single or limited number of eye diseases and utilise only one imaging modality, such as fundus photographs, preventing their broader application in clinical diagnosis.

To address these challenges, the CUHK and Beijing Tongren Hospital research teams developed VisionFM, a groundbreaking AI ophthalmic imaging foundation model. VisionFM was pre-trained on the world’s largest ophthalmic data cohort containing 3.4 million images from eight different ophthalmic modalities, covering a wide range of ophthalmic diseases, imaging modalities and devices, as well as clinical scenarios. This innovative model has been tested for multiple applications, including ophthalmic disease diagnosis, progression prediction, systemic biomarker prediction through ocular imaging, intracranial tumor prediction, and lesion, vessel, layer segmentation.

VisionFM outperforms existing models in ophthalmic disease diagnosis, achieving diagnostic accuracy comparable to an ophthalmologist with 4-8 years of clinical experience. Additionally, VisionFM demonstrates remarkable few-shot learning capabilities in diagnosis and anatomical segmentation, enabling it to adapt to imaging modalities and devices not encountered in the pre-training phase, or require only a small  number of gold-standard samples for rapid fine-tuning. The model also reveals for the first time the association between intracranial tumors and retinal images, enabling the prediction of tumors directly from low-cost retinal images. This holds great potential for the early detection in community and primary care. The model has been deployed to diagnose common eye diseases in Henan Province.

Furthermore, the research result indicates that high-quality synthetic ophthalmic data, which passed the Turing test, can significantly enhance the pre-training of foundation models like VisionFM, further improving their efficacy. With its open-source codebase and model, it is poised to address the global challenges and improve patient outcomes through advanced AI technology.  

The full research paper can be found at: https://ai.nejm.org/doi/full/10.1056/AIoa2300221

 

About NEJM AI and the research article

About NEJM AI

NEJM AI is a new journal launched by the New England Journal of Medicine (NEJM) in 2024, focusing on groundbreaking research in artificial intelligence in the medical field. As the newest member of the most influential family of journals in the medical community, NEJM AI upholds NEJM's rigorous academic standards and is dedicated to publishing AI innovations that can truly transform medical practice. The launch of this journal marks the recognition of the importance of artificial intelligence in the field of medicine by the mainstream medical community.

Research Article

 “ Development and Validation of a Multimodal Multitask Vision Foundation Model for Generalist Ophthalmic Artificial Intelligence” by Professor Scott Yuan Wu’s team at CUHK in collaboration with Professor Wang Ningli’s team at Beijing Tongren Hospital

Authors:Qiu Jianing†, Wu Jian†, Wei Hao†, Shi Peilun, Zhang Minqing, Sun Yunyun, Li Lin, Liu Hanruo, Liu Hongyi, Hou Simeng, Zhao Yuyang, Shi Xuehui, Xian Junfang, Qu Xiaoxia, Zhu Sirui, Pan Lijie, Chen Xiaoniao, Zhang Xiaojia, Jiang Shuai, Wang Kebing, Yang Chenlong, Chen Mingqiang, Fan Sujie, Hu Jianhua, Lv Aiguo, Miao Hui, Guo Li, Zhang Shujun, Pei Cheng, Fan Xiaojuan, Lei Jianqin, Wei Ting, Duan Junguo, Liu Chun, Xia Xiaobo, Xiong Siqi, Li Junhong, Lam Kyle, Lo Benny, Tham Yih-chung, Wong Tien-yin, Wang Ningli ∗, Yuan Wu ∗

† Co-first;∗ Co-corresponding

 

VisionFM was developed by the research team from CUHK, led by Professor Scott Yuan Wu at the Department of Biomedical Engineering, in collaboration with Beijing Tongren Hospital. 


VisionFM is able to predict 38 common biomarkers related to complete blood count, liver and renal functions, as well as serum glucose and lipids, directly from eye images. It can also predict the presence of intracranial tumours directly from fundus photographs.

VisionFM is a generalist ophthalmic image foundation model. It was developed using 3.4 million ophthalmic images from over half a million of individuals. VisionFM can process eight common ophthalmic image modalities. It can diagnose multiple eye diseases, forecast the progression of diseases, predict systemic biomarkers and intracranial tumours from eye images, detect anatomical landmarks, and segment vessels, layers, and lesions..

 

Filter: Dept: 
Faculty
BME
Media Release
Name: 
LI Shaohua
Title ( post ): 
Assistant Professor
Department: 
Computer Science and Engineering
email: 
shaohuali@cse.cuhk.edu.hk
phone: 
3943 8418
website: 
https://www.cse.cuhk.edu.hk/people/faculty/shaohuali/
Avatar: 
Class: 
faculty_member
glossary_index: 
L
Name: 
LU Songtao
Title ( post ): 
Assistant Professor
Department: 
Computer Science and Engineering
email: 
3943 8434
phone: 
stlu@cse.cuhk.edu.hk
website: 
https://www.cse.cuhk.edu.hk/people/faculty/songtao-lu/
Avatar: 
Class: 
faculty_member
Chinese Name: 
盧松濤
glossary_index: 
L
Name: 
WANG Dongan
Title ( post ): 
Professor
Department: 
Biomedical Engineering
email: 
donganwang@cuhk.edu.hk
phone: 
3943 5404
website: 
https://www.bme.cuhk.edu.hk/new/dongan.php
Avatar: 
Class: 
faculty_member
glossary_index: 
W

Faculty of Engineering received visit of mentees from the Strive and Rise Programme Alumni Club

Date: 
2025-01-02
Thumbnail: 
Body: 

Led by the Chief Secretary for Administration, the Strive and Rise Programme (「共 創明『Teen』計劃」) has been taken forward by an inter-departmental Task Force through tripartite collaboration between the Government, the business sector and the community.  The Strive and Rise Alumni Club (共創明 teen 校友會) provides a platform for graduating mentees to broaden their horizons through activities and internship programs continuously.

Organized by Tencent, being one of the supporting organization, CUHK Faculty of Engineering had been receiving over 80 secondary school students from the Strive and Rise Alumni Club on a programming event held on 27 December 2024 (Friday). This series of programme aim to provide opportunity for participating students to understand recent development of the technology industry and inspire their life planning through different elements such as technology learning, university life experience, and career enlightenment.

 

Professor WONG Kam Fai, MH, Associate Dean (External Affairs) of the Faculty of Engineering introducing the Faculty and development of Engineering field to the participants.

 

Filter: Dept: 
Faculty
Media Release

Two CUHK Engineering Professors elected IEEE Fellows for 2025

Date: 
2024-12-27
Thumbnail: 
Body: 

Two professors from The Chinese University of Hong Kong (CUHK)’s Faculty of Engineering have been elected Fellows of the prestigious Institute of Electrical and Electronics Engineers (IEEE) in the class of 2025. They are Professor CHAN Yuen Yan Rosanna from Department of Information Engineering and Professor PAN Sinno Jialin from Department of Computer Science and Engineering. 

Professor Chan has been elevated to IEEE Fellow for her contributions to learning technologies for special education needs and social inclusion. Professor Chan’s research focuses on integrating innovative technologies into educational practices, enhancing learning experiences for individuals with special needs. Her dedication to promoting social inclusion through technology has established her as a leader in the field, inspiring both educators and students alike.

Professor Pan was elected for his contributions to transfer learning methodologies. Professor Pan is a Professor and Global STEM Scholar with the Department of Computer Science and Engineering at the Chinese University of Hong Kong (CUHK) in Hong Kong. He is also the Director of the JC STEM Lab of Machine Learning and Symbolic Reasoning. Prof. Pan’s research interests focus on machine learning and artificial intelligence, especially transfer learning and its applications.

About IEEE

The IEEE is the world’s largest professional organisation dedicated to advancing technological innovation and excellence for the benefit of humanity, with more than 430,000 members in over 160 countries. IEEE Fellow is the highest grade of membership and is recognised by the technical community as a prestigious honour and an important career achievement.

Professor CHAN Yuen Yan Rosanna from Department of Information Engineering

Professor PAN Sinno Jialin from Department of Computer Science and Engineering.

 

Filter: Dept: 
Faculty
EE
IE
Media Release

2024 CUHK–Mainland Optics & Photonics Workshop successfully held

Date: 
2024-12-20
Thumbnail: 
Body: 

2024 CUHK–Mainland Optics & Photonics Workshop successfully took place in Guangming, Shenzhen during 13-15 December 2024. The two and half day in-person workshop provided a free platform for academic and industrial interactions, joined by over 20 distinguished scholars from Tsinghua University, Peking University, Zhejiang University, Huazhong University of Science and Technology, Shenzhen University, CUHK, and other renowned universities/institutions in the mainland. The workshop was also attended by many CUHK alumni in academia and industry and current students and postdocs, guests from mainland institutions and industries. Compared with last year’s event, this time we featured a mini-industry exhibition, showcasing high-tech products by startups from CUHK and mainland, which also sponsored the workshop.

The workshop further fostered the existing collaborations between CUHK and mainland partners, while exploring new frontiers in research innovations and new initiatives in cultivating the next generation of leaders in the field of optics and photonics. In addition, we promoted collaboration with the Greater Bay Area and brought relevant officials from Guangming district and Prof. Jianbin Xu (Associate Dean for Mainland Affairs) to our event. The workshop was coordinated by Prof. Renjie Zhou with support from Prof. Xiankai Sun, Prof. Wei Ren, Prof. Chaoran Huang, Prof. Scott Yuan, and many senior colleagues in the Faculty of Engineering.   

Workshop group photo with invited speakers

Mini-industry exhibition. 

 

Filter: Dept: 
Faculty
Media Release

Pages