In a groundbreaking initiative, Smart Lab at The Hong Kong University of Science and Technology (HKUST) has unveiled four AI-driven models poised to transform the medical and healthcare fields. These AI models can assist both general and specialist medical practitioners in diagnosing and prognosing up to 30 types of cancers and diseases, with some achieving accuracy comparable to that of medical experts with five years of experience or more.

Supported by the University’s new AI supercomputing facility, which offers robust computing power, these large AI models surpass many existing systems due to their foundation on extensive data sets and novel machine training strategies. Prof. Hao Chen, Assistant Professor in the Department of Computer Science and Engineering and project lead, said that one model processed over 160 million images across 32 cancer types for pathological diagnostic tasks.

The four models include:

• MOME (Breast Cancer Diagnosis) - This is the first adaptation of a large foundation model for 3D mpMRI fusion. Targeting breast cancer, one of the most prevalent cancers among females in Hong Kong, MOME distinguishes malignant from benign breast lumps using MRI scans, achieving accuracy levels comparable to radiologists with 5 or more years of experience. It provides an alternative to the more intrusive diagnostic form of biopsies. Additionally, it predicts patients’ responses to neoadjuvant chemotherapy, enabling non-invasive and personalized management of breast cancer.

• mSTAR (Pathology Assistant Tool) - Pathological examination is the gold standard in cancer diagnosis, but generating pathology reports can be time-consuming and prone to errors. mSTAR, recognized as one of the world’s leading foundation models, aims to streamline this process. Unlike existing ones that analyze slides by splitting them into individual patches, mSTAR stands out by directly modelling whole slide images and augmenting with multimodal knowledge, further enhancing accuracy. It assists pathologists in performing up to 40 diagnostic and prognostic tasks, significantly reducing examination time and improving diagnostic consistency.

• MedDr (Generalist) - This versatile AI medical generalist functions like a “medical GPT”. The multimodal language model can answer questions, generate medical reports, and provide initial diagnoses based on medical images. As one of the largest open-source generalist foundation models tailored for medicine, MedDr helps physicians make faster and more accurate diagnoses. A recent study by Shanghai AI Lab recognized it as one of the best existing generalist models.

• XAIM (Explainable AI) - This novel framework enhances healthcare professionals’ confidence in adopting AI by explaining how medical AI models reach their decisions. While many AI models offer high accuracy, they often lack transparency, leading to skepticism. XAIM addresses this by providing visual and textual explanations of the models’ analyses.

Prof. Chen, also the Director of the HKUST Smart Lab, stated, “We hope our AI tools will serve as valuable assistants for doctors, improving diagnostics, personalizing treatment plans, and streamlining different tasks. Building on our success, we are developing a wide range of AI applications targeting various clinical tasks, particularly precision oncology and computer-assisted intervention, while continuously enhancing our models with more data and modalities. By collaborating closely with clinical partners, we aim to drive improvements in patient care over the long term.”

About Smart Lab

Smart Lab, led by Prof. Hao Chen, is committed to pushing the boundaries of trustworthy AI technologies for healthcare and science. Our research directions include large-scale models for healthcare, computer-assisted intervention, AI for science, and bioinformatics, etc. Our ultimate goal is to spearhead a transformative revolution in medical practices and scientific discoveries, paving the way for a brighter and healthier future.

About The Hong Kong University of Science and Technology

The Hong Kong University of Science and Technology (HKUST) is a world-class university that excels in driving innovative education, research excellence, and impactful knowledge transfer. With a holistic and interdisciplinary pedagogy approach, HKUST was ranked 3rd in the Times Higher Education’s Young University Rankings 2024, while 12 of its subjects were ranked among the world’s top 50 in the QS World University Rankings by Subject 2024, with “Data Science and Artificial Intelligence” being ranked first in Hong Kong and 10th in the world. Our graduates are highly competitive, consistently ranking among the world’s top 30 most sought-after employees. In terms of research and entrepreneurship, over 80% of our work was rated “internationally excellent” or “world leading” in the latest Research Assessment Exercise 2020 of Hong Kong’s University Grants Committee. As of October 2024, HKUST members have founded over 1,800 active start-ups, including 10 Unicorns and 14 exits (IPO or M&A).

Media Links

RTHK: Link (Traditional Chinese only)

Mingpao: Link (Traditional Chinese only)

TVB News: Link (Traditional Chinese only)

Hong Kong Economic Journal: Link (Traditional Chinese only)