Researchers at Harvard Medical School have unveiled a new AI model called CHIEF (Clinical Histopathology Imaging Evaluation Foundation) that can diagnose and predict outcomes for multiple cancer types with remarkable accuracy.

According to the study, CHIEF outperforms existing AI systems, achieving up to “96% accuracy” in cancer detection across 19 different cancer types. The researchers liken the versatility of CHIEF to that of ChatGPT, the language model that has captured widespread attention for its ability to tackle a wide range of tasks.

CHIEF is basically a very specialized AI Vision model—a model capable of understanding visual inputs—trained to be super detailed in images of cancer cells instead of the generalist approach we see in traditional models like GPT-4V or LlaVA.

So instead of being trained to recognize general elements like “cats” or “oranges,” CHIEF was trained on a massive multimodal dataset, including 15 million unlabeled images and 60,000 whole-slide images of tissues from 19 different anatomical sites.

“Through pretraining on 44 terabytes of high-resolution pathology imaging datasets, CHIEF extracted microscopic representations useful for cancer cell detection, tumor origin identification, molecular profile characterization and prognostic prediction,” the research reads.

The approach seems to work better than expected. “Our ambition was to create a nimble, versatile ChatGPT-like AI platform that can perform a broad range of cancer evaluation tasks,” said study senior author Kun-Hsing Yu. “Our model turned out to be very useful across multiple tasks related to cancer detection, prognosis, and treatment response across multiple cancers.”

The researchers tested CHIEF on over 19,400 images from 32 independent datasets collected globally, and it outperformed state-of-the-art AI methods by up to 36.1% across these tasks. It was also more accurate at separating patients with high and low survival rates, and was able to provide accurate insights of different analyzed tissue samples.

The researchers plan to further refine CHIEF by training it on images of rare diseases, non-cancerous conditions, and pre-malignant tissues to improve its accuracy. They also expect to feed the model with more data so it gets better at identifying cancer aggressiveness and predicting the effects of novel treatments.

AI’s expanding role in cancer detection and beyond

Researchers have been leveraging AI to advance the detection, diagnosis, and treatment of cancer and other diseases for a while now.

For example, researchers from Cambridge introduced EMethylNET, an AI model that uses DNA data from tissue samples to detect 13 types of cancer with a 98% accuracy rate. Trained on over 6,000 tissue samples, EMethylNET highlights the potential of AI in identifying cancer early through DNA methylation, which plays a crucial role in cancer growth.

Another earlier model named CancerGPT (we’re not making that name up) used a large language model to predict how drug combinations may affect rare tissues in cancer patients. It demonstrated that pre-trained models can be invaluable when structured data and samples are scarce. CancerGPT is able to generalize predictions and leverage prior medical research to offer significant insights, although researchers were still concerned about potential AI hallucinations.

Google and iCAD also teamed up to use AI to enhance cancer screening. Their AI-powered system outperformed expert radiologists in accuracy, offering a viable solution amid a global shortage of radiologists and improving accessibility to life-saving breast cancer screenings.

And finally, another AI technology named Sturgeon is being used by brain surgeons to assist in diagnosing central nervous system tumors in real-time with 90% accuracy.

CHIEF is open source and available to download on the project’s Github page so researchers (or anyone else) can input their own images and run it locally.