Google’s working on an updated version of its medical A.I. that can answer health questions

Google on Tuesday announced new health initiatives and partnerships at its annual event called “The Check Up.” The Google Health team shared updates about features coming to search, tools for building health apps and the latest in artificial intelligence-powered health research.

During the event, the company discussed new partnerships to help develop AI-assisted ultrasounds, cancer treatment and tuberculosis screenings, but there was a particular buzz around the latest version of its medical large language model called Med-PaLM.

Google first introduced Med-PaLM late last year. It’s designed to provide high-quality responses to medical questions. Med-PaLM was the first AI system to successfully receive a passing score, or more than a 60%, on multiple choice style questions similar to the ones used in U.S. medical licensing exams.

The company said the second iteration of the technology, Med-PaLM 2, consistently performed at an “expert” level on medical exam questions. Med-PaLM 2 has reached more than 85% accuracy, scoring 18% higher than its earlier results.

Dr. Alan Karthikesalingam, a research lead at Google Health, said the company is also testing Med-PaLM’s answers against responses from real doctors and clinicians. He said Med-PaLM’s responses are evaluated for factual accuracy, bias, and potential for harm.

Karthikesalingam showed controlled examples — the demo wasn’t live — of how Med-PaLM 2 might answer questions like “what are the first warning signs of pneumonia?” and “can incontinence be cured?” In some cases, Med-PaLM 2’s answers were on a par, and even more detailed, than the answers that clinicians had provided. But in other cases, Med-PaLM 2’s responses were not as accurate.

“You can see from this sort of work that we’re still learning,” Karthikesalingam said during the event.

Given the sensitive nature of medical information, Karthikesalingam said it could be a while before this technology is at the fingertips of the average consumer. He said it is important to innovate responsibly and in a controlled environment.

Google will continue to work with researchers and experts on Med-PaLM, and Karthikesalingam said the company will share more updates about it in the future.

“The potential here is tremendous,” he said, “but it’s crucial that real-world applications are explored in a responsible and ethical manner.”