In December, last year, when OpenAI’s ChatGPT was struggling to find real use cases, Google decided to explore the use of large language models (LLMs) for Healthcare, resulting in the creation of Med-PaLM —an open-sourced large language model designed for Medical purposes.

Since then, the team has released scaled-up versions of healthcare LLMs, including Med-PaLM-2 and Med-PaLM-M, both of which have had a direct impact on human lives. Currently, Med-PaLM-2 is also undergoing testing at renowned healthcare institutions such as the Mayo Clinic. One of the prominent contributors to these projects is Vivek Natarajan, an AI researcher at Google Health.

Currently, based in the San Francisco Bay Area, the Tamilian with deep Bengali roots, began his journey as an engineering intern at Qualcomm, progressing to a role with Meta AI, and ultimately finding a fulfilling place at Google Health.

However, there is a story behind why he chose to transition into the field of medical AI.

How it All Began

It is 2023, and India’s healthcare system still faces significant hurdles with insufficient medical infrastructure and a severe shortage of medical professionals, especially in rural regions. The ratio of doctors to patients falls well below global standards, with a mere 0.7 doctors per 1,000 people. Adding to that, we have only 0.9 beds per 1,000 population, and out of those, only 30% are in rural areas.

Most had to walk tens of kilometres, often in extreme conditions, leading to delayed diagnoses, poorly managed chronic conditions, and even untimely deaths. This healthcare disparity affected both the underprivileged and affluent individuals, underscoring the stark healthcare inequalities in these areas.

Having grown up in different parts of India, Natarajan witnessed these immense challenges faced by people in small towns and villages when it came to accessing medical care. “It always bothered me that people should not have to suffer so much to receive basic healthcare, and I always wanted to do something about it,” Natarajan told AIM in an exclusive interaction.

From starting out by building ‘Ask the Doctor, Anytime Anywhere’, an app aimed at democratizing healthcare access in 2013 to being the research lead behind Google’s state-of-the-art LLM for medicine, Med-PaLM 2, Natarajan has come a long way. “I guess the name gives away what we were trying to do. Ask the Doctor was bootstrapped using older AI techniques and a lot of rules, and it clearly did not work well, leading to its discontinuation,” he said.

The app was made by leveraging pre-deep learning ML techniques — a combination of expert systems and rules. However, even back in 2013, he had this intuition that AI would be the most important piece of solving this healthcare problem.

How Google Happened

After completing a bachelor’s degree at NIT Trichy in Electronics Engineering and graduating with a master’s degree in Computer Science from UT Austin in 2015, Natarajan joined Meta AI. Despite being in the pre-transformer era, Natarajan’s time at Meta AI, which was his first job, taught him the potential of deep learning. At Meta, he worked in various areas, from speech recognition to conversational and multimodal AI, and on various business-critical platforms such as Newsfeed and Messenger.

However things took a different turn. Unfortunately, it was during this period that his father began showing signs of an aggressive form of Parkinson’s disease, which couldn’t have been identified sooner due to the limited care options and resources. “That persuaded me to go back to the problem that I always deeply cared about — using AI to democratise access to healthcare and put world-class medical expertise in the pocket of billions,” said Natarajan.

Coincidentally, this was also the time when researchers from Google Brain and DeepMind (now referred to as Google DeepMind), after some seminal medical AI papers, were coming together to form Google Health AI, aligning with his aim. “So when Greg Corrado, co-founder of Google Brain and head of Google Health AI, offered me the chance to join, I took it up without hesitation,” he added.

Since then, he has collaborated with esteemed AI researchers like Greg and Dr Alan Karthikesalingam to work toward the vision of making an AI doctor accessible to billions.

Behind the Making of Med-PaLM

If not an AI researcher, Natarajan would have probably been a cricket commenter like Harsha Bhogle. Well, let’s take a moment to appreciate that he didn’t embark on that career, otherwise, we might have missed out on his stellar work in building Med-PaLM, Med-PaLM 2, Med-PaLM M, and related projects. 

The core concept driving the development of Med-PaLM is the utilisation of general-purpose language models like PaLM and GPT-4, which excel in predicting text but lack specialised medical knowledge. However, the challenge lies in transforming these models into medical experts. “So, we need to do the same with AI and ‘send them to medical school’ if we want to use them for medical applications. Make them learn from high-quality medical domain information spanning human biology to practice of medicine as well as from clinical expert demonstrations and feedback — similar to residency after medical school,” he added. 

However, the primary obstacle was the scarcity of large-scale medical datasets due to privacy concerns and healthcare in the global south not being digital. Additionally, there’s a pressing concern about bias in LLMs used in healthcare. These cultural, social, racial, and gender biases can result in unequal access to care, misdiagnoses, and treatment disparities. The root of this problem lies in the reliance of healthcare LLMs on extensive datasets that mirror historical healthcare inequities, potentially leading to inaccurate diagnoses and treatment recommendations for marginalised communities.

The Med-PaLM models, derived from the PaLM general-purpose language models, are tailored for medical applications through fine-tuning with high-quality medical datasets and clinical expert demonstrations, covering areas like professional medical exams, PubMed research, and user-generated medical questions. These datasets, including the openly available HealthSearchQA dataset from Google, are instrumental in the development of Med-PaLM and its likes. 

In the Med-PaLM paper, researchers introduced an evaluation rubric for assessing LLMs in medical applications, with bias being one of the key dimensions. “Additionally, in Med-PaLM 2, we introduced adversarial questions evaluation, specifically targeting sensitive topics like vaccine misinformation, COVID-19, obesity, mental health, and suicide. These topics have a high potential to exacerbate bias and healthcare disparities through the spread of medical misinformation,” said Natarajan. 

“Our approach to mitigating bias involves rigorous evaluation and expert clinician demonstrations to train the model. While it’s a complex challenge, we are steadily making progress in this area,” he added.

Consequently, he added that the fine-tuning approach used depends on the available data. In the case of the first Med-PaLM, prompt tuning was employed, wherein the majority of the LLM parameters remained fixed, and only a small set of additional parameters were learned. However, for subsequent versions such as Med-PaLM 2 and Med-PaLM M, the team had access to more data, enabling them to fine-tune the models end-to-end in order to enhance performance and align them more closely with medical expertise. 

AI Doctor for Everyone

As we continue to ride the generative AI wave, Natarajan believes that understanding LLMs is crucial, as they differ from human intelligence and require specialised methods, such as “mechanistic interpretability or artificial neuroscience”, posing a plethora of new challenges that need to be solved. According to him, there lies immense potential for exciting research beyond large language models. He is particularly excited about LLMs’ potential in biology and neurology, such as analysing the human genome and decoding brain signals.

Although he has no plans to directly revisit building a similar app like Ask the Doctor, he believes that his work on Med-PaLM and medical AI as a whole at Google will eventually lead to something very similar. “While there is still a long way to go, given the incredible progress made in LLMs just last year, it appears that my dream of making an AI doctor accessible to billions is no longer science fiction. Fingers crossed!” Natarajan concluded.

Read more: Pushmeet Kohli On Solving Intelligence at DeepMind for Humanity & Science

The post Meet the Genius behind Med-PaLM 2 appeared first on Analytics India Magazine.