DeepMind Is Using AI To Pinpoint The Causes Of Genetic Disease
Google Deepmind says it's trained an artificial intelligence that can predict which DNA variations in our genomes are likely to cause disease—predictions that could speed diagnosis of rare disorders and possibly yield clues for drug development.
DeepMind, founded in London and acquired by Google 10 years ago, is known for artificial-intelligence programs that play video games and have conquered complex board games like Go. It jumped into medicine when it announced that its program AlphaFold was able to accurately predict the shape of proteins, a problem considered a "grand challenge" in biology.
Now the company says it has fine-tuned that protein model to predict which misspellings found in human DNA are safe to ignore and which are likely to cause disease. The new software, called AlphaMissense, was described today in a report published by the journal Science.
As part of its project, DeepMind says, it is publicly releasing tens of millions of these predictions, but the company isn't letting others directly download the model because of what it characterizes as potential biosecurity risks should the technique be applied to other species.
Although not intended to directly make diagnoses, computer predictions are already used by doctors to help locate the genetic causes of mysterious syndromes. In a blog post, DeepMind said its results are part of an effort to uncover "the root cause of disease" and could lead to "faster diagnosis and developing life-saving treatments."
The three-year project was led by DeepMind engineers Jun Cheng and Žiga Avsec, and the company said it is publicly releasing predictions for 71 million possible variants. Each is what's known as a missense mutation—a single DNA letter that, if altered, changes the protein a gene makes.
"The goal here is, you give me a change to a protein, and instead of predicting the protein shape, I tell you: Is this bad for the human that has it?" says Stephen Hsu, a physicist at Michigan State University who works on genetic problems with AI techniques. "Most of these flips, we just have no idea whether they cause sickness."
Outside experts said DeepMind's announcement was the latest in a string of flashy demonstrations whose commercial value remains unclear. "DeepMind is being DeepMind," says Alex Zhavoronkov, founder of Insilico Medicine, an AI company developing drugs. "Amazing on PR and good work on AI."
Zhavoronkov says the real test of modern artificial intelligence is whether it can lead to new cures, something that still hasn't happened. But some AI-designed drugs are in testing, and efforts to create useful new proteins are a particularly hot sector, investors say. One company, Generate Biomedicines, just raised $273 million to create antibodies, and a team of former Meta engineers started EvolutionaryScale, which thinks AI can come up with "programmable cells that seek out and destroy cancer," according to Forbes.
Better modelsDeepMind's new effort has less to do with drugs, however, and more to do with how doctors diagnose rare disease, especially in patients with mystery symptoms, like a newborn with a rash that won't go away, or an adult suddenly feeling weaker.
With the rise of gene sequencing, doctors can now decode people's genomes and then scour the DNA data for possible culprits. Sometimes, the cause is clear, like the mutation that leads to cystic fibrosis. But in about 25% of cases where extensive gene sequencing is done, scientists will find a suspicious DNA change whose effects aren't fully understood, says Heidi Rehm, director of the clinical laboratory at the Broad Institute, in Cambridge, Massachusetts.
Scientists call these mystery mutations "variants of uncertain significance," and they can appear even in exhaustively studied genes like BRCA1, a notorious hot spot of inherited cancer risk. "There is not a single gene out there that does not have them," says Rehm.
DeepMind says AlphaMissense can help in the search for answers by using AI to predict which DNA changes are benign and which are "likely pathogenic." The model joins previously released programs, such as one called PrimateAI, that make similar predictions.
"There has been a lot of work in this space already, and overall, the quality of these in silico predictors has gotten much better," says Rehm. However, Rehm says computer predictions are only "one piece of evidence," which on their own can't convince her a DNA change is really making someone sick.
Typically, experts don't declare a mutation pathogenic until they have real-world data from patients, evidence of inheritance patterns in families, and lab tests—information that's shared through public websites of variants such as ClinVar.
"The models are improving, but none are perfect, and they still don't get you to pathogenic or not," says Rehm, who says she was "disappointed" that DeepMind seemed to exaggerate the medical certainty of its predictions by describing variants as benign or pathogenic.
Fine tuningDeepMind says the new model is based on AlphaFold, the earlier model for predicting protein shapes. Even though AlphaMissense does something very different, says Pushmeet Kohli, a vice president of research at DeepMind, the software is somehow "leveraging the intuitions it gained" about biology from its previous task. Because it was based on AlphaFold, the new model requires relatively less computer time to run—and therefore less energy than if it had been built from scratch.
In technical terms, the model is pre-trained, but then adapted to a new task in an additional step called fine-tuning. For this reason, Patrick Malone, a doctor and biologist at KdT Ventures, believes that AlphaMissense is "an example of one of the most important recent methodological developments in AI."
"The concept is that the fine-tuned AI is able to leverage prior learning," says Malone. "The pre-training framework is especially useful in computational biology, where we are often limited by access to data at sufficient scale."
Biosecurity risksDeepMind says it's provided free access to all its predictions for human genes, as well as all the details needed to fully replicate the work, including computer code. However, it isn't releasing the whole model for immediate download and use by others because of what it calls a biosecurity risk if it were applied to analyze the genes of species other than humans.
"As part of our commitment to releasing our research breakthroughs safely and responsibly, we will not be sharing model weights, to prevent use in potentially unsafe applications," the authors wrote in the fine print of their paper.
It's not obvious what those unsafe applications are, or what non-human species the researchers had in mind. DeepMind didn't spell them out, but risks could include using an AI to design more dangerous bacteria or a bioweapon.
However, at least one outside expert we spoke to, who asked for anonymity because Google invests in companies he's started, characterized the restrictions as a transparent effort to stop others from quickly deploying the model for their own uses.
DeepMind denied it was throttling the model for reasons other than safety. The work was assessed both by the Google DeepMind Institute, which studies responsible AI, and by an "outside biosafety expert," a spokesperson for DeepMind said.
The restriction on the model "primarily limits making predictions on non-human protein sequences," DeepMind said in a statement. "Not releasing weights prevents others from simply downloading the model and using it in non-human species … hence reducing the likelihood of misuse by bad actors."
Google Gemini: What We Know So Far
At the Google I/O developer conference in May 2023, CEO Sundar Pichai announced the company's upcoming artificial intelligence (AI) system, Gemini.
The large language model (LLM) is being developed by the Google DeepMind division (Brain Team + DeepMind). It could compete with AI systems like ChatGPT from OpenAI and possibly outperform them.
While details remain scarce, here is what we can piece together from the latest interviews and reports about Google Gemini.
Google Gemini Will Be MultimodalPichai stated that Gemini combines the strengths of DeepMind's AlphaGo system, known for mastering the complex game Go, with extensive language modeling capabilities.
He said it is designed from the ground up to be multimodal, integrating text, images, and other data types. This could allow for more natural conversational abilities.
Pichai also hinted at future capabilities like memory and planning that could enable tasks requiring reasoning.
Gemini Can Use Tools And APIsIn an update to his professional bio over the summer, Google Chief Scientist Jeffrey Dean said Gemini is one of the "next-generation multimodal models" he is co-leading.
He stated it will utilize Pathways, Google's new AI infrastructure, to enable scaling up training on diverse datasets.
This hints at Gemini potentially being the largest language model created to date, likely exceeding the size of GPT-3 with over 175 billion parameters.
It Will Come With Various Sizes And CapabilitiesAdditional details came from Demis Hassabis, CEO of DeepMind.
In June, he told Wired that techniques from AlphaGo, like reinforcement learning and tree search, may give Gemini new abilities like reasoning and problem-solving.
Hassabis stated Gemini is a "series of models" that will be made available in different sizes and capabilities.
He also mentioned Gemini may utilize memory, fact-checking against sources like Google Search, and improved reinforcement learning to enhance accuracy and reduce hazardous hallucinated content.
Early Gemini Results Are PromisingIn a September Time interview, Hassabis reiterated that Gemini aims to combine scale and innovation.
He said incorporating planning and memory is in the early exploratory stages.
Hassabis also stated Gemini may employ retrieval methods to output entire blocks of information, rather than word-by-word generation, to improve factual consistency.
He revealed that Gemini builds on DeepMind's multimodal work like the image captioning system Flamingo.
Overall, Hassabis said Gemini is showing "very promising early results."
Advanced Chatbots As Universal Personal AssistantsIn an interview with Wired, published a few days later, Pichai provided the most unambiguous indication of how Gemini fits into Google's product roadmap.
He stated conversational AI systems like Bard are "not the end state" but waypoints leading towards more advanced chatbots.
Pichai said Gemini and future iterations will ultimately become "incredible universal personal assistants" integrated throughout people's daily lives in areas like travel, work, and entertainment.
He reiterated that Gemini will combine strengths of text and images, stating that today's chatbots will "look trivial" in comparison within a few years.
Competitors Are Interested In Gemini's PerformanceOpenAI CEO tweeted what appeared to be a response to a paywalled-article reporting that Google Gemini could outperform GPT-4.
There was no official response to the follow-up question by Elon Musk on whether the numbers provided by SemiAnalysis are correct.
Select Companies Have Early Access To GeminiMore clues about Gemini's progress this week: The Information reported that Google gave a small group of developers outside Google early access to Gemini.
This suggests Gemini may soon be ready for a beta release and integration into services like Google Cloud Vertex AI.
Meta Working On LLM To Compete With OpenAIWhile the news about Gemini is promising thus far, Google isn't the only company reportedly ready to launch a new LLM to compete with OpenAI.
According to the Wall Street Journal, Meta is also working on an AI model that would compete with the GPT model that powers ChatGPT.
Meta most recently announced the release of Llama 2, an open-source AI model, in partnership with Microsoft. The company appears dedicated to responsibly creating AI that is more accessible.
The Countdown To Google GeminiWhat we know so far indicates Gemini could represent a significant advancement in natural language processing.
The fusion of DeepMind's latest AI research with Google's vast computational resources makes the potential impact challenging to overstate.
If Gemini lives up to expectations, it could drive a change in interactive AI, aligning with Google's ambitions to "bring AI in responsible ways to billions of people."
The latest news from Meta and Google comes a few days after the first AI Insight Forum, where tech CEOs privately met with a portion of the United States Senate to discuss the future of AI.
Featured image: VDB Photos/Shutterstock
Lasker Award For Revolutionizing Protein Structure Predictions
Today (September 21), the Lasker Foundation announced this year's award winners. John Jumper, a computational biologist at DeepMind, and Demis Hassabis, cofounder and CEO at DeepMind, were awarded the 2023 Albert Lasker Basic Medical Research Award for "the invention of AlphaFold, the artificial intelligence (AI) system that solved the long-standing challenge of predicting the three-dimensional structures of proteins from the one-dimensional sequence of their amino acids," announced the Lasker Foundation.
Jumper and Hassabis led the AlphaFold team that revolutionized the field of structural biology by accelerating the process of protein structure prediction with speed and accuracy. Their approach melded together different backgrounds and disciplines, and researchers have adopted the platform to answer diverse biological questions.
Small molecule, big problemLasker Laureate John Jumper played a crucial role in leading the AlphaFold team to new heights in protein structure prediction.
DeepMind
Between 1957 and 1960, Nobel Prize laureate John Kendrew, a biochemist at the Medical Research Council Laboratory of Molecular Biology, resolved the first structural model of a globular protein, myoglobin.1 After studying the results, 1972 Nobel Prize laureate Christian Anfinsen, a biochemist at the National Institutes of Health, postulated that, in theory, a protein's amino acid sequence should fully determine its structure. However, protein structure was notoriously difficult to study.
Scientists heavily relied on X-ray crystallography for decades for protein identification studies, but researchers could spend years attempting to crystalize proteins. Then, the invention of cryogenic electron microscopy (cryo-EM) shed some light on proteins' elusive structures, but the microscope images often had low resolutions. It took years to slowly advance cryo-EM, but by 2019, scientists used cryo-EM to determine the structures of almost 4,000 proteins in the Protein Data Bank (PDB) out of 150,000 entries.2 This is only a fraction of the estimated tens of millions of protein sequences.
Automating the protein pathTo scale up protein structure predictions, researchers turned to artificial intelligence. In 1994, John Moult and Krzysztof Fidelis, both computational biologists at the University of Maryland, founded the Critical Assessment of Structural Prediction (CASP) competition, a biannual test designed for groups to predict the three-dimensional structures of several proteins that were already verified experimentally but not released publicly. Teams received accuracy-based scores out of 100 using the Global Distance Test (GDT).3 Since CASP's inaugural event in 1994, the average score steadily increased from 20 to more than 50. According to the organizers, 90 is the threshold for meeting experimental values.
One of the earliest approaches was developed by David Baker, a biochemist and computational biologist at the University of Washington. He used short segments from the PDB to predict protein structures. Using this model, Rosetta, Baker and his team made several iterations that consistently improved the program's performances in early 2000s CASP competitions. However, historical progress in CASP stagnated.
Now, AlphaFold's effects can been seen as a transformative technology that can be as big as a new microscope technology. Now you can see things that you couldn't see before.-Martin Steinegger, Seoul National University
DeepMind, an artificial intelligence company cofounded by Hassabis in 2010, succeeded in designing AI that could beat human players at chess, and the more challenging game of go (AlphaGo). As Hassabis watched AlphaGo play, it reminded him of Baker's online game FoldIt, which was released in 2008, where players explored and created accurate protein structure models. Shortly after AlphaGo's success in 2016, DeepMind aimed to tackle the next challenge: protein folding.
In 2016, Hassabis believed that his team could create a protein prediction system with machine learning as a core component of the system. This would be one of the first of its kind, debuting at the competition in 2018 as AlphaFold1.4 Machine learning contrasted the traditional AI approaches that relied on preconceived logic by running through iterations of the data to discover patterns.
Lasker Laureate Demis Hassabis led the AlphaFold team to unparalleled speed in accuracy in protein structure prediction.
Hassabis' and Jumper's team won that year's CASP, and AlphaFold1 left quite an impression for creating highly accurate structures for 24 out of 43 modeling domains. AlphaFold1 starkly outperformed the next best method, which achieved 14 out of 43 domains. However, the AlphaFold team knew that it hadn't reached its full potential to serve biologists; there was more work to be done.
Soon after, Jumper took the lead in redesigning the AlphaFold algorithm with an interdisciplinary team of biologists, chemists, and biophysicists. Hassabis, Jumper, and the AlphaFold team brainstormed ways to finetune the algorithm to ensure that AlphaFold2 learned efficiently.
Incorporating a larger database for training greatly helped increase the accuracy of the software's prediction capabilities. "My contribution was done mainly to deliver them these bigger, more comprehensive metagenomic protein databases that they can use for the training part or also for the inference part at the end," said Martin Steinegger, a computational biologist at Seoul National University who helped develop AlphaFold2.
At the next CASP competition in 2020, AlphaFold2 stunned the attendees with staggering accuracy and achieved a median score of 92.4 GDT overall across all targets.5 This kind of accuracy rivaled experimental techniques and boasted an average error comparable to the width of an atom. This iteration of AlphaFold succeded in part because of the complicated architecture built by researchers from different backgrounds and disciplines.
See also "DeepMind AI Speeds Up the Time to Determine Proteins' Structures"
Tobin Sosnick, a biochemist, biophysicist, and Jumper's doctoral advisor at the University of Chicago, said, "He took the folding principles he worked on here [for his doctoral work] and successfully applied them in combination with AI at DeepMind."
Then in 2021, DeepMind publicly released the source code for AlphaFold and its impressive database of more than 350,000 proteins in collaboration with the European Bioinformatics Institute at the European Molecular Biology Laboratory.6 This database has since grown to more than 200 million structures.
"It was AlphaFold that pushed the accuracy over a critical limit where people were now saying the sequence-to-structure problem has largely been solved," said Sosnick. The accessibility of this powerful tool inspired its widespread adoption and allowed researchers to fill in the gaps of their own experimental research.
It's an honor to receive this award in recognition of the work of our team. The work on AlphaFold has been such an incredible experience, and we're only just beginning to see how AI will help us transform biology.-John Jumper, DeepMind
"Normally, computational work was seen as the sidekick for experimental work," said Steinegger. "Now, AlphaFold's effects can been seen as a transformative technology that can be as big as a new microscope technology. Now you can see things that you couldn't see before."
Jumper, Hassabis, and their team tackled a problem that stumped scientists for half a century. This AI tool ushered in a new era of studying proteins for understanding biological functions and guiding drug development. These advances in AI technology fundamentally changed the ways that scientists address problems.
"It's an honor to receive this award in recognition of the work of our team. The work on AlphaFold has been such an incredible experience, and we're only just beginning to see how AI will help us transform biology," said Jumper.
See also "2022 Lasker Award Winners Announced"
References
