When it seemed like the innovation in Smartphones had reached its threshold flattening the sales curve, a few Qualcomm engineers managed to deploy the text-to-image AI Model Stable Diffusion on a smartphone. It was a significant moment both for AI and smartphones. 

In a demonstration video, the engineers ran the text-to-image model on a Sony Xperia 5 II handset, which comes with a Qualcomm Snapdragon 865 (Adreno 650), 8GB RAM and more than 30 GB of storage, to generate a 512 x 512 pixel image in less than 15 seconds. 

Generally, foundational models require significant compute to run and are generally deployed on the cloud. Enabling generative AI models to operate on smartphones has the potential to herald the next significant breakthrough in AI. This achievement addresses a fundamental challenge — cost-effectiveness.

Moreover, the generative AI innovation could give the smartphone market a significant boost. Qualcomm CEO Cristiano Amon believes generative AI could breathe new life into smartphones. 

While smartphones already incorporate AI to some extent, the implementation of generative AI models on these devices has the potential to profoundly transform how we interact with and utilise smartphones. It could help developers explore app development like never done before. Running a model on a device could mean all the apps on your smartphone, including your camera, could be powered by a foundational model.

Jim Yang, senior AI scientist at NVIDIA stated that LLMs could change how smartphone keyboard functions. LLMs are good at predicting the next text in a sentence and this very capability could be a game changer, according to him. Interestingly, three months after Yang tweeted about the same, Apple revealed something similar at its Worldwide Developers Conference 2023. The phonemaker said autocorrect will now use a transformer language model and on-device ML model to make autocorrect better. 

Moreover, generative AI can analyse user behaviour, preferences, and patterns to create personalised user interfaces on smartphones. This can result in customised app layouts, recommendations, and user experiences tailored to individual users. This advancement could effectively transform your smartphone into a personalised assistant. Picture your phone autonomously composing and responding to emails on your behalf. With AI processing taking place directly on your device, the realisation of this scenario becomes increasingly plausible.

Democratising AI 

LLMs, if deployed locally on smartphones, have the potential to democratise AI on a global scale, considering there are more than 6.5 billion smartphone users worldwide. With LLMs deployed on a smartphone, users can access AI-powered language capabilities even without an internet connection. This is particularly valuable in areas with limited connectivity or when travelling. Processing AI tasks locally on a smartphone significantly reduces latency compared to cloud-based solutions. This results in quicker responses, making real-time applications more efficient.

LLMs running on a smartphone also consume less data because they don’t rely on constant communication with cloud servers. This is cost-effective, especially in regions with expensive or limited data plans. Moreover, local AI processing can lead to a smoother and more responsive user experience in applications such as virtual assistants, chatbots, and predictive text input.

Making AI economical 

To keep ChatGPT running and for free, OpenAI could be burning as much as USD 7,00,000 per day, according to SemiAnalysis’ chief analyst Dylan Patel. Today, ChatGPT, even though available as a mobile application, still runs on the cloud. For AI companies, the next big breakthrough could be to run these models locally on devices, as it would lead to a significant reduction in cost.

This is because there is no additional cost involved when it comes to running AI models on smartphones, the users already pay for the hardware cost upfront, unlike running AI models on the cloud. “You need to make the AI hybrid, which means running it both on the data centres as well as locally, otherwise it will cost too much money,” Amon told Financial Times.

Similar to Qualcomm, Google too, earlier this year, revealed that it successfully ran a version of PaLM 2, its latest LLM, on a Samsung Galaxy smartphone. Mediatek, a Taiwanese fabless chipmaker, has also partnered with Meta to bring generative AI to smartphones. Interestingly, Qualcomm has also partnered with Meta to make Llama 2 implementations available on-device, harnessing the capabilities of the new AI-enabled Snapdragon chips. 

Emad Mostaque, not only believes it could be the next big leap in AI, he even said it could happen within a year. “I believe you will see a ChatGPT-level language model (on at least some metrics) on a mobile phone next year and GPT-4 level year after.” 

Is Qualcomm the new NVIDIA?

To run foundational models on smartphones, suitable hardware support is imperative. Currently, several startups are dedicated to developing edge AI chips. For instance, SiMa.ai, is developing chips to run AI models on the edge. 

However, Qualcomm is renowned for its smartphone processors, which power multiple Android smartphones in the market developed by Samsung, OnePlus, Oppo, Vivo, Realme, Redmi, Xiaomi, Lenovo, and Motorola among others.

During the Qualcomm Summit in Hawaii, this October, the San Diego-based company is poised to unveil new chips with the capability to run foundational generative AI models. It firmly advocates that the optimal approach to alleviate the considerable costs linked with cloud-based generative AI is by relocating these models to edge devices. Just as NVIDIA has capitalised on the AI revolution through the development of AI hardware, particularly GPUs, Qualcomm stands to gain substantially by creating chips that enable AI processing on smartphones.  

The post GenAI Will Fundamentally Change How we Use Smartphones  appeared first on Analytics India Magazine.