“If you look back at Meta’s history, we’ve been a huge proponent of open source,” believes Ahmed Al-Dahle, Meta’s vice president for generative AI. 

The AI company has unexpectedly become the Robinhood of the open-source community since its language model LLaMA was leaked on 4Chan earlier this year. Some falsely predicted the leak will have troubling consequences and blamed Meta for distributing the technology too freely. Contrarily, since developers and researchers got their hands on their first really capable foundation model, they have made significant breakthroughs in the landscape.  

In May an anonymous (apparently a Google researcher) memo concerned how open source software was quietly eating the big tech’s lunch was leaked. The memo specifically discussed the bleak chances of Google surviving the LLM battle against open source. It argued that, while the big tech execs squabble about the competitive threat of text-generation technology from OpenAI, the open-source is secretly snatching the spot; with its best contender LLaMA. 

What Makes LLaMA Stand Out

“Suddenly anyone is able to experiment,” stated the leaked document. 

In February 2023, LLaMA was initially available only to researchers by invitation but in less than two weeks leaked, and quickly became popular with programmers who adapted and built on the project. Within weeks of its release, variants surfaced on the internet. Stanford’s Alpaca and Vicuna-13B were nearly as good as OpenAI’s model but agile enough to customise on a computer. 

During the entire month of March, the model made it to the headlines from being ‘fast enough to be practical’ to serving as the foundation for a GPT ecosystem, GPT4ALL. One of the reasons behind its popularity was that the training cost a few hundred dollars compared to the millions usually funnelled for language models. 

Fast forward to July when Meta released the extremely anticipated LLaMA 2. The most significant breakthrough introduced by LLaMA 2 is overcoming the commonly observed tradeoff between safety and helpfulness, achieving superior performance on both criteria.

The model also clarifies that feedback works better for LLMs than supervised data which has long been considered the gold standard. Meta’s LLaMA 2 was made using 40% more data than the original, and a chatbot built with the model is capable of generating results on par with OpenAI’s ChatGPT, Meta claims.

While the v1 model was lauded, it became infamous for being in a legal grey area regarding its commercial use. In a similar vein, LLaMA v2 has resulted in Meta becoming a frontrunner in open-source AI, not all elements of the launch can be labelled as fully ‘open’. The nature of the model isn’t precisely “open source,” but rather an ‘open innovation’. The company previously criticised for its AI strategies, has now become a major contributor, surpassing even the company whose name implies being ‘Open’. 

The training data used to create the model is described in release materials only as “publicly available online sources,” but no further details have been published about the model’s creation. Meta’s licence for the second iteration also requires companies with more than 700 million monthly active users to establish a separate licence agreement with Meta. The reason behind this clause isn’t apparent, yet it creates a hurdle for other tech giants wanting to build upon the system.

Additionally, the model has an acceptable use policy that forbids the generation of harmful code, promoting violence, or facilitating criminal endeavours, misuse, or harassment. It remains unclear what action will Meta take in case of violation of the policy by users of Llama 2.

Enterprise-wise

Notably, Meta isn’t offering up Llama 2 alone. It has support from some major partners that are already making the model available to their customers, The AI startup that releases open-source machine-learning software Hugging Face is the host of the Llama 2 models from Meta. 

Alongside it is also available on IBM’s WatsonX, easing the cost of adoption. Developers can fine-tune a 70B LLM on a single GPU which was not even thinkable a few months ago.

Amazon’s cloud, AWS, also offers access to LlaMA 2. Even Microsoft, the sugar daddy of OpenAI will also be offering LLaMA download to lone coders for use in the cloud and PC. Meta also partnered with Databricks, a software provider and OctoML, an AI optimization startup; to help enterprise-level companies leverage their proprietary data through LLaMA 2. 

A recent study titled, ‘Challenges and Applications of Large Language Models’ states that the capability gap between fine-tuned closed-source and open-source models pertains. With LLaMA2, the gap is narrowed for the community to develop equal competitors of OpenAI’s GPT models. The bottom line is while LLMs and big tech continue to make it to the headlines for reasons right and wrong. Relying on open-source models looks like the right way forward for corporates as well as lone coders. 

The post How Meta Broke the Barriers To Entry With LLaMA 2 appeared first on Analytics India Magazine.