In a leaked letter, a Google senior software engineer has highlighted the remarkable progress made by the open-source community in natural language processing (NLP). The engineer argues that open-source models are outpacing the progress of larger tech companies like Google and OpenAI, particularly in areas such as scalable personal AI and responsible release. The letter was later published by SemiAnalysis and has since sparked discussions about the future of NLP and the role of collaboration between companies and the open-source community.
The author of the letter suggests that the open-source community has solved many of the major problems in NLP, and they are doing so at a faster rate than traditional AI research organizations. The letter highlights the breakthrough in low rank adaptation (LoRA), which reduces the time and cost of model fine-tuning. The engineer argues that Google and OpenAI should pay more attention to the work being done by the open-source community and prioritize enabling third-party integrations. These advancements are putting pressure on traditional AI research organizations like Google and OpenAI to adapt to the new reality.
The article notes several examples of accomplishments made by the open-source community, such as running foundation models on a Pixel 6 phone and creating responsible models with no restrictions. The author suggests that while Google and OpenAI may still hold a slight edge in terms of language model (LLM) quality, the gap is quickly closing, and open-source models are becoming more capable, customizable, and private.
In response to this new reality, the engineer claims that Google and OpenAI need to adapt by collaborating with outside researchers and prioritizing third-party integrations. The letter emphasizes that the best models are those that can be iterated upon quickly, and companies should focus on creating small variants that can be updated easily rather than giant models that require significant resources to train. The author also suggests that companies need to rethink the value they offer to customers since people may not be willing to pay for restricted models when free and unrestricted alternatives are available.
The letter has sparked discussions about the importance of collaboration between large tech companies and the open-source community. Some argue that open-source models offer more transparency, privacy, and customization, while others point out that larger companies like Google and OpenAI have the resources and expertise to develop more advanced models. However, the engineer’s call to action for collaboration highlights the potential benefits of combining the strengths of both groups to drive further advancements in NLP.
In conclusion, the letter from the Google engineer urges companies to adapt to the new reality of NLP advancements and collaborate with the open-source community to drive progress. The engineer suggests that companies should prioritize enabling third-party integrations and focus on creating small, easily updatable models. As the gap between open-source and traditional AI research organizations narrows, it is crucial that companies work together to leverage each other’s strengths and create more advanced NLP models.
