AI Enterprise At Scale: Faster, Surer Rollouts

AI enterprise at scale: Faster, surer rollouts Strategic efforts to expand AI into production systems are exploding. Here's how you can accelerate operationalization.

Encouraged by industry success stories and the results of their own initial efforts, enterprises worldwide are investing heavily in expanding strategic AI initiatives into production. Top bosses and boards want low TCO with rapid ROI and time-to-value — including faster innovation, improved productivity, revenue growth, and ideally, a lift in stock price.

AI-based products and embedded capabilities will explode across the mainstream in 2023, many analysts predict. Indeed, aggressive AI expansion is already underway in manufacturing, healthcare, financial services and many other industries. Global researcher Omdia forecasts the number of companies actively developing live AI deployments will double by the end of 2025.

It's go-time for mainstreaming AI. Ready?

More than ever, IT leaders and builders are being called upon to quickly develop, deploy and scale robust AI production systems to support key corporate goals like improving customer experience (CX) and service, developing new products, and driving sales and marketing.

In this in-depth coverage, we'll look at current realities, challenges, technologies and strategies for successfully accelerating and expanding AI that can quickly deliver clear value. (Bottom line: A key to success is the ability to consistently scale pilots into production.)

A great place to start is with a close-up of Wayve. The UK global pioneer is working to scale next-generation autonomous vehicles (AVs) guided by embodied AI to 100 cities around the world.

Wayve: New road for AVs runs through the cloud

Becoming a cabbie in London requires memorizing the city's famously bewildering labyrinth of 25,000 streets, along with every landmark and business. It's considered to be a grueling test, taking years of learning. It's an ideal place for Wayve to test and train its next-generation, fully autonomous driving system based on deep learning and embodied intelligence.

For the last six years, Wayve has pursued an ambitious mission to reimagine autonomous vehicles (AVs). They've pioneered the software, lean hardware and fleet learning technology platform needed to support development of large foundation, AI-based models that can quickly and safely adapt to new conditions without explicit programming. Instead of relying on the traditional AV stack, high-definition (HD) maps  and hand-coded rules, the data-driven driver uses cloud-based supercomputing (HPC) that allows Wayve-powered AVs to scale, adapt and generalize their driving intelligence to places it has never seen before.

Commercial trials in London, including with grocery delivery partners, are a key step in the company's plan to introduce Wayve-powered cars and vans to 100 cities across the globe. It's AI scaling at a scale that few organizations will need or attempt. Yet Wayve's innovations offers a dramatic lesson for any company about the importance of planning for foundational AI infrastructure to enable growth.

"Can't get there from here"

After Wayve set out in 2017, Kendall had a growing revelation. It amounted to: "You can't get there from here." More specifically, it was becoming clear that on-premise AI infrastructure would not be up to the task of handling company goals.

In the small bedroom of a house in the startup's headquarters in Cambridge, U.K., 50 or so GPUs cranked away, busily training prototypes of the next-generation autonomous driving system at the core of the company's mission.

Above: Wayve Co-founder and CEO Alex Kendall

Kendall noticed that the small staff of 15 Machine Learning (ML) and robotics experts working on the venture-funded project was burning precious time servicing the on-premise AI processing and infrastructure.

"In a disruptive startup," says Kendall, who holds a PhD in Computing Vision and Robotics from the University of Cambridge, "timing and speed are everything." Wayve's strategy has been to keep focus on their core product, an embodied AI for driving, and partner with others for speed and cost-effective development.

In 2019, the company decided to shift to a cloud-based approach to accelerate development and scaling of the AI that is the engine of the business. Working with Microsoft, they developed a platform that uses the PyTorch open-source machine learning framework with Microsoft Azure Machine Learning and "AI-first" infrastructure to handle end-to-end machine learning, rapid prototyping and quick iteration. Expansion continued over the next few years, including introduction of cloud-based NVIDIA GPUs (like T4s) and development platform.

Illustration courtesy of Wayve 90% faster training on neural nets with billions of parameters

Today, the entire Wayve ecosystem runs within the purpose-built AI environment – compute, storage, networking and software – gathering, managing and processing millions of samples of driving data per year, including petabytes of images, GPS and sensor data. Kendall says the scalable capacity of the cloud-based environment makes it faster to build, iterate and deploy large foundation models for driving in complex urban environments, adjust models more nimbly and adapt to new environments more readily.

Each week, according to Kendall, Wayve trains neural networks with billions of parameters —  three orders of magnitude larger than before – and all 90% faster than previously. "None of this would have been attainable if we were on premise," he says.

Simulation leads to faster insights never before possible

It's not just faster development and scale. Working in a cloud-based AI environment also lets the company do things not previously possible. Take simulation. Real-world testing is a critical part of the development process but comes with major limitations. It costs significant time and money, edge-cases can be rare and scenarios cannot be recreated.

To overcome these challenges, Wayve developed its Infinity Simulator. With the push of a button, the simulator procedurally creates synthetic training data from diverse, large-scale virtual worlds. Using reinforcement learning and foundational models, Infinity generates complex and challenging driving scenarios that allow Wayve teams to train, understand and validate the AI model's driving intelligence. Says Kendall: "We have orders-of-magnitude more ability to elastically create simulated scenarios, which gives us insights that would be vastly slower or even impossible to get from real-world testing."

Countless variants of the same initial scenario can be run in parallel in Azure to provide huge numbers of training and test cases for driving intelligence models. 

The purpose-built AI cloud infrastructure also extends to in-field data collection. Wayve deploys its hardware stack on vehicles operated by Wayve and its fleet partners, which send driving experiences back to the company through Azure cloud and IoT services.

Overall, Kendall says the ability to use ML to generate operating environments and run atop a Kubernetes service "lets us run at a fast distributed scale… and spin different systems up and down depending on internal demand." Further, he says, the purpose-built, AI cloud infrastructure improves consumption flexibility and cost management by enabling quick switching to optimize for a particular GPU or to co-locate with data, for example.

"It's all about scaling, speed and leveraging the latest technology."

Armed with these leading-edge technologies, Wayve is currently focusing on scaling AV2.0, using Azure to further increase the size and complexity of its neural networks by "many orders of magnitude". Last year, the company announced it is working with Microsoft to leverage Azure supercomputing infrastructure and technologies to further accelerate and power.

Kendall is confident Wayve is on the right track. "Building and safely deploying autonomous driving technology on a global scale requires powerful AI infrastructure that one day can train models with trillions of parameters and exabytes of image data," he says. "It's all about scale, speed and leveraging the latest technology. Trying to run all this on-premise infrastructure would distract our focus and mean we have to hire 100 people just to build a data center. That's not our business."

Challenge: Prioritize AI production with clear value

Enterprise AI is stuck. A widely cited survey by Gartner in 2019 found that 53% of enterprise AI projects made it from pilot to production. An update in mid-2022 reported an increase of only 1%. "Scaling AI continues to be a significant challenge," concludes Frances Karamouzis, distinguished VP analyst at Gartner.

Only 54% of AI pilots will make it into production. Gartner 

Large global studies by McKinsey and others also show that despite rapidly increasing investments, operationalization and adoption of AI has plateaued. Many enterprises are trapped in what the consulting firm calls "pilot purgatory".

What's behind the stubborn difficulty of turning "science experiments" and tactical pilots into production AI that can drive impactful business gains?

Stymied by complexity, talent and cost

For starters, says John Lee, AI Platforms & Infrastructure Principal Lead for Microsoft Azure, "the inherent difficulty, complexity and cost" make deployment of AI at scale challenging for all but the most advanced, deepest-pocketed enterprises like Tesla and hyperscalers.

"People see the opportunities, but not everybody has the same resources, capabilities, talent pool and understanding to implement and monetize it." Affordable tools, platforms and processes are big barriers to adoption, he notes.

Beyond cost hurdles, industry studies say that enterprises trying to scale AI also struggle with inadequate data, slow model training, workforce resistance, poor alignment with key organizational goals and unclear ROI. In some cases, AI efforts divert technical resources and cycles from core operational systems, impeding and slowing both.

The key to avoiding stalled AI

Failure to advance from POC to pilot to production is an important issue for IT and business teams. Stalled, unscalable, poorly targeted AI without clear value burns precious funding, dampening management and shareholder support, jeopardizing further investment, stifling innovation and endangering competitiveness.

The solution lies in improving the technological, process and organizational maturity of enterprise AI. Doing so enables rapid, reliable, cost-effective development and deployment that delivers greater value with fewer resources. Industry research is clear:

Enterprises that prioritize scaling pilots into production systems that solve critical business problems and spot opportunities (with an eye toward eventual "industrialization" of AI) will enjoy continuing advantage and funding for innovation.

AI growth needs a solid technology foundation

With greater demands for data volume, speed and processing power, AI development and deployment requires different technology, processes and skills than traditional software. To quickly build and operate reliable solutions at scale, McKinsey and other experts underscore it's crucial that enterprise leaders make the right investments in tech stacks and teams. Unfortunately, few appear well-positioned.

Only 20% of companies have the technology infrastructure in place to make the most of AI's potential. Bain & Co.

End-to-end environments. For many organizations, efforts to mature and industrialize AI production will focus on a foundational, "AI-first" platform and infrastructure that includes cloud and data platforms and tools, GPUs and accelerators, software, architecture and services. As Accenture notes, this "AI core" works across the cloud continuum (e.G., migration, integration, growth and innovation), provides end-to-end data capabilities (foundation, management and governance), manages the machine learning lifecycle (workflow, model training, model deployment) and provides self-service capabilities. Pre-integrated, full-stack environments optimized for AI help accelerate building, operationalization, management and expansion.

The focus on the cloud is important. "Where" AI workloads run is rapidly evolving, says Robert Ober, Chief Platform Architect, NVIDIA.  He notes that more enterprises are choosing public clouds and Infrastructure as a Service (IaaS) to build and deploy AI-enabled services and maximize infrastructure investments.

Cloud infrastructure from Microsoft and NVIDIA, purpose-built for AI, provides an end-to-end, full-stack solution that includes on-demand global access to the latest GPUs. This "AI-first" foundation combines the support, agility, simplified IT management and scalability of the cloud and performance-optimized, enterprise-grade software stack, delivering the highest scalable performance to accelerate the whole AI pipeline – from start to finish.

High-performing organizations have prioritized putting these foundational platforms in place for AI-related data science, data engineering and application development. Many have also adopted advanced scaling practices, such as using standardized tool sets to create production-ready data pipelines. AI workflow management can be simplified and accelerated with curated collections of use-case based content, which speed identification of compatible frameworks containers, models, Jupyter notebooks, detailed documentation and other resources.

AI services. Pay-as-you-go AI services offer another way for enterprises to speed delivery of cloud and intelligent edge AI applications and capabilities – in days instead of months. Microsoft Azure AI, for example, speeds development by giving data scientists and others access to vision, speech, language, decision-making models and built-in business logic through simple API calls.

The new Azure OpenAI Service lets organizations quickly create cutting-edge applications or modernize business processes without ML expertise. Access to advanced AI models — including GPT-3.5, Codex, DALL•E 2,  — is backed by trusted enterprise-grade capabilities and AI-optimized infrastructure.

Accelerated hardware. Custom combinations of new, energy-efficient GPUs and DPUs accelerate compute performance and deliver the parallelism, scale and efficiency needed to build language models, recommenders and other leading-edge AI applications more quickly — with lower TCO, reduced carbon footprint and faster ROI than legacy architectures.

Software. "Enterprises often must choose between cloud computing and hybrid architectures, which can stifle productivity and slow time-to-value and innovation," says Manuvir Das, Vice President, Enterprise Computing at NVIDIA. Advances in AI platform software can help businesses unify AI pipelines across all infrastructure types — on-prem, private cloud, public cloud, or hybrid cloud — and deliver a single, connected experience, says Das.

This ability to run AI software across different infrastructures brings several benefits, according to Das: Ending AI silos, allowing enterprises to balance costs against strategic objectives, regardless of project size or complexity, and providing access to virtually unlimited capacity for flexible development."

Another advance: curated AI software stacks for horizontal use cases. These make it faster and easier for enterprises to bring AI into common business workflows via SaaS and other delivery modes. Similarly, industry-specific AI solutions for life science and other fast-growing verticals promise to speed go-live and expansion schedules by freeing enterprises from the time-consuming work of creating specialized but common capabilities and training models.

Accelerate production AI with autonomous systems

In addition to establishing a solid, end-to-end-core foundation for AI, several other fast-emerging new autonomous technologies and approaches can help significantly speed development, scaling and time-to-value of enterprise production AI.

Large language models (LLMs).  Open AI's GPT-3.5 (soon 4.0) and DALL•E 2 thrust the powerful capabilities of LLMs into the spotlight in 2022. LLMs find hidden patterns in unstructured data to support healthcare breakthroughs, advancements in science, better customer engagements and even major advances in self-driving transportation.  Wayve, for example, is creating a single large foundation model similar to the large language models on the market today, but instead of text input, they use image. They believe the best approach to solving autonomous driving is a large-scale foundation neural network that's trained using self-supervised learning that can really address diverse sets of data. Rapid R&D and commercialization will bring a universe of new business applications – from summarizing medical notes, generating catalog descriptions, more natural and helpful chatbots, instant translation into hundreds of languages — across multiple domains.

96% of businesses surveyed plan to use AI simulations this year. PWC

Of particular interest to AI scalers: the ability of these autonomous models to self-train without supervision and to be acquired pre-trained, ready for customization if desired. Removing the time-consuming work, normally done by hard-to-find data specialists, provides a huge advantage to enterprises pressed to show rapid progress and results. The emerging capabilities of LLMs to automate software coding and editing (including for AI) further increases their appeal.

Simulation. The ability of digital twins to accurately simulate the real world, capture and process massive amounts of data and encode autonomy, without the need for deep expertise, is another powerful tool for speeding testing, training and propagation of production AI.

Low-code/ No-code programming. These popular new development tools help enterprises sidestep talent bottlenecks and resulting delays. Using visual authoring capabilities, engineers and others can quickly build and add AI to systems, equipment and processes without writing code or algorithms. Result: fast solutions to complex problems and accelerated innovation.

AI high-performers are 1.6 times more likely than other organizations to engage nontechnical employees in creating AI applications by using low-code or no-code programs. McKinsey

Low-code/no-code development tools let non-specialists transform whiteboards into AI, speeding go-live dates and innovation.  

Bottom line: Scale or fail

As with any adolescence, becoming a more mature AI organization inevitably will bring awkward moments, dashed hopes and growing pains. Microsoft's Lee reminds about the importance of having realistic expectations for AI. And patience.

"If you think you're going to scale once and hit a home run, you need to reconsider," he cautions. "Expect that your first deployments might be science projects. Learn from them, and develop the needed muscles internally, so that your organization gets really good at it."

For perspective, Lee believes it's helpful to look at other disruptive technologies. The first mass-produced electric car, GM's EV1, debuted in 1996. Yet it wasn't until 2017 that Tesla's Model 3 became business feasible. Similarly, he notes, pilot-to-production percentages for another modern technology, IoT, is less than 40% — lower than AI.

A growing gap between achievers and laggards

Even so, Lee and other industry experts have no doubt that the race to develop AI-infused systems and applications will continue to accelerate as more companies and industries invest. Eventually, many believe it will be easier to adopt AI and harvest greater and greater benefits.

And as with any major technology change, companies advancing their AI maturity will need to look at processes and people to succeed. Fortunately, there's a fast-growing body of best practices for deployment and other AI challenges. AI standards remain rare, but here again, various industry, professional and government groups are busily working on developing them.

Finally, take note: Several industry studies highlight a clear and growing gap between laggards and AI high achievers seeing high financial returns from AI. Top performers are making larger investments in AI, adopting advanced practices known to enable faster AI scaling and development, and show signs of faring better in the tight market for AI talent.

Cynics dismiss these warnings at their own peril, says Lee. "If AI is the future and you don't invest, your organization risks becoming irrelevant. History is a graveyard littered with the tombs of famous companies that failed to capitalize on the next big thing like smartphones, digital photography and media streaming. What do you want your legacy to be?"

Make AI Your Reality Learn more here

AI Opportunities: Transforming Coverage Of Live Events

The AI in Production team at BBC R&D is looking at some of the ways that Artificial Intelligence (AI) and machine learning could transform the business of producing media. These are new forms of automation, and we want to know what the opportunities are for using them to significantly increase the range of programmes that broadcasters like the BBC could offer. Could we build a system which would allow us to cover the hundreds of stages at the Edinburgh Festival, for example, or broadcast every music festival in the UK?

Image above by jcburns (cropped) on Flickr, cc licence.

We started our research with a project aimed squarely at broadening coverage in this way, and opening up access to events that it would be impractical or un-affordable to cover using conventional techniques.  In our prototype system, which we have named "Ed", a human operator sets up fixed, high resolution cameras pointing at the area in which the action will take place, and then the automated system takes over.  It attempts to create pleasingly framed shots by cropping the raw camera views.  It then switches between those "virtual cameras" to try and follow the action.  In many ways, this project is a successor to our prior work on automated production: the basic concept of covering live events by cutting between virtual cameras was explored previously by our Primer and SOMA projects.

One of the things that working with AI technologies really highlights is that there are big differences between how even "intelligent" computer systems view the world and how people do.  If we think about the "unscripted" genres of television, such as sport, comedy and talk shows, most people would have little difficulty in identifying what they want to see depicted on the screen – it'll usually be the action around the ball in a game of football, for example, or the people who are talking in a televised conversation.  AI systems have no idea what we humans are going to find interesting, and no easy way of finding out.  We therefore decided to keep things simple: this first iteration of "Ed" looks for human faces, and then tries to show the viewer the face of whoever is talking at any given point in time.  These relatively simple rules are a reasonably good match for any genre consisting of people sitting down and talking – in particular, comedy panel shows, which is therefore the genre we have been targeting.

BBC R&D - AI Production

IBC - Video Analysis and Machine Learning for Expanded Live Events Coverage

IBC TV - Interview with BBC R&D's Craig Wright of the AI Production project

Our first version of Ed is entirely driven by rules like these.  We generated them by asking BBC editorial staff about how they carried out these tasks in real productions.  To frame its shots, Ed rigidly applies the kinds of guideline that students get taught in film schools: the "rule of thirds", "looking room", and so forth.  Selecting which shots to show and when to change shots is similarly rule-based.  Ed tries to show close-ups when people are speaking, and wide shots when they aren't. It tries not to use the same shot twice in quick succession.  It changes shots every few seconds, and tries not to cut to or from a speaker shortly after they start speaking or shortly before they stop again.

Having created a working system, we needed to test it.  We're proponents of "user-centred" approaches, and we believe that ultimately, the only test of our system that matters is what real audience members think of it. We want to compare our system's decision-making, and the quality of the ultimate viewing experience, to that of human programme-makers.  We have a series of formal studies planned to evaluate and improve Ed, and we started with an evaluation of shot-framing.

To compare Ed's shot-framing to some human professionals, we took four directors and camera operators and asked them to frame some shots for us, based on footage from a "panel show" of our own that we created as test material.  We asked Ed to do the same thing.  We then mixed all the shots up and put them into pairs. Each pair consisted of two framings of the same shot – either both framed by humans, or one by a human and one by Ed.  We showed them to 24 members of the public, asking them which one they preferred. Sometimes we asked these participants to think aloud as they decided, and we interviewed them afterwards to try to get a better understanding of their preferences.

We've already learned a lot by analysing the results of this study.  We plan to write it up in full as a conference or journal paper, but just looking through the things people said to us has helped us come up with a number of additional rules that would improve Ed's ability to frame shots attractively.  People disliked having objects and people framed half-in and half-out of the shot, for example, or having unnecessary empty space within the frame.  We hope to be able to pull even more insights from the data when it is fully analysed, and we plan to run further studies to evaluate Ed's ability to select and sequence shots.

What's next?  Well, we intend to improve Ed, both by implementing the findings of our studies, and by replacing some of our rules with machine learning approaches, using the BBC archives as a source of training data.  In addition, there are many aspects of a production that Ed does not currently attempt to address: lighting and sound, for example.  Most importantly, we need to think about other genres – in particular, productions that require creative decision-making that can't be approximated by simple rules, or by today's machine learning techniques, which think very much "inside the box" defined by their training data.  Shows for which a simple narrative must be assembled by whittling down a large set of potential material, for example, or which start off with a vision for a story and need to work out how best to tell it, will need humans and AIs to work together, posing new challenges.  We also want to explore a "bottom-up" approach, working with real-world productions to identify tedious and time-consuming aspects of their work that would be good candidates for less ambitious but more immediately useful forms of AI automation.

We'll be talking about Ed at this year's IBC conference. (The conference organisers have been kind enough to give us their "Best Paper" award for our work.)  If you want to learn more about the Ed system and our initial study, you can now read the paper 'AI in Production: Video Analysis and Machine Learning for Expanded Live Events Coverage' which has now been published.

Tweet This - Share on Facebook

BBC R&D - AI Production

IBC TV - Interview with BBC R&D's Craig Wright of the AI Production project

IBC TV - Tech Talks: AI in Production

IBC - AI in Production: Video Analysis and Machine Learning for Expanded Live Events Coverage

BBC R&D - AI and the Archive - the making of Made by Machine

BBC R&D - Artificial Intelligence in Broadcasting

BBC R&D - Data Science Research Partnership

BBC R&D - The Business of Bots

BBC Technology + Creativity Blog - Fighting bias in the machines

BBC Technology + Creativity Blog - AI for Common Good

BBC News Labs - What we talk about when we talk about fair AI

BBC R&D - Editorial Algorithms

BBC R&D - Using Algorithms to Understand Content

BBC R&D - COMMA

BBC R&D - Content Analysis Toolkit

BBC R&D - Natural Language Processing

BBC R&D - CODAM

BBC iWonder - 15 Key Moments in the Story of Artificial Intelligence

Wikipedia - Artificial Intelligence

BBC News - Artificial Intelligence

BBC News - Intelligent Machines

Machine Learning and Artificial Intelligence Training and Skills from the BBC Academy including:

The history of machine learning

Get ready to work with AI

What does AI mean for the BBC?

AI at the BBC: Hi-tech hopes and sci-fi fears

44 Of The Most Promising Generative-artificial-intelligence Startups Of 2023, According To VCs

Adept

Adept's founding team. Adept

Startup: Adept

Recommended by: Saam Motamedi, Greylock; Jill Chase, CapitalG; Jake Saper, Emergence Capital

Relationship: Motamedi is an investor, and Chase and Saper have no financial interest in the company.

Total funding: $415 million, according to the company

Notable investors: Greylock Partners, General Catalyst, Spark Capital, Addition, Root Ventures, Air Street Capital, 10X Capital

What it does: Adept is a machine-learning research-and-product lab aiming to build AI technology that can automate any task a human can do, from running software to browsing the web.

Why it's on the list: With an impressive founding lineup hailing from top companies like OpenAI and Google Brain, Adept is perhaps one of the buzziest startups in the space, taking on the ambitious goal of not just generating text but also action. Adept creates a strong competitive moat by being a "full-stack" AI company, or one that creates its own foundation model versus being built on other providers like OpenAI, Chase told Insider. In doing so, the company makes it more difficult for competitors to replicate its offerings, she explained. Adept further differentiates itself by training its models on unique, private datasets, Motamedi said. Even skeptics are excited — although there is "lots of risk in implementation," Saper told Insider. But if Adept "can put in the proper bounds and lower risk," the product "could be super powerful," he added.

Arena

Arena's cofounders Engin Ural and Pratap Ranade. Arena

Startup: Arena

Recommended by: Leigh Marie Braswell, Founders Fund

Relationship: Investor

Total funding: $32 million, according to the company

Notable investors: Founders Fund, Initialized Capital Management, Goldcrest Capital

What it does: Arena leverages AI to help consumer-goods companies turn their sales processes and supply chains into more autonomous, self-learning systems.

Why it's on the list: When thinking about generative AI, many people's minds go automatically to the technology's creative applications, like AI-generated art or content, Braswell told Insider. However, generative AI startups like Arena are not generating art, but rather "simulations of human behavior," Braswell said. In doing so, Arena is able to help enterprise customers "create pricing models and inventory-management models" based on unique simulations rather than "having to just rely on past data," she explained.

AssemblyAI

Dyan Fox, CEO and founder of AssemblyAI Dylan Fox

Startup: AssemblyAI

Recommended by: Steve Loughlin, Accel

Relationship: Investor

Total funding: $64.8 million, according to PitchBook

Notable investors: Y Combinator, TechNexus Venture Collaborative, MAVA Ventures, Accel, Insight

What it does: AssemblyAI's API uses AI models to translate and understand speech.

Why it's on the list: The demand for AI products is bigger than ever, but it's still very complex to build it into existing products, Loughlin told Insider. AssemblyAI solves that problem for companies of all sizes with its production-ready models. It's benefited greatly from the AI buzz of the last few months, seeing a 291% growth in registered users since November 2022, the company told Insider.

"They're starting with the massive category of audio and video to power transcription, summarization, moderation, and compliance, but are setting up a framework to deploy across other core areas as well," Loughlin said.

Banana

Banana cofounders Kyle Morris and Erik Dunteman. Banana

Startup: Banana

Recommended by: Jill Chase, CapitalG

Relationship: No financial interest

Total funding: $3.2 million, according to the company

Notable investors: Pioneer.App, Founders Inc, CapitalX, AVG Basecamp, Outset Capital, Spice Capital

What it does: Banana outsources the infrastructure for AI companies by providing them with hosting services for machine-learning models, allowing customers to streamline the process of putting models into production by copying and pasting an API. 

Why it's on the list: Despite all of the excitement around generative AI, large AI models are still relatively difficult to use and build. This is why Chase has been especially interested in infrastructure companies like Banana that democratize access to large models for developers, she told Insider. Banana itself frames its mission as providing the "picks and shovels" to technologists looking to get in on the "biggest technological gold rush of the 21st century," according to its website. Its usage-based pricing allows developers to pay only for the computing they use, and its three-step deployment process aims to make the process cheaper and more efficient for customers.

BigHat Biosciences

BigHat Lab Biosciences cofounders Peyton Greenside and Mark DePristo BigHat Lab Biosciences

Startup: BigHat Biosciences

Recommended by: Andy Harrison, Section 32

Relationship: Investor

Total funding: $104 million, according to the startup

Notable investors: 8VC, MBC BioLabs, Amgen, Andreessen Horowitz, Section 32

What it does: BigHat Biosciences uses machine learning and synthetic biology to develop safer and more effective antibody therapies.

Why it's on the list: Section 32's investment in BigHat represents the firm's interest in "hardcore computational biology," Harrison told Insider. The startup is so revolutionary because it can step in when the human body doesn't produce exactly what it needs when it comes to antibodies. 

"We just expect the body to give us the right antibody that grabs the drug candidate that grabs the right tissue at the exact right receptor. Now, we can design these proteins from scratch," Harrison said.

Character.AI

Character.AI cofounders Noam Shazeer and Daniel De Freitas. Character.AI

Startup: Character.AI

Recommended by: Jill Chase, CapitalG

Relationship: No financial interest

Total funding: $193 million, according to the company

Notable investors: Andreessen Horowitz, SV Angel, A Capital, Nat Friedman, Elad Gil, Paul Buchheit

What it does: Character.AI allows users to chat with a multitude of AI-powered characters, ranging from Billie Eilish to William Shakespeare.

Why it's on the list: In March, Character.AI announced its highly anticipated funding round, landing an $150 million Series A after months of speculation. The startup met with a number of top-tier VC firms, including Sequoia Capital, according to The Information, before Andreessen took the lead for its latest round. Character.AI was granted a $1 billion valuation, a mind-boggling number for a startup whose product is still in beta. One reason investors may be so excited about the company is that by building its own foundation models, Character.AI has created a strong competitive moat around itself, setting itself apart from competitors building on top of third-party models, Chase told Insider.

Chroma

Chroma cofounders Jeff Huber and Anton Troynikov. Chroma

Startup: Chroma

Recommended by: Saam Motamedi, Greylock

Relationship: No financial interest

Total funding: 20.3 million, according to the company

Notable investors: Quiet Capital, AIX Ventures, Bloomberg Beta, AI Grant

What it does: Chroma enables companies to add their own data, state, and memory to AI-enabled applications. The startup achieves this through "vector embeddings," a numerical representation of any kind of data including text, images, video, and audio, which can provide models with new knowledge without expensive fine-tuning, human feedback, or hallucinations.

Why it's on the list: Databases like Chroma are an important component of the AI ecosystem, especially for developers building AI tools, Motamedi told Insider. While building Chroma, its founding team aimed to create an easy-to-use and lightweight product that would allow users to prototype rapidly instead of using existing solutions that were more arduous to operate, Chroma's cofounder, Anton Troynikov, said on Twitter. The project is also open source — this broad accessibility was important to the company in its mission to achieve a "flourishing of humanity that will be unlocked through the democratization of robust, safe, and aligned AI systems," its website says. The company's mission seemed to resonate with investors, gaining the startup a $75 million post-money valuation in its most recent round, Insider learned.

Codeium

Exafunction cofounders Varun Mohan and Douglas Chen. Exafunction

Startup: Codeium

Recommended by: Leigh Marie Braswell, Founders Fund

Relationship: Investor

Total funding: Exafunction, Codeium's parent company, has raised $28 million, according to the company

Notable investors: Exafunction's investors include Greenoaks Capital, Founders Fund, Spencer Kimball, Neha Narkhede, Sahir Azam, Carlos Delatorre, Howie Liu, Richard Socher

What it does: Codeium auto-completes code based on developers' instructions and comments written in natural language. 

Why it's on the list: Codeium is a product built by deep learning startup Exafunction. This solution differentiates itself from competitors like Copilot, GitHub's code-generating AI tool, by being free for individual users and training its own models instead of using OpenAI's, which the company claims reduces latency, Braswell told Insider. Already, the product is being used by developers at companies like Adobe, IBM, Meta, and Tesla and academic institutions like Berkeley, Harvard, and Stanford, according to the company's website.

Cohere

Cohere cofounders Ivan Zhang, Nick Frosst, and Aidan Gomez. Cohere

Startup: Cohere

Recommended by: Andy Harrison, Section 32

Relationship: Investor

Total funding: More than $170 million, according to the company

Notable investors: Radical Ventures, Section 32, Index Ventures, Tiger Global Management, Geoffrey Hinton, Fei-Fei Li, Pieter Abbeel, Raquel Urtasun

What it does: Cohere offers easy-to-deploy large language models that have the ability to classify, generate, and uncover trends and patterns in text.   

Why it's on the list: Cohere is in a category of infrastructure-AI companies that can "help developers start to program in the space," Harrison said. Cohere's cofounder and CEO, Aidan Gomez, was an author on the groundbreaking "Attention Is All You Need" paper, which spawned the transformer-model architecture, an invention critical to today's AI models, and a number of high-flying startups like Adept and Character.AI, founded by other authors. Cohere's other cofounders, Ivan Zhang and Nick Frosst, are equally pedigreed, having spent time conducting AI research at FOR.Ai and Google Brain. Although the startup is taking on the AI giant OpenAI in its offering of large language models, it's remained "crazy under the radar," Gomez said in a VentureBeat interview. That might not stay true for long, though, as Cohere is rumored to be raising a new round valuing it at more than $6 billion, according to Reuters.

Cursor

Startup: Cursor

Recommended by: Leigh Marie Braswell, Founders Fund 

Relationship: No financial interest

Total funding: Not available 

Notable investors: Not available

What it does: Cursor offers software developers a development environment in which they can command changes to code using natural language.

Why it's on the list: Cursor is taking a revolutionary approach to aiding software developers by building a new development environment from "the ground up" and rethinking the whole concept of an "AI-first framework," rather than just offering a plugin, Braswell told Insider. The startup allows software engineers to do everything from making complex changes across multiple files to asking questions about their codebase, all through instructing the software in everyday language. Cursor currently has a waitlist for users and is looking to hire engineers, researchers, and designers, its cofounder tweeted.

Darrow

Darrow cofounders Gila Hayat, Evyatar Ben Artzi, and Elad Spiegelman. Darrow

Startup: Darrow

Recommended by: James Currier, NFX

Relationship: Investor

Total funding: $24 million, according to the company

Notable investors: Y Combinator, F2 Venture Capital, NFX, Entrée Capital

What it does: Darrow leverages AI to scan publicly available data to identify potential claims and automatically generate likely cases for litigators.

Why it's on the list: An example of one of the many vertical-specific applications of generative AI, Darrow analyzes public data to "figure out where consumers are being hurt and then form collective action to help them get compensation" or to stop perpetrators, Currier told Insider. For the investor, it's a fascinating and practical use of generative AI, as Darrow's tech is able to "plow through hundreds of thousands of pages of content and distill and organize them in a way that would be just too expensive for humans to do," he said.

DeepL

Jarek Kutylowski, CEO and founder of DeepL Jarek Kutylowski

Startup: DeepL

Recommended by: Karthik Ramakrishnan, IVP

Relationship: Investor

Total funding: $100 million, according to PitchBook

Notable investors: btov Partners, TA Ventures, Niko Waesche, Cinco Capital, Tamiva Ventures, Benchmark, Blue Fund Consulting & Invest, IVP

What it does: DeepL offers free translation services across a number of European and Asian languages.

Why it's on the list: DeepL is seeking to unseat Google as the dominant translation service provider — a process that is "insanely difficult" and surprisingly manual, Ramkrishnan told Insider.

"We can just go on Google Translate and get whatever we need to, but for businesses who are translating documents and legal contracts and things like that, most people are still hiring manual consultants," he said. That process is incredibly expensive and can include weeks of turnaround time.

"DeepL helps