Sramana Mitra: Now, let’s double click down on the concept of experiments. Once you were in these POC situations and started to gain some traction, what kinds of experiments are customers running?

John Wallace: There’re already a lot of early adopters that have bought in as we need to have a culture of test and learn, right? There’ve been a lot of these types of transformations done by the big consultants like the McKinsey, Bain, and BCG to teach brands to embrace Experimentation. You can already run some Experiments in the ad platform for free.

So, if I can run an experiment for free, why would I need LiftLab?

That’s the starting point of how do we find our white space? We think of experimentation as a means to customize it and adapt it for understanding the economics of paid media. A lot of experimentation’s a little more multipurpose, a lot of experimentation fits under the rubric that we all learned in science class around AB testing. And that’s a winner take all kind of thing. Is A better than B? Based on a lot of what we’ve done, that’s not the answer. There’s never a day where I’m only gonna spend on Facebook and never spend on Google.

So, we’ve generalized the testing to embrace a part of economics that’s been around for decades called diminishing returns. We’ve invented a diminishing returns experiment that helps markers answer the question, how much should I spend on these ad platforms?

Sramana Mitra: So, how much of your product is kind of productized around the kinds of experiments that you were seeing as use cases? Are you selling kind of a generic experimentation platform, or are you selling kind of experimentation use cases that are specific, like you just explained one?

John Wallace: Yes, they’re very specific. The piece of context here in terms of the early adopters and the value that they got out of experiments is that we got critical feedback from them that said, this is really cool. I’m making a lot of money with it, but there’s another use case right next to what you’ve delivered that would be equally compelling, if not more.

And so we decided to really increase the footprint of our product. We doubled the footprint of our product. And so that was our journey into another adjacent field called marketing mix models.

So, it’s not a pivot.

We didn’t fail with experiments, but on its own, it didn’t have all the product-market fit that we were after.
And that was a sobering kind of conversation to have internally to debate whether we cut the price of our experiment software and just be a standalone, or do we do a whole bunch more product development?

We clearly went in the direction of increasing the footprint of the product, and then we were first to market to combine these. So, we have competitors that only have the experiment side,
and we have competitors that only have the modeling side, but we’re in a really unique space
where we put these two together. It’s pretty formidable.

Sramana Mitra: And the specs for this, the extension of what people were looking for came from the customers. Were they paying you to develop those additional functionalities?

John Wallace: Well, they were already paying customers of the experimentation. So, once we decided we had kind of a prototype or a high level design, we approached a couple of them and said, we want to use your data to develop this. We’ll keep the price at the same point, but you’ll have a whole lot more functionality.

They didn’t really hesitate.