Investors have poured $1B into Experimentation this past year. Here's why.
Dissecting two different trends behind a flurry of M&A and investment activity
It’s been a busy 12 months in experimentation and its adjacencies: add up all the investment and M&A activity and it’d be reasonable to estimate that investors have poured over $1B into our space.
In these last 12 months,
VWO sold to private equity for $200M
Optimizely acquired warehouse-native analytics company NetSpring
Of course, my home team of choice at Eppo raised a $28M Series B last August and then was acquired by Datadog in May
Statsig raised a $100M Series C at an eye-popping $1B valuation
LaunchDarkly acquired observability company Highlight and warehouse-native analytics company Houseware
Marketing experimentation platform Haus raised an extra $20M in funding
…and just outside of this time window, Harness acquired Split.io and Webflow acquired Intellimize.
As of last week, there’s a new addition to the parade of headlines: Monetate2 (owned by private equity firm Centre Lane Partners) has taken out a $75M line of financing to refinance its own debt, and acquire SiteSpect.
All this activity has a lot of voices riled up on LinkedIn. Mostly vendors doing their work as spin doctors3 and trying to turn each moment into a pitch for their product, but a few professional analysts and concerned customers are chiming in too.
What the heck is going on here? The market is growing? Struggling? Consolidating?
I think there are really two broad shifts going on simultaneously. One is happening in the mature “traditional A/B testing tools” world, where companies have been on the market for anywhere from 8-20+ years. The second is continuing growth-stage evolution in the newer “experimentation platform” market.
It’s also worth acknowledging that tech M&A activity as a whole is spiking right now. While we’re nowhere near the pandemic-era insanity, global deal volume in tech/media/comms is nicely recovered from its slump the last few years, and even slightly above 2019 levels.
EY Parthenon shows a month-by-month view of US M&A activity that shows spikes in March and May of this year rival with the hottest months in H1’22. They report $39B of deal volume in US Tech M&A in May, up 22% from this time last year4.
Part of this might be driven by an expectation that interest rates, which began to rise in 2022 after fourteen years of being near-zero5, will go down again soon. In short — when interest rates are high, a company’s valuation tends to fall because their future cash flow is subjected to a higher discount rate when calculating said valuation.6 If rates do go down again soon, this presents a bit of an arbitrage opportunity for would-be acquirers: to buy a company at a lower valuation than it would have tomorrow. (You can read a more thorough exploration of the relationship between interest rates and M&A here.)
But even taking the macro into consideration, I don’t think it’s over-extrapolation to say that we’re having a bit of a moment in the experimentation space right now given how many companies are involved. So what seems to be going on?
1) The website-focused A/B testing tool space is starting an early consolidation phase
Core to my thesis is that there are really two different types of experimentation vendors attracting all of this investment interest. The first we’ll look at is the “legacy” A/B testing tool: these are the website-focused solutions that have been on the market for an average of 10+ years and have names like Optimizely, VWO, Kameleoon, Convert.com, AB Tasty, Monetate, Webtrends Optimize, and many more.
The core capability for these tools is what’s called client-side experimentation. When you want to run an experiment on a technology product like an app or website, you need to write code to create the experimental “treatment”. That code can either live on your server (in your codebase), or in the case of a website experiment, it can live somewhere else and be delivered alongside your codebase to a user’s browser (the client) as they load your site (usually via a Javascript snippet).
(Although several of these vendors have built expanded offerings that include server-side experimentation — i.e., feature flagging — for use by engineering and product teams, their largest SKUs by-and-large remain client-side.)
Decoupling the ability to run experiments from the need to be inside a company’s codebase is a great proposition for non-engineering teams like marketing. They can simply install a Javascript snippet on their website, set off to build some tests using a provided WYSIWYG editor, and live happily ever after.
There are just a few problems in this space. One is that the value proposition has gotten harder and harder to deliver on over time, technologically-speaking. Modern development frameworks occasionally break the code from these WYSIWYG editors, some ad blocker plugins stop these tools from loading or executing, and they incur a notable cost to website speed/performance, which has become increasingly important to user experience and search engine optimization.7
But even with these very real complications stacked against it, this approach remains the only viable way for many marketing teams — who lack engineering resources or other important types of support — to run experiments. And so they continue to buy and utilize these tools. (One glimmer on the horizon: AI codegen models present a real opportunity to help marketers leave this approach behind and move their experimenting, too, to the server-side. A topic for another issue.)
The second problem, economically-speaking, is that there are many companies who service this space, and seemingly more pop up every year as the underlying tech gets easier to build8. Just looking at who I’m connected to on LinkedIn, I could easily name 25-30 contenders in a ~$1B market. (more on that market size later)
This kind of mature, fragmented market presents a prime opportunity for private equity firms to come in and consolidate it — especially considering how many of these businesses are cashflow positive. And given how long many of these companies have been in operation (10-20 years), selling to anybody can become an increasingly attractive option to the original investors/founders/owners.
If you're unfamiliar with how private equity buyouts operate, the important thing to know is that PE firms hope to only hold these companies for a couple of years, maybe ~5, before exiting them (at a profit). One of their most common playbooks is a rollup: if they can grab a few smaller companies in the space and combine them (both to boost market power and cut costs), they can turn around and resell them at a neat profit later. Very few of their acquisitions will make it to the public markets in an IPO of their own. Ideally most will be sold to strategic corporate acquirers. And some will end up shuffled around in a secondary buyout/sale to another PE firm.
A rollup may be as straightforward as combining the operations of two A/B testing tools like Monetate and SiteSpect, or as sprawling and ambitious as creating a MarTech juggernaut, as Insight Partners have done by buying Episerver, then having Episerver acquire Optimizely, then having the new Optimizely buy up tons of smaller MarTech companies like Zaius (CDP), Welcome (marketing orchestration), and NetSpring (warehouse-native analytics).
I fully anticipate plenty of private equity activity to come here. For example — given that they’re now 2 years in to the investment, I wouldn't be surprised to see Centre Lane Partners (the firm who owns Monetate) look for one or two more acquisition targets in the next 18-24 months now.
A Tale of Two Markets
In contrast to the traditional, website-focused A/B testing tools we’ve discussed are the new kids on the block: broader experimentation platforms.
See, while the traditional tools served marketers well, they did not serve the rest of an organization so well. And while a few forward-thinking companies invested heavily in building their own infrastructure for experimentation, everyone else was stuck. Davis Treybig named the phenomenon in a 2022 article that’s since been cited in keynotes, courses, and LinkedIn posts thousands of times: The Experimentation Gap.
You can read Davis’ original article for the full story, but the short version is that the myriad use cases that website tools left unserved, plus tech advancements in the data stack (and the advent of data teams), opened the door for commercial experimentation platforms to take shape.
Eppo, Statsig, Growthbook, ABSmartly, and Spotify Confidence all appeared around roughly the same time, with slightly different approaches. The broad theme that unifies them all was an attempt to close the Experimentation Gap and replicate the wide-ranging capabilities that tech giants like Microsoft, Netflix, Airbnb, Meta, and Booking.com had built in-house.9
In practice, these vendors certainly seem to constitute a distinct market from the website tools. The buying committee is different, the day-to-day users are different personas, and (in my experience) they are not perceived by customers as substitutable goods. At Eppo, we were not competing in sales deals against VWO or AB Tasty, and I’m sure they weren’t hearing about us from their prospects either.
(Re: lack of substitutability… these experimentation platforms aren’t putting a meaningful dent in filling demand from marketers for user-friendly, non-technical solutions, maybe because their focus has been on previously unserved needs like product experiments, AI model evaluations, or backend use cases. The growth of experimentation platforms doesn’t seem to be visibly eroding the market for website A/B testing tools right now.)
It’s a relatively new space — less than 5 years old maybe. And while two or three players in the traditional space are making product moves to help them serve these customers too10, it’s still a far less fragmented market.
You really only have to look at the investment activity itself to see the difference: it’s not private equity buying up companies here, it’s large venture capital investments and strategic acquisitions by publicly-traded companies.
2) The growth-stage experimentation platforms are expanding into parallel markets
In 2024, I conducted the second-most job interviews of anybody at Eppo — 73 to be exact. (I lost first place by one interview, to the great Eric Metelka)
I remember leaving one of my interviews with a VP Marketing candidate aghast because the guy had the audacity to suggest that Experimentation, categorically, should be positioned and sold as a child category of DevOps. Actually, he used some far stranger buzzwords too that I had never heard before and haven’t heard since — I can’t actually remember what it was, “value stream engineering” is the best my memory can conjure — but it all sounded like nonsense to me. What about all the marketing, product, ops use cases? “This guy must have such a small view of what experimentation really is,” I told myself.
Now I work for Datadog11.
And while I still don’t think that experimentation should be categorically positioned as subservient to… anything, really, I do see increasingly see the Go-to-Market power of coupling it with a neat partner like that.
It’s been just shy of two months since the acquisition, and I’m happy to report that my enthusiasm around the Datadog + Eppo strategic narrative has only grown. After a few weeks of conferences talking to Datadog customers in the field, it is very apparent to me — somebody who has spent a decade trying to get people to run experiments — just how powerful of a lever “feature flags + experiments + product analytics inside your DevOps + observability platform” is going to be in getting companies to actually run experiments.
Statsig has taken to a very similar product strategy (i.e. expansion into parallel markets) to justify their shiny new $1B valuation. Their homepage today gives equal billing to experimentation, feature flagging, product analytics, and session replays.
This is the great thing about experimentation: it’s a critical tool with so many applications. It's what makes product analytics actionable (taking us from correlations to causation), gives DevOps the ability to test in production, allows AI evals to go beyond eyeballing output to quantitatively measuring real business metrics.
It just so happens that all of these applications — DevOps, Evals, Analytics — have much larger markets.
Publicly available estimates of the A/B testing/experimentation market peg it around $1-1.3B today. And while these types of estimates I’m linking to are all junk, that order of magnitude does seem about right compared to my personal informed guess and the estimates of VCs I’ve talked to. In comparison, the market for Product Analytics is estimated at something like 7-8x that. DevOps at $10-15B. Who even knows how big the market for AI evals will become?
The companies in each of those other markets who can provide robust experimentation alongside their offerings will have unbeatable competitive advantage. In a few years’ time, it'll be a ridiculous notion to even consider buying a product analytics or AI eval or DevOps platform that didn't also enable experimentation.
This is why Amplitude built an experimentation product, why Datadog bought Eppo, why LaunchDarkly bought an observability company12, and why Statsig raised at a unicorn valuation. For the venture capital-funded companies, taking this path is essentially necessary to hit the growth targets that come along with that capital.13
And while I think the market for experimentation platforms should be $10B or $15B instead (no brainer, IMO), I’m not mad about taking any path that’s successfully going to get companies testing everything. Even if, in the most cynical view possible, it feels like jumping into a Trojan horse.
I prefer the way Nils Stotz framed it in a LinkedIn comment: companies are building “platforms that make testing invisible because it’s already built into everything”. Or Marcel Toben in the same thread, opining that this strategy provides “a strong signal that experimentation is becoming a first-class citizen in modern product development”.
This is a high-growth market serving early adopter customers today, with plenty of headroom, IMO well beyond the made-up estimate of 10.9% CAGR in that graphic above.
It’ll take a couple years at minimum to see how these investments are playing out - we’ll likely see signal from the private equity investors and strategic acquirers first, maybe the venture capital investments later. But it’s quite the consortium of forces driving money into experimentation right now, and lots of untapped potential still to get more companies running experiments. As for me, I now have 30,000+ Datadog customers to go reach with the good word.
I include OfferFit here since they’re fundamentally a contextual bandits product, functionality they share with many experimentation platforms. One could argue to also include Hightouch in this category, who raised $80M this year at a $1.2B valuation, but Hightouch is doing a lot more CDP and Reverse ETL work at its core.
A few ex-Eppo colleagues have also joined Aampe, which has a similar 1:1 personalization focus (powered by “agentic AI”) and raised a Series A in December - their Chief Scientist says the tech is “bandit-like” but “moved pretty far from textbook bandits”.
Monetate had itself been purchased by private equity in 2019, when Vista rolled it into Kibo. This lasted for almost 3 years exactly, when it was then spun out again and sold off to Centre Lane.
the “ZIRP (Zero Interest Rate Policy) era”, if you’re unfamiliar - I realize I’ve used that term in at least one newsletter issue already and readers may not be familiar
One of the hardest parts of building a product like this used to be building the capabilities to collect and store data for customers to run experiments on. A company like Optimizely essentially had to build mini-event tracking and data warehousing and identity resolution products. Today, many potential customers have all that infrastructure already from tools like Segment, Snowflake, or Heap — or their platform of choice, e.g. Shopify. New contenders can pop up and offer a basic but usable set of experimentation tools with much less effort.
Of course, there was also plenty of variance in how this inspiration took root. Statsig, for example, stayed very close to Meta’s internal tool in their early product. Eppo CEO Che Sharma chose intentionally not to recreate Airbnb’s.
“Eppo decidedly did not transplant Airbnb's ERF but instead did a from-scratch rethink of the whole workflow. The Eppo team and the Eppo product pulls from a much wider diaspora than Airbnb and Webflow: Uber, LinkedIn, Stitchfix, and a host of smaller in-house setups like Angi, Big Fish Games. In a way, the most Airbnb influence in Eppo is the focus on design, on quality, on the user.”
Optimizely and Kameleoon come to mind: both have stood up some sort of relationship with the data warehouse (Optimizely a true Native option, albeit currently detached from their main UI, and Kameleoon a two-way integration), implemented CUPED, etc.
Datadog’s homepage hero bills it as “modern monitoring & security”, and the major product categories in its website navigation are Observability, Security, Digital Experience, Software Delivery, Service Management, and AI.
what we in the business call a “reverse Datadog”
Growthbook (as an open-source tool), ABSmartly (bootstrapped), and Spotify Confidence (internal startup/business unit) are free to continue focusing on experimentation/flagging on a standalone basis.
Wow! This is one of the most thoughtful article that I read focusing on our community and where we are heading! Absolutely love it (not saying this because I am quoted btw haha). You are doing amazing work, Ryan! Please keep it up!
Filling the experimentation gap and strategically using it to expand to larger or more "traditional" markets makes so much sense actually. I guess the assumption (or hope) is then also when experimentation becomes seamless and invisible it does not need to go too deep into the "Why" anymore but that everyone already understands this and we can focus on the "How" :)