Growth hacking with GenAI: the engine has three strokes, and AI only turbocharged one
The promise that AI transforms growth is true for one stroke of the engine and dangerous for the other two. Testing became nearly free. Measurement collapsed. Deciding is still yours. Running more tests you trust less is not progress. I show it with a real case where I cut a Target CPA 20% without a single hack, just by measuring the game.
Growth hacking was never a bag of tricks. It was always a three-stroke engine: test, measure, decide. GenAI turbocharged the first and is breaking the second. And almost everyone celebrates the turbo without noticing the speedometer stopped working.
The strongest version of "AI changed everything"
Start with the strong version of the opposing argument, because it is not nonsense. The cost of experimenting collapsed. What used to take a team a quarter (a hundred ad variants, ten landing pages, a user-interview script) now ships in an afternoon. AI generates hypotheses from your data, simulates segments, analyses results in seconds. Speed of learning was always growth's edge, and the idea, repeated since Amazon, that whoever runs more experiments wins favours whoever iterates fastest. If AI multiplies experiments tenfold or a hundredfold, it multiplies the advantage. A solo founder now does what used to need a whole team. This argument is real and, on the execution side, it is right.
What that argument misses: growth hacking was always a three-stroke engine
The argument's flaw is not what it says. It is what it ignores. Sean Ellis, who coined growth hacking in 2010, never described tricks: he described a profile obsessed with a single growth metric as its north star (Ellis, 2010; later in the book Hacking Growth, 2017). Earlier, Dave McClure had already handed over the ruler with AARRR, the pirate metrics. The engine was always the same: form a hypothesis, test it, measure the effect, decide to kill or scale. Testing faster only compounds if you can still measure what the test did and still have the judgement to decide. AI turbocharged the testing. It did not touch the deciding. And it is sabotaging the measuring.
AI turbocharged testing and broke measurement
While the cost of testing fell to zero, the ability to measure the result crumbled in parallel. The click was already vanishing before the LLMs: a large share of search ends with no click at all, with everything that does to your KPIs (I covered it in zero-click search and SEO KPIs). Then the referrals themselves began to evaporate: OpenAI altered its referral strings and a good share of that traffic stopped carrying a source, showing up as Direct (I wrote about this around Avinash and OAO). Add modeled conversions and Consent Mode in GA4 and you get a dashboard that guesses more than it measures. So yes, you can run a hundred tests in a week. The uncomfortable question is a different one: do you trust what any of them tells you?
Most experiments were never trustworthy
Here is the part nobody wants to hear. Even before AI, most growth experiments were not trustworthy. Ron Kohavi, Diane Tang and Ya Xu, who ran thousands of tests at Microsoft, Google and Amazon, wrote the field's reference book on how to do them right (Trustworthy Online Controlled Experiments, Cambridge, 2020). And in a paper dedicated to the pitfalls, A/B Testing Intuition Busters (Kohavi, Deng and Vermeer, 2022), they catalogue why so many results mislead: underpowered tests, sample ratio mismatch, flexible data processing that inflates false positives. They invoke Twyman's law: a figure that looks interesting is usually wrong. Multiplying a machine that already produced noise by a hundred does not give you a hundred times more truth. It gives you a hundred times more noise, faster.
What gave me an edge was measuring the game, not a hack
The best growth play I made for a client came from no new tool. It came from measuring what the competitor was doing. The client was pure lead generation, run to a Target CPA, with the duty to improve cost per lead to the cent. We built an analysis of direct competitors' daily spend on Google Ads, and a pattern appeared that they themselves probably could not see: they spent most of the budget in the first three weeks of each month and slowed down in the last, because they got there with the monthly budget almost gone.
Most would have followed the market's rhythm. We did the opposite. We renegotiated a budget that was no longer monthly. In the first three weeks we held back, deliberately restraining spend. In the last week, when the competition was already out of budget and the auctions got cheaper, we went in concentrated and hard, funded by what we had held. We did not spend less. We spent at the right moment.
The Target CPA dropped 20%. The edge was not a hack. It was measuring the opponent's rhythm and having the judgement to restructure the budget to exploit their constraint. (The same logic of watching signals and acting early that I use in Google Ads scripts.)
And here is the point for 2026: GenAI would have generated the hundred ad variants in an afternoon. It would not have measured the competitor's daily rhythm, would not have connected the dots, and would not have renegotiated the budget. The insight and the nerve do not come in the prompt.
The measurement growth needs in the AI era
If testing became free and measuring became hard, the edge moved to whoever can measure. Three layers, concretely.
First, real incrementality, not click attribution: holdouts and geo-experiments, the methodology Google itself documented more than a decade ago (Vaver and Koehler, Measuring Ad Effectiveness Using Geo Experiments, 2011). You split geographies or audiences, hold a control group, and measure what the spend actually added, not what the pixel claims.
Second, guardrails before looking at any result: a hypothesis written before the test, enough statistical power, a sample ratio mismatch check, guardrail metrics to catch collateral damage. It is Kohavi's discipline, and it is what stops AI from selling you noise faster.
Third, the process: a hypothesis backlog prioritised by ICE (impact, confidence, ease), tested at high tempo, and a growth model built as a loop with actionable input metrics, not a static funnel you look at once a month. It talks to what I have already written on North Star and AARRR and on attribution models, without repeating it here.
Where GenAI actually fits, stroke by stroke
To avoid both the hype and the disdain, map GenAI onto the engine's three strokes.
On testing, it changes the game: it generates hypotheses from your data, produces variants at near-zero cost, fills the backlog faster than any team. Use it freely here.
On measuring, it does not belong. Worse: if you trust the outputs without your own instrumentation, it accelerates the error. AI does not create the target metric or the measurement infrastructure you lack. That is yours.
On deciding, it helps at the margins: anomaly detection, summaries, catching a sample ratio mismatch you missed. But killing or scaling is still a judgement, not an output.
Whoever accelerates the testing with GenAI and ignores the measuring is not doing growth. They are making noise faster and calling it strategy.
Your account's next edge is not in the tool that generates more variants. It is in what you can still measure when everyone around you has stopped seeing.
Sources
- Kohavi, Tang & Xu - Trustworthy Online Controlled Experiments (Cambridge, 2020)
- Kohavi, Deng & Vermeer - A/B Testing Intuition Busters (KDD 2022)
- Vaver & Koehler - Measuring Ad Effectiveness Using Geo Experiments (Google, 2011)
- Growth hacking: origin of the term (Sean Ellis, 2010)
Frequently asked questions
Doesn't GenAI solve measurement with first-party data?
It helps collect, not interpret. First-party data improves the input; it does not replace causal design. Without a holdout or a geo-experiment, you keep confusing correlation with effect, however clean your base is. See attribution models →
Can you measure incrementality without an enterprise budget?
Yes. A simple geo-experiment (turning spend on in one region and keeping a comparable one as control for a few weeks) gives you an honest causal signal with no expensive stack. The hard part is not the tool, it is the discipline to hold the control and not touch it halfway.