Mobile game ad creative testing: how to run tests and scale the winners

Most of the spend on gaming video ads goes to a small share of the creative variations. In AppsFlyer's data, the top 2% of gaming video creative variations took 60.7% of video spend in Q2 2026, up from 53.5% in Q1 2025, and the top 10% took 85.6% (AppsFlyer, State of Creative Optimization 2026). Finding those few winners, and replacing them before they wear out, is what creative testing is for.

This guide covers the method: what to test, how to set a test up, how long to wait before reading it, when a winner is real, how to scale it and when to refresh. For producing better creatives in the first place, see our earlier post on maximizing the results of your mobile game creatives.

Why testing volume matters

Because spend piles onto so few variations, the studios that test more find more of the winners. AppsFlyer's figures for gaming show the gap widening with budget. Apps spending $7M+ a quarter on UA averaged 3,977 creative variations in Q1 2026, up 45% on a year earlier. Apps spending under $100K averaged 84, down 20%. Our own arithmetic on the same figures: the top tier ran 4.5 times the variations of the $1-4M tier, up from 3.5 times a year before.

A smaller budget tests well when every test answers a question worth answering.

What to test: concepts before iterations

There are two kinds of creative test, and they need different budgets.

  • Concept tests compare different ideas: a gameplay video against a fail-ad hook, a character story against a puzzle tease. A new winning concept can change a game's CPI, so these are where the big gains are. They also fail most often.
  • Iteration tests change one element of a proven concept: the first three seconds, the end card, the music, the copy, the aspect ratio. Gains are smaller and more reliable, and iterations extend the life of a concept that already works.

A healthy testing plan runs both. A common failure is to only iterate on one old winner until it fatigues, with no new concept ready to replace it.

How to set up a test

  1. Write the question first. "Does a story hook beat a gameplay hook for this audience?" is a test. "Let's try some new videos" is not.
  2. Change one thing at a time in iteration tests. If the hook, the music and the end card all change, you will not know which one moved the result.
  3. Keep a control. Run the current best creative alongside the new ones, so you compare against what you would otherwise spend on.
  4. Hold the rest steady. Same network, same geo, same audience settings, same bid strategy, same dates. Differences in setup will show up as differences in creative performance.
  5. Use the network's own testing tools where they exist. Meta's A/B testing and TikTok's Split Testing show each version to a separate part of the audience, so nobody sees both, and TikTok's works with App promotion campaigns. Google Ads rates App campaign assets Best, Good or Low against assets of the same type in its asset report, and for Android apps an App asset experiment can test added video assets against the original campaign. A split test controls for delivery better than two campaigns set up by hand.

The statistics behind this are the same as any A/B test. Our guide to A/B testing in mobile games covers sample size and significance in more depth.

How long to read a test

A creative that looks strong on day one often settles once the network's algorithm moves past the users most likely to click anything. Three rules keep you from calling tests too early.

  • Wait for volume, not days. Set a minimum number of installs or a minimum spend per variation before you read the result, and do not decide before it is reached.
  • Read the right metric. Click-through rate tells you the ad gets attention. Installs per mille (IPM) and CPI tell you it brings players in. Retention and early ROAS tell you they are the right players. A creative that wins on CTR and loses on day-7 ROAS has found curious users, not paying ones.
  • Cover a full week. Weekday and weekend audiences behave differently in most genres, so a test that ends on a Friday is reading half a week.

When a winner is real

A variation has won when it beats the control on the metric you chose, with enough volume that the gap is unlikely to be noise, and it holds when you repeat it. Before you move real budget, check three things.

  • Does it hold on a second run? Re-run the winner against the control, or in a second geo. A real winner repeats; a lucky one falls back toward the control.
  • Does it hold downstream? Check retention and revenue of the players it brought in, not only the install metrics.
  • Is the gap worth the change? A 3% CPI improvement inside the noise band does not justify retiring a proven creative.

How to scale a winner

  • Raise budget in steps. Large jumps can reset a network's learning phase and make a winner look like a loser for reasons that have nothing to do with the creative.
  • Adapt it for each network. A winner on one network is a strong candidate on the next, not a guaranteed one. Re-cut it for each network's formats and placements, and test it there against that network's control.
  • Localize the proven version. Translate and adapt a winner for new markets before you invent new concepts for them.
  • Build iterations early. Start producing variations of a winner while it is still winning, so the replacements are tested before it fades.

Creative fatigue and when to refresh

Every creative wears out as the audience that responds to it is used up. The signs are steady: IPM drifts down, CPI drifts up and frequency rises, while targeting and bids stay the same. Fatigue arrives faster on smaller audiences and at higher spend.

Watch the trend for each creative rather than the daily number, and treat a sustained IPM decline against the creative's own early level as the trigger to rotate in the next tested iteration. The goal is a pipeline where a tested replacement is ready before the current winner fades, so spend never sits on a tired ad while you wait for production.

A creative testing checklist

  1. Every test starts with a written question.
  2. Concept tests and iteration tests both run, with separate budgets.
  3. One variable changes per iteration test, against a live control.
  4. Results are read after a minimum volume and a full week, never on day one.
  5. Winners are judged on retention and early ROAS as well as CPI and IPM.
  6. Winners are re-tested before budget moves.
  7. Budget scales in steps, and each new network gets its own test.
  8. Iterations of the current winner are in testing before it fatigues.

How SuperScale runs creative testing

Our creative team runs this method for the games we manage, briefed from live campaign data. When a studio wants production and testing handled for them, creative production is scoped around user acquisition and billed separately. The pricing page covers how our two offers work.

SuperScale

Loading…