TL;DR
Creative testing is not free: it takes budget, time, attention, and a decision that someone is prepared to act on. That does not make testing less valuable. It makes prioritisation more important.
On the second episode of Boringly Effective, Andrew Tindall of System1 offers a useful question for marketers: when should you not measure creativity? His answer is not a universal percentage or a case for abandoning research. It is a practical rule: concentrate serious creative data on the work that has enough reach, investment, and consistency to have an effect.
The implication is simple. Do not give every asset the same research treatment. Give the consequential work enough attention to make a better decision.
The question marketers rarely ask
Most discussions about creative testing begin with the opposite question: how can a team test more work, earlier, faster, or more cheaply?
Tindall starts elsewhere:
“When not to measure creativity?” — Andrew Tindall
That question matters because more content does not automatically mean more strategically important content. A campaign can contain a large number of social cut-downs, adaptations, variants, and executions. But those assets do not necessarily have the same media support, expected audience, duration, or commercial importance.
The useful decision is not whether testing is inherently good or bad. It is whether the result of this test could change a meaningful decision about work that will actually be seen.
Test the work that has a realistic chance to matter
In the interview, Tindall describes the volume of assets created by the industry, particularly as AI makes production easier. He says he would focus creative data on the smaller share that will receive sufficient support.
“I would say only about the top 10% of that is going to get enough reach, enough investment enough consistency to even do anything I would not waste creative data on this 90 wash of shit I would only bother focusing on that on that 10 if I'm completely honest because most of the tools you get on that bottom 90 aren't really there yet there's a lot of people selling ai creative testing stuff that's just like an llm plugin that's made up and will change any time you bang it in.” — Andrew Tindall
The 10% is Tindall's interview estimate, not a planning standard to copy into every business. Its value is the underlying prioritisation principle.
Before commissioning or interpreting a test, ask four questions:
- Will this work get meaningful reach? A test result is more consequential when the work will be seen at a scale that can affect the outcome.
- Is there enough investment behind it? The larger the production or media commitment, the more expensive a poor creative decision may be.
- Will the idea run consistently? A one-off execution and a long-running platform do not carry the same decision risk.
- Can the result still change the work? Testing is most useful before the decision is locked. If nobody can alter the asset, placement, or investment, be honest about what the research is for.
These are planning questions, not claims that a campaign below a particular spend threshold should never be tested. A small piece of work can still be important. The point is to match the level of evidence to the consequence of the decision.
Do not confuse a fast prediction with a complete answer
The episode also addresses AI-based creative prediction. Tindall describes models as potentially useful for providing an initial read, but says that they do not remove the need for human response data when the objective is to understand how advertising is performing.
“So far, I think a lot of those AI testing tools have been, as I said, either trained on shite data or simply LLMs, open AI cloud plugins that are guessing from their vast model.” — Andrew Tindall
His reason is tied to creative difference and surprise. In the conversation, he says those elements are difficult for AI to resolve exactly and that the difference between AI and human response may itself be informative.
For a marketing team, this creates a useful distinction:
- A quick prediction can help sort, compare, or prompt a discussion.
- A decision about consequential work may require evidence from real people.
- Neither output is valuable unless the team knows what decision it will inform.
The right question is not whether a tool is labelled AI or research. It is whether its evidence is fit for the decision in front of you.
In B2B, audience precision is not the whole brief
B2B testing can be difficult and expensive when the intended audience is highly specialised. Tindall discusses the practical challenge of recruiting particular decision-makers, including senior finance audiences and specialist buyers.
But he also argues that a narrowly defined research sample should not distract marketers from a more basic advertising requirement:
“What I actually find is that good advertising makes a broad set of people, anyone, feel something, and it interests them.” — Andrew Tindall
This does not mean that B2B marketers should ignore who buys, who influences, or what the buying context demands. It means that technical relevance is not a substitute for being interesting. A test plan should investigate whether the work communicates to the intended audience, but the creative brief should still demand an emotional and distinctive response rather than accepting dullness as the price of specificity.
A practical creative-testing triage
Use this sequence before adding a test to the plan.
1. Name the decision
Write down the decision the test will influence: whether to make a creative change, select one execution, change the level of media investment, or stop a piece of work. If the team cannot name the decision, it is not ready to specify the research.
2. Estimate the consequence
Consider the expected reach, investment, duration, and strategic importance of the work. This is where a flagship campaign, a new brand platform, or an expensive launch usually merits more scrutiny than a low-support adaptation.
3. Decide what evidence is needed
A low-stakes comparison may need only a fast directional input. A high-stakes creative decision may need direct evidence of human response. The methods should follow the risk, not the other way round.
4. Preserve time to act
Build testing into the production plan early enough that the result can change something. A post-launch scorecard can be useful for learning, but it is not a substitute for a pre-launch decision process.
5. Record the learning
The value of a test should not end with one asset. Record what changed, what did not, and why the team made the decision. Over time, that makes the next brief and the next test more useful.
What this approach does not say
It does not say that only large brands can test advertising.
It does not say that every high-reach asset will be effective if it receives a good score.
It does not say that an AI tool cannot be useful.
And it does not turn Tindall's “top 10%” comment into an evidence-backed universal ratio. It is an interview perspective about where he would focus attention in a high-volume content environment.
What it does say is more modest and more useful: research effort should be proportional to the decision it supports.
Key takeaways
- Test the decisions that matter. Reach, investment, consistency, and the ability to act on the result are better starting points than asset volume.
- Treat the “top 10%” as Tindall's rule of thumb, not as a universal threshold.
- Use fast prediction and human testing for the jobs each can actually do.
- Do not let audience specificity excuse uninteresting B2B advertising.
- Design research around the decision, then leave enough time to use it.
Source and further listening
- Andrew Tindall, Boringly Effective episode 2: creative testing, AI, emotion and marketing effectiveness. The quotations and interview-specific claims in this article are taken from the published episode transcript.