How to test Facebook ads creative one change at a time

Say you sell candles. Ad A is the candle on a shelf, with a headline about how long it burns and no offer. Ad B is a hand lighting the wick, with a headline about gifts and free delivery. You run both for a week and B wins.

Was it the picture, the headline or the delivery? Nobody can say, so the next ad has nothing to copy. Meta's page on testing more than one variable says the same: you cannot tell which one led to the winning ad set.

To test Facebook ads creative, change one thing between two versions of an ad, show them to separate groups of people on the same budget, and leave the test running for at least a week. Meta's page on A/B testing says its tool does the splitting and ensures nobody sees both versions.

Test Facebook ads creative by changing one thing between the two ads

Meta's best practices for A/B tests say the two ad sets should be identical except for the variable you test. Everything else stays put: the headline, the offer, the audience, the placements and the budget.

A different crop is a second change, so cut both versions to the ratio Meta lists for each placement.

Make the change big. Meta asks for versions with distinct differences, and its example tests include a 1:1 video against a 9:16 one. A headline with one word swapped is a difference a small account is unlikely to see in so few orders, as the arithmetic below shows.

The change can be a bundle when the bundle is the question. A candle as a gift against a candle for its long burn moves picture and headline together, which is the kind of difference Meta asks advertisers for. The test cannot say which half did the work.

How to do creative testing on Meta ads

In Ads Manager, duplicate the ad you already run as an A/B test, change one variable in the duplicate, set a schedule and a budget, and choose the metric that decides the winner. Meta then shows each version to a separate part of your audience and names the one with the lower cost per result.

Version A stays as it is and you edit Version B. Meta recommends the same budget for both and takes a schedule of 1 to 30 days. If the duplicate option is missing, Meta lists the Ads Manager toolbar and the Experiments tool as other ways in.

Name your hypothesis first, as Meta asks: the hand lighting the wick gets a lower cost per purchase than the shelf photo. Keep the test audience out of your other campaigns, because Meta's ads guidance on A/B testing says overlap can cause delivery problems and contaminate the results. Once the test runs, Meta locks the variable you chose.

How long to leave a test running

Meta's help centre recommends at least 7 days and says shorter tests may be inconclusive, while its advertiser pages say at least two weeks and up to 30 days. If your buyers take over a week to buy, run longer; Meta's example is 10 days.

Wait for the end. Meta says results start to show at 100 events observed for the cost per result you test, and to judge them after the test has ended. Leave the ad set alone meanwhile, since adding an ad or changing the creative is a significant edit that can send it back into the learning phase.

A winner ages too, and how to check an ad for creative fatigue covers what to look for.

What winner and confidence mean

The winner is the version with the lower cost per result. Confidence is Meta's name for the chance of similar results if you ran the test again, and Meta's page on confidence treats 65 per cent or more as a winning result in an A/B test, against 90 per cent for a lift test.

A fair reading is that 65 per cent gives you a lead to follow with a second test.

Why ads in one ad set are not a fair test

The informal test is several ads in one ad set, with a watch on which gets the spend. That shows which ad Meta chose to deliver, not which ad is better.

Meta's page on ad delivery says that with several ads in an ad set, Meta shows the ad most likely to achieve the lowest cost per optimisation event for the given person, so each ad is not necessarily delivered the same number of times. An A/B test, Meta says, helps keep audiences evenly split and statistically comparable.

Separate ad sets do not fix it. Meta's measurement page says ad sets run side by side without the A/B tool are not split evenly, and its page on auction overlap says that when two ads from one advertiser would enter the same auction, only the one with the highest total value competes. That makes an ad set a way to run ads, and a poor way to learn why one beat another.

What a small budget cannot learn

A winner needs results, and a small account buys few. These numbers are arithmetic on Meta's figures, with a purchase costing 14 in ad spend, as in how many ads Meta lets you run.

Meta's learning phase page puts the point where an ad set settles at about 50 results in the week after its last significant edit. If each version is its own ad set, that is 700 a week in spend for one and 1,400 for two. The 100 events Meta mentions cost 1,400: 100 a day over two weeks, or about 47 a day at the 30 day limit. Meta does not say whether the 100 counts both versions together.

Spend 20 a day across two versions and each gets about 10 purchases in two weeks, or 5 a week against Meta's 50. One order more or less moves a version's cost per result by about a tenth, from 14.00 to 12.73 or 15.56, and 30 days buys about 43 purchases in all.

So a small account can test a difference big enough to show in ten orders, such as a different reason to buy or a video against a still. It cannot rank two good ads that sit close together, and every extra version splits the same orders again.

One of Meta's fixes for a Learning limited ad set is an event that happens more often, such as add to cart, which gives more events and asks a different question. The setup screen shows estimated power, which Meta recommends keeping at 80 per cent or more, and a test that ends with no winner may only mean the versions differ by less than your budget can see.

Choosing what to test and making the versions

Before Ads Manager comes the slow part: choosing which reason to buy deserves a test, and making the versions. Ask Nomi for a campaign and the strategist reads the ads in your market, then comes back with what it would run before it draws anything. You change the plan in a sentence and approve each ad it makes.