You know how important A/B testing is — you’ve read all the case studies about millions of dollars made and maybe had shocking results of your own. You even sleep with a copy of Lean Startup under your pillow.
But testing takes a lot of time, and when you run A/B tests on Facebook Ads, it takes a lot money. So are you sure that you’re setting your Facebook A/B tests up in a way that maximizes learning given your limited resources?
It’s a problem our clients at Ladder face every day, and figuring out how to deal with it was important to increasing ROI on the Facebook Ads channel.
So I came up with a few tips and tricks that stand as a quick and dirty way to solve exactly this problem for them. I’m sharing those tips and tricks with you in the hopes that you can take better advantage of the powerful A/B testing capabilities of the Facebook Ads platform.
Along the way, you’ll find Facebook Ads tactics drawn straight from the Ladder Planner. You can use those unique ads approaches, powered by the targeting capabilities of Facebook’s robust ads platform, to run smarter A/B tests on different audiences and with different copy/creative assets.
Remember that while these tips are a walkthrough of Facebook ads specifically, you can apply the principles to any ads platform. With that said, let’s start with…
Before you start testing, have a clear idea of WHY you’re doing it. Good questions are “How should I price my product”, “What audiences are most interested?” or “Is loss-aversion or gain-seeking a better hook for my ad-copy?”.
This tactic requires a media budget
Pointing to a potential loss of not buying your product can be an effective way to drive clicks. This drives a sense of urgency for a customer to avoid losing out on an item they’re interested in or a money-saving opportunity. Why? people are naturally risk averse and would prefer to avoid losing out on an opportunity to get a better deal or miss out on an item they want.
Examples:
“Sale ends in 30 minutes!”
“Last chance to get the designer-branded jacket line!”
“Only 10 spots left in our email marketing webinar!”
Try to focus here — you’re wasting your time and money testing 50 shades of blue unless you’ve got millions of users and dollars (as you’ll see below).
This tactic requires a media budget
Pointing to a tangible gain in performance (rather than avoiding a loss) can drive action effectively. For example, highlighting a BOGO deal on a product can drive a visitor to convert into a customer by targeting their desire to own more of something than they may actually need. They’ll be driven to act by the prospect of gaining something for seemingly nothing.
Examples:
“For a smooth shave, try our razor”
“Buy 2, Get 1 Free!”
So how do you figure out what you can test with your budget and still get a result you can trust? You can read up on statistical significance, but I find a good rule of thumb is to try for 30 conversions per variation. Ideally in this case a conversion a purchase or signup, with a focus on getting as close to revenue as possible.
Now for a quick calculation using your best guesses:
The above figure sounds really, really expensive, right? Don’t have that kind of money? There’s still a ton you can do.
Try running the same test, but concluding based on what variation gets the most clicks (instead of conversions). This would cost 1/20th the amount — only $270 total — because getting to 30 clicks costs $30 per variation, vs $600 per conversion.
Of course you should trust the result less, because some variations are prone to driving clicks that will never convert. This approach requires a good bit of knowledge and understanding of what drives people to become paying customers for your business.
My best advice here? Use your best judgement.
This tactic requires a media budget
An appealing bold color in the background of an image can attract attention to your ad. If the image stands out from the rest of the page of a social network or site where the ad is served, it has a chance to draw eyes directly to it and your ad will be more likely to be seen. This can improve ad performance and help you better reach your audience.
Another way to save budget is to bandit test — dropping variations as you go, based on early performance. So if Variation A gets 300 clicks in the first few days while Variation B gets only 100, you can use the bandit trick to just drop Variation B with the assumption that it’s a worse performer.
The caveat here is that you’ll be much more likely you’ll drop a variation that could have performed well (false negative), but it costs less and will get you to a good enough result quicker. And beyond that, you should always be testing and re-testing variations that you believed weren’t performing well, trying them with different audiences or slight tweaks.
Pro tip: Facebook already uses this trick to automatically find the best ad-copy in an ad set. So if you need performance more than rigor, dump everything in one ad set and let the Facebook gods decide.
This tactic requires a media budget
Fans / followers of your business on social media are a strong indicator of the audience you should target. Create a lookalike audience of your social media following by setting your ad to target people similar to them. This will give you access to a highly qualified audience with the same traits as people who have already indicated interest in you. If you have fans who aren’t part of your ideal audience, you can narrow down further with demographic targeting.
Another neat trick? Deprivation testing. This works well for anything that’s difficult or impossible to A/B test side-by-side, like app store copy or the price of your app (it’s also one of the few ways to conclusively test SEO tactics).
Here’s how it works:
Every week for 6 weeks we’d cycle through the 3 variations, starting fresh by recreating the campaigns to erase history and make it a fair test.
This way you can parse out the relative performance without worrying too much about the effect of a single ‘bad’ week.
This tactic requires a media budget
Once you have a solid lead list, you’ll be able to build an audience for your advertising based on what those leads “look like.” Say you find that a majority of your leads are startups in the New York City area – you’ll be able to build and target an audience of other NYC startups, founders, and their employees. With this approach you’ll drive qualified traffic to your site – businesses and individuals similar to ones that have already displayed interest in your product.
This one is a more advanced (read: time-consuming) version of deprivation testing. For it to work, you have to get a list of geographic areas (zip codes / states / DMAs) and randomly assign them to each variation.
Each needs the same number of areas and a similar total population size:
You should also choose a specific ‘control’ population, that won’t get any treatment — to compare the results against. This is much more trustworthy method than simple deprivation testing, which is why it’s commonly used to measure the ROI of TV advertising.
The reason this works is you can isolate specific geographies where you’re getting the absolute best performance from a variation. Based off the results, you can determine which geography gives the best return on investment and double down with your advertising spend there.
This tactic requires a media budget
Targeting an audience at a more local level can help you better target your audience and avoid serving ads to businesses or consumers that you’re not trying to reach. Especially if your business is a retail store with locations in specific geographies or if you want to focus an ad campaign on audiences in specific states or countries, narrowing your ad’s geographic reach will focus your ad spending on the most relevant audiences.
What if you’re testing on Facebook but the results have wider implications? For example, trying taglines to use in a TV ad, or testing whether a startup idea is worth pursuing? If a little extra spend now can potentially save you millions down the line, it pays to be rigorous.
In this case, your best bet is to set up one ad set per variation to make your test as precise as possible. This gives you the best shot at serving each variation to an equal amount of people and arriving at a valid result.
It’s a quick and dirty way to validate a variation without necessarily breaking the bank, but does require a bit more spend than most of the other tricks above.
That’s the end of the quick and dirty A/B testing tricks, but I have a few more tips for you to use when creating Facebook Ads.
These are best practices that can make sure you’re using the tactics and ideas above to get actual, valid results and higher click rates / conversion rates.
And that’s our quick primer on how to run effective A/B tests with your Facebook Ads. Remember, you don’t have to spend thousands to get valid results from your A/B tests. What you do have to do is move a bit higher up the funnel and have a strong understanding of your business and audience. With that combination, you’ll be able to make valid assumptions about a variation’s performance even if you haven’t reached statistical significance.
Need to run A/B tests on your Facebook Ads? Ladder can help.
Review your growth goals with a Ladder Strategist:
Sign up to our newsletter to stay up to date with all the latest movements in the field.
More from Ladder Blog
The art world is being disrupted by generative AI, and artists aren’t happy. Generative AI models like DALL-E, Midjourney, and Stable Diffusion were trained by scraping millions of images from the internet, without permission. Artists claim this violates copyright, while the companies training these models argue this falls under fair use.
Read More →In the world of digital marketing and data-driven decision-making, creative testing is a pivotal tool in achieving business growth. Gone are the days of relying on gut feelings or guesswork; now, business decisions are powered by data-validated insights, meticulously collected, analyzed, and validated. This transformative process empowers businesses of all sizes, from established enterprises to budding startups, to thrive in an ever-evolving digital market. This article looks at the practical applications, challenges, and innovative transformations associated with creative testing, offering you valuable insights and actionable strategies to implement in your own digital marketing efforts for achieving growth and success.
Read More →
