Performance and testing split – how to do it right?
A/B Testing
Creative Testing
Audience Testing

Performance and testing split – how to do it right?

Jon BrodyJon Brody

July 15, 2021

There will always be a need for growth. The size and market value of your business don’t matter here, as there always will be a place to improve something, to test new opportunities, new channels, and try to reach new audiences. That’s why growth marketing, even though still new, is evolving so fast, as more and more companies experiment and look for the best possible way to grow their businesses with a mix of performance and testing.

The truth is that most of the time, it’s not about reinventing the wheel. It’s about making that wheel fit your situation and evolving it accordingly.

That was also our case. After being on the bumpy road for the last three years, lots of experimentation presented us with our happy medium – our take on the Pareto Principle.

But let’s start from the beginning.

Should you test or optimize?

You probably know that you can divide growth into two types of activities: optimization and testing. While the former is a low-risk strategy focused on steadily growing performance, the latter has higher risks and rewards as you need time to see their results, yet a successful experiment will spike up your growth.

Believe it or not, my colleagues and I are continually learning – more often than not, from our own mistakes. One of the most important lessons that we learned is that you shouldn’t focus all of your attention just on one thing – be it tests or optimizations. In our case, it was too many experiments, too little optimization and performance.

Here’s something we learned from that time what’s worth remembering:

  1. You get continuous progress through continuous testing – you just need to balance it against optimization;
  2. In general, companies need performance;
  3. Optimizations surely will produce better results, but it won’t hockey stick growth;
  4. Most tests fail…
  5. …but the successful ones result in sharp spikes up;
  6. Not everything is or should be a test, i.e. optimizations, changes on landing pages with low traffic, etc.

These seem like obvious things everyone should know. But, even though you may think you know that, you probably don’t stick to it. We knew it and needed some time to acknowledge it and change the way we work.

Speaking of which.

Optimization and testing model

Adaptive Growth is different from our previous approach. We took the Pareto Principle and translated it into a growth marketer’s language. Initially, the principle says that 80% of your results come from 20% of your efforts, and the other 20% of the effects come from the other 80% of your actions. Seems reasonable and applicable in every aspect of your life. At least, that’s what the wise heads say.

To understand how it can be adopted in growth marketing, let’s get down to the nitty-gritty with our adaptive growth system.

Add successful tests to the optimizations stream and scale them to boost account performance.

Add successful tests to the optimizations stream and scale them to boost account performance.

As you can see, the system includes five stages, of which the first three will enable you to split your focus on optimization and testing according to the 80/20 rule. So, you’re taking a top-down approach that leads you to your goal – growth.

1. Strategy

It all starts with a strategy. There are multiple types of strategies in marketing focused on its various aspects, i.e. content, social media, digital, creative, etc. What’s more, there are even various types of growth strategies from market penetration to product development to market segmentation. In general, though, a strategy is your plan of action with all the answers for the whys, whats, whos, wheres, and hows. Especially, the “how much”.

In this stage, you decide what are your main goals and how you’re going to achieve them (or at least try to). Also, you fix the budget for those actions and note down all data that you will need later on in the process, i.e. audience, channels, keywords, locations, etc.

With all that information in place, you need to allocate your resources.

2. Resource allocation

Here’s where the 80/20 rule comes in. You devote most of your time on performance and optimizations as they are easier to predict and constantly bring more or less positive results. So, the statistics present a steady growth. In contrast, experiments are unpredictable, yet can produce impressive outcomes. For example, indeed 20% of your time spent on tests may spike up your performance by 80%, just like Pareto Principle states, but you have no idea if it will happen in a month, three, six, or 12 months. Often, you don’t have that comfort to wait and see how it will work out. That’s why you should limit the resources devoted to experimentation, but don’t back out from it completely. Validate on a small scale not to waste too much time and budget, and remember about the overall performance and profitability of the project.

However, how you split the budget into your activities depends on your long- and short-term goals. If you need to see the results in performance as fast as possible, then you should focus more on optimization with little resources allocated in tests. This way, you will gain a steady growth boosted by occasionally successful experimentations. Remember that some part of your budget will be allocated to an agency if you cooperate with one. Then it’s good to divide the allowance into media spend (money assigned to the campaigns, optimizations, tests, etc.) and team budget (a fee paid agency).

Some companies turn towards testing to better understand their core users, who they are, why do they convert, what makes them click. You can use this data later across different stages of the funnel and channels. That is when you can afford to focus more on tests rather than optimizations due to the fact that the business is profitable.

3. Performance

After the budget split, you should specify the channels you will use: social, search, CRO, CRM. Research each of them and base your decision on this data, i.e. keywords in SEM, what are your audiences on LinkedIn and Facebook, or checking the e-mail addresses of your clients in CRM. Evaluations and audits of your channels based on the available budgets and business model will help you decide which one has the best potential to close in on your goal. For example, mobile apps get good results from Facebook Ads and Google Universal App Campaigns, while B2B often relies on Google Search Ads and sometimes LinkedIn ads.

Add to that competitor analysis and working closely with your clients and you will know exactly what worked in the past, what resonates with your audience, etc. With so much information, you will have an inkling on what to optimize and what to test in the next step.

4. Optimization and testing

Optimization

Just like mentioned previously, optimization is a low-risk strategy that maintains the current level of performance, whilst slowly improving results, it may be volume or efficiency, over time. It includes such actions as turning off underperforming ads and campaigns, shifting budget towards more promising ones, as well as keywords or bidding.

When it comes to optimization, it’s important to remember about regular check-ups or, as Ladderians like to call it, health checks. The purpose of these health checks is to:

  1. check volume, efficiency and budget pacing on the account, looking for any anomalies and detecting them to quickly act on them, i.e. a drop in spend or a card refused by an ad platform,
  2. review campaign and ad volume and efficiency performance,
  3. execute optimizations to scale the most efficient audiences and campaigns.

How frequently you will check up on your optimization process solely depends on the budget spent. The bigger the budget, the more optimizations are needed, therefore sometimes it needs to be daily as there is so much happening across the account on a daily basis or even a few times a day, that it has to be regularly and frequently checked. Meanwhile, an account spending, for example, $1k a month can be checked less frequently.

We also prepared a specific process for a health check:

  • Every quarter, fill out your goals,
  • Each week, report your results and check the setup.

Here’s a little cheat sheet with what to focus on during such a check-up.

Goals
  • Conversions
  • Cost per conversion
  • Budget
Report
  • Number of conversions month to date
  • Cost of conversions month to date
  • Pacing against budget
  • Predicted spend on the end of month
  • Additional KPI metric to conversion, e.g. ROI or revenue
  • Performance overview from the last reporting period
  • Questions, errors, additional notes
Check
Google Ads
  • Campaign budget
  • Campaign performance
  • Ad group performance
  • Ads performance
  • LP performance
  • Extensions
  • Keywords, search terms, placements, and audiences with the high cost and those which are gaining conversions
  • Demographics, device, keywords, location, placement bidding
Social
  • Campaign budget
  • Campaign performance
  • Ad groups performance
  • Ads performance
  • Ads frequency

…And start optimizing these fields.

Testing

When testing marketing tactics, on average only 3 in 10 tests produce higher results than the control. Therefore, most tests will produce worse results than your current baseline. However, if a test is successful, it may dramatically outstrip your baseline performance, resulting in high rewards for the testing process.

To increase performance this way, you must run a high volume of experiments at a high velocity, which requires a lot of data that a few companies have. When it produces better results than the control variant, which could be a page or ad, this test should be added to the optimizations stream and scaled to boost account performance. Here, you can test a creative story, channel, audience, format, etc.

When you include testing in your processes, you should also remember about regular check-ups here. I would recommend doing so depending on the budget, every few days or once a week so that you can quickly react in case a test doesn’t perform well.

Also, at the start of tests, you should provide a clear hypothesis and target KPI that you want to directly influence (i.e. CTR, CPC, conversion rate) for each test you want to do. This way, you will know exactly what to expect and look for during the experiment. Plus, you will be able to easily present your results and insights.

Here’s a short overview of each of these elements.

Hypothesis

You form a hypothesis based on the available data, but also on your experience or competition analysis. Remember that it must have a KPI which will help you determine if it was correct or incorrect.

Here’s an exemplary structure you can use:

Action A will improve metric A by X%.

For example, adding more fields in the contact form will increase conversion from visit to submission form by 25%.

Results

Each test needs to have a control group in the shape of data from the past (it may be less reliable, though, as many variables possibly changed over time) or A/B tests:

  • On the website, the percentage of the traffic lands of the variant A, and another percentage on the variant B;
  • In the case of ads, provide 3-5 ads in each ad group so that the algorithm can choose which one to show and thus, will get more clicks or conversions, winning the tests.

At the end of the tests, you compare the results of each variant, which one won, and check their statistical significance.

Remember that, in some cases, results may be either conclusive or inconclusive. For example, there is no difference between the experiments’ outcomes. Then, you need to decide whether you continue or end tests and try out another idea.

Insights

No matter if the test was conclusive or inconclusive, you still gain valuable insight from it:

  • Inconclusive tests show you which elements don’t have that much of an impact on a specific metric, i.e. a button color may not affect conversion so there’s a chance that other components may influence it more.
  • Conclusive tests allow you to learn about what resonates with your customers, thereby how they make decisions. For example, if a majority of users preferred an ad with balloons and champagne, next time, your ads should include derivatives of a successful test. Here, you mostly depend on your imagination, experience, and knowledge about the market/channel/industry to produce new ideas on why the specific variant won.

5. Growth

Finally, there’s growth. Here, you should choose between volume or efficiency. Increasing efficiency means you get less conversions and vice versa when you want to increase the volume you will be paying incrementally more for each subsequent conversion.

It’s vital to base your growth on more than one channel to ensure stability so that, i.e. a new Google update or ban of your Facebook account won’t end up killing your company. So, decide which channels you will try next and set a budget for them. Also, you may base your analytics, attribution, and user journey on multiple platforms, so you need to check which ones bring more new and returning visitors, and which ones give the most conversions. For example, the fact that all conversions come from Google Ads doesn’t mean that you should resign from Facebook as it may fuel the top of funnel which later converts via Google Ads – which is exactly what happened for one of our clients.

Truth is, it’s not the final phase of the process. The optimization and testing model is a loop. This is just an end of one of many iterations of your growth. Each quarter, you should review what you did and didn’t do in the last three months, plan goals for the next quarter, update your strategy, channels, and budgets, and immerse yourself in performance and testing processes once again. So, get ready to truly thrive.

One model to rule them all?

With such a model in place, we’re able to meet our clients’ expectations by personalizing the optimization and testing ratio for each customer. However, we’re also aware that this model isn’t for everyone, and so should you. There are at least two situations that require a different approach:

  1. You’re starting from scratch and you have yet to build your business’ performance, or
  2. Your business is all set and running but you have no idea who your customer is and how to target them.

In these cases, all you should care about are learnings. You need to test various target groups, designs, copies, etc. to find out what resonates with your audience. Only after you have that data, you can answer all your customers’ questions (create value for them) and scale those campaigns which performed the best during tests.

Nevertheless, if you decide to try this model out and make it work for you, here are some best practices we prepared for you:

  • Focus on delivering performance. Bosses tend to be skeptical about tests, but if there is a good performance in place, it will be easier for you to convince them to start experimenting. But…
  • Start small. It’s important to test small elements and see how they perform. If they’re bringing positive results, scale them.
  • Base your website testing on the test calculator to know how much time it will take to see any outcomes, for example, if you want to grow 73% in 3 months, you need to do 67 experiments each month. Sometimes, it will be better if you just change specific elements, note down what and when was changed (a simple excel sheet may come in handy here), and check out after some time if it helped. This way, you won’t have to wait X months for a statistical hypothesis.

And remember, some platforms, like Google or Facebook, make testing easier for you with their native solutions.

Ladder powers strategy and performance solutions for fast-growing brands

Talk to a strategist →

Growth is a high velocity game.

Sign up to our newsletter to stay up to date with all the latest movements in the field.

More from

796+ open sourced growth tactics used by the best marketers in the world.
Explore our playbooks →