Today most brands start as online-only, and that gets you pretty far. As you scale, your marketing mix gets more complex – you start working with influencers, sponsoring podcasts, running TV ads and billboards... all things that are difficult to track digitally (you can’t ‘click’ on a TV ad!).

You ask around, and inevitably someone turns you on to Marketing Mix Modeling (MMM), a technique from the 1980s that actually does work to measure all marketing channels (even offline channels). However it’s kind of archaic, and comes with a whole host of problems and pitfalls. It's expensive, takes 4 - 12 weeks and is prone to human bias.

Thankfully being startup people, teams at companies like Monday.com, Thirdlove and Harry's (where one of the authors of this post led the marketing science team) set about reinventing MMM for the modern age: combining Bayesian statistics and data pipelines to build robust, real-time automated models. Even Google and Facebook are getting in on the action, publishing papers and releasing open-source code.

Over at Recast we're building one of these next-gen MMM solutions, and you can take simulator-based courses on MMM at Vexpower.  But you don’t have to use our products to benefit -- this post should tell you enough about what's going on to decide how it impacts your business, and whether to dig further.

WTF is MMM?

In a post-cookie, adblocked, and privacy legislated world, knowing how to use MMM (which doesn't require user-level data) is fast becoming a popular solution. It works by attempting to match spikes and dips in sales to events and marketing actions. For example, if you consistently see that sales increase after launching a TV campaign, that’s good evidence that the TV campaign is driving sales for the business.

Of course, things are a bit more complicated than that. Marketing mix modelers attempt to leverage the 4 Ps of marketing (product, price, place, and promotion) in order to validate how a TV campaign in Boise lifted the sales in stores in that area, and how promotional activity (e.g., discounts) worked in conjunction with those advertising campaigns.

Media mix modelers built statistical models to try to measure the impact of these different activities. The most common tool is linear regression, which you can do with a simple Excel function. More advanced practitioners will use automated scripts in R or Python, which allow for more accurate and flexible methods like Bayesian MCMC models, of the kind we use at Recast

The Comeback

In recent years Media Mix Modeling has been making a comeback. This comeback has been fueled by a few important developments in how goods are sold:

• The rise of omni-channel sales (online and offline working together)
• Increasing protections for consumer privacy (GDPR, ad blockers, iOS14)

The rise of omni-channel sales means many businesses are selling both online direct-to-consumer as well as through brick-and-mortar outlets (like Harry’s launching in Walmart, Target, and eventually everywhere else). Some have said that “rent is the new CAC” which reflects this truth: even digitally-native brands that grew up on the internet are turning to brick-and-mortar stores to scale their business. And of course traditional brick-and-mortar retailers are all developing strong online presences to meet customers wherever they want to shop.

The growing importance of protecting online privacy makes traditional digital-tracking methodology less effective. iOS 14.5 prevents certain types of tracking related to app downloads, GDPR / CCPA both place limitations on how easily consumers can be tracked across the internet, and the growing prevalence of ad blockers (40% of all traffic by some estimates) means that it’s more and more difficult to take multi-touch or last click attribution at face value. We expect that this trend will continue to make digital tracking increasingly less effective as an attribution methodology over time.

Of course, no one is throwing out their last-touch CAC in favor of using only media mix models: rather, as the ability of digital-tracking methodologies to tell the whole story decreases, marketers are looking to expand their toolset in order to be able to see “the whole picture”.

Benefits of MMM

• No reliance on user-level data or cookie-based tracking
• Independent from ad platforms and other vendors
• Works for offline / hard to measure marketing channels
• Able to quantify the impact of non-marketing factors
• Can handle ‘what if?’ scenario analysis

User Privacy

Because marketing mix models are top-down models, they don’t rely on user-level data at all. That is, to build an MMM model you don’t need to share your sensitive client data with a vendor and it also means that you don’t need to track your customers at all to use it. This means that whatever happens to Facebook or Google’s ability to track conversions in the future, all you need from them is spend and impressions by day by campaign.

Independence

Dropping the requirement for user-level data also leads to another benefit of MMM: it’s completely independent. Whatever marketing campaign you’re experimenting with, you can include it within your model for one single source of truth. You get an answer on how many incremental sales you’re getting from that activity without having to trust that ad platform or vendor. No more conflicting reports: MMM brings everything together in one unified view.

Offline Channels

Because there’s no reliance on digital tracking, MMM works just as well for offline (untracked) channels like TV, radio, and podcast as for online channels. They don’t rely on coupon codes or vanity URLs or other hacky assumptions around customer behavior. And the same is true for offline sales! Your online advertising may be driving brick-and-mortar purchases but many companies don’t have a good way to measure that effect and end up under-investing in marketing spend that they could be using to grow their business.

External Factors

Ever see your performance dip for no reason? Or is your agency claiming credit for good results that you suspect were driven by something external? MMM allows you to flexibly include external factors in your model. For example it can tell you where your numbers would have been without COVID-19, so you can forecast the impact of a future potential lockdown. It can also properly quantify the effect your competitors have on your results, which in some cases can have a bigger impact than your own campaigns!

What-If Forecasting

Finally, good marketing mix models can be used for complex scenario analysis and forecasting. Based on the results of the model, marketers can explore what might happen to their overall business performance under a number of different proposed marketing budgets or strategies. And once a budget is finalized, the MMM can be used to forecast top-line business performance over the coming months. This is a super-power for marketers who want to be taken seriously by finance and leadership teams.

Challenges with traditional MMM

Unfortunately, media mix modeling is not a panacea. It requires a new mode of thinking for marketers used to deterministic last click models. It comes with a number of challenges which fall into a few buckets:

• Traditional MMM vendors are (relatively) expensive and  only refresh their models once per quarter or 6 months
• Refreshing data can be slow
• Model accuracy is critical and difficult to verify

Expensive

Because media mix modeling requires lots of statistical expertise, traditional media mix model vendors are very expensive. They are often hand-tuning a model for your business and they need to be able to pay the staff of data scientists they have on-hand to do this for your business.

Slow

Because the models are expensive to update, they tend to be updated very infrequently. A few times a year at most!

And data collection can be another challenge, especially for  businesses that have not historically done a good job of tracking their marketing spend historically. For example, if you need to call up your radio agency to get your spend from last year and they send you an ugly looking excel file, that can be a real nightmare to deal with.

However, when compared with digital tracking, the data requirements for doing MMM actually aren’t that stringent -- if you have been keeping track of your daily marketing spend for the last few years (which you should be doing to track your blended CAC anyway!!) then you probably have a good starting point for an MMM model.

Model Accuracy

The most challenging part of doing media mix modeling is determining if the model is accurate or not. Unfortunately, measures like R² (“r squared”) or even holdout accuracy from backtesting are not sufficient to determine that an MMM model is accurate. This is because marketing is very complex, and MMM models tend to be “over-determined” which means that there are many different plausible sets of results that all make equally good predictions.

There are lots and lots of ways that this can go wrong, and to some extent you have to rely on your vendors or in-house data scientists to just get this right. The best way you can guard against inaccurate models is to put in place a robust testing and validation plan so that you can externally validate the results of the model via lift-tests or other experiments.

The Next-Generation

The next generation of media mix models is attempting to address a number of the limitations described above by leveraging advances in statistical model fitting (like machine learning) as well as access to huge amounts of computer infrastructure (e.g., via AWS).

These next-generation models focus on a few areas:

• Reducing human-error and costs through software-based automation
• Gaining real-time insights through more frequent model updating
• Improving model accuracy via improved statistical methods

Automation

One of the main reasons that traditional MMM products are so expensive (and often inaccurate) is that they rely on humans to “hand tune” each model and each refresh. This is both costly from a time perspective and prone to error and bias. The next generation of MMM products attempt to rely much less on human judgement and much more on machine learning and simulation-based optimization to fit their models. While this requires considerably more computing power (just trying running Facebook’s Robyn on your laptop!) it also allows for more accurate (and more timely) model results.

Real-Time

Because many people are aware that the slow feedback-loop of the once-per-quarter MMM powerpoint deck led to a lack of actionability, the next generation of MMM products attempts to solve this by making model refreshes much more frequent -- this allows marketers to see how changes to their marketing programs are evaluated in real time and allows them to make changes “in flight” to marketing programs as the media mix model gives them results.

Improving Model Accuracy

We aren’t going to get into the weeds of the underlying statistical techniques that enable improved model accuracy, but if you’ve heard about the growing importance of machine learning and AI in recent years then you know that there has been lots of research effort poured into making these types of statistical models more accurate. The new generation of MMM products leverages these technological developments to extract more signal from the noise of your marketing data.

Multiple Attribution Methods

With more crossover between the traditional practitioners and modern data practices, we’re seeing more creative collaborations. For example using the results of a deprivation test to calibrate your marketing mix model, or developing heuristics (rules of thumb) from your MMM to apply to your Facebook ads results to optimize campaigns in real time.

Conclusion

Leadership teams are starting to understand the ‘single customer view’ is a pipe dream that won’t come true. We’re all waking up to the fact that we can’t outsource attribution to the ad platforms or our analytics vendor – nor should we, lest we inherit their biases. In the post-iOS14 era you must use multiple methods to triangulate the truth, and MMM should be one of them. Combining good statistical practices with modern data pipeline management has breathed new life into this old technique, and presents an opportunity to data-driven marketers looking to get full credit for the performance of their campaigns.

If you’ve made it this far and you want to learn more about how a next-gen MMM could benefit your company, check out Recast where we’re helping modern brands do omni-channel media measurement in real time. And if you’re an analyst looking to add MMM to your statistical toolbox, definitely sign up for Vexpower MMM training.