Confounders: machine learning's blindspot
7 min read
Today’s marketing attribution models aren’t good enough and that’s not ok
We’ve all seen the familiar graphs preaching to the choir that “correlation is not causation.” As marketers and data scientists, we know this, it isn’t a novel concept and has long since had its time as a meme. We’re ready to look beyond correlation, causation is what we’re after.
Through researching on Google, there’s a plethora of content on how to approach the challenge of marketing attribution. Generally it would seem there are two broad paths we, as marketing leaders and teams, can take, heuristics based and data-driven attribution models, or causal attribution.
The first path are the heuristics based attribution models we all grew up with or data-driven, correlation ML approaches more popular today. The heuristic based models are the “Touch” models and more recently, multi-source models (formerly known as “multi touch attribution”), time-decay, and position-based. The trouble with these heuristics based models is that they make broad assumptions about the customer journey.
89% of marketers have reevaluated their channel mix since the pandemic to try to adapt.Salesforce, 2020
Data-driven or correlation ML approaches to attribution bring with it a lot of potential, but also a host of issues, namely trust. “ML models are black boxes that marketers can’t verify or scrutinize. They tend to overfit to the current market conditions. And to function at all they need very big data — emerging trends, rare purchases, and unusual customer behaviors can lead to tech meltdowns.” (causaLens)
I’ve lost sleep trying to resolve the voice in my gut telling me something is off with our attribution model. I know there are variables that are not being taken into account, and even if I had questions about the model’s recommendations, a further question remains: why can’t I see an explanation. I have domain knowledge, an understanding of current events, and who our customer is. Not utilizing this knowledge leaves value on the table. A human-in-the-loop approach is what I’m after.
For Kelley, attribution theory “… is about how people make causal explanations, about how people respond to questions starting with” why? “ It focuses on the information that people use to make causal inferences, and what they do with that information to answer the causal questions. “Kelley ,1973
The second involves getting down to first principles: we need to clearly define the problem. Let’s start with Harold Kelley, on the subject of attribution theory to make clear what we are after.
When it comes to the marketing attribution challenge, the questions we ask as marketers are also causal questions: what channels and engagements are driving the most revenue, what channels are delivering positive ROI, and “which ones should [we] outright ditch?”
Can we all say it in unison now….
At the heart of it, marketers need to know the cause of a customer’s action (that’s the ‘why’ behind the decision) in order to attribute credit in a way that accurately values a given channel or engagement. For example, we need to anticipate the effect changes to budget allocation will have on various target KPIs.
Causal AI is not a new field of artificial intelligence, but until recently, it hasn’t been put into production in a way that is productive for enterprise level challenges and questions. causaLens has delivered this by developing Decision Apps powered by Causal AI. These are built for specific problems across a business. For marketers, Decision Apps autonomously discover causal models that map out the cause-and-effect relationships between channels, messaging, on and offline engagements, socio-economic variables, and target KPIs.
These causal models are also highly explainable. That conversation with leadership at the end of the quarter no longer requires weeks of data requests and hair pulling. The answers to what caused certain behaviors, why a channel is performing better than another, where should budget be allocated next quarter, are all easily accessible and backed by explanations.
They also keep humans-in-the-loop, enabling marketers to provide domain context, for instance, competitor dynamics and business constraints. This is a step-change versus current measurement approaches which offer no scope for human-machine collaboration. There’s a wealth of knowledge sitting in an organization’s people, not taking advantage of that is leaving value on the table.
If you know what already affects your targets, this knowledge isn’t wasted. It can be used to inform the causal model with causal drivers that are specific to your domain.
Let’s walk through a scenario around planning marketing strategy and budget allocation leading up to a busy sales period. What are some of the questions that we might want to ask:
Notice anything about these questions? They are all dealing with the causal relationships between our marketing activities, customer behavior, and achieving our target KPIs. For marketers using causaLens’ Marketing Attribution Decision App powered by Causal AI, these questions are answerable directly via the apps interface.
Causal models are at the heart of Causal AI. A causal model discovers and maps out all the qualitative cause-and-effect relationships in a domain, combined with quantitative information about the strengths of those relationships.
If you don’t have these quantified cause and effect relationships, when you set out to build your attribution model, you are assuming that all variables have an effect on our target KPIs in the same way. In the real world, we know that this isn’t the case.
As we can see from the image above, causal attribution takes the causal model for your domain, taking into account the macro-economic variables that are not readily available.
Scenario testing for a deeper understanding of what levers are the most impactful towards our target KPIs.
Attribution is a cause-and-effect problem, shout it from the rooftops. By naming the problem, we can move towards solving it. Causal AI makes this path not only possible but materialized in causaLens’ Marketing Attribution Decision App.
Causal attribution represents the full picture of why consumers behave the way they do.
It’s time to consider a different way to model our marketing attribution. One that utilizes all the existing domain knowledge within an organization, takes into account the cause-and-effect relationships between channels, engagements, and customer behavior to maximize our ability to drive growth.