When Vital Marketing Analytics Data Goes Missing

What happens when a vital data point, a vital data source, goes missing? In the world of marketing analytics, this isn’t uncommon, but the consequences can be dire.

Why Marketing Analytics Data Goes Missing

Marketing analytics data tends to go missing for one of two major reasons. The first reason is instrument failure: a tracking code goes missing, a piece of software doesn’t operate correctly, a vendor has a service interruption. This missing data tends to be temporary, but depending on the duration of missing data can wreak havoc on your analysis. Even something as simple as a month-over-month comparison can be severely impact by a few days’ missing data.

The second reason is data availability. This is when a system or vendor discontinues data, such as removing it from an API or service. This tends to be more permanent and, depending on the data source, can be a significant disruption to your operations. For example, when Facebook’s Crowdtangle software discontinued its coverage of Twitter data, marketers had to scramble to locate other services that provided similar features.

What To Do When Marketing Data Goes Missing

The most important thing to do with marketing data goes missing is to ascertain whether it’s temporary or permanent. The situation will determine your response and how you’ll recover from the missing data.

Temporary Missing Marketing Data

To deal with temporarily missing data, we can use one of two techniques – omission or imputation. Omission is the exclusion of analysis from the time period in question; if we’re doing month-over-month analysis, and we have two days of missing data, we can exclude the same time period of data in the previous month so as to still be comparing fairly. This approach tends to be less accurate and very manual, but easy for any practitioner to do – as long as they remember to do it. Therein lies the greatest difficulty of omission as a technique: governance. If you don’t have strong analytics governance and documentation in place, very likely you’ll forget about the missing data, and subsequent analyses will be wrong.

The second approach, imputation, uses sophisticated statistical and machine learning techniques to impute, or guess, at what the missing data should have been. Using techniques like predictive mean matching, random forests, and extensive cross-validation or out-of-bag error checking, algorithms can often complete missing data with high accuracy. Imputation requires a parallel data set that’s complete and correlated to the missing data during the period of missing data in order to do the imputation well. For example, if you have several days of Twitter data missing, but you have complete data about traffic from Twitter in your Google Analytics, the Google Analytics data will provide “guardrails” for the imputation algorithm to soundly infer what’s missing from your Twitter data.

Permanent Missing Marketing Data

Dealing with permanent, ongoing missing data requires us to understand the importance of the missing data. How vital is it to our overall outcomes? How important is it to our goals? If it’s not important, then we simply move on and accept the loss. If it’s important, then we need to establish whether we can find an acceptable proxy.

Finding an acceptable proxy involves first determining whether any of our other data is highly correlated to our permanently missing data, either alone or in combination. You’ll use advanced statistical and machine learning techniques like multiple regression, gradient descent, or gradient boosting to identify feature importance of the missing data and its correlates. Then, as best as you can, statistically test your proxy data against outcomes you care about over time to ensure that the relationship of the proxy variable(s) have a causal impact on your outcomes.

For example, we recently lost a significant piece of data from a provider on website traffic for sites we don’t own. However, using these techniques, we identified a correlated variable from some SEO metrics that delivered similar outcomes, and after testing, we validated that the proxy variable was as good or even better at generating the outcome we were after than the original data. We’ve since substituted that variable entirely, with no disruption of operations in our software.

Missing Marketing Analytics Data Isn’t The End of the World

The good news is, in either temporary or permanent missing data cases, we have so much data available to us that with some creative thinking and sound science & mathematics, it’s possible to overcome many missing data challenges in marketing analytics. Master the techniques to fix your missing data problems, and you’ll set your data analysis up for success in the foreseeable future.