Maggie asks, “Would you use CHAID analysis to discover relationships between multiple categorical predictor variables? For example, using GA dimensions (traffic channel, device type, country, landing page, etc.) and group them according to their likelihood to convert. What else can be used for a similar analysis?”
Maggie knows her stuff when it comes to statistics. CHAID (Chi-square Automated Interaction Detection) analysis is a great classical method for doing multiple outcome statistics; unlike decision trees, which typically have binary splits in decisions, CHAID can do multiple splits. CHAID is also useful for data that is non-parametric, meaning it isn’t in a normal distribution, which a lot of marketing data is, but as a technique for marketing analysis, it’s more for interpretability than prediction. That said, it has fallen behind more advanced classification and regression trees (especially with random forests) and newer techniques like XGBoost and stacked ensemble methods in tools like IBM Watson Studio AutoAI and H2O AutoML.
Can’t see anything? Watch it on YouTube here.
Listen to the audio here:
- Got a question for You Ask, I’ll Answer? Submit it here!
- Subscribe to my weekly newsletter for more useful marketing tips.
- Find older episodes of You Ask, I Answer on my YouTube channel.
- Need help with your company’s data and analytics? Let me know!
- Join my free Slack group for marketers interested in analytics!
Machine-Generated Transcript
What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for watching the video.
In today’s episode Maggie asks, would you use chat analysis to discover relationships between multiple categorical predictor variables, for example, using Google Analytics dimensions like traffic channel, device type, country landing page, etc, and group them according to their likelihood to convert What else can be used for a similar analysis.
So, Maggie knows her stuff when it comes to classical statistics Chad, which stands for chi square, automatic, automated interaction detection analysis is a classical statistics method for doing multiple outcomes statistics is similar to decision trees.
But what makes it different than decision trees is that at any given point, when you’re trying to split up data, it can have more than one split, where’s decisions, trees are always binary, those split you one part goes this way, one part because this way.
It’s also the reason why it’s so popular in classical statistics is because it creates a model that is very easy to interpret a human being can look at a trade analysis on paper and go, Okay, I understand what’s going on here.
Whereas when you look at something like if you have a massive decision tree, it can be very hard to figure out like what exactly we doing this tree is, you know, 200 layers deep and has 40 splits each way.
And, and it is it is more like a tangled hedge than a tree.
That said, it has fallen out of favor, the technique has fallen out of favor, really since more computational power became available in the 90s.
Now it’s available.
It’s in tools like IBM Watson and IBM, SPSS, it’s available.
And for those people who know what it is know how to use it, it is still useful.
And there are times that we’ve talked many times there are times when a classical technique may be better than you know the state of the art because the state of the art may be computationally intensive, harder to interpret and only offer a couple of percentage points improvements.
Over the over the classical methods, this is especially true and things like time series forecasting, classical methods are still on par with with state of the art for the ability to predict.
Shade is useful for data that is not in a normal distribution, because it uses Chi squares for it splits, which is important because a lot of marketing is not in a normal distribution, which means it doesn’t look like a bell curve.
Most marketing data looks like a power law curve, you’re tall head long tail.
In fact, they’ve been marketing books titled the long tail.
And as a result, it’s not a normal distribution.
And so a technique like Chad, or any technique that’s non parametric, is a important for being able to work with marketing data.
But CHAID is not particularly good at prediction.
And that’s what a lot of marketers are trying to figure out is not necessarily the relationships between variables, but what variables in combination predicts the outcome we care about.
So in Google Analytics, you may be looking at, I care about conversions, and I care about goal completions, what are the variables and their interactions that predict a goal completion so that I know I need to spend more money on mobile ad bids, or I need to spend less money on Facebook.
That’s the kind of information we want and for that, CHAID has fallen behind a classification or and regression trees or cart techniques, especially because the newer techniques, newer because something like a random forest is still only 50 years old, not exactly new, at large scale, random forest are much more powerful for that type of prediction.
And then even newer techniques that still fall in the family.
But things like gradient boosting Stochastic gradient descent, extreme gradient boosting.
And the state of the art for the average machine learning professional who has access to the technology really are strong ensemble methods.
tools like IBM Watson studios, auto AI, or eight shows auto ml are the best at doing this kind of analysis today, because they can do all the hyper parameter optimization, they can do all the feature engineering or the mall, I should say the feature extraction because they can’t add any net new features.
But most importantly, they can do all the algorithm testing and selection for us and put together as a stacked ensemble of the best techniques which may include shade at some point in the stack, but then also have all the cart techniques above or below or in parallel to that.
And that’s what is the best in class today are the stacked ensemble techniques where a machine looks at our data.
h2o is auto auto ml, for example, does take the categorical predictor variables does one hot encoding turns them into their still category poker numerical and then runs them through gradient boosting GLM GBM.
It does neural networks.
And then it says, here’s this combination of methods that work best for this data set without too much overfitting, because it does, you will use specify doing like a 60 2020 split.
So from a a hardcore data science and machine learning perspective, using these automated methods for classical statistical problems is the best choice today.
Because they can do you know, 50, or 100, or 200 different tests on the data in order to give you the best combination of variables, deploy a modeling, and then have that model, taking new data from the data source.
Google Analytics is a great example.
Once you build a model, you want to have data streaming into it, and then updating and retraining the model and tuning it so that you get accurate predictions about like what is what’s happening with your data.
If you need to be able to explain step by step, what a model is doing to somebody, chain is still a great choice.
Because you will incur some performance penalties, you will incur some accuracy, penalties, but you will have a model that you can literally draw on a whiteboard, and somebody can understand it.
If you need maximum performance, you need to use stacked ensemble methods.
That’s pretty much the way things go these days.
Great question.
Very, very good technical question.
So if you are new to stats and data science and machine learning, most of this episode is probably like, I don’t know what’s going on here.
And that’s okay.
Just know that there are that technique selection is rapidly being handed over to machines because they’re better at it, they can do more variations.
It’s like fundamentally what we’re talking about if you are baking cookies, a machine will go through and test bake every possible variation of the cookie with all the ingredients and then test bake and every possible temperature your oven and tell you this is the best combination of ingredients and the best combination of oven time and temperature to yield the best cookie you can get.
And that’s where machines really out shine humans today for any kind of prediction.
Just that that rabbit tuning.
So great question Maggie.
Glad to see that there are folks out there who are working in this realm as well.
Based on the content of your questions, always always fun.
As always, please leave your comments below and please subscribe to the YouTube channel and to the newsletter.
I’ll talk to you soon.
Help solving your company’s data analytics and digital marketing problems.
Visit trusted insights.ai today and listen to how we can help you
You might also enjoy:
- You Ask, I Answer: Reliability of LLMs vs Other Software?
- Almost Timely News: Principles-Based Prompt Engineering (2024-02-25)
- Mind Readings: Most Analytics Data is Wasted
- Almost Timely News, January 28, 2024: Copyright Must NEVER Apply to AI-Made Works
- Almost Timely News, February 4, 2024: What AI Has Made Scarce
Want to read more like this from Christopher Penn? Get updates here:
 Take my Generative AI for Marketers course! |
Leave a Reply