Glenn Granger, CEO at Marketing QED, talks data with Darren and its role in informing fact-based decision making rather than simply relying on intuition. They discuss the challenges, processes and benefits of taking a ‘Math Men’ approach to marketing instead of the traditional ‘Mad Men’ style. And provide insights into why we work the way we do and how to improve.
You can listen to the podcast here:
Or, follow Managing Marketing on Soundcloud or iTunes
Podcast transcription
Darren:
Alright, so welcome and I’m here today with Glenn Granger, CEO of Marketing QED which is a marketing effectiveness technology company. Welcome Glenn.
Glenn:
Hello Darren.
Using data to create marketing insights
Darren:
I’m really looking forward to this conversation because with my science background and then getting into marketing, one of the key areas for me has always been the idea of using data because in science, medical science, we used experimental data a lot as a way of being able to validate or get insights. But it’s an area in marketing that’s only really taken off in the last few years.
Glenn:
That’s right, yeah. Well I actually have a science and maths background myself and so bringing in those kinds of techniques to the business world was kind of a neat way for me to move from academia into the business place.
But you’re right, marketing has only really embraced analytics as a way of trying to create some value over the last, I guess, the last ten years really. There have been people pioneering with stuff for about twenty-five years, and I was lucky enough twenty-five years ago in fact, to get into this area but the reason really was twofold.
One was that the data wasn’t readily available a long time ago and the people with the skills and the know-how who had made that journey from the mathematics world into the marketing world were quite few and far between. But there was a guy.
Darren:
It was almost, sorry Glenn, but it was almost like two tribes wasn’t it? Because I remember thirty years ago, when I went from science into being a copywriter and in fact I wrote a blog about this, having a researcher there who had done quant, no, qual groups, talking about the statistical results from the qual groups of thirty people. I started talking about the statistical significance of it and everyone just looked at me like I was talking, sort of, Mandarin. They had no idea what I was talking about so there’s definitely tribes, isn’t there?
Glenn:
Definitely.
Darren:
You know, math men and mad men.
Glenn:
Yeah, exactly. And that distinction has actually broken down over the years because marketing has been pushed on the back foot to try to justify itself more and more.
There was a report done by Deloittes a good few years ago now, that actually questioned other executives in companies about what they thought of the marketing teams, and it didn’t make very happy reading because a lot of them said, “Well they just don’t seem to have any evidence for what they’re doing”. And so I think, and the recession as well, really has shaken people up and made them have to justify their budgets.
So I think what we’re seeing, is that coupled with a lot more people going to business school and/or business degrees of some sort, and then finding their way into marketing teams. There tends to be more of, should we say, an analytical education along the way.
The education system has changed a little bit so people are getting a taste of statistics and other things like that, so that’s what’s come to a head and then all the data has actually gotten a lot better as well.
Finding better ways to interpret data
Darren:
And technology is driving that isn’t it? I mean, we talk about big data because there’s lots and lots and lots of data.
Glenn:
Yes, that’s right. I think it was IBM who have got some statistic that says in the last two years there’s been more data created than in all the years of mankind’s existence up to that point. And that’s probably always going to be true.
We’re always going to be developing more and more data. We’re getting to the point now where yeah, lots of data’s been created and we now need to really find ways of interpreting it.
In my company, we actually have developed a suite of tools that can make that process as simple as possible. We’re finding a lot of people are interested in taking that technology in-house to be able to make sense of all this data that they’re drowning in at the moment. We’ve actually got to the point now where there’s too much data and what you need is a lot of information, you want to create insight.
Darren:
Yeah, that’s right, I mean data itself is not actually very useful until you actually create insight. I like that, the distinction that there’s data and then there’s information and then there’s, what was the next one? Insight and then knowledge. Oh and then the final one was wisdom.
Glenn:
Yeah.
Darren:
So that you can actually build or ladder up from the most basic form of data because all data is, is a collection of numbers, facts, digits that actually relate to a particular event.
The emergence of new skills for interpreting data
Glenn:
Yeah, that’s right. And of course what makes it more complicated is the analytical skills needed to interpret this data don’t tend to reside in most companies. There are, we are seeing the emergence of a new title that’s emerging of the Chief Analytics Officer.
Darren:
Yeah. Another chief.
Glenn:
Yeah, another chief.
Darren:
Lots of chiefs, no Indians.
Glenn:
And so, they are starting to recruit, and the universities are starting to put on marketing or business analytics courses and masters courses. It’s the US that’s ahead of the game, as they normally are on these things but we’re starting to see there’s a new emergence of people with skills. So that’s the essential requirement really, that you’ve got to be able to turn that data into all those steps you mentioned.
Darren:
So Glenn, this is a fundamental shift for marketing because I know over many years, marketers were pretty reliant and felt very positive about their ability to instinctively know what should be done.
Glenn:
Yeah.
Darren:
The gut instinct was often the thing that drove their decision making. So isn’t this going to have a profound impact on those types of marketers?
The value of analytics in companies
Glenn:
It is, you’re absolutely right. There’s this sort of gut feel and intuition.
There’s an academic in the States called Brynn Neilson who at MIT, wrote a really interesting paper on the value of analytics in companies and he talks about people using experience and intuition to make their decisions, rather than facts and data.
And you’re right, the world of marketing has been dominated by those kind of approaches but unfortunately, it isn’t as effective as using facts and data. When people start using facts and data, they get the edge. They find that they can actually improve what they do.
Those that want to cling tenaciously to the old ways, are finding that they are now sort of slowly becoming in the minority because other people are starting to embrace all this analytics, seeing some real value from it and actually pulling ahead in terms of competing.
So this is what Brynn Neilson and his colleagues found. They found that actual productivity in companies that used analytics was 5 or 6% higher. Now that might not seem very much, but over time, that starts to have a real impact.
There’s another great book, called ‘Competing on Analytics’ by Davenport and this was a good ten years or so ago now, but he outlines this idea that analytics and data is one of the new forms of competitive advantage.
You know, companies can come up with new innovations, but they can be copied very quickly. But using the customer data that you have, or the market data that you have available, to mine for sources of insight that are not readily available on the surface, you’ve got to dig down into it.
But once you do that you can find all sorts of interesting little nuggets of insight which help you with your business tactics and strategy, and that’s what’s giving people the advantage. So it’s paying back and the world is being dragged whether they like it or not, some people are being dragged in that direction.
Darren:
So, speaking of books, I mean, you have your own book that you published a couple of years ago.
Glenn:
A couple of years ago, yeah.
Darren:
Called ‘Raindancing: Why Rational Beats Ritual’, so that’s why obviously this is a core area of interest for you, because one of the things that I often see, even in choosing agencies, is that marketers will often default to their emotional response rather than a rational response.
Why do marketers default to an emotional response?
Darren:
My question is, why do you think that it’s human nature to do that? Or particularly, marketing nature to do that?
Glenn:
Well, I think it is human nature, it’s not just, you know, marketers are humans. I guess in that sense they do what a lot of humans do. Like thinking and looking at data and hypothesising how things are really working is hard work and it requires certain skills, which a lot of people don’t possess.
Darren:
That critical thinking.
Glenn:
Well, you can put it like that. I mean, I guess some people who don’t have any kind of analytical background tend to be a little bit afraid of delving into numbers and how to approach it.
That’s why we’ve actually created the tools we’ve created to make that process a lot easier for people, because we realised you’ve got to tool up people so that they can do it themselves. It’s like you’ve got to strap on that jetpack so that they can fly.
Darren:
Yep.
Glenn:
Without that, they’re not going to fly, right? But they need to be able to fly and I think people are wanting to embrace it but they’re just not sure.
Darren:
Fit or fearful. I mean, I often see when companies especially go down this path of becoming more fact or data driven decision making, that often there’s a fear within marketing and I flippantly say that perhaps the fear is that Lord Leverhulme or John Wanamaker who get quoted all the time, were actually optimists.
Maybe the fear is that, if we move to a data driven rather than an intuition driven decision making process, we end up finding that it’s not 50% of our budget’s wasted, it’s actually something more like 70%.
The absence of real information can lead to false decision making outcomes
Glenn:
Well, okay so I’ve got some insight on this. As you say, I’ve spent about twenty-five years analysing marketing campaigns and looking at the actual paybacks from them using analytics, and what we found was a couple of interesting phenomenon. Because in the absence of real information about what’s working, what’s not, people still make decisions.
Darren:
Yep.
Glenn:
And those decisions tend to get made by the kind of group dynamics that sometimes lead to false decision making outcomes. We’ve all been in a situation where we sit in a room and nobody really has any information or understanding of what’s the reality of something, but we still have to make a decision and the boss says something or somebody with experience says, “Well, I think it’s this”, and we all just go along with it.
Darren:
Yep.
Glenn:
They call it conformity in psychology. And then there’s all sorts of other phenomenon where people just kind of end up believing in things, quite fervently sometime,s without any real evidence for that. And what happens is, traditions and myths about what’s right for them and so on builds up.
This happens in every industry so in analysing different campaigns from different industries, various patterns emerge. What you find is that each industry tends to have its own pet media channel that it uses so typically, consumer goods is a very heavy television advertiser.
Darren:
Yeah.
Glenn:
And what we find, this is true for all different industries; financial services often rely on it.
Darren:
And it’s often called the sort of collective wisdom of that category. That’s what everyone else is doing, so we should do it too.
Glenn:
Yeah, absolutely and there’s nothing that’s come along to shake their faith that it’s true. That’s why I called the book ‘Raindancing’, because somebody did a dance, it rained once…
Darren:
…so everyone dances.
Glenn:
And the myth continues, right? And you’ve got to be able to break that cycle so financial services often use direct mail, pharmaceuticals use sales forces, electronics, consumer electronics often use print.
They all have their little pet channel and what we find is that they tend to overspend. Nine times out of ten there’ll be overspending on their pet channel. But, this is a really interesting other point, is that overall, the budget, they tend to underspend.
Darren:
Right.
Glenn:
So about 80% of the companies that I’ve seen over my career underspend overall, but they’re just spending in the wrong place.
Darren:
Yeah, they put so much of that into one channel which may not be necessarily the right channel.
Glenn:
That’s right. Correct.
Darren:
Because they’ve chosen it because that’s what they’ve always done.
Glenn:
Yeah exactly.
How do you break that cycle of people making decisions without enough information
Darren:
So how do you break, I mean if this is as you say, not just marketing behaviour but human behaviour, how do you, and I’ll go back to the point, maybe marketing over-indexes on humanity. But how do you break that cycle of people making decisions without enough information in a world where you can drown in information, how do you break it when there are such social pressures to conform within organisations?
Because I see that many times when decisions get made in the work that we do with our clients, they’ll often just, whoever the most senior person is in the room, they’ll do that, follow that. How do you, what do you need to do as an organisation to actually break this behaviour?
Glenn:
Yeah, so it turns out, there’s some good books been written on this subject on group dynamics like this and how to break up group think as it’s called.
Darren:
Yeah.
Do you have the courage to stand up for new ideas?
Glenn:
Sometimes, what you need is to have sort of diversity of opinion at the very least. You need people who have got the courage to stand up for new ideas and to say, “No, no no no, are we sure that’s true? How about we do something different?” And just stir it up.
Darren:
So that like the guy in Life of Brian, you know, you’re all different, we’re all different. You’re not the same, I am. So you need the guy that goes, “I am”.
Glenn:
Yeah exactly, you’ve got to have, and actually it turns out that just one person who’s fairly dogged in their resistance if you like, can make a big difference. It can stir everything up. It disrupts the…
Darren:
You mean the person that’s about to get fired.
Glenn:
Yeah.
Darren:
Or promoted to I don’t know, New Zealand.
Glenn:
For all our New Zealand listeners. So yeah, so absolutely right. So now, yeah, that’s a good point. So of course, why does nobody do that? Why does nobody stand up right, and say, “I’m different”, and it’s because they actually don’t have enough information to be certain. So they’re taking a bit of a flyer on the fact that they might be right.
Darren:
Because it breaks down to my opinion versus your opinion and then the most senior person in the room decides whether you stay or go has a different opinion, you go.
Glenn:
And in some cultures of course, in Asia particularly, that’s a complete no-no that you would ever challenge.
Darren:
You would never challenge the boss. Even if it means the whole company goes under.
Glenn:
Yeah.
Darren:
You just all go down together.
Glenn:
I told you it was a nice book. Yeah, so what analytics can do is bring that hard facts and information to the table, which can’t be easily ignored.
Darren:
Yep.
What do the numbers say?
Glenn:
Right, as long as you’ve got a reasonably rational bunch of people which generally in businesses you do have. So that’s what analytics can bring. It can bring that descending voice in the corner of the room, that awkward voice that says, “Hold on, that isn’t right, let’s look at it a different way”.
Darren:
And let’s look at the numbers because it’s actually the numbers that are saying that’s not right, not that person, isn’t it?
Glenn:
That’s right.
Darren:
So in that way, it gives them the support and the courage because it’s no longer a personal thing, they can actually refer to this third party which is the data.
Glenn:
That’s right, exactly. They can point to something and say, “Look, this is saying something different so we need to investigate or at least trial out some new stuff”. And that’s why analytics is such a great disrupter of these myths and traditions that have built up over the years. There’s a great film which is kind of parallel film but called, ‘Money Ball’ which is about baseball.
Darren:
So it uses data to actually select a team. Everyone says that that’s the wrong team, because it’s not the way you select a baseball team, and they end up winning the world championship.
Glenn:
Series I think.
Darren:
Series, that’s it. Only America can have a competition that’s only played in America and is called the World Series, but that’s a separate issue. Anyway, Money Ball.
Glenn:
Yeah, so that’s right. So in there, he shows some great scenes which reminded me of sitting with marketing teams where all these scouts who used their gut feeling, experience, intuition to make decisions and they have done forever. They’re the ones saying, “No, no no no no, I’m right”, and they come up with all sorts of rationalisations, not rationality, but rationalisations as to why their intuition is right, and there’s a huge distinction.
Darren:
And everything else is wrong.
Senior sponsorship is needed to push the facts and the data through the organisation
Glenn:
Yep. So these guys are, and they’re the ones that when you present the facts and the data, they suddenly don’t like it and they kick off sometimes, but that’s why ultimately in the case of Money Ball, you had Billy Bean who was the coach, the general manager and he had the authority and that’s the key. So to talk a little bit about how companies can embrace this stuff, you need senior sponsorship.
Darren:
Yep.
Glenn:
It can’t be the junior guy in the corner of the room because he might make some inroads, but actually, he can be overruled. So you need senior sponsorship to push it through the organisation.
Darren:
And one of the issues that I’ve noticed socially is that it’s often very painful for very senior people to feel like they’re being proven wrong.
Glenn:
Yeah.
Darren:
So one of the ways that it can be done and moving to fact based, data driven, decision making, is actually to use the data to drive the insights so that the people can still make decisions but they’re now based on data but they’re making decisions that are now more correct.
I think the danger to bring about change, is this process where someone makes an intuitive decision and then someone else comes in and goes, “Oh yes, but the numbers say you’re wrong”. And immediately, you’ve got resistance, right?
Glenn:
Yeah.
How to bring about change
Darren:
But to actually start to bring about change, which is, rather than making decisions to your point in a lack of information, and in an environment that lacks information, actually bring the information into this in the form of insights and understanding that can then inform those decisions.
Glenn:
Yeah, that’s right. So you don’t want to kind of call somebody out on decisions that they’ve already made. Ideally, you want to be thinking about future decisions, so that there are no egos at stake. It’s more, “Okay well what shall we do?” Okay and that’s where you can then start to introduce it, “The evidence suggests that this works well, let’s try some more of that”, and then people can get behind that and increasingly we see that people saying that, what do the models say?
Now there’s always a caveat here and the great thing is that marketers are not about to be put out of a job by kind of black boxes and mathematical models and so on, because ultimately, analysis and statistics can only give you correlation, it can’t give you true causality. It can show you that when we did this, this appeared to happen.
Darren:
Yep.
Glenn:
But you need that human understanding of context of the marketplace that they’re operating in to look at that and say, “Yes, that actually makes sense and it is coherent with all the other facts and information we have about it from other sources, market research and other experience we have”.
So the ideal is that you blend it together and it becomes, but there’s an idea that you kind of mix ideas together. You take some analytics, you take some experience, you take some other market research that you’ve got, and you find the sweet spot where they all are consistent with each other.
Darren:
Well, again, you’ll know this, the scientific method which I was trained in to be able to work in medical research – there are those distinct steps.
Experiment and see what the outcomes are
There is the observation. You observe data, you observe the real world, you observe and try and identify trends and patterns, and when you have those observations, which is what data provides, you then draw a hypothesis. What is the cause or effect that is driving that?
But it’s only a hypothesis. It’s not real because it’s just your interpretation. Your interpretation is also being influenced by your experience, your bias, all of the things that make you a human being. This is the whole observer principle in any scientific environment. The observer actually brings themselves into the experiment.
So then you’ve got the observation, you’ve made the hypothesis, then you have to set up a test. Now this is where science and business go two separate ways because in science, you either need a double blind or you need a control group to be able to compare what happens when I do nothing, compared to what happens when I do the experiment. In business, it’s very hard to set up test groups.
Glenn:
Yeah.
Darren:
And increasingly because of the complexity, you’ll never ever get a group that is not affected by something you do, but you can still start to do things, think of it as an experiment and just observe what the outcomes are.
Glenn:
Yes.
Darren:
The outcome is, what your hypothesis predicted or are they different? And if they’re different, then rethink the data that’s come in from the experiment to see what is a new hypothesis, to draw this and if they are exactly what you predicted, then do it again to see if you can reproduce it. That’s the other interesting thing about science – nothing is actually proven until you can reproduce it independently.
Blending the scientific approach with the social sciences
Glenn:
That’s right, and in fact, that’s a very good distinction. In the physical world, we can, within a given context, create deterministic laws. We know that if we do this, something will happen and you think about chemical reaction or breaking point of a particular material.
Darren:
Oh yeah, I remember chem lab, dropping sodium metal into water and having it shoot around and explode, and we’d all laugh except for the chemistry professor who would be very disappointed. But anyway, yeah, there are certain things that we know and have laws attached to them.
Glenn:
Yeah and because the world follows the law of identity and the law of causality and so on, it means that these things are repeatable and we can get a good understanding of what’s happened within a given context. In the human world, the social sciences, people are not subject to immutable eternal laws right? The deterministic laws, so people’s behaviour can change. So what you could only ever hope to do is to measure how they’re behaving now.
Darren:
Yeah.
Glenn:
And hope that in the near future they’ll carry on behaving in a similar way and generally they do. You know, if we’re talking about buying Corn Flakes or baked beans or mobile phone tariffs, by and large, people’s behaviour is fairly stable.
It’s quite interesting, even in war zones, the kind of day to day life still carries on. It’s quite interesting. But of course, it can change, it can evolve and so the kind of models that we use to measure human behaviour are what we can say are stochastic models.
They are models that acknowledge that there are errors, that there’s a bit of noise in the data. But we make sure that the noise isn’t hiding other important factors that we’ve missed and so we discount that, and as long as the noise is fairly random, up and down, we can live with that.
We’re not going to have a model that’s 100% absolute like a physics law or chemistry law would be. And so long as we keep taking our measurements, we keep dipping into the data to see how things are evolving, it’s very useful for near term, and by near term I mean six months to a year kind of forecasting.
Darren:
Which is about the time length of a budget, isn’t it?
Glenn:
Yeah, exactly. So, the analytical approach or the scientific approach can work by and large, in the social sciences, so long as you’re not expecting it to be immutable laws that will be true for all time.
Darren:
Yeah. And you know, this is where there’s this crossover. We started off talking and I raised the issue of mad men and math men, but this is where the two coming together actually work quite well.
People would say to me, having gone from medical research to working as a copywriter, they’d say, “Well how do you make the transition?”, but in actual fact, there is creativity in science.
Glenn:
Oh yeah.
Data, insights and hypotheses equals informed decision making
Darren:
To come up with a hypothesis is actually a creative process. I think it was Einstein that said it was 1% inspiration and 99% perspiration. That is still a significant amount of what marketers would largely champion as their strength, being able to hypothesise, create, have intuition, or response.
But I guess what we’re talking about here is the opportunity to not use that skill in a vacuum, but to use it in a world that is rich with data which should be then rich with insight, which then allows you to use those insights to create these hypotheses and make these decisions.
Glenn:
That’s absolutely right. Nobody quite understands how human beings can make this game, this cause, or understanding of how the world works. It’s a bit of a mystery but we seem to be able to do that and if you look at the context, if you look at data and you look at the correlations that that data can tell you, you then need to layer on top of that your understanding of what a true cause or path would look like.
You can’t just trust in blind correlations, you need to be able to say, “It makes sense that that happens like that and here’s how I would hypothesise what’s happening. That is causing this, that’s causing that, and that all makes sense and it’s consistent with other data we have.
And you’ve got to see the pathway in the real world in the context of the business that you’re creating. And that’s why it’s a blend, it is a blend absolutely of know-how.
The decisions that you make are not the end game
Darren:
The other important thing is that people think that decisions that they make are the end game, whereas what I like about science and a scientific approach is that a hypothesis is not the end game, it’s another step, right?
Glenn:
Yeah.
Darren:
And that if we could start to get into the process, this belief that when you make a decision, it’s to follow a particular path. That doesn’t mean that everything else has been eliminated because constantly, as you say, monitoring that environment and seeing what lessons can be learned and responding to those, actually makes it a much more dynamic, and I believe, ultimately more effective approach than making a decision and just following that path blindly long term.
Glenn:
Yeah it is. It’s the current hypothesis idea. It’s the where are we at and it evolves. And we’re seeing that when companies that I’ve seen have taken analytics in-house in marketing departments. They are now doing ten times the amount of analysis that they were previously doing.
That one question leads to an answer which leads to another question and the hypotheses, and the complexity of where they’re delving and going is it keeps unravelling the more you pull it. It’s like pulling a string on a jumper, the whole thing just keeps unravelling and it leads sometimes to some very profound insights, which completely change the business model.
Darren:
Yep.
Developing insights from your data
Glenn:
So I’ll give you an example. I know an insurance company where their business was suffering, in the UK, and they have these price comparison websites for insurance companies.
So you go onto one of these websites, you type in what you’re looking for, and then there’ll be a ranking of all the companies and the offers and prices of all the policies.
What a lot of people were thinking in this particular company was that the online people, the people that bought their policies through these aggregators, these digital worlds, were a different type of customer from the people that just picked up the phone and called direct. And what they discovered was that they weren’t a separate audience at all.
Darren:
Yep.
Glenn:
They discovered that people were, if they were looking at the rankings on the aggregator website, picking the top two or three and then calling direct. They weren’t going through the website. Which when you say it, you think, “Oh that’s obvious, I’d do that”, of course you do.
Darren:
But that’s the insight brought to the data. That’s the human insight isn’t it?
Glenn:
Well they found a correlation between the direct line and the ranking in the aggregator. So then they said, “Well what we need to do is find a way to make sure that we are at the top end of the rankings so that we’ll get more direct business”, and that’s what happened and they’ve had great results.
Darren:
You see the same thing when people build attribution models. You know, when they’re looking at media and trying to work out where the attribution comes from. Because if you work on last click, the only place you’d look is Google, but in actual fact a lot of display advertising actually then leads to search for particular phrases because people don’t want to click on the ad, because they know that that will take them through your sales funnel. Whereas they can do a search and then find their own way through the process.
Glenn:
Yeah.
Darren:
So it’s that insight that you need to develop from looking at the data, and then making an hypothesis, and then testing it to start to look at how many people do a search for a particular phrase that was in your display ad. Within two or three minutes of that display ad being shown is the way of testing it and looking at that data.
Glenn:
Yeah, in fact, I’ve seen a case study on this. We helped an electronics retailer look at this and they were looking at last click as their method, and it completely undervalued display advertising.
When you do a proper analysis of this over, let’s say a month before purchase, what you discover is a different picture. What you discover is display does its work a month ago to stimulate.
Darren:
And then they do more on their own.
Glenn:
Then closer to the day of purchase they’re doing searches to find more information, then they go to affiliate sites and so on, then the purchase happens. But when this company looked at the whole picture, they actually saw that the effect of display was twice what they thought it was.
Data in isolation doesn’t tell you what the cause is
Darren:
Now you mentioned it before, and I just want to pick up on it now, is that often people will look at data and then jump to a cause and effect which I think you need to be careful about, and it’s beautifully demonstrated by that series of books on freakonomics.
You know, where data in isolation doesn’t, as you say, tell you what the cause is, and that people can quickly, without the proper level of interrogation, consideration and analysis, will start to look at data and looking at last click, you go, “Well, all our money should be on Google”, whereas in actual fact display advertising has an impact.
Glenn:
Yeah.
Getting started with data analysis
Darren:
What are some things that you would recommend to people when they’re going, starting out, looking at data and analytics to stop that type of behaviour, that freakonomics behaviour happening?
Glenn:
What you mean kind of confusing correlation with causality?
Darren:
Yeah, that’s it.
Glenn:
Yeah. Well, there’s no easy answer to that. I think the first thing I would say is that you don’t have to boil the ocean when it comes to data analysis. You can start with some big buckets of spend, analyse how they appear to be driving behaviour.
And if that doesn’t give you any results that make sense, you then have to say, “Well okay, let’s look at another, let’s look at a deeper level of data”. So, I would always suggest you start with quite high aggregated level of data and drill your way down as need requires, until you start getting something that makes sense from a causal point of view, rather than just pure correlation point of view.
And this is kind of segues into this whole discussion about what is big data, right? People talk about big data and actually, the truth is, big data doesn’t have to be big. And in fact, too much data is a real pain to get your head around.
Darren:
Well there’s the mathematics law of big numbers. First of all, the bigger the sample, the more likely it is to be an average result.
Glenn:
Yes, if you dig too deep into let’s say, individual consumer level data, you can get interesting analysis there, but you will get a lot more noise at that lower level. The law of large numbers as you aggregate up and you get a total picture, you can see the bigger picture like that.
So that’s what I would suggest. I’d suggest you go in, start with some fairly simple data sets, at a fairly high level, see what kind of correlations you’re getting and then interpret those and then just drill down as you go.
Darren:
Sense making, I like the term sense making because it is – how does this make sense to me?
Glenn:
Yeah exactly.
Darren:
Well look, I think we actually have to wind it up there, but thank you very much Glenn; it’s been a fascinating talk. I was going to say, I think this has worked in being a terrific discussion but I don’t have any data to support that.
Glenn:
Well maybe you will. Do you get the kind of rankings of your podcast downloads?
Darren:
I’ll let you know! Okay, thanks very much.
Glenn:
Thank you Darren.