Which person, from another field, do you think advertisers could learn most from?
One name that might not spring to mind is Abraham Wald.
Wald, who Matthew Syed wrote about in Black Box Thinking, was the grandson of a rabbi, born in the Austro-Hungarian Empire in 1902. He was a mathematical prodigy – at primary school he was correcting his teachers and at secondary school he was correcting his textbooks.
After a glittering early career at the University of Vienna, his success stalled in the 1930s with the rise of anti-Semitism. In 1938 he fled to America. That decision probably saved his life. By the end of the war his whole family, bar one brother, had been killed.
Once in the US he worked for the government’s Applied Mathematics Panel, putting his talents to the war effort. Specifically, how to reduce the death rate among European bomber crews. Theirs was such a dangerous job that about half of bomber crews died during the war.
Wald’s team needed to determine which parts of the planes should be armored. It was a tricky calculation. Too much and the bombers would be slow and cumbersome, easy targets for German anti-aircraft defenses. Too little and the crew were exposed – a handful of hits might bring them down.
The researchers set to work collecting data. As planes returned from a sortie his team recorded which parts had been punctured by bullets. When hundreds of bombers had been logged, a pattern emerged. As the diagram clearly shows, the areas most regularly hit were the wings, fuselage and tail.
The top brass was ecstatic; Wald’s methodology had convincingly identified the critical parts to armor. Case closed.
But was it?
Wald’s interpretation was radically different. His bosses had fallen victim to “survivorship bias”: assuming returning planes were representative of all aircraft. That was a mistake. The data collected omitted the crucial planes: downed ones. They held the clue to solving the problem, not the returning planes.
The parts peppered with bullet holes could be ignored on returning planes, after all they had limped home. It was the unscathed spots that needed armoring. Any plane hit in these places was at the bottom of the English Channel.
Wald’s counter-intuitive thinking was quickly tested and shown to improve survival rates. His approach became standard US policy until the Vietnam War.
This is more than a historical anecdote. It’s a guide to how best to use data, whether in the military or marketing. There are four key principles that can be drawn from Wald’s experience:
1. Don’t Rely On Intuition
Wald was a genius (a genuine one, not in the way we now bandy about the term) yet even he didn’t rely on introspection alone. He collected data, and only then, developed his hypothesis.
Far too many plans and approaches are based on gut feeling alone. This is an issue because agency staff are not representative. A study by Newsworks, the trade body for British newspapers, investigated this problem by recruiting 30 young media planners to complete the IPA Touchpoints diary for a week. This meant that their media habits could be compared to the broader population.
The differences were stark. Take TV. Young planners watched half the volume of TV compared to the population as a whole. Other media showed a similar variation. This discrepancy is a concern because of what psychologists term the false consensus effect. It’s the idea that we over-estimate how much others share our beliefs and behaviors.
Again this bias is prevalent in agencies. Claire Linford and I tested this by asking agency planners to estimate what proportion of the population had an iPhone. We then compared that data with whether they had an iPhone or not. Those who owned one thought half the population did, while those who didn’t put the figure at just a third.
Relying on intuition is dangerous – it leads to plans that would influence us but not the consumer. Instead, every plan needs data at its heart. And Wald gives us guidance on how to achieve that.
2. Collect Data In The Simplest Manner That Answers The Question
Wald’s methodology required pen, paper and a numerate assistant. Basic even for the 1940s.
This is important for marketing as too often we obsess over the complexity of our methodologies: econometrics, machine learning, artificial intelligence and so on. It’s as if we believe we can replace the hard work of thinking with the high costs of measurement.
But insights don’t need high budgets. For a brief into a male incontinence drug we wanted to help the planners understand the target audience. We had no budget so we used a technique we call ‘method planning’.
Over a weekend we texted the planners at random times. Each time they received a text they had to stop what they were doing and get to a toilet within two minutes. This helped the planners understand the experience of the target audience. From this experiment we uncovered two useful insights. First, incontinence is not a great concern when people are at home. After all, a toilet is a few seconds away. It’s when they’re out of home that it’s a worry. This led us to recommend media which reached people at the maximum moment of concern, such as tube (subway) car panels.
Second, our participants mentioned that while the experiment was inconvenient for them, it was also a burden on their families. This led us to the insight that maybe it was better to encourage older, male sufferers to rectify the issue not for their benefit, but for their family’s.
The cost of all this? About 70 cents on my phone bill.
Good insights can come from the simplest of places. Only if we recognize this will we undertake insight generation as regularly as we should.
3. Focus On The Representativeness Of The Data
A seemingly simplistic point but the majority of online measurement collects short term effects only: sales, views or visits. This is a problem as what we measure shapes what we do. If we just measure the short-term then that is what we optimize to: more budgets go to the best performers, while the worst are struck off the plan.
And if we optimize just to short-term metrics then we under-perform in the long term. Binet and Field have showed in their analysis of the IPA Effectiveness database, that advertising that works best in the short term isn’t ideal in the long term.
4. Data Alone Are Worthless, Analysis Is Key
Data are nothing but raw material. They’re a necessary condition for insight, but not a condition that necessarily creates insight.
Henri Poincaré, the 19th century French mathematician, pointed out the difference between science and facts. What he said is as relevant for insight and data:
“Science is built of facts the way a house is built of bricks: but an accumulation of facts is no more science than a pile of bricks is a house.”
And just as facts require theory to become science, so do data require theory and analysis to become insight.
If agencies want a competitive advantage they need a combination of best in class data collection and interpretative skills. The best source of those interpretative skills is the study of human decisions making: behavioral science and social psychology. These tell us of biases like the survivorship bias but also hundreds of others that affect consumer decision makings.
Stop Worrying About The Changing Man
The experience of Wald is useful. But there are plenty of other historical figures to learn from. We shouldn’t ignore them because our technology has changed – fundamental human motivations are the same as they ever were.
“Human nature hasn’t changed for a million years. It won’t even change in the next million years. Only the superficial things have changed. It is fashionable to talk about the changing man. A communicator must be concerned with the unchanging man with his obsessive drive to survive, to be admired, to succeed, to love, to take care of his own.”
The agencies that focus most on the unchanging man will win the insight war.
Contributed by Branding Strategy Insider by: Richard Shotton, the author of The Choice Factory, a new book on how to apply findings from behavioral science to advertising.
The Blake Project Can Help: Discover Your Competitive Advantage With Brand Equity Measurement
Branding Strategy Insider is a service of The Blake Project: A strategic brand consultancy specializing in Brand Research, Brand Strategy, Brand Growth and Brand Education
