draliman on life

Taste tests – seen regularly on TV adverts, “randomly” selected “members of the public” choose between two or three unmarked brands and decide which they prefer.

I imagine that the various manufacturers perform their taste tests and if they find that their brand is not the preferred one, they keep it quiet. Otherwise they hire some actors and make an advert. Which is fine.

However, from my own personal experience in switching brands, I’m not at all sure we can actually infer any usual information about which brand is best from a taste test. I see things falling into three main categories:

1. Brand A really does taste better than Brand B (a statistically significant sample is required!)
2. The taste tester has been buying Brand A for years and it tastes “normal” – and Brand B just tastes a bit “off” – these people don’t like change in their food
3. The taste tester has been buying Brand A for years and is ready for a change and so chooses Brand B

I recently changed my brand of “spreadable butter” – those are the ones which taste butter-like but aren’t really butter. I bought a new brand but had some of the old left, so I tried them both side by side and decided the new one tasted more butter-like. To my taste buds, that constituted a proper, successful taste test.

By the way – spreadable butter? Butter and margarine both have pluses and minuses – butter is natural but high in fat, margarine is lower in fat but full of chemicals (to make it spreadable). Is spreadable butter not the worst of both worlds?

But I digress.

Here’s a personal example of case (2) above. I used to drink regular cola. I didn’t like the taste of diet. Then I looked at the calories in regular cola and how much I drank a day, and discovered I was absorbing several hundred calories just through what I was drinking. People tend to discount drinks as being “mainly water” but try adding all the calories up and you might be surprised. So I switched to diet. Ugh! After a couple of weeks it tasted fine and now I hate regular drinks instead – far too sweet!

If I’d taste-tested regular versus diet a few years ago I would have preferred regular, now I would prefer diet – because I’ve become used to diet, not because it actually tastes any better.

It’s all very well getting a statistically significant sample for a taste test, but if more people already use Brand A, will they tend to prefer Brand A because they’re used to it?

And that’s statistics in a nutshell. Useful at face value, but think about what they’re not telling you and take them with a pinch of salt. Now, is that a pinch of Brand A – table salt, Brand B – rock salt, or Brand C – low-salt salt…?