Blind Taste Tests

No one believes that all whiskies or wines taste the same. They have different mash bills, yeast strains, barrel wood sources, and charring. The rickhouses have different environmental conditions. So whiskies certainly evolve varying flavors.

But can we rely on experts to find out which are better?

2016 Whiskey shelf

In test after test, we find that even experts don’t rate the supposedly best whiskies significantly higher than other whiskies. Expectations overwhelm our experience.

Below is a set-up for a blind taste test of bourbons sponsored by The Bourbon Quest.

This is how a blind taste test is set up.

Blind taste test of a 1964 Black Bowmore. What happens when you put what’s arguably the most legendary single malt into an otherwise unassuming, unhyped blind tasting? Not a whole lot… Any “legendary” whisky is good-to-excellent, but mind-blowing is impossible. Whisky can only get so good, and the rest is added in your head.  Blind Black Bowmore: A blind tasting of Scotch whiskey yields a great lesson

Feeling baffled when it comes to wine? Turns out the experts don’t know anything either. (And I bet the same is true for whiskey)  Adam Ruins Everything – Why Wine Snobs Are Faking It

How to run a blind taste test. Whiskey with Wes: A blind taste test to reveal the best whiskey for the Grove

Blind Taste Test – Including Pappy Van Winkle’s 20 Year, Michter’s 10 Year, Blanton’s, Basil Hayden’s and Jefferson’s Small Batch Bourbon. Guess what happened?!  Taste Test: The (Cult-Like) Pappy Van Winkle’s 20-Year Reserve Bourbon

Study on the Chill Filtration of Scotch Single Malt Whiskies: Comparative Assessment of a Blind Tasting, by Horst Lüning

In a blind tasting 111 experienced whisky connoisseurs from Germany tasted a total of 1,331 samples of high-quality Scotch single malt whiskies and evaluated them according to numerous parameters. …The study shows that even among experienced single malt whisky connoisseurs, only 45% identify chill filtration correctly. Respectively, non chill filtration is identified correctly by 55% of the participants. At 50.0%, the total identification rate is exactly at the statistical mean. Thus chill filtration cannot be identified by the consumer.
Study on the Chill Filtration of Scotch Single Malt Whiskies

Wine-tasting: it’s junk science. Experiments have shown that people can’t tell plonk from grand cru. Now one US winemaker claims that even experts can’t judge wine accurately. What’s the science behind the taste?  Wine-tasting: it’s junk science

Judge Reliability at a major U.S. Wine Competition

An analysis of over 4000 wines entered in 13 U.S. wine competitions shows little concordance among the venues in awarding Gold medals. Of the 2,440 wines entered in more than three competitions, 47 percent received Gold medals, but 84 percent of these same wines also received no award in another competition. Thus, many wines that are viewed as extraordinarily good at some competitions are viewed as below average at others. An analysis of the number of Gold medals received in multiple competitions indicates that the probability of winning a Gold medal at one competition is stochastically independent of the probability of receiving a Gold at another competition, indicating that winning a Gold medal is greatly influenced by chance alone.

Concordance Among 13 U.S. Wine Competitions. Robert T. Hodgson, Journal of Wine Economics.

———————-

“Wine judge performance at a major wine competition has been analyzed from 2005 to 2008 using replicate samples. Each panel of four expert judges received a flight of 30 wines imbedded with triplicate samples poured from the same bottle… About 10 percent of the judges were able to replicate their score within a single medal group. …only about half of the panels presented awards based solely on wine quality.“ – That’s literally no better than random guessing!
An Examination of Judge Reliability at a major U.S. Wine Competition, Robert T. Hodgson, June 2012, An Examination of Judge Reliability at a major U.S. Wine Competition

Do more expensive wines taste better?

“Individuals who are unaware of the price do not derive more enjoyment from more expensive wine. In a sample of more than 6,000 blind tastings, we find that the correlation between price and overall rating is small and negative… Our results indicate that both the prices of wines and wine recommendations by experts may be poor guides for non-expert wine consumers.” American Association of Wine Economosts, AAWE Working Paper, No. 16, April 2008, Do More Expensive wines taste better? Robin Goldstein et al

Richard Feynman at Caltech in 1974

The first principle is that you must not fool yourself—and you are the easiest person to fool. So you have to be very careful about that. After you’ve not fooled yourself, it’s easy not to fool other scientists. You just have to be honest in a conventional way after that.
Richard Feynman, Nobel Prize winning physicist, from “What is and What Should be the Role of Scientific Culture in Modern Society”

 

 

Here you can check out our reviews of  bourbon, Scotch,  Irish whiskey,  Canadian whiskey,  ryes or flavored whiskys.
We have articles on science & health, and a plethora of other topics. Feel free to learn more about me, Distilled Sunshine.