Calling Bullshit
December 1, 2021

Calling Bullshit: a book to develop your Critical Thinking skills

By Paul Laughlin

Apologies for my use of language, but this book focuses on a key challenge of our times, calling bullshit on lies. To put it more diplomatically, developing the critical thinking skills to spot & challenge misinformation.

The latest book I am reviewing is another in the Penguin Science series (alongside “Moonwalking with Einstein“). In the traditional Penguin paperback size it is ideal to take with you anywhere & so enjoyable you’ll want to complete it ASAP. The writing style is warm, human and packed with entertaining examples. A book on data & critical thinking that often makes you laugh. Who could ask for anything more?

The authors of “Calling Bullshit: The art of scepticism in a data-driven world” are Professors Carl Bergstrom & Kevin West from the University of Washington. Carl is a theoretical & evolutionary biologist, whilst Kevin is a data scientist. Their own critical thinking skills shine through this fun book, as well as plenty of honesty about their own mistakes. More importantly it is packed with heuristics, lists & tips to help us all better spot bullshit & even call it out.

Why this book matters and who should read it

Both in the preface and the first two chapters the authors layout a clear diagnosis of the problem. From historical quotes to modern examples of misinformation, they make clear how often humans are being misled by apparently data-based claims. That is what makes this book so pertinent for data leaders.

As the authors highlight, many people are confident in challenging & debating rhetoric or ideas from the humanities. However, once data, tables of numbers, statistics and graphs are introduced – many people are intimated. They assume the person talking knows best or that “the numbers don’t lie“. This malaise is only exacerbated by the rise of the internet, self publishing & social media. Now every crackpot idea & misuse of data or stats can have a megaphone.

For that reason, I recommend this as a book for both data leaders & all non-technical leaders. It is a helpful protection for all leaders and explained in a language that enables them to better fulfil their role. All leaders should be able to understand, critique & challenge the information presented. Data leaders should also hear this book as a clarion call to their role as watchmen. Those who first spot & when needed call out misrepresentation (as well as not producing it).

What is covered? (from defining Bullshit to Diplomacy)

For a small book (under 300 pages) this is a usefully comprehensive book on the subject. One that I suspect will become a classic. Its 11 chapters can be roughly divided into parts covering defining the issue, identifying 6 different forms of bullshit and finally what to do about it.

After careful definitional work that help us recognise why we are where we are, the really gold dust in this book is the clarity of that middle part. This brings to light common mistakes (or deliberate misrepresentation of data & how to spot it). Some of the content reminded by of the checklists I’ve shared previously from Tim Harford & Professor David Speigelhalter. However, the former is more about our own mindset/biases and the latter mainly an introduction to statistics. So, this book goes deeper and better equips us to spot these cases.

Those forms of bullshit are:

  1. Causality (understanding correlation is not causation & spotting when this is misleading)
  2. Numbers and Nonsense (the concept of ‘Mathiness‘ and how even some leading academics publish nonsense equations)
  3. Selection bias (the need to think critically about data sources, wider context & selection process)
  4. Data visualisation (great chapter that Tufte would love, including spotting ‘ducks’ & ‘glass slippers’)
  5. Big Data (well actually this is more about Machine Learning, a similar critique as “Rebooting AI”)\
  6. Science Journals (written as a critical friend who still believes but calls out pitfalls of incentives in the system)

The final two chapters include vital checklists to arm yourself against such a tsunami of brown stuff. The authors list 6 pointers to spotting bullshit in general and a checklist of 10 points to help you online. Then the final chapter takes the focus to the world of diplomacy. With some wise words about not alienating others or speaking up to try and appear clever. They also go on to layout what is really a moral challenge for us all. When the time is right, how can you call out bullshit in a way that helps. Well worth reading & practicing those tactics.

My favourite tips for better Critical Thinking

There is so much in this book that I fear my review will not do it justice. But I guess that is ok, as I’d prefer to whet your appetite to read the book yourself than replace it. But I will call out a couple of sections of the book that I enjoyed most.

The Data Visualisation chapter was a great example of covering a lot of material (and potential bullshit) in under 50 pages. This is a good complement to Alberto Cairo’s “How Charts Lie“. Their list both confirms and builds on the pitfalls that Alberto highlighted. They cover familiar problems like misleading axes, trends, 3D & proportional ink. They also highlight cases of Edward Tufte’sducks“(graphics where decoration/aesthetics overwhelm/obscure the data). Plus, the great but gruesome definition of “glass slippers“(data shoehorned into inappropriate visual form, lie periodic tables of everything). they build on this with the concept of “ugly sisters” (meaningless use of analogies in schematic diagrams, e.g. parts of a unicorn form different technologies). Well called out!

But my favourite tips were from the final chapter. Ideas on how to call out bullshit in a way that works. They include good advice on psychological approach. But I really liked the tactics for exposing the error. These include:

  • Use Reductio Ad Absurdum (with an example of extrapolating to future sprinters running in negative time)
  • Be memorable (with an example of MRI scans on a dead salmon)
  • Provide analogies (with an example from baseball for understanding Seattle traffic flows)
  • Redraw data visualisations (with an example showing more data in context)
  • Deploy a null model (with an example of how to compare age bands better)

What are your best stories of Calling Bullshit to protect others?

I hope you found that review helpful & it has enticed you to read the book (or buy it as a present).

What has been your experience of both spotting bullshit & calling it out? Do you have any great stories of how you’ve managed to expose misinformation and so protect others from being mislead? If so, I’d love to hear your stories and perhaps publish a collection of the funniest or most touching examples.

As a final thought let me leave you with a quote that is repeated in this book. One that reminds us of the scale of this challenge (especially in the age of social media) and why we need to continue the good fight.

“Falsehood flies, and truth comes limping after”

Jonathan Swift (1710)