The Iron Law of Confirmation Bias   2 comments

Ben Goldacre posts over on Bad Science about the distressing tendency of people to reject that which does not confirm their already-held beliefs.

Their views on each issue were added together to produce one bumper score on the extent to which they thought science could be informative on all of these questions, and the results were truly frightening. People whose pre-existing stereotypes about homosexuality had been challenged by the scientific evidence presented to them were more inclined to believe that science had nothing to offer, on any question, not just on homosexuality, when compared with people whose views on homosexuality had been reinforced.

When presented with unwelcome scientific evidence, it seems, in a desperate bid to retain some consistency in their world view, people would rather conclude that science in general is broken. This is an interesting finding. But I’m not sure it makes me very happy.

This is in and of itself not a terribly astonishing finding (depressing though it may be).  There is a very large body of evidence to show that people are resoundingly poor at objectively measuring evidence using a consistent standard.

Simply put, if a study has a conclusion with which you agree, you tend to ignore the limitations of the study and place more trust in the conclusion.  If a study with the same exact design has a conclusion with which you disagree, you have a tendency to focus on the limitations of the study and place less trust in the conclusion.

Even if the methodology is precisely the same.  Depressing, isn’t it?

Unfortunately, this tendency is not minimized by intelligence quotient.  Smart people, as I’ve said before elsewhere on this blog, are not guaranteed to be wise.  As I observed over the weekend to the pater familias, people who have invested huge quantities of time and training in a field (and thus have a justifiable belief in their own expertise in that field) unfortunately have a tendency to assume that competence possesses fantastic osmotic capabilities.

Which is fascinating, as they’d jump straight to how much work they had to complete to gain basic competency in their own field as a justification for their position for their beliefs in their field… but the expertise acquired by someone else through their own study of some other field is trumped by… well, I’m not really sure.  The “obvious truth”, I suppose.

People who are mathematical mavens assume that they understand economics better than they actually do.  People who are biologists assume that they understand psychology better than they actually do.  Successful politicians assume that they can understand engineering, everybody assumes they understand everybody’s theology, and so on.

You see this a lot when you start arguing about the philosophy of science.  Scientists, as a class, make fun of other fields in order of their likelihood to line up with postivist standards of measure.  Mathematicians famously make fun of scientists (and the philosophers make fun of the mathematicians).  I’ve thrown this XKCD comic up before, it’s a classic which illustrates the situation fairly well:

Sadly, as you move away from mathematics towards physics, you move away from an axiomatic system to constructive empiricism.  You lose truth, but you gain facts, something I’ve mentioned before.  The problem, of course, is that your facts are based upon your ability to observe, which is largely contingent upon the accuracy of your measurements.  The farther away from physics you go, the more uncertainty you get in your measurements, and the more qualifications you need to put upon your observations (which has the distressing tendency to produce the, “Social scientists aren’t *real* scientists” attitude among the hard science crowd).

This follows, of course, when you have a biologist who has a particular ideological stance, but some whippersnapper sociologist comes along and challenges that ideological stance.  The biologist, of course, depends upon science for their livelihood, so they can hardly disclaim science.  They can cheerfully disclaim sociology.

The point?

You know, I’m really not sure I’m going to bother to say.  If you’re reading this, and you agree with me, you’re already going to know what the point is.

And if you don’t, you’re going to disagree with the point, right?


Posted July 7, 2010 by padraic2112 in philosophy, rants, science, Uncategorized

2 responses to “The Iron Law of Confirmation Bias

Subscribe to comments with RSS.

  1. I’m pretty sure your point is that sociologists are really smart because sociology is actually really much harder than physics because of the measurement issues. Right?

  2. I liked that cartoon the first time I saw it, and I still like it now.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: