Which one of the following three sequences of births (six baby boys and girls) registered at a hospital is most probable?
For many, the third stands out as the most likely, but with a bit of thought, we realize that each sequence is equally likely. This is one of the quizzes served up by Nobel Laureate Daniel Kahneman in his book, Thinking, Fast and Slow, in which he explores the tendency to make quick decisions that are influenced by the cognitive biases harbored in all of us.
When co-teaching a course called Conceptualizing Research at the doctoral level I include a few such quizzes, many of them taken from the work of Kahneman and his research collaborator, Amos Tversky, conducted in the 1970s and 1980s. And while these kinds of quizzes may seem rather distant from the type of content taught in research methods courses, I believe they have close links to the types of potential assumptions and conclusions graduate students and researchers can make when analyzing their data.
The example above illustrates how easy it is to look at data, notice a pattern and then be seduced by it, sometimes to the exclusion of more significant meanings that may lie deeper in the data. Although this may seem obvious, we should not underestimate the powers of our cognitive biases.
In my ‘Points of Departure’ essay in Teaching in Higher Education, I begin with the results of a real research study which reports that school children who eat a full breakfast tend to score higher grades than their peers who didn’t. In my course, I invite the students to propose reasons for this finding and they very quickly make connections between the nutritional value of breakfast and blood sugar levels and link these to alertness in class. This response appears very reasonable and probably has some bearing. But a more compelling analysis goes in a different direction: children who eat breakfast are more likely to come from stable families with two parents present, and this in turn may be a proxy for families that have higher incomes, the resources to pay for tutoring and the time to spend helping with homework. In the same vein, children who don’t eat breakfast are more likely to come from families bereft of these qualities.
Research methods courses are a standard offering in graduate programs, yet, based on an informal survey of popular textbooks, which presumably parallel these types of courses, the focus is mostly on how to properly follow procedural operations. However, I contend that an important element may be missing: the avoidance of cognitive biases and how to make good decisions because these biases are not at all obvious. Among grad students, whose data collection and analysis is largely unfunded with small sample sizes, there may be a greater tendency to be attracted by patterns that immediately stand out.
My ‘Points of Departure’ piece is thus a bit of advocacy for raising the profile of several named biases and how to avoid them.