It's been a while since the last blog post. With the last post dated on 7th June, the World Cup held between 11th June to 11th July and this post just on the final day, one could easily infer that we have just been busy watching the football matches. But as the mantra goes, correlation does not imply causation and there is not much we can get from the apparent relationship.
Another related case of a spurious correlation is well illustrated in Paul the Octopus, or the psychic octopus: Paul gained fame for his robust predictions on German matches. While even information markets and bidding sites did not foresee many of the results (such as Germany losing against Serbia or Spain), Paul managed to correctly predict the outcomes of all German 2010 World Cup games. These apparent predictive powers (7 correct guesses out of 7) have even resulted in death threats from disappointed fans and superstition among the generally reasonable.
However, even 100% precision in 7 games is no evidence for any supernatural power. Of course, we would intuitively expect Paul to guess about half of the games right, and the probability of getting 7 games right would be about 0.8% (0.5^7*100), hence very unprobable and suggesting supernatural powers. But is it?
First, consider the results Paul predicted in the last Euro Cup: Out of the 6 games played by Germany, Paul correctly predicted 4 games, i.e. 66% precision. Now getting one more game right than half of it (which would be assumed pure guess) could just be noise or randomness. This is the small sample size problem Mlodinov refers to as the "law of small numbers". In order to reliably identify the precision of Paul's estimates, we would need to repeat the precisons for a larger sample.
Second, remember that Paul only predicted German games. Obviously, the best guess to increase your precision would be to always bet on German wins. Why? The reason is simple: If Germany is out, Paul won't be asked again, so his prediction are always conditional on Germany having won few times before. Now if winning does not exhibit regression to the mean but is a signal of the team's overall strength, it is clear that betting on future wins is the best strategy. This is mixing up conditional and unconditional probability.
Third, keep in mind that Paul is just one of many oracles used to predict games. Among the amateur predictors found in various pubs and online platforms, Paul is also joined by a parrot Mani, who is also renown for making correct guesses. But why do Paul and Mani gain prominence, whereas countless other oracles have never made it into the press? Obviously, Paul and Mani are selected because they have done good guesses. In a world with a million amateur oracles guessing, it should not seem special if there are at least two who got almost everything right. This is the typical selection bias we face, whereby we selectively pick the evidence in order to argue for our case.
Why all this fuss, then? The example of Paul highlights many of the logical fallacies human face in general. All these methodological issues (small sample size, mixing up conditional and unconditional probability and the selection bias) are concerns that hinder us to conduct rigorous scientific work. The physicist Mlodinov (2009) offers a pile of example for logical fallacies, and most of the current criticism the development economist Easterly brings forward are based on exactly these points. Latter economist is also worthwhile quoting to conclude the post: "Humans are suckers for finding patterns where none really exist, like seeing the shapes of lion and giraffes in the clouds." (Easterly, 2009).
PS: On the other hand, Sen blames Easterly for being "The Man without a Plan"
PPS: Paul has predicted that Spain will win against Netherlands tonight (first non-German game prediction). However, we should not be concerned with verifying this single result. Instead, we should let Paul predict the next 20 or so World Cups. Unfortunately, the life span of an octopus is about 4 years maximum, so Paul will not be able to statistically prove his powers.
Posts
