Why the Prisoners' Dilemma Has Not Been Tested In Any Way Whatsoever

Right, so the last few days or so, a minor socio-behavioral experiment conducted by two economics researchers in a German prison (where they compare how inmates and students reason) has been kicked around the web and social media via a brief article in the web magazine Business Insider (bounced further or cross-posted by numerous sites and blogs, including Improbable Research and Freakonomics) as news of the famous game theoretical result called the prisoners' dilemma having now been tested and, as it where, faulted – or, at least, put into question. The report in Business Insider underscores this notion of what the results of the experiment mean by the headline "They Finally Tested The 'Prisoner's Dilemma' On Actual Prisoners...". However, all of this is a misconception. 

The prisoners' dilemma is specific illustration of a general result in the mathematical discipline of game theory. It is famous, since it illustrates that it is possible for two self-interested and perfectly (instrumentally) rational parties to succeed acting in their own best interest, while the joint result of that is worse for these parties than if they had both abstained from acting in their own best interest (in a particular way). In economics, the importance of this is that even a perfectly free market with perfectly informed, self-interested and rational actors cannot guarantee an optimal overall outcome – i.e. Adam Smith's famous theory of the invisible hand is false. The classic illustration of this phenomenon is a story of two prisoners, held by the police on suspicion of a crime with a substantial sentencing value and who are both offered a plea bargain, provided that they confess and thereby rat on their pal. If they both confess, the discount will be much less (since no one will get the max sentence), but still clearly less harsh than for any of them that abstains from confession while ratted on by his or her pal. However, at the same time, if they both keep silent, the prosecutor will only be able to make minor stuff stick and they can escape with a slap on the wrist. This is often illustrated by a decision matrix of the following sort (I nicked this image from this page):

So, assuming that you are one of the culprits, the example is here constructed so that confession will earn you parole (or acquittal) if your pal does not confess, while said pal gets life, but 20 year in prison if you both confess (since then, the prosecutor need not award anyone of you as handsomely for helping him secure life for someone). However, if you both hold your water, you will be able to escape with a mere 1 year sentence. In the matrix, PA's outcomes are in the lower part of the box and PB's in the upper. The logic is that for each of you, the rational strategy is to choose a dominant strategy (if there is one), that is a strategy that is the best one regardless of what the other party does. To find out if there is one, you reason like this. Suppose you are PA and that you first assume that PB would choose not to confess. In that case you can choose between confessing, in which case you are acquitted (or paroled), and not confessing, in which case you have to do 1 year. Clearly, confessing is better. Next step in the argument is to instead assume that PB would confess. In that case, you have to choose between confessing, in which case you get 20 years, or not confessing, in which case you get life. Again, clearly, confessing is the better option. This means that confessing is the preferable option no matter what PB would do, which means that confessing is the dominant strategy and thus the rational way to behave. Now, since the strategic situation of PB exactly mirror's that of PA, we need not go through all these steps again, but merely conclude that the same reasoning and conclusion applies to PB as well. The end result is that, jointly, PA and PB realise a collective strategy that secures them 40 years of total prison time, rather than the total of 2 years that would have resulted, had they both instead kept quiet (what is often called "cooperation" when the example is discussed). In other words, on the collective/societal level they fail splendidly in spite of both succeeding perfectly on the individual level. This result is a useful one in a great many ways, not least as an inspiration for understanding various collective coordination problems, as well as for explaining why, in social settings, it may be rational for initially self-interested people or beings to evolve into more cooperative sorts of actors than what an assumption of self-interest and rationality would otherwise lead one to assume. I will not go into the abundance of interesting social science research that has been coming out of this, merely point to the reference available via the links already provided above. 

What happened in the experiment under consideration, is that inmates and students where put to decide in different versions of prisoners' dilemma-style decision situations and that, in fact, prisoners were found to be slightly more bent on going for cooperation (that is, not rat on your pal) than the students.

Now, what may be underscored, is that the application of the prisoners' dilemma or any other game theoretical result will always have to observe the assumptions necessary for deducing it. So, in a prisoner's dilemma it is assumed that the actors are (1) perfectly instrumentally rational (meaning that they maximise the satisfaction of their own interests, or the expected such satisfaction), (2) having no interests relevant to the evaluation of the option besides those illustrated in the matrix – that is they care only about the harshness of the sentence. As mentioned in the article in BI, in so-called repeated prisoners' dilemmas, where the outcome of each matrix is assumed to be fed into a new similar choice situation, there is sometimes also added the interest of not being punished by one's pal for ratting and as long as that punishment is not so much worse than 1 year inside that this sentence would be preferable to being acquitted (or paroled) + punished by your pal, this does not change the logic described above. 

This means that we may expect that no actual person fits the assumptions of the prisoners' dilemma, since we all, criminals or not, have a wide collection of interests. In fact, this is the conclusion of the research article that is the source of this whole buzz: 

...we find a similar and significant fraction of inmates and students to hold social preferences.

In spite what Business Insider tries to have us think through its suggestive headline ("The Results Were Not What You Would Expect"), this in fact looks very expected to me, but what it means is, in effect, that whatever has been going on in the experiment with the prisoners and the students, it has not been a prisoners' dilemma of any kind. Why? because in a prisoner's dilemma it is assumed that the acting or choosing parties do not hold "social preferences", that's why.

However, it seems to me that the authors of the article miss an alternative explanation, namely that the parties in the experiments are not even approximating perfect instrumental rationality. So, even if we (implausibly) assume that all they care about is harshness of punishment (from pals or society), they may be so little inclined to choose the dominant strategy that it looks like as if they had social preferences. In view of the many results out of the field that has become known as behavioural economics that puts the instrumental rationality of ordinary people into question, this is not an a priori implausible hypothesis. But this would not change anything visavis the prisoners' dilemma, for if the parties' rationality is lacking, assumption no. 1 above is not met, so again we may conclude that the result implies that it is not a prisoner's dilemma that has been tested.

Finally, if one understands the nature of the prisoner's dilemma (as I would have assumed several of the people reporting this buzz to do), one would also understand immediately that no empirical experiment of this or any similar kind be a "test" of the prisoner's dilemma. This is due to the simple fact that the prisoners' dilemma is not an empirical hypothesis and thus makes no claim to any such dilemmas occurring in any place at any time in the real world. In particular, it makes no claim whatsoever about prisoners. It is a part of the nature of a prisoners' dilemma that if a party (prisoner or student or what have you) is caught in it, there is but one way out – namely the one described earlier. All other suggestions imply that a prisoners' dilemma has been, in some way or other, avoided. We may, of course, congratulate the inmates for accomplishing this and pity the students for their slightly lesser success. We may, if we want, also speculate on the nature of social life in prison, as well as on that of economics education at universities. What we may not do, however, is to infer that the prisoner's dilemma has been disproved, tested, applied or even illustrated.