Decision Making In Prisoner’s Dilemma

9.6 Discussion

Tests of hypotheses H5, H6, H10, and H11 confirmed the existence of end-effect in iterated Prisoner’s Dilemma with known finite end.

We did not confirm hypothesis H7 and we did not find significant difference between “28” and “26”, and “28” and “27” either. This means the amount of cooperation was the same in closed, semiclosed, and open games in moves 20-25. So, there is no “early” end-effect (in moves 20-25) in semiclosed games, as postulated in hypothesis H7.
On the other hand, test of H7 and the additional tests (comparing “28” and “26”, “28” and “27”) indicate that end-effect is specifically present only in the very last moves of the closed-end game. Analogous conclusion can be drawn from our test of hypothesis H9 (i. e. the overall frequency of cooperation during the game – save the very end, i. e. in moves 1-25 – was the same in closed, semiclosed, and open games).
Hypothesis H8 is related to hypothesis H7. We originally supposed (in hypothesis H7) that there might be an “early” end-game effect in semiclosed games, and that this effect will probably be weaker than in closed games (hypothesis H8). Since hypothesis H7 was not confirmed, the fact that hypothesis H8 was confirmed only tells us more or less the same thing as confirmation of H6 (that is: end-effect exists). That hypothesis H8 was confirmed and hypothesis H7 was not (together with the tests added to testing H7), then, tells us that the decline of cooperation due to end-game effect is specific for the very last moves (i. e. moves 25-30 in our case), as pointed out in the previous paragraph.
---
End-effect was found for example by McClintock et al., 1963; Rapoport & Dale, 1966; Rapoport et al., 1976; Andreoni & Miller, 1993; Hauk & Nagel, 2001; Hauk, 2003; Bó, 2005; Bereby-Meyer & Roth, 2006. Silverstein et al., 1998 found end-effect when subjects played against ALL C, ALL D, and Random (the average rate of cooperation against these strategies taken together dropped from 20% in the first eight trials to 15% in the last eight trials). They found an increase of cooperation when subjects interacted with Tit for Tat (48% cooperation in the first eight trials and 59% cooperation in the last eight trials).
End-effect has been called into question by some empirical results (Morehouse, 1966). Oskamp (1974) found that after gradual initial increase the cooperation of subjects (in a 50-run game) was stable (i. e. no significant end-effect). In an experiment by Jones et al., 1968 cooperation in an iterated game (150 trials) decreased lineary as a function of time (rather than abruptly towards the end of the game). Scodel et al., 1959 and Minas et al., 1960 found an overall decrease of cooperation in the development of an iterated game, rather than a pronounced “end-effect”.
End-game effect is probably caused by the fact that defections towards the end of the game are not (easily) punishable in the prospective moves (see section 6.19).

10. Effect of learning

10.1 Task

Our task here is to test whether subjects play the Prisoner’s Dilemma game more successfully (i. e. receive higher gains) as they encounter certain strategies repeatedly and thus have the opportunity to learn through experience how to react more successfully to them. When playing against an unknown opponent, adapting to his or her strategy is usually preferable to playing a predetermined stationary strategy (Airiau et al., 2007; Fudenberg & Levine, 1998). We will test whether subjects adapt more successfully to their “opponents” (the five strategies we employed) as they learn more about them (i. e. in repeated encounters).

We are not trying to determine the exact nature of the adaptations the subjects develop (a more detailed study of the 20.000+ trials of the Prisoner’s Dilemma game we performed would possibly enable this, though – but as pointed out by Rapoport, 1968, such an inquiry tends to be decidedly difficult). We can, however, at least tentatively hypothesize that adaptive decision making consists of (A) adopting and gradual improvement of some “heuristic” rules, such as: (1) if your opponent fights back, return to cooperation; or (2) if the opponent does not reciprocate your first defection, defect again in n moves, if he does not reciprocate your second defection, defect again in m moves, if he reciprocates your third defection, defect again in n moves, if he reciprocates your fourth defection, return to cooperation, etc. Subjects might also (B) keep a more or less precise frequency count of cooperative/competitive responses they received in past interactions, classify new interactions using this frequency counts database, and adapt their own responding accordingly. And/or subjects might learn to (C) recognize patterns in their opponent’s responses and adapt to them.
Decision making optimizing can be explained by learning to make more adaptive responses to changes in perceived utility of a prospective outcome (and/or to its perceived probability – assessed probability of an outcome influences its perceived utility, see section 4., paragraph (1)), see for example Cohen & Axelrod, 1984; Wilson, 1986; Moore & Moore, 1989.
After suffering losses an individual is likely to change her behavior (because of her “perception” of further prospective losses). If receiving satisfactory gains, the present behavior is likely to be reinforced and maintained (because the decision maker “perceives” further prospective gains). When decision making optimizing (effect of learning) takes place, less losses are needed to change an individual’s maladaptive behavior and less gains are needed to reinforce/maintain adaptive behavior.
If there is an effect of learning on decision making in our subjects (and decision makers in general), they would respond more effectively, as they learn more about a given strategy/opponent (or other features pertaining to the whole strategic situation). Such an effect of learning will lead to higher gains in later games as compared to earlier ones.