CABLab Blog

Vary One at a Time: A Thumb-Rule for Comparison

Critiquing prior work is an essential part of research. It helps in developing the correct forum for discussion of research ideas and a detailed explanation of the appropriate methods of conducting experiments. This piece of writing is an attempt to inform young researchers about appropriate ways to conduct experiments and issues that may create obstacles in the way of reaching the targeted objectives of a given study. Before you proceed to read this post kindly read the article titled “The effects of feelings of guilt on the behaviour of uncooperative individuals in repeated social bargaining games: An affect-as-information interpretation of the role of emotion in social interaction” by Ketelaar and Au, published in 2003 by the journal ‘Cognition and Emotion’.


Ketelaar and Au (2003) investigated whether feelings of guilt, either manipulated or self-assessed, would compel non-co-operators to become co-operators in repeated social interactions. In order to test this hypothesis, the authors conducted two studies, one using the Repeated Prisoner’s Dilemma and the second using the Repeated Ultimatum Game. Both the studies were divided into three stages. The first and the third stages were identical where the participants played the respective game. The two studies differed crucially in the second stage of play where feelings of guilt were either manipulated or self-assessed.


In the first study, feelings of guilt were manipulated through a writing exercise, by asking participants to write about a recent incident when they felt guilty or ashamed or blame-worthy. However, the feelings of guilt were only implied and not objectively assessed before the third stage of the play. Hence, the authors conducted a second study where they asked the participants to self-report their feelings of guilt, among other emotions, immediately after the first stage of play. With this variation, the authors could, with some confidence, conclude that the change in cooperative strategy by non-co-operators could be attributed to the feelings of guilt. This conclusion does not sound all that convincing since absence of other emotions, such as shame or blameworthiness, was not established during the course of the second study.


An important point to note about the research study is that there were very few commonalities between the two parts conducted. Except for the second stage of both studies, where guilt feelings were either manipulated or self-reported, the extent of overlap between the two studies was minimal. The most glaring difference was the nature of the games that were implemented. The Prisoner’s Dilemma is a very different game compared to the Ultimatum Game. The Ultimatum Game measures bargaining behaviour, whereas the Prisoner’s Dilemma is used to study cooperative behaviour in the face of a dilemma. Therefore, using only the Prisoner’s Dilemma, across both studies, would have served the purpose of examining the effect of guilt feelings on cooperative behaviour. The Ultimatum Game is a serious misfit in the second study as the variables it measures vary considerably from those measured by the Prisoner’s Dilemma.


Quite a few other, minute but crucial, details that varied between the two studies are listed below:

  • The Prisoner’s Dilemma was played for four blocks of 10 trials each in both the first and third stages. On the other hand, the Ultimatum Game was played only once at each stage. Therefore, the ability to generalise the results for selfish/generous behaviour in the second study is weaker compared to the first study, which is capable of representing more generalizable cooperative behaviour.

  • Study 1 was conducted at a single stretch in a single day, but study 2 spanned over a week. This variation makes sense if there is proof that feelings of guilt once elicited can be sustained for over a period of seven days. When there is such a long gap between stages in a study, other confounding variables can also affect the results.

  • Study 1 involved participants playing against a computer, but believed to be playing against an unknown human partner. On the other hand, study 2 involved participants playing with familiar partners, their classmates. Hence, there was deception involved in the first study, but not in the second study.


In the first study, categorisation of participants, as cooperative and uncooperative, was done based on the median level (46.25 percent) of cooperation. One other possible method could be to use a 50 percent cut-off rate for cooperation. Therefore, using the median level of cooperation seems arbitrary, especially as the median level could have been any number from 0 to 100. Such a cut-off rate can differentiate between co-operators and non-co-operators in a relative sense at best. It is important to note that practical concerns- such as equal representation of individuals as co-operators and non-co-operators- may demand a median level for the cut-off rate, but theory can lead to a more justified cut-off rate. Hence, it becomes important to understand and differentiate between practical demands and theoretical justification behind choosing a cut-off rate, which can turn out to be quite controversial among researchers.


The self-report method used in the second study was able to capture the intensity of guilt feelings as a continuous measure. However, any indication of guilt was taken as a single category and the level of intensity of guilt was not considered for analysis. Moreover, for the bargaining task, all offer sizes were categori