Critiquing prior work is an essential part of research. It helps in developing the correct forum for discussion of research ideas and a detailed explanation of the appropriate methods of conducting experiments. This piece of writing is an attempt to inform young researchers about appropriate ways to conduct experiments and issues that may create obstacles in the way of reaching the targeted objectives of a given study. Before you proceed to read this post kindly read the article titled “The effects of feelings of guilt on the behaviour of uncooperative individuals in repeated social bargaining games: An affect-as-information interpretation of the role of emotion in social interaction” by Ketelaar and Au, published in 2003 by the journal ‘Cognition and Emotion’.
Ketelaar and Au (2003) investigated whether feelings of guilt, either manipulated or self-assessed, would compel non-co-operators to become co-operators in repeated social interactions. In order to test this hypothesis, the authors conducted two studies, one using the Repeated Prisoner’s Dilemma and the second using the Repeated Ultimatum Game. Both the studies were divided into three stages. The first and the third stages were identical where the participants played the respective game. The two studies differed crucially in the second stage of play where feelings of guilt were either manipulated or self-assessed.
In the first study, feelings of guilt were manipulated through a writing exercise, by asking participants to write about a recent incident when they felt guilty or ashamed or blame-worthy. However, the feelings of guilt were only implied and not objectively assessed before the third stage of the play. Hence, the authors conducted a second study where they asked the participants to self-report their feelings of guilt, among other emotions, immediately after the first stage of play. With this variation, the authors could, with some confidence, conclude that the change in cooperative strategy by non-co-operators could be attributed to the feelings of guilt. This conclusion does not sound all that convincing since absence of other emotions, such as shame or blameworthiness, was not established during the course of the second study.
An important point to note about the research study is that there were very few commonalities between the two parts conducted. Except for the second stage of both studies, where guilt feelings were either manipulated or self-reported, the extent of overlap between the two studies was minimal. The most glaring difference was the nature of the games that were implemented. The Prisoner’s Dilemma is a very different game compared to the Ultimatum Game. The Ultimatum Game measures bargaining behaviour, whereas the Prisoner’s Dilemma is used to study cooperative behaviour in the face of a dilemma. Therefore, using only the Prisoner’s Dilemma, across both studies, would have served the purpose of examining the effect of guilt feelings on cooperative behaviour. The Ultimatum Game is a serious misfit in the second study as the variables it measures vary considerably from those measured by the Prisoner’s Dilemma.
Quite a few other, minute but crucial, details that varied between the two studies are listed below:
The Prisoner’s Dilemma was played for four blocks of 10 trials each in both the first and third stages. On the other hand, the Ultimatum Game was played only once at each stage. Therefore, the ability to generalise the results for selfish/generous behaviour in the second study is weaker compared to the first study, which is capable of representing more generalizable cooperative behaviour.
Study 1 was conducted at a single stretch in a single day, but study 2 spanned over a week. This variation makes sense if there is proof that feelings of guilt once elicited can be sustained for over a period of seven days. When there is such a long gap between stages in a study, other confounding variables can also affect the results.
Study 1 involved participants playing against a computer, but believed to be playing against an unknown human partner. On the other hand, study 2 involved participants playing with familiar partners, their classmates. Hence, there was deception involved in the first study, but not in the second study.
In the first study, categorisation of participants, as cooperative and uncooperative, was done based on the median level (46.25 percent) of cooperation. One other possible method could be to use a 50 percent cut-off rate for cooperation. Therefore, using the median level of cooperation seems arbitrary, especially as the median level could have been any number from 0 to 100. Such a cut-off rate can differentiate between co-operators and non-co-operators in a relative sense at best. It is important to note that practical concerns- such as equal representation of individuals as co-operators and non-co-operators- may demand a median level for the cut-off rate, but theory can lead to a more justified cut-off rate. Hence, it becomes important to understand and differentiate between practical demands and theoretical justification behind choosing a cut-off rate, which can turn out to be quite controversial among researchers.
The self-report method used in the second study was able to capture the intensity of guilt feelings as a continuous measure. However, any indication of guilt was taken as a single category and the level of intensity of guilt was not considered for analysis. Moreover, for the bargaining task, all offer sizes were categorised as either generous or selfish, without consideration of the size of the offer and also the variations within the generous or selfish offers. Had generosity been taken as a continuous measure based on the changing offer sizes, rather than determined by a cut-off, a more fine-grained analysis could have been carried out using a regression model where offer size could have been regressed on the intensity of guilt feelings. A positive significant regression coefficient would have indicated that more intense guilt feelings lead to greater generous behaviour in social interactions.
The second study had a huge drawback in terms of its small sample size. They recruited only 36 pairs of participants to play the Ultimatum Game. In other words, there were 36 independent data points. Out of these, only 21 individual Proposers were classified as selfish. Out of 21 selfish Proposers, only 12 Proposers felt guilty to some extent. Therefore, only 9 Proposers did not feel guilty at all. Thus, the final comparison was between 12 and 9 individuals only, and the power of any analysis based on such small samples will be abysmally low and lead to weak conclusions.
To reiterate the point of central importance, when comparing two studies which are related to each other, it is important to remember that the number of parameters that vary across the two studies need to be minimal; ideally not more than one. If the goal of the authors was to find the change in the rate of cooperation by uncooperative participants post guilt manipulation/self-assessment, every parameter mentioned above should have been kept constant across the two studies, except for the method of guilt induction in the participants. Otherwise, any observed effects can become contaminated by confounding variables. The two studies clearly did not fit well with each other and thus, do not constitute as parts of a single research inquiry. Therefore, this work could be categorized as a pilot study at best. In order to be able to conclude their findings with greater confidence, a more in-depth experimentation is required where only one parameter is changed at a time.
(Disclaimer: Any views or opinions represented in this blog article are personal and belong solely to the author, and do not represent those of people, institutions or organizations that the author may be associated with in professional or personal capacity, unless explicitly stated.)
References
Ketelaar, T., & Tung Au, W. (2003). The effects of feelings of guilt on the behaviour of uncooperative individuals in repeated social bargaining games: An affect-as-information interpretation of the role of emotion in social interaction. Cognition and emotion, 17(3), 429-453. https://doi.org/10.1080/02699930143000662