Thursday, December 24, 2015

Are todays experiments more unethical than Milgrams?

Everyone knows about Milgrams famous experiment. At least the patients I can hear from my office discussing it do. In case you don’t know it, here’s a short summary: 

Milgram has tested in the 60th of the last century how willing his participants (ordinary people without any special interest in harming others) were to follow orders from an authority (the experimenter) even if they had to put another person in danger for that. 

In his experiments the participant was given the role of a teacher whose task was to give electrical shocks to a learner whenever the learner did a mistake in a word-paring task. The learner was a conferderate and not really harmed by the electrical shocks. However the subject didn’t know this: he was given a trial-shock himself before the experiment in order to make the electrical-stimulator more believable and then placed in a separate room than the learner. 

In Milgrams first experiment 65% of participants continued until the final voltage of 450 V, even though they probably believed the learner was in danger(1, 2): The stimulator was labeled with “Danger: Serve Shock” at 375 Volt (and the four steps following that) and only with “XXX” at 435V and 450V. Furthermore the (supposed) learner pounded against the wall at 300V and 315V and is not heard afterwards, i.e. he doesn’t answer anymore. 

This experiment has been repeated several times by Milgram himself and others under slightly different conditions to find out which factors lead to obedience. However, in all the experiments a high proportion of participants “cooperated” until the final shock, despite experiencing high stress while doing so. (2)

Today, it is said, ethic committees would not allow this study anymore, precisely because of the high stress that was inflicted on the participants.

However, I wonder, what differentiates “modern” studies from that of Milgrams. Participants are still stressed and potentially harmed. I’m thinking about PTSD-studies where participants suffering from post-traumatic-stress-disorder are shown pictures related to their trauma, stress-studies where participants are stressed as much as possible in order to investigate the biological and psychological responses to stressors in healthy participants as well as in participants suffering from various disorders, pain-studies where the conditions which lead to more or less subjective pain or the pain-inhibitory system(s) is/are examined, conditioning-studies which involve learned helplessness and so on…

Now obviously the potential harm that is done to the participants is weighted against potential benefits of the study: the goal of such studies is to find mechanisms that make people more vulnerable to disorders or those that would potentially lead to the development of new treatments: after all, something has to be learned about disorders or suffering in general in order to understand and reduce it. 

But, I don’t know… 

Milgram explains (as one of 13 potential contributing factors for the obedience of his subjects) that “the experiment is, on the face of it, designed to attain a worthy purpose – advancement of knowledge about learning and memory. Obedience occurs not as an end in itself, but as an instrumental element in a situation that the subject construes as significant, and meaningful. He may not be able to see its full significance, but he may properly assume that the experimenter does.” (3)

All experiments should be designed to “[A]ttain a worth purpose – advancement of knowledge”, aren’t they? I think it would be bad if they weren’t (4)… but are they also significant in meaning?

I think, a study of which we don’t have any clue whether or not the reported results are (likely) true or not can’t be meaningful. (5)

Reproducibility in psychology is low (6) and most neuroscientific studies are underpowered (7). Ioannidis (8) famous paper shows that “most published research findings [in biomedical research] are false.” Negative findings remain often unpublished.

Yet, for all the p-hacked and unreproduceable studies a lot of resources were used (or wasted): Money that could have been spend otherwise, as well as time and effort of the participants and the people conducting, writing, publishing and reading the study. 

And maybe worse than that, some subjects suffer under the study – like the fake subjects (the learners) in Milgrams study did: they are given electrical shocks, shown awful pictures, brought into situations they fear or reminded on their worst times. 

Is this right? If there were true meaning, such experiments might be justified. The subjects sign a consent form and they know they are free to leave any time. But just like the real subjects (the teachers) in Milgrams studies, they usually don’t leave because they think they are helping science advance and develop new treatments. They don’t know about statistical problems, p-hacking and the pressure to publish anything out of a pile of underpowered noise. They don’t know the study they are participating in might be meaningless.

But I do. (9) I have tortured participants knowing the experiment is worthless. I will probably do similar things again. New study new hunt for statistical significances on the cost of participants. This is very false.

I don’t know what to do about it, I really don’t. 

Since not every study involves mental of physical pain for the subjects, I could concentrate on such studies or search for another job. But while that might allow me more sleep at night (likely not) it wouldn’t solve the problem (10). After all the studies are not stressful/painful/frightening for the participants because we want to torture them, but because it is seen as necessary for the “advancement of knowledge” about these states (11).

Therefore what remains is that studies should have sufficient power and be carefully designed to detect effects when present and to avoid unnecessary harm. Everybody agrees with that, yet it is not done.

As a PhD-student I'm not in the position to change that (and I don't know if anyone is). It is false to conduct worthless experiments, that (potentially) harm the participants and it is false to do nothing just to reduce own stress.

So what should I do?

 click on image to enlarge
______________________________________________

(1) In an interview after the experiment the subjects were asked what they think how painful the last shocks where for the learner on 14-point scale from “not painful at all” to “extremely painful” and the mean answer was 13.42. Milgram, S. (1963). Behavioral Study of Obedience. The Journal of Abnormal and Social Psychology, 67(4). [PDF-link] (see page 5 of pdf / page 375)

(2) Furthermore, according to Milgrams description many subjects were extremely nervous upon administering the high electrical shocks. They “sweat, tremble, stutter, bite their lips, groan, and dig their finger-nails into their flesh.” Milgram, S. (1963). Behavioral Study of Obedience. The Journal of Abnormal and Social Psychology, 67(4). [PDF-link] (see page 5 of pdf / page 375)


See also: 
Haslam SA, Reicher SD (2012) Contesting the “Nature” Of Conformity: What Milgram and Zimbardo's Studies Really Show. PLoS Biol 10(11): e1001426. doi:10.1371/journal.pbio.1001426 [PDF-link] (page 3)
“However, some of the most compelling evidence that participants' administration of shocks results from their identification with Milgram's scientific goals comes from what happened after the study had ended. In his debriefing, Milgram praised participants for their commitment to the advancement of science, especially as it had come at the cost of personal discomfort. This inoculated them against doubts concerning their own punitive actions, but it also it led them to support more of such actions in the future. “I am happy to have been of service,” one typical participant responded, “Continue your experiments by all means as long as good can come of them. In this crazy mixed up world of ours, every bit of goodness is needed” (S. Haslam, SD Reicher, K Millward, R MacDonald, unpublished data). […] what is shocking about Milgram's experiments is that rather than being distressed by their actions, participants could be led to construe them as “service” in the cause of “goodness.” […]”

(4) Which of course is possible: A goal of scientific experiments can also be to have something to publish or to “show” that one is right (even if that is not clear).

(5) The reasoning behind a study can still be meaningful of course. E.g. if a treatment were tested, that might be meaningful. But any study which tests it with getting a true result at or below chance-level isn’t imo.


(7) Button, K. et al. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14, 365-376. doi:10.1038/nrn3475  


(9) And others know too. I just don’t want to speak for other people, because I don’t know what they really think.

(10) That’s what horrible persons always say, right? But I don't know whats right.

(11) It would of course still be possible to invite participants that feel stressed at the moment to the lab when they experience that emotion. But obviously this has clear disadvantages since there were much more confounding variables then. Probably the disadvantages are so big that this would do even more harm, when it can be done otherwise. But I don’t know. In some/lots of instances this is of course the only possibility anyways. In others I don’t know if it would make any sense at all. That would then be a total waste as well.

Also interesting: 
Blass, T. (1999). The Milgram Paradigm After 35 Years: Some Things We Now Know About Obedience to Authority. Journal of Applied Social Psychology, 29(5), 955-978. [PDF-link]
Milgram, S. (1974). Obedience to authority: An experimental view. New York: Harper & Row. [PDF-link]

10 comments:

  1. Maybe you could think of what you want to achieve doing science. An interesting perspective may come from thinking about your phd time as a chance to perform the best research you can do, without looking at publishing in the most glamorous papers and all that stuff, and without assuming you'll do anything science-related after your phd time.

    What kind of research would you want to do then?

    Which studies would you like to perform then?

    I (like to) think there are more and more people and places for whom/ where the quality of research is worth more than the quantity, or where you published.

    I know that at the end of my life, i would only be truly content with some real, and useful, contributions to science, and not a cv filled with non-replicable, p-hacked, low-powered studies. If that means i won't get a science-related job after my phd, so be it. At least i can say to myself that i tried my best to contribute something useful. Perhaps that's naive, but that's the way it has to work or else it's not really worth it for me.

    Good luck!

    ReplyDelete
    Replies
    1. Thank you.

      I don’t know much useful to respond, because the consequence (of my reasoning) is that I should do something else than working in science. But I don’t want to do anything else!! (Nor am I sure that I could do anything else.)

      (I studied psychology/neuropsychology because of the experimental approach and the basis it has on data and scientific methods (as opposed to most of philosophy for example). But now I have doubts about the practical application of these methods. Maybe its better elsewhere, but I won’t find out if I don’t finish my PhD... which involves publishing two studies as first author*... which I don’t want to do, because I don’t want to draw any unjustifiable conclusions...)

      (*there're exceptions but not for me atm)

      Delete
  2. (1/2)
    If you really have tortured experimental study participants, you should turn yourself in to the police. If you've shown violent pictures to experimental participants after telling them "I am going to show you some violent pictures and I will stop doing that the very instant you tell me to" then you should instead turn yourself in to the hyperbole police. Sometimes, we need to be skeptical of the skeptics and this looks like one of those times. With support and empathy in my heart, let me try to talk you off the ledge here.

    "I’m thinking about PTSD-studies where participants suffering from post-traumatic-stress-disorder are shown pictures related to their trauma"
    -This is the exact method of the best-validated form of therapy for PTSD, which is why it's called "exposure therapy." This is the reason that simple participation in PTSD research has been shown to reduce PTSD symptom severity. I know, I know: such studies are probably just bullshit because research is more than half lies, right? Have you ever seen anybody get better from PTSD? I have. It happens. More than half the time they stick with it, it happens after exposure therapy. Which looks a lot like a symptom provocation study without the PET scanner. Both of which were undertaken with INFORMED CONSENT.

    "...stress-studies where participants are stressed as much as possible in order to investigate the biological and psychological responses to stressors"
    -OK, what do you mean by "as much as possible"? That's not even close to being true. Dunking a consenting participant's hand in ice water for 3 minutes to trigger an HPA-axis response is hardly making them "stressed as much as possible" - nor is asking a consenting participant to give a speech to stern-faced confederates. You're p-hacking to reach a significant level of angst here.

    "pain-studies where the conditions which lead to more or less subjective pain or the pain-inhibitory system(s)"
    -The level of pain in these studies, pain experienced by participants who gave informed consent, is pretty mild. Have you ever put it on yourself or signed up for such a study? Do it. It's usually heat pain in which the participant themselves chose the level. Consent, consent, consent. Consent is the key difference here. Informed, active, ongoing consent that can be withdrawn at any time.

    ReplyDelete
  3. (2/2)
    "Ioannidis famous paper shows that 'most published research findings [in biomedical research] are false.'"
    -No, his paper doesn't show that, it *asserts* that, which is a very different thing. That paper engages in at least as much stat-hacking, straw-men arguments, and spinning of data as the literature he's criticizing. He consistently just sort of brushes over the fact that most studies are, in some way, replication studies. Part of an ongoing research program that builds very slightly upon previous findings. That have been replicated by several different groups across several different institutions. Very few studies are purely exploratory or fishing expeditions. Professional skeptics have gotten a whole lot of mileage out of the (problematic) fact that the highest-impact papers (i.e., those subjected to highly publicized replication studies) *are* usually the highly novel experiments - in other words, those studies are almost never replication studies themselves. There's a reason that the big replication studies didn't show fearful faces to participants in an fMRI to see if their amygdalae really do activate. It replicates. Like, hundreds of times. It's strange that skeptics give such wide benefit of doubt to Ioannidis - if the same focused skepticism leveled at meditation were leveled at Ioannidis' assertions, I'd suspect that more than half of his published assertions were false.

    "And maybe worse than that, some subjects suffer under the study – like the fake subjects (the learners) in Milgrams study did: they are given electrical shocks, shown awful pictures, brought into situations they fear or reminded on their worst times. Is this right?"
    -If by "right" you mean "factually correct" then the answer is: no, what you said is not factually correct. The average level of shock applied to the fingertips of a consenting participant in a fear conditioning study is less than 3 milliamperes. Milgrim's confederates received 450 volts. Not even close. And no, it isn't a matter of degree any more than spilling water on someone's shirt is just a matter of degree less severe than waterboarding them. And of course, Milgrim's confederates repeatedly screamed for the study to stop. Again, if you've done anything even close to that then please turn yourself into the police. In fact, if you've ever continued an experiment after a consenting participant turned to you and calmly demurred, "I am rather bored and want to go home" then please turn yourself into the police. If not, then turn down the hyperbole and get a grip. Written, of course, with support and empathy in my heart.

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. This comment has been removed by the author.

      Delete
    3. This comment has been removed by the author.

      Delete
    4. This comment has been removed by the author.

      Delete
  4. This comment has been removed by the author.

    ReplyDelete