Archive for July, 2012

h1

another one bites the dust…

July 31, 2012

It seems that the past year has been a rough one for psychological science – several researchers have had their work called into question based on evidence suggesting that they had falsified data. This is a tricky issue. For science to work, the community of scientists has to trust that everyone is on the up-and-up. Honesty in reporting data is paramount. So, it is critical that individuals that are not living up to this standard are removed from the community. At the same time we are looking out for these nefarious researchers, we have to trust that everyone else is on being honest. So, there is obviously tension – I must trust, but I also must be skeptical…

The tricky part is how do we know? Well, we can feel a bit more confident because individuals like Uri Simonsohn are on the case. Here is an interview with him about how he detected fraudulent data in one high profile case this past year. This morning I saw that there was a retraction notice in the recent issue of Psychological Science, and following the story a bit found out that Simonsohn had identified another batch of suspect data. I found this situation particularly interesting because of the simplicity of his approach. Lawrence Sanna, a professor of psychology at the University of Michigan, had published a study purporting that the level of altruism an individual felt was related to his or her actual level of elevation – higher elevation = higher level of atruism. The method was interesting – it involved escalators and hot sauce, but that is a story for another post… Anyhow, Simonsohn noticed that although the means for the different conditions were very different, the standard deviations across the conditions were nearly identical. Simply put, different groups of individuals had been tested at different levels of elevation – the participants at higher elevations on average had much higher levels of altruism. The problem in the data was that although these means differed significantly, the range and distribution of responding within each group of participants was essentially identical. Simonsohn correctly noted that this is highly unlikely, so he located some other studies using similar methods and found that not only were the differences between groups far smaller, the variability within each group also differed more appropriately. Several other papers written by Sanna showed a similar lack of variability in the reported variability across the conditions, so Simonsohn contacted Sanna and some of his co-authors with his concerns during the fall of 2011. As of June 2012, Sanna has resigned from U of Michigan, retracted at least four papers, and now maintains his silence (under legal counsel).

All of this occurred because of some lowly, little standard deviations. By the way – you should use this as a prompt to review just what a standard deviation is and why your friendly research methods instructor would spend so much time making sure you understood that statistic…