Open Science Collaboration Blog · Articles by Denny Borsboom, Han van der Maas, Eric-Jan Wagenmakers, Department of Psychological Methods, UvA

May 29, 2014

Questions and Answers about the Förster case

by Denny Borsboom, Han van der Maas, Eric-Jan Wagenmakers, Department of Psychological Methods, UvA

By now, everyone is probably familiar with the recent investigation of the work of Dr. Förster, in which the Landelijk Orgaan Wetenschappelijke Integriteit (LOWI) concluded that data reported in a paper by Dr. Förster had been manipulated. In his reaction to the newspaper article NRC Dr. Förster suggested that our department would be involved in a witch-hunt. This is incorrect.

However, we have noticed that there are many questions about both the nature of the case and the procedure followed. We have compiled the following list of questions and answers to explain what happened. If any other questions arise, feel free to email them to us so we can add them to this document.

Q: What was the basis of the allegations against Dr. Förster?
A: In every single one of 40 experiments, reported across three papers, the means of two experimental conditions (“local focus” and “global focus”) showed almost exactly opposite behavior with respect to the control condition. So whenever the local focus condition led to a one-point increase of the mean level of the dependent variable compared to the control condition, the global condition led almost exactly to a one-point decrease. Thus, the samples exhibit an unrealistic level of linearity.

Q: Couldn’t the effects actually be linear in reality?
A: Yes, that is unlikely but possible. However, in addition to the perfect linearity of the effects themselves, there is far too little variance in the means of the conditions, given the variance that is present within the conditions. In other words: the means across the conditions follow the linear pattern (much) too perfectly. To show this, the whistleblower’s complaint computed the probability of finding this level of linearity (or even more perfect linearity) in the samples researched, under the assumption that, in reality, the effect is linear in the population. That probability equals 1/508,000,000,000,000,000,000.