Everything that is stated in scientific publications is always absolutely true. Although sometimes such an article is less readable because of all the careful references and solid evidence. Or as Paulien Cornelisse wrote last Thursday:[Ik] I was surprised at how boring and long winded I found it.
Or wait, Cornelisse was talking about BFG Written by Roald Dahl and not based on a scientific article. The first sentence of my column turns out to be incorrect upon closer examination. Let me start again.
Scientific research often builds on previous research – it’s the infamous “standing on the shoulders of giants.” You would expect scientists to be very careful when referring to the results of previous research.
Last week’s results a Stady Psychologist Cory Cobb and four colleagues examined whether scientific publications correctly referenced previous work. It included a total of 3,347 references to prior research in 89 scientific articles on psychology. The good news is that 2,718 of those citations accurately described the findings of the original study. The somewhat less good news is that 311 Signal was a bit less accurate. For example, findings of Chinese immigrants have been generalized to other Asian populations.
Even worse, 318 signals were completely wrong. This mainly relates to claims that were completely absent from the original study – as it did with Paulien Cornelisse’s column. Even worse, about a third of the false references claimed something completely contradictory to the cited article. For example, a study that found no evidence for a particular phenomenon is quoted as “This study shows that this phenomenon does occur.”
In short: approximately 10 percent of the references checked were incorrect in terms of content. This study was about psychology, but previous studies have shown that similar misery occurs in fields from medicine to marine biology and from earth sciences to education (although the scientific paper did not use the term “misery”).
Cobb and colleagues offer a number of tips for improving citations, particularly the importance of checking citations closely in the future. The question, of course, is who should do it. Scientific articles are reviewed by other researchers (often in their own time) and there are often dozens of references in a single publication. It is unrealistic to expect these reviewers to proofread all of these essential articles.
Perhaps – as so often – we should start at the other end: with the young researchers we train. This week I have reviewed two research proposals from students. I wrote several times in the margins of the phrase: “Do you have a reference for this?” And I suddenly realized that this question might prompt students to quickly find an article that seemed to more or less fit their claim. So from now on I’m going to put in the margin they should see if they can prove their claim. What does previous research say about this? Perhaps the correct conclusion is that their claim is wrong and they have to start over. Pauline Cornelis would call it “boring and tedious”.
About the author
Ionika Smits is Professor of Communication Sciences at Leiden University.
