Sketchy science: open access is the solution, not the problem.

More and more science is becoming freely available to the public, or open access (OA). I love the movement towards OA, mainly because I like to be able to find and read papers online, even when I’m not on campus. I like being able to use the internet as my interconnected library of science articles, rather than having to store them as isolated PDFs on a hard drive. 

OA is clearly a good thing for readers, but there are dangers as well. The first problem is that, like all new and useful media technologies, scammers quickly find ways to exploit it. The first purchases by mail quickly led to new kinds of business fraud, credit cards led to identity theft, email led to all variety of strange messages from wealthy Nigerian investors, informational websites quickly led to ad-filled pseudo-informational sites. The authority of scientific organizations can also be hijacked. The Intergovernmental Panel on Climate Change (IPCC) led to the similarly-sounding Nongovernmental International Panel on Climate Change (NIPCC; i.e. the scientific source that Fox News can cite for their information on climate change). 

Similarly, the rise of open access publishing (sparked mostly by the Public Library of Science, PLoS) has led to all kinds of copycat predatory OA journals that charge authors large amounts of money to publish crap science. These unvetted science articles might be read by people who don’t know what journals are legit vs fake.

There’s also a second more subtle problem. Even for legitimate publishers that are not trying to scam anyone, the OA model changes the incentives for publishing and rejecting. If OA journals make more money by publishing more papers, then they have every incentive to accept papers rather than reject them. So they have every reason to have lower standards and publish flawed work.

Science Magazine performed a “sting operation” to reveal a world of corruption and predatory OA publishing. A science writer created a sham paper about a new cancer drug (extracted from lichen) littered with scientific, statistical, and even ethical problems. Then they sent it to about 300 open-access online journals. Many of these targets were already listed on published lists of predatory journals. The project verified what everyone already knew: there are a lot of scammers and crappy peer-review systems out there in the open access publishing world. Of the targeted journals, 157 took the bait and published the crap article, with 60% not even doing peer review (these are the obvious scammers). Of the 106 journals that did conduct peer review, 70% still accepted the paper (these are a mix of the truly dishonest and the self-deceived).

Take home message: you shouldn’t believe everything you read on the internet, even if it comes from a journal with a scientificky-sounding name.

But here’s the problem I see. Let’s talk about incentives. Traditional publishers are losing a lot of money to open-access in all its forms, not to mention it just makes them look bad for charging readers money. The open access movement is to traditional publishers, as climate change activism is to gas and oil companies. At best, it’s something the industry has to adapt to; at worst, it’s a threat. Unsurprisingly then, traditional publishing magazines like Nature and Science usually seem to discuss open access as something potentially full of peril, both to readers and scientists. The message is often “Watch out readers, OA journals publish bad science” and “Watch out scientists, OA journals will take your money and exploit you.” These are both valid concerns, but the issue of predatory science journals is not the same as the issue of open access, and it’s dishonest to draw a simple connection between them. There’s a lot of bait-and-switch going on here: “predatory” becomes “open access”.

Unfortunately, some readers will get the impression that there’s something unreliable or corrupt about the notion of *open-access*. Here are some example headlines:

A “sting” operation found that open-access journals will accept anything — for a price [Salon]

Open-Access Journals Hit By Journalist’s Sting [NPR]

The Wild West world of open-access journals [LA Times]

Hundreds of open access journals accept fake science paper [The Guardian]

First of all, “open access” does not = crappy scam online journals. If I take any paper and post it on my website so that anyone can read it, I’ve just made it “open access”. Some of the best journals like PLoS Biology and PLoS Genetics are open-access journals. PLoS ONE (the largest OA journal in the world by a large margin) pointed out both the scientific and ethical issues in the hoax paper and rejected it in under 2 weeks (that’s fast).

Second, I think it’s a bit hypocritical for mainstream journalists to talk about the lack of rigorous peer-review and reliability of science journals. Yes, predatory journals are absolutely terrible. Why? Because at worst they have the same terrible standards of accuracy as the mainstream media. They’ll publish almost anything without peer review, regardless of the quality of the evidence, just because someone out there reports it. But yeah, that’s another issue.

The most glaring problem is that this “experiment” did not compare open access with non-OA journals. It only targeted OA journals, so it cannot possibly show any effect of being open-access on peer-review quality. We may all agree that OA publishing is where the problem is probably worst, but this study doesn’t even show that. What would have happened if the sham paper was sent to a bunch of small journals that were not OA? What would be the acceptance rate? My guess is that it would be much much lower but still shocking in how many times it would get through peer-review. I once accidentally took a large knife in my knapsack on 2 planes through 3 international airports (post 9-11). No screening procedure is perfect.

Peer-review is a system that ensures that at least 2 other people in your field think your paper is not fundamentally flawed. It doesn’t ensure quality; it just helps. As a reviewer, I’ve seen the other reviewer completely miss giant flaws in a paper, and I’ve also seen the other reviewer catch obvious flaws that I didn’t notice (shame on me). You also read papers that make you think, “How did this get through peer review?”

Ultimately, as a consumer of science, you can’t keep outsourcing skepticism and judgement. If you always have others do the assessments, then who will assess those assessors when there’s a reason for them to be biased? Sometimes you just have to look at the evidence directly, and make up your own mind without resorting to the authority of the journal brand or website or author. That’s why all science papers have methods sections. There’s only one real solution: every individual should develop a healthy skepticism and an intellectual self-defense, rather than a reliance on authority– scientific or otherwise. Crap science published in some sketchy journal should not fool anyone. And if it does, why should “open access” receive any of the blame?

If someone makes a claim, like say, cell phones cause brain tumors (cite: Smith 2013 Journal of Brain Tumors), I personally think the most important service of publishing is that I can go and read that Smith 2013 paper myself to see if I actually agree with the evidence. That’s what open access allows and facilitates.

The problem of bad information does not stem from open access. Flawed science gets published all the time, even when good peer-review is in place, and even in prestigious journals such as Nature and Science. The fact that there’s more bullshit in open-access journals is predictable given the incentive structure. But there are many other examples of poor incentive structures in publishing. For instance, Nature and Science benefit when they publish overly-ambitious or very controversial ideas, because these will attract more citations (even if the majority are citing the paper to say it’s wrong); the effect for the journal is a higher impact factor. Another problem that nobody wants to publish negative results. Another is that scientists can be rewarded by making their papers so technical and hard to read, that peer-review is difficult or only possible by a small group of close associates. Yet another problem is that drug companies pay scientists to publish results they like (e.g. 21% of medical papers published in 6 leading medical journals in 2008 were likely written by honorary authors or ghost writers).

To summarize the larger problem– there’s just a lot of bullshit. In science. In academia. On the internet. In journalism. In blogposts, like this one. Maybe in this very paragraph. Indeed, there are entire fields or subfields of academia that are built entirely out of bullshit. That’s what the Sokal Affair attempted to demonstrated with their original sting operation. 

An important difference between a paper published in some small subpar open access journal and some small subpar traditional journal, is that open access allows anyone to read the paper and decide for themselves. Predatory journals will eventually gain bad reputations and won’t do well. That’s why all of them are tiny and have names you’ve never heard of. But there are also new, good progressive OA  journals that are just as honest and reliable as any traditional journal. And many new OA journals are doing cool and interesting things. One example is PLoS ONE and another is PeerJ. These are not the end of scientific integrity, I personally think they are the future.

I do think this study is terrific because it will spark a lively discussion. All scientists need to discuss this issue more, and also be more aware of predatory science publishing. But I hope that people stop talking about this finding as if it says something directly about the “dangers” of open access.

People talk about the dangers of the internet, blogging, and social media. Ok, great, interesting. But when people who work in publishing (and make money by charging access to science papers) want to talk about the danger of… open access to scientific information, you have to wonder…