My girlfriend took me to task after I wrote my rant on peer review: “You have to have some assurance that the scientific literature is believable.” I agree. In my rant, I publicly publicly promised that what I produced would be believable, my reputation on it. But she persevered, and I eventually disgorged my core belief on this subject: no intervention, whether it be peer review or something else, at the level of publication, can produce credibility.
Shocking, yes? But consider: all a peer reviewer has to go on is what the author wants him to have. They both know the same background material, and both know that unknown factors can make seemingly identical experiments done by different people contradict. All a reviewer can do is ascertain that, if there was fraud committed, it was skillfully done. Our potential fraudsters are scientists, highly trained and intelligent. I believe them capable of producing a skillful fraud if they wish. Peer review spreads the responsibility editorial decisions for journals around on the theory that, with enough heads, someone will be able to decide well. It does no more.
How can we have credibility, then? Remove the social forces that reward fraud, and then punish the act of fraud with shame and ostracism. If there is no benefit and only risk, only a few pathological minds will bother. What are the social forces? Essentially, temporal rewards: promotion, tenure, prizes, funding, minions, and not being fired.
I want to digress on the firing of academic scientists. At this point in time, all the funding comes from the scientist. Universities provide at best partial salary support and some money to get set up, and thereafter demand rent from the scientist, in exchange for teaching and committee duties in perpetuity. The university has only prestige and facilities to offer in return. Facilities are cheap, so why does the university hold power in this relationship? It has to be the prestige, which is a fine inducement to fraud. The power universities hold beyond what any other landlord renting facilities would needs to end. As for the rest of it, promotion and prizes and minions, get rid of it all. Tenure, in the civilized world, should go. Civilized countries today have freedom of speech built into their legal foundations. The rights tenure originally guaranteed are now guaranteed to all citizens.
What does the resulting scientific establishment look like? It should be relatively easy to get enough funding for a single person to run a frugal lab for four or five years at a stretch, and horribly difficult to get more. There is no official recognition. There are no prizes to compete for, no plum positions or sinecures unless you convince private citizens to endow you luxuriously. There are no armies of graduate students being driven through programs to support the prestige of their professors.
Inevitably someone will protest that without these incentives, why try to do important science? I feel slightly ridiculous even addressing the point, but it must be done: I have trouble imagining the mind that gets up every morning and says to itself, “I think I’ll do something humdrum and unimportant today.” Perhaps they exist, and for those minds we can install basic checks. At the end of your funding period your work has to be reviewed for renewal. If you haven’t produced anything at all, not even interesting, intermediate results, then you’re on probation. If you don’t produce anything twice around, you’re not funded. The standards need not be that high. We’re not talking about a large amount of money, no more than $80,000 to $100,000 a year to pay for salary and all research expenses. There’s plenty of room at the bottom.
Scientists positioned like this, with no hope of official recognition, prestige, or promotion, have no reason to commit fraud. Add to that a strong discouragement—being barred for life from any government research funding if convicted of fraud—and you have a system where the literature will probably be credible. There will be crackpots, but that’s a whole different problem, and one that can be handled gracefully with minimal measures at the level of publishing.
I have spoken here only of individual scientists, not of large projects. What of major initiatives like CERN or USGS surveys? Projects of sufficient size have their own means of avoiding fraud. Papers coming from the experiments on CERN’s LHC are not peer reviewed by the journals. They have already been checked and rechecked internally, along with the data and the analyses themselves. I’m not worried about projects on this scale. They function just fine. The worry about credibility of the scientific literature is purely directed at the independent researcher working on a small project, and for that I can see no other solution than a reengineering of the scientific community to remove the inducements to fraud.