Peer-review is essential to keep science on track. But it can be too demanding on scientists, who are asked to do it for free and anonymously. How many papers are fair to review or to decline?

Sometime ago, I asked a top-notch scientist how often he accepted papers to Review. His answer was curious. He told me: “I publish thirty to forty papers a year. Each one takes two or three reviewers, making about 100 reports. I feel that I have to give this same quantity of reports back to keep the system working. Therefore, I review about 100 papers a year.”

I was a bit surprised. First by the reasoning, which was new to me; second by the number. I asked him how could he find time to review 100 reviews per year, roughly two papers per week. And I was again surprised by his answer: “I don’t really do it myself. I ask my students to review them.”

I think he read the ethical concerns on my face, as he continued somewhat defensively: “I think that this is a good exercise for my students. They learn how to critically read a paper, survey references, write a technical report. Of course, I check each review before submitting it.”

I don’t know numbers for that, but I don’t think that outsourcing a review to students is rare. Many of my colleagues told me that, as students, they were asked by their advisors to review papers. The practice is even captured in this PhD Comics cartoon.

When I asked that professor about his “review policy,” I was trying to get some insight to myself. I usually get two or three review requests per week. Naturally, I have to decline most of them, otherwise I wouldn’t do anything but peer-reviewing, which would ironically grant me to be excluded from the “peer” club in no time.

I basically only accept to review papers that are of immediate interest for me. It may sound opportunistic (and it really is), but seeing what other people in the field are doing ahead publication gives me some competitive advantage.

On the other hand, I don’t ever ask students and co-authors to do the job for me. What I often do is to write back to the editor declining the invitation and suggesting that the review request is sent instead to one of my co-authors, who I think is qualified to do the job. Maybe, in few occasions, I also suggested that some especially boring and troublesome papers were sent to people who I didn’t have in good esteem. (Don’t look at me like that. Nobody can be nice 100% of time…)

To transfer the review job to students is against the rules of most journals. Normally, they explicitly state that the person invited to review must not show or discuss the paper with anyone else, and if an additional opinion is required, the editor should be informed.

Even more serious, to transfer the review to students most probably lowers the review quality. I’ve received many reports in the past, filled of primary statements, which clearly denounced that someone with little experience wrote them.

Especially concerning postdocs, who are usually qualified to review a paper, it’s much better to suggest them as alternative reviewers than to ask them to informally do the job. This will help them to improve their CVs and also help editors to be aware of new people coming into the field.

In any case, I liked that professor’s rational that to keep the system working we should give back the same amount as requested for our own papers, although I don’t agree that the number of reports is the measure of this work.

Then, how many reports should I give to break things even?

Suppose I submit a paper and this paper gets, say, two reports before final decision (no matter whether is was accepted or rejected). Suppose yet that I co-author this paper with two other colleagues holding a doctorate (and are eligible as reviewers). Therefore, we three owe the system two reports. And I alone owe 2/3 of a report for that paper alone, where 2/3 is obviously the ratio between the number of reports and of PhD authors.

Then, to know the number of papers I should review in a certain year, I should compute the sum over the reports-number/PhD-authors ratio for each of my papers submitted in the previous year. For me in 2014, for instance, it makes ten papers.

But this is too complicated. I don’t want to keep tabs on these ratios. I want a simple rule of thumb.

I think it is fair to assume that the mean reports-number/PhD-authors ratio is approximately one. Therefore, to break things even and keep system running, I should write one report for each paper I submit.

Putting everything together, I’ve been sticking to the following rules:

1. I will not ask any other person to do the review for me.

2. If I should decline an editor’s invitation, I may suggest one of my young co-workers holding at a doctor degree as an alternative reviewer.

3. For each paper I submit, no matter whether it is accepted or not, I should review one paper.

In principle, these rules sound fair, but it still a lot of work to cope with them. It takes me about five hours to review a paper. Then, I should dedicate a total of more than one work-week per year just reviewing papers. (And there is still the question whether it is fair to transfer these costs to my university, which is paying me for these hours.)

As for the last rule, still with my previous example, I submitted (and published) ten peer-reviewed papers in 2014, therefore, I should have reviewed ten papers in 2015. I just checked my records, and ,in fact, I reviewed fourteen papers. This means I overworked… and I hate overworking. Probably I should compensate for that now in 2016.

MB

• I published the first version of this post about one year ago, but there was some wrong and complicated reasoning there. This is a much clear full revision.
• Take also a look at Sophie C. Lewis’s post on this topic.
• The painting opening this post is the Nicolaes Maes’ An Old Woman Dozing over a Book, 1655.