Can we use empathy to bridge the gap between authors and peer reviewers? Erin Owens shows us how — R Voice

Can we use empathy to bridge the gap between authors and peer reviewers? Erin Owens shows us how

Jayashree Rajagopalan
Jayashree Rajagopalan Member, Administrator, Moderator Posts: 319 admin
edited October 2021 in Publication Support

Two terms – Empathy and Peer Review. These were the highlight of the webinar by @Erin Owens on day 2 of Peer Review Week 2021. If you attended Erin’s talk, you may recall how she used her personal experiences as an author, peer reviewer, and journal editor to explain how a little empathy can go a long way in making peer review smoother less stressful for authors and reviewers. Erin was flooded with questions during the session and we couldn’t address all of them. So Erin took the time to look at some of these questions and share her thoughts on them. And if you missed her session, here’s your chance to tune in to the webinar recording for some timeless tips on dealing with peer review as an author AND reviewer.  

How can I trust my reviewer?

At least initially we should assume that the editor has done their due diligence in selecting a reviewer who has expertise in the proper areas. But if we're reading their comments and starting to feel like “I don't think they interpreted my paper right,” then maybe they were not the right reviewer and we can't trust their feedback fully. That's a good moment, then, to go back to the editor and to express that concern, share some examples of reviewer comments that don't feel right and give the editor that opportunity to work with you. If they agree that there's an issue, they can go and find another reviewer. So I like to start out assuming the best. But I also remain aware if there's a moment where I need to ask questions. 

Our reviewers are supposed to correct grammar mistakes?

On some level it may depend on the journal they're reviewing for and what their guidelines include. Most of the journals I've worked with have dedicated copy editors who will perform detailed corrections of grammar and punctuation on accepted papers. So in general we ask reviewers not to worry about trying to correct all of the grammar. However, while I am doing a peer review, if I notice a problem reoccurring throughout the paper, I may highlight one or two examples and leave one general comment that says, “There's a problem with the subject and verb agreement that needs to be looked at through the paper.” I will mark every instance that needs to be corrected, but I may just identify an overall theme. As a reviewer, we really want time to be dedicated to big issues: Is there something wrong with the methodology? Is there something wrong with the logic used in drawing conclusions? 

As a reviewer, I always wanted to give authors a second chance to or, even though the papers were not that good. As a reviewer where you draw the line between being lenient and being really strict?

The first thing I would probably mention, as a journal editor, is that if I see a paper that is clearly not within the scope of our journal, or nowhere close to publishable, I'm probably going to decline that paper before it would go to peer review. If I have decided to send the paper out for review, even if it's not great yet, it usually means that I've seen something in there that I think is valuable. There's some core to it that could be really informative and unique, and I just feel like it needs strengthening. So if you keep that in mind as a reviewer – that a paper came to you because the editor saw something there – then I think you can feel comfortable with being lenient. Go ahead and recommend a “revise and resubmit,” and give the authors that opportunity to do it again to make it better. If the editor disagrees with you, they'll make their own decision. 

How can an author really understand if a reviewer is being subjective?

That can be really hard. Sometimes it feels like a reviewer is giving us a comment that's just their preference. But is maybe not really a necessary change. In such situations, I try to read it through once and feel grumpy that they're being subjective. Then I step back from it and come back and read it again. I try to take myself out of it as much as possible. I try to pretend that I'm reading someone else is paper and comments and look for any value to this comment. I also compare the two versions. When I do this, I sometimes end up feeling like that comment is subjective and is not essential and may not make that change. And when I submit my revisions to the editor, I'll include a comment there that explains exactly why I chose not to make that change. 

Do you think reviewers can be biased about the country of origin of the authors?

I definitely think it happens it. Obviously, we want to say it shouldn't, but I think the reality is that there is sometimes an unconscious bias - even if it is double anonymous and reviewers cannot see the name, country of origin of an author. If they're reading a paper where the English writing style is different from the way that they express themselves, and even if reviewers don’t mean to, they may unconsciously react to that and say, well, this this doesn't feel like the way that I express my ideas. It sounds strange to me. I think another possible piece, from my experience at the journal that I edit for, is that often the papers we get from smaller, more developing countries are often focused on local problems in that country or community. I think this is great and very important. But sometimes when our reviewers read it, they may have a certain bias towards this and feel that this doesn't seem big enough or universally applicable enough. Yet another piece of the education as a reviewer is learning – to realize that those kinds of local focused things can still be very informative. Reviewers can help authors identify such areas and ask them to expand on certain points or explain how this could be generalized to other environments, but without discouraging the local focus. 

How can I reject my reviewer in a polite way if I want to stick to my original plan or my additional idea and I don't really agree with the reviewers?

I can definitely relate to that. There are some great resources out there with suggested phrasings. My focus would just be on being as respectful as possible, letting them know that I appreciate their opinion, and then trying to provide a clear explanation for why I've chosen not to change it because it benefits the overall paper. 

I have a manuscript that has been rejected several times. I've now almost given up. Is it OK to give up publishing a manuscript because it has been rejected over and over again, even when I still feel it contains a great idea?

That is a really tough one. I do a lot of workshops with graduate students that are trying to break into publishing their research, and this is one of the things we talk about that if you get a paper declined once that doesn't mean it's bad. You may want to try another venue. Going through multiple rounds of peer review really does improve a paper over time, so you can think of those declining rounds as constructive. But at the same time, I think it eventually reaches a point where you just have to make a decision and ask yourself, “How emotionally invested am I in publishing this particular paper? How much is it emotionally damaging me to continue to see it rejected?” And while I encourage people to try again, I also encourage people not to feel bad if you eventually reach a point where you say, “I'm just ready to put this one down and focus on something else,” because it does take a lot of emotional energy to keep working with that same piece over and over again. So I don't think there's really a right answer as to how many times you should submit it, but trust your instinct for when you're ready to move on. 

When I submitted my manuscript to one journal or one reviewer voted my paper 90% and another 34% and the paper was declined. What should the editor-in-chief have done?

I will respond to this question as an editor. It is frustrating when one reviewer says it's fabulous and another reviewer says it's terrible. If as an editor, I have two reviewers that are really that different in their opinions, I will usually opt to get a third review. This a challenging decision because it's going to make the process take longer. I hate to make the authors wait even longer in suspense for a decision. But if I'm really looking at their comments and feel like I just can't reconcile those differences of opinion, then getting a third review can help me to get more information and make me feel a little more confident in making my decision. But the other piece of it too is that the editor is also a reviewer in a sense. As an editor, I can look at both reviews objectively and decide where I agree or disagree with the reviews that have been submitted or whether I can get a third reviewer. So at some point the buck stops with me (the editor) and I have to make up my mind and be accountable for that decision. 

Coming back to the theme of your talk today - the humans behind peer review. Peer reviewers are human beings too, like you said. So I have two questions for you. How long do you spend reviewing the paper – how do you know when to stop? And, do you feel bad when you have to reject the paper?

Very good question! The amount of time I spend will vary, depending on how long the paper is. But I usually try to read it at least twice. I'll read it through once without pens in my hands, to get a sense of the overall context and purpose, and then I will read it again and actually go through making my comments and my questions. So it's a couple of hours per paper, depending on how many pages it is. But it's definitely several hours. And that could be tricky in terms of recognizing that we have a lot of responsibilities. I do have to kind of control my urge to go through line by line and correct every single thing. I have to limit the amount of effort that I could put into it, but I want to make sure that authors get a fair reading from me. I don't want to just zip through it and be done in 30 minutes and then feel like maybe I didn't really give it enough thought, so I'm very comfortable with the idea of spending three hours on a paper.  


The second part of the question was whether I feel bad if I reject something. I do! It doesn't feel good, because I know that that author put a lot of work into that. I know that they put their heart out there on those pages. And that's why I said earlier - if I can find that redeeming value in the core, I'll recommend a “revise and resubmit” instead of a “decline,” and really give them that opportunity to try and achieve the full potential of their work. 

The same paper was refused due to reviewer’s opinion in one journal, i did not changed it and send it to another journal

This can definitely happen! We should remember that (a) different reviewers bring different perspectives, and (b) different journal have different missions and give their reviewers different criteria. So a paper might have been judged not a good fit for one journal and yet be accepted by another journal even without revisions. If you experienced that, it's probably a sign that the second journal was a better fit for your paper. 

Is there a standard criteria for a reviewer to follow?

Great question! Each journal will usually provide its reviewers with guidelines regarding what the journal is looking for and what they want the reviewers to focus on. Authors can often find the reviewer guidelines on the journal's website and get a sense of what the reviewers are asked. But additionally, there are "best practices" and standards that often apply. I link to several of those in this Scholarly Peer Review Guide, in the box titled "Reviewing Others: Standards and Best Practices"  

I am also interested about the academic culture that you've mentioned. Are the peers in your department expected to conduct the review similar with the external peer reviewers?

I can only speak for my own department. We don't set expectations that anyone "must" review their colleagues' work, and we don't necessarily ask that they follow any formal guidelines as an external reviewer would. It's more of a reciprocal courtesy among us: if you take a little time to read over this paper for me, I'll read over one for you as well. They just act as a fresh pair of eyes before submission to catch errors or unclear explanations that we would prefer to have polished out before peer review. 

What is the meaning of "The manuscript does not reach the high priority to be sent for peer review”?

I don't remember the context of this statement, but I assume I must have been talking about some papers that get rejected by the editor without being sent to peer review (we often call this a "desk reject"). This often happens with papers that are out of scope for our journal, and we will just send a letter indicating that it does not fit with what we publish. Sometimes I will suggest titles of others journals that might be a better fit, especially if it is within my discipline and just not my specific focus. (However, in some cases we do get submissions that are totally outside our discipline, and I simply tell them that; I don't necessarily try to go research journal options in the other field.) Occasionally we will also give a desk rejection for a paper that simply doesn't meet our standards; for example, the journal I edit for looks for rigorous methodology. If a particular paper has very weak methods, even if the topic is technically in scope, we may reject it for that reason, since methods are not easily revised! So there might be several reasons that we would decide a manuscript does not meet our standards to go out for peer review and we simply send a decision letter right away. 

What are the criteria that reviewers should follow?

Each journal will usually provide its reviewers with guidelines regarding what the journal is looking for and what they want the reviewers to focus on. Authors can often find the reviewer guidelines on the journal's website and get a sense of what the reviewers are asked. But additionally, there are "best practices" and standards that often apply. I link to several of those in this Scholarly Peer Review Guide, in the box titled "Reviewing Others: Standards and Best Practices."  

What should the author do with the editor’s decision to “Reject” but with NO comments?

This is tough; I'm sorry that you experienced this. In a perfect world, an editor should always have SOMETHING to say to explain a rejection. They might say its out of scope for their journal, they might say that it needs too much work to be publishable, but they should say SOMETHING. You can try asking them for feedback if it was not initially provided; perhaps say something like, "I appreciate your time in reviewing my submission. I wonder if you could give me some feedback to help me improve the work? 

How to respond to and address conflicting reviewer comments?

It can feel really frustrating to receive conflicting reviewer comments. You can try weighing them and seeing if you agree more with one reviewer than the other. You can try reaching out to the editor and asking them to reconcile the conflict: Which way would they prefer you go? Whichever way you go, be sure to address it clearly in your response to the reviewers. Point out the way in which the two comments conflicted with each other so that you couldn't please both reviewers, then explain which advice you decided to take and what change you made as a result. Just try to remain polite in your explanation. Some tips for responding to reviewers are available here.  

I submitted my manuscript in 2019. I got my first comment in December 2019 and responded in January 2020. This is more than one year, almost to the second when I sent my review but no response yet. What should I do?

That is a long time to wait without any communication on the paper's status. I would definitely encourage you to reach out to the editor. Be respectful, but curious. You can always phrase the question along the lines of, "I wanted to inquire about the current status of my paper (revisions submitted on #date#). Do you need any further information or action from me to help this move forward?" Using a question which implies that maybe YOU need to do something to help can feel less accusing, and if you're lucky it will help them realize that THEY need to do something. I will say that I have experienced some poor communication from editors in the past, and it turned out there was an explanation, such as dealing with serious medical problems. So if you don't get a response from the editor, you may want to find someone else at the journal to reach out to. You could check their website for a list of the Editorial Board members and reach out politely to one to say, for example, that you have not received responses from the editor are concerned about their well-being. Eventually you may decide to withdraw the submission and send it somewhere else; that is your right, but just be sure that you clearly communicate that the submission is being withdrawn. 

What is the best advice you can give to newbie in reviewing papers and mentoring researchers?

My first recommendation is to read the journal's reviewer guidelines and follow them. Even if you have reviewed for other journals, this one may take a different approach. They may want you to focus on different aspects of the paper. Be sure that you are considering the work the way they want you to, and not in some other way that you are bringing with you. My second recommendation would be, as you write comments, imagine that you are the author instead of the reviewer, and speak as you would want to be spoken to. The best way to mentor researchers with your reviews is with empathy, not "tough love." 

Why do some reviewers ask authors to cite and reference their works?

Oh, this is such a good question, and a really tricky matter. In ideal circumstances, our papers are being reviewed by others with expertise in the subject, meaning they are often publishing their own research in related areas. In some cases, a request to add a citation to recent, related work may just be a legitimate suggestion for strengthening our coverage of existing literature. But in other cases, it may be what we call "coercive"--a reviewer or editor pushing you to cite their article, or their journal, in order to benefit them. (Read more about coercive citation practices here.) My best advice is to go and thoroughly investigate the work they are asking you to reference. Does it actually have significance for your work? Would it make your discussion stronger? If so, go ahead and cite it for its own value and ignore its authorship. But if you cannot objectively agree that it makes your work stronger, then don't let yourself be pressured to add an irrelevant citation. If they would refuse to publish your work without that citation, you would be better off working with another journal anyway. 

After being asked to do minor corrections for the paper and submitting the required corrections, does the paper go for reviewing again or the decision of acceptance is taken by the editor?

Great question! This actually varies. In the journal that I edit for, we have two options when we request revisions: Either revisions with another round of review, or revisions without another round of review. This gives us more choices depending on the strength of the paper. If the revisions are minor and I believe it is otherwise ready to be published, I will not require another round of review. If the revisions are more substantial, I may require another round of review so that the reviewers can help me judge whether the goals of the revisions have been achieved. I often try to get the same reviewers to read the revised version, so that they can judge how well the changes have answered their original concerns. With my editorial decisions, I try to be very clear about stating whether or not the revised paper will have to undergo an additional round of review. Unfortunately I realize that some editors are not as transparent about that. 

What happens to the papers that are submitted and no response or feedback is not given? It’s so frustrating!

I would definitely encourage you to reach out to the editor. Be respectful, but curious. You can always phrase the question along the lines of, "I wanted to inquire about the current status of my paper. Do you need any further information or action from me to help this move forward?" Using a question which implies that maybe YOU need to do something to help can feel less accusing, and if you're lucky it will help them realize that THEY need to do something. I will say that I have experienced some poor communication from editors in the past, and it turned out there was an explanation, such as dealing with serious medical problems. So if you don't get a response from the editor, you may want to find someone else at the journal to reach out to. You could check their website for a list of the Editorial Board members and reach out politely to one to say, for example, that you have not received responses from the editor are are concerned about their well-being. Eventually you may decide to withdraw the submission and send it somewhere else; that is your right, but just be sure that you clearly communicate that the submission is being withdrawn. 

As a reviewer, what is your recommendation for other reviewers?

My first recommendation to other reviewers is to read the journal's reviewer guidelines and follow them. Even if you have reviewed for other journals, this one may take a different approach. They may want you to focus on different aspects of the paper. Be sure that you are considering the work the way they want you to, and not in some other way that you are bringing with you. My second recommendation would be, as you write comments, imagine that you are the author instead of the reviewer, and speak as you would want to be spoken to. 

If a reviewer is not grounded in a particular area how can the author ascertain the objectivity of the feedback given?

This is tough! In a perfect world, the editor would always invite a reviewer who is well grounded in the subject. But finding sufficient numbers of reviewers can be tough. And sometimes a reviewer has a strong foundation in one aspect of a paper and not another--for instance, maybe they understand the subject matter but not the specific methodology used. The best reviewers will admit their lack of sufficient knowledge on certain points, but of course that doesn't always happen. Sometimes we just have to trust our gut as authors. Try to read and consider the feedback as objectively as possible, but if you cannot reconcile it with your work, if it is clearly illogical or confused, then don't make the related revisions. Just try to be polite in the response to the reviewer: obviously we don't want to say, "The reviewer doesn't know what he's talking about." Instead we want to express our gratitude for the reviewer's time and feedback, but gently explain why the feedback doesn't make sense in this case and why we are not changing the content. Unfortunately there may be other cases that are less clear: the reviewer isn't obviously wrong, but you're just not sure she's right either. That is the hardest. I recommend taking the paper and the feedback to a trusted mentor and asking them to talk through it with you.  

Authors who split their data and publish in multiple journals. Should the reviewers be concerned about it?

This can be difficult. Depending on the nature of the data collected in a study, there might honestly be too much for just one article, so it might be appropriate to discuss findings in multiple papers. And if those papers have different angles, they might fit more appropriately in different journals. But at the same time, we want to be cautious about cases where people are really just stretching limited data too thin to try to "game" the publication system and get more articles and more citations. If the reviewer is aware of multiple works being produced, they should use their judgement to decide whether those works are different enough and substantive enough to warrant multiple papers. If they do not believe that is the case, and it seems like the authors are just trying to increase their paper count, the reviewer should express that concern to the editor in their review. In most modern review systems, the reviewers has choices to enter comments for the author to see or comments for only the editor to see, so they could include that concern in a private comment to the editor. Or they could just express it as a concern directly to the author: maybe they request particular revisions, for instance to the Introduction or Objectives, where the author addresses the similar work and justifies the differences in the two papers.