What is Peer Review and why is it important?

We asked Emma Soneson, postdoctoral researcher at the University of Oxford, about Peer Reviewing.

What is Peer Review, and why is it important?

Peer review is a fundamental part of scientific publishing whereby individuals with relevant expertise who were not involved in a particular piece of research critically appraise others’ work to ensure that it meets standards for ethical conduct, quality, and rigour. In fact, peer review is such a fundamental part of the research process that it’s hard to imagine a world without it!

In the field of child and adolescent mental health, decisions about policy and practice rely on having high-quality evidence readily available. As peer reviewers, we help in this decision-making process by assessing the quality and implications of research. Without peer review, it would be far more difficult (and time-consuming) for clinicians, policymakers, educators, researchers, and members of the public to evaluate research in our field. Peer reviewers therefore play a critical role in helping appraise the evidence and ensure that it’s acceptable for publication and dissemination.

The theme of this year’s Peer Review Week is ‘Research Integrity: Creating and Supporting Trust in Research’. How do you think we can create and support trust in research through the peer review process?

If we’ve learned anything over the past few years, it’s that rigorously conducted and thoroughly peer-reviewed research is essential in promoting and protecting people’s health. Unfortunately, we’ve also seen how poor-quality research that has ‘slipped through the cracks’ in terms of peer review can have serious and wide-reaching consequences. When done properly, peer review can help build trust in science by serving as a ‘quality control’ measure that ensures that research – and especially the research that hits the front page of the newspapers and/or forms the basis of health policy and practice – has been conducted to a high standard. Rigorous peer review can lend credibility to research and ensure that decision-making is based on a strong foundation of high-quality evidence.

As an early career researcher, what influenced you to be a peer reviewer and what impact has it had on your own work?

As is likely the case with many early career researchers, I was initially nervous that I wouldn’t be a ‘good enough’ peer reviewer. After all, how could I, as a new researcher, judge the work of others who have been in this field for decades? However, I knew that participating in the peer review process was an important part of being a researcher and in ‘giving back’ to the scientific community, and so was eager to contribute.

Luckily, the first review I ever wrote was in partnership with my supervisor, who helped guide me through the process. When I sent back pages of line-by-line comments, he clarified that it’s more useful to focus on a few overarching issues rather than to worry about things like grammar and formatting. Having his guidance was immensely helpful for this first review (and I really hate to think how the authors would have reacted after receiving five pages of comments!) Since that first review, I’ve reviewed for nine journals and have become a lot more confident – and hopefully more useful! – as a reviewer.

While I’m glad to be contributing to improving the science behind child and adolescent mental health, I also gain a lot from peer review myself. Through critically assessing others’ work, I’ve learned what reviewers are looking for and can therefore make sure that my own papers address those points. I’ve also learned a lot about our field this way! Doing peer reviews helps make sure that I stay on top of the most recent research in my area, which helps me to develop my own research ideas and make sure that my work is impactful. Finally, peer review helps expose me to new methods that I can use in my own research. Just recently, I reviewed a systematic review that used a style of quantitative synthesis that I’d never seen before but that I will definitely consider using moving forward!

What is your approach when it comes to peer reviewing?

The first step for me is always determining whether I’m a suitable reviewer for the paper. Any time I get an invitation, I always read the abstract to make sure that I am confident enough in the content area and methods used to be able to write a useful, well-informed review.

If I feel it’s a good fit, I start by reading the entire paper through once without writing a single comment. This helps me get a sense of the bigger picture of the article and its narrative before focusing in on the detail (this is something that I’ve changed since I first started reviewing, and I highly recommend this strategy!) On the second readthrough, I start making notes to myself about potential issues and check the methods against the authors’ pre-registration, if available. In this iteration, I always try to comment on objective issues, rather than personal preferences (e.g. ‘This paragraph is technically correct, but I would have worded it differently’). It’s important to remember that different authors have different styles, and your job as a reviewer is to assess the merit of their work, not to say how you would have written the paper!

When I finally write up the review for the authors and editors, I always structure it in a way that assesses both the strengths and weaknesses of the paper, as from an author perspective it’s never nice to receive a page full of solely-critical comments! I put a lot of care into how I phrase all of my points and make sure to offer potential suggestions for improvements and solutions rather than just raising issues. My general motto is to never write a review that I wouldn’t be happy to read aloud to the authors!

Can you give us a few examples of what you think makes a good peer reviewer?

Having been on the receiving end of many peer reviews, I’ve learned a lot about what makes a good review! Regardless of the final recommendation (i.e. to accept/revise/reject), it’s always useful when a reviewer provides constructive criticism on how the paper can be improved. As I mentioned above, nothing is more disheartening than receiving a laundry list of negative comments. It’s so much more useful to receive suggestions on how to improve! For example, can the methods be clarified? Can the recommendations for practitioners be made more impactful? These are the types of suggestions that can help make the paper – and science as a whole – better. Relatedly, I find it useful to hear what was good about a paper. This helps me understand how to progress my research moving forward and to ensure that it’s useful and relevant to the field.

I also appreciate reviews that are specific. Again, regardless of the final recommendation you make, it’s always helpful to give the precise details and reasoning behind your decision. As an author, I’ve been on the receiving end of the ‘single sentence review’ (in my case, an unfavourable assessment of our paper), and I can say with complete certainty that this is not helpful to anyone! It doesn’t help the editors to make an informed decision about publication, it doesn’t help the authors to improve their paper, and it doesn’t help to move research forward. Authors spend a lot of time conducting research and writing it up for submission, and they deserve more than a single sentence in return!

Last but not least, a good review is one where the reviewer has acknowledged any limitations they may have. Are you an expert in a topic area, but not 100% sure about whether the statistical methods used were appropriate? Your review can still be incredibly valuable, but do make sure to acknowledge your uncertainties about the methods! This can help the editor to search for a reviewer with the specific expertise you’re lacking. None of us are experts in everything, and acknowledging our limitations is an important part of enabling a rigorous review process!

Why do you think early career researchers should get involved in peer reviewing, and how can they go about this?

There are so many benefits of peer reviewing, especially for early career researchers. Peer review is valuable for improving your own research through learning about how papers are assessed as well as keeping on top of your topic area and innovative methodology!

Given my own experiences, I’d always recommend doing your first peer review with supervision from someone who’s done them before. This can help make sure you’re on the right track with your comments and can boost your confidence if you’re feeling nervous about completing your first review. After this, you can sign yourself up as a reviewer for journals you like, so that they know they can contact you as a reviewer. By the time you’ve written a few papers as corresponding author, don’t be surprised if the journals contact you directly!

But, as with all things, it’s important to remember that it’s a balance. At this point, I receive an average of two peer review invitations a week. It’s important not to take on too many, as doing a good peer review takes time. Instead, I recommend focusing on one review at a time, to ensure that you have the time and energy to do a thorough job.