Pros and cons of different models of peer review

/0 Comments/in /by

Peer Review Week is an annual international event celebrating the essential role peer review plays in maintaining scientific quality.

Veterinary Evidence‘s Editorial and Production Assistant, Jennifer Morris, explains some of the pros and cons of different models of peer review.

You can also read her blog on the importance of peer review.

There are different types of peer review, normally separated into three categories:

Double-blind – the author does not know who the reviewers are and the reviewers do not know who the author is.

Single-blind – the reviewers know who the author is but the author does not know who the reviewers are.

Open – the author knows who the reviewers are and the reviewers know who the author is.

Since its inception three years ago, Veterinary Evidence (VE) has operated under the open peer review model, and we have published the reviewers’ names alongside the published paper. The reasoning behind this is that it offers a clear and transparent critique of papers. It also makes conflicts of interest easier to ascertain, allowing people to be open and honest regarding their disclosures. There are other arguments in favour of open peer review, for example Publons suggests that movement from blind to open review is a “significant culture shift” that makes peer review an “activity” the reviewer can be openly recognised for, incentivising the act of peer review itself. It also suggests that “open peer review is an important element of open science, as it leads to increased transparency and accountability in the research process”.

Although at VE we do not publish the reviewer comments alongside the published paper, it can also be argued that this method imposes a kind of quality control on the type of reviews submitted. If a reviewer knows their comments will be published, they are more likely to conduct a thorough and helpful review of the paper. However, there are also some downsides to open peer review, such as reviewers declining to review because they do not want to be identified as the ‘source of a negative review’; or Early Career Researchers being reluctant to openly criticise senior researchers’ work, especially when an area of scientific research has such a small community of researchers.

Even though, according to Publons author Jo Wilkinson, the open model of peer review is being adopted across more journals, most reviewers still prefer the more traditional model of double-blind peer review. In a recent survey we conducted with our reviewers, 73% said they would prefer a double-blind type of peer review, with only 13% saying they preferred the current model. This is not a trend specific to VE. The Publishing Research Consortium carried out a study of 3,000 academics, finding that “71% said that they have confidence in double-blind peer review, while 56% prefer it over any other form of reviewing”.

But why is this? Is it a case of ‘this is how we have done it for 200 years’ and people being scared of change? In a world of online profiles and instant messaging where people’s views are available to everyone at the click of a button, do reviewers really have the right not to be held accountable for their critiques? And how blind can a review truly be when sometimes writing style or specificity of the content will ‘give away’ the author or reviewer’s identity?

What is apparent, however, is that reviewers still want to exercise their right to anonymity, and there are some advantages to blind review. The publisher Wiley argues that blind review eliminates any kind of bias that aspects such as gender, affiliation or social attitudes can impose on a paper, and that both “author and reviewer benefit from some level of protection against criticism”.

Papers should be assessed on the merit and clarity of the content alone. As a journal, VE acknowledges its responsibility to make its reviewers feel comfortable with the task assigned to them. With that in mind, Veterinary Evidence will consider adopting a double-blind model.

It is important to stress that peer review is not an exact science – by definition alone it is a critical appraisal of an author’s ideas, and therefore a judgemental comment that may conflict with another’s assessment of the work. Although it is an essential part of the publication process assuring a journal’s academic rigour, peer review will always be flawed and subject to debate because it is a system based on opinion and people’s good will. For the most part however, and certainly from a Veterinary Evidence editor’s perspective, peer review is there to help. When we asked reviewers in our survey what motivates them to review a paper for a journal, they responded with comments such as:

“I am interested in what people are researching and writing on in my subject area.”

“To ensure highest standards possible for publications in veterinary medicine.”

“Giving back to the veterinary community.”

“The desire to make a contribution in the field.”

Peer review is there to help your paper be the best that it can be, and 99% of the time it is being conducted by people who genuinely have an invested interest in the area of veterinary medicine you are contributing to.

As a reviewer, you are helping to advance science within your subject area by providing crucial feedback to authors. Your review is vital to improving the quality and scientific standard of papers submitted to and published in VE, therefore you are playing a very real part in progressing your area of research. If you are interested in reading more and becoming a reviewer, please see our reviewer guidelines.

Peer Review Week: Q&A with Veterinary Evidence reviewers

/0 Comments/in /by

Peer Review Week is an annual international event celebrating the essential role peer review plays in maintaining scientific quality.

For some authors, their submission to Veterinary Evidence is their first submission to any journal. So, for Peer Review Week 2019, we asked four of our reviewers to give authors and readers an insight and overview of what it’s like on the other side of the Veterinary Evidence peer-review process and why it’s important.

You can also read our Q&A with published authors.

Introducing the Reviewers

Bruce A. Smith BVSc, MS, DACVS

Currently Director of the Small Animal Veterinary Teaching Hospital at the University of Queensland, I have worked in private and university referral practice in Australia since 1997. I have had a career interest in orthopaedic surgery, clinical career development for veterinary specialists and, more recently, evidence-based veterinary medicine (EBVM).

Constance White DVM, PhD

I graduated with honours in 1997 from Oregon State University’s College of Veterinary Medicine, after first earning a PhD in genetics. I have experience working in mixed large and small animal practice. My particular interests are in surgery/critical care and in diagnostic imaging. I’m currently working as a small animal emergency and critical care vet at Fremont Veterinary Clinic in Oregon.

Louise Buckley PhD, RVN

I am a long standing veterinary nurse, with a background in research, clinical practice and lecturing in higher education. My particular enthusiasms are in the area of companion animal welfare and behaviour, EBVM and the evidence behind non-pharmaceutical products promoted in veterinary practice. Currently based at Edinburgh University in the postgraduate education team, I am joining the Bristol Vet School in September as Programme Director of Veterinary Nursing, where I will be supporting our undergraduate veterinary nurses to publish their dissertations and write Knowledge Summaries for Veterinary Evidence.

Jacqueline Cole BVetMed MRCVS

I qualified from the Royal Veterinary College in 2014 after completing a degree in Zoology in the USA. I am originally from New York, but have made England my home. I have worked in a multitude of practices around the country. Following a small animal rotating internship in 2016 at the University of Bristol, I locumed until making Sanctuary Veterinary Centre my home. I have a passion for internal medicine, and I am currently pursuing an advanced qualification in feline medicine via the University of Sydney in order to advance my knowledge and deliver the best care to my patients.

Is peer review worthwhile and important? What are its limitations? Does it help advance science?


A qualified ‘yes’ on both worthwhile and important. The limitations are very much those of what is the ‘expectation’ of peer review. An explicit statement would provide clarity on definition, goals and expectation for authors, readers, reviewers and editorial staff. I am not convinced that peer review per se ‘advances science’, however it does assist authors to hone their manuscript and present it in a way that maximises its message to the readers – to put it another way, it improves the quality of the communication.

Bruce

Peer review is a bit like democracy: flawed and inefficient but superior to all of the other systems of governance that have been tried. In a perfect world, science is an objective, iterative process aimed at establishing the ‘truth’ by the consistent application of hypothesis testing and ruthless rejection of prior beliefs which are not supported by a well-designed study. Peer review is but one step in that process, in which the scientific product is scrutinised for any potential missteps in the testing method. Problematically, peer review is done by humans, with their own biases and blind spots. Veterinary medicine is a particularly thorny area, since paltry research funding constrains most of our published literature to observational and descriptive studies in which authors must do their best to sort through dirty and often incomplete data for kernels of evidence. Knowledge Summaries are especially difficult since heterogeneous outcome reporting, study design, and patient populations make evidence synthesis immensely challenging. Veterinary medicine lags behind human medicine in shifting from ‘eminence-based’ to ‘evidence-based’ and reviewers may sometimes disagree with the ‘result’ due to their own deeply-entrenched practice beliefs or because the manuscript authors are not recognised specialists within the reviewer’s domain. Unlike our physician colleagues, most of us do not receive focused training in clinical study design and statistics adequate to task; this reviewer frequently finds errors in both study classification and statistical methods in the veterinary literature. Though these may seem like pedantic pitfalls, proper analysis and evaluation is challenging in this context.

Connie

I am going to have to answer yes to this one or I would have to claim that I spend hours of time engaged in a pointless activity! For me it is an important part of both quality assurance and the process of refining paper content. Authors, particularly new-to-research ones, often get so up close and personal to their work that they forget to write in a way that is accessible to the reader, or they omit information that someone who does not know the research would need in order to make sense of the work. The reviewer can play a part in moderating this. The quality assurance aspect though is only, really, as good as the knowledge/experience of the reviewer in the area in which the research is situated, their knowledge of the research methodologies employed and the time they have to review the papers. In all three cases this can be tricky to find reviewers that can deliver on all components, and if there is a reviewer skill deficit in one or more of these areas, it will limit the value of peer review. Plus, reviewers are only human; we have our off days and there is an element of subjectivity in any reviewing – I have often (politely) disagreed with some of a reviewer’s comments on a draft manuscript. Even reviewers often disagree between themselves as to a paper’s quality! But an imperfect quality assurance is better than no quality assurance at all, and an increase in poor quality papers in the public domain to be reviewed there instead.

Louise

Peer review is very worthwhile. Instead of just a single person deciding whether an article has merit, multiple people can evaluate it. It also means more eyes to pick up errors or mistakes or give suggestions to improve the writing. This process advances science; by bringing people together to look at questions it means more innovative solutions can be found.

Jacqueline

What motivates you to review a paper? How does reviewing a paper benefit you individually?


Motivation to review stems from my personal understanding of my professional responsibilities. I have found ‘joy’ in my work and part of that is being of use and help to others. Reviewing, in my view, is a service that I can provide to others. It feels a ‘right’ thing to do; that is my personal reward.

Bruce

These limitations combine to make peer review in veterinary medicine a bit more arduous than in other fields, particularly for articles which attempt systematic evidence synthesis.  I believe that there is a crying need for first-opinion veterinarians to participate in this process.  Our literature is primarily derived from referral centres, where patient mix is substantially different from those which we see in primary care.  Our specialists, who author the bulk of our primary research, are not always aware of the pitfalls of referral bias, nor are necessarily well trained in clinical research.  The movement towards establishing consensus guidelines is well-intentioned, but currently most guidelines are written without transparent evidence appraisal and without the input of first-opinion veterinarians on guideline panels.  I believe that generalists are perhaps better equipped to evaluate the evidence objectively and, most importantly, to assess generalisability to first-opinion populations. Peer review is one area in which generalists can have a seat at the table for helping to determine best practice for our populations.  For me personally, peer review offers an opportunity for exhaustive review of specific topics not refreshed in years. Recently, I reviewed a Knowledge Summary on a surgical procedure which I will never perform, yet my improved knowledge of the clinical questions surrounding this procedure will be helpful to my clients, who may need to know what options are available prior to their visit with a specialist.

Connie


Where to start? I think reviewing papers makes me better at understanding the scientific process and develops my critical evaluation skills, so it develops me as a professional. I also get a real buzz out of helping other veterinary professionals to develop their skills in this area, and I also really like being one of the first people to hear about new research that is being undertaken. I currently review for quite a wide range of journals so am reviewing work submitted by all sorts of people, from experienced researchers heading their own research group, to final-year student vets and nurses submitting their undergraduate dissertations. I would guess I spend about 250–300 hours a year reviewing papers, conference abstracts, etc. and this is all unpaid (as it usually is for most reviewers) so it must be pretty intrinsically motivating!

Louise

As I have published a paper in the past and therefore had people donate their time to review mine, I believe in the name of science and collaboration, that peer reviewing in turn is the way forward. It is interesting to see the thought processes of others, which in turn will improve my own writing.

Jacqueline

What would you say to authors unfamiliar with and daunted by the thought of the peer-review process?


All of our communications – bar none – can benefit from the feedback of others. After all, the core purpose of a manuscript is communication for the benefit of others. Keeping this perspective, ALL feedback (favourable, unfavourable or irrelevant) helps us communicate better. We naturally take criticism personally and a new author is certainly more vulnerable. There is no place for incivility on the behalf of reviewers, irrespective of truth. This is a role for the editor to police; personally I would have a zero tolerance for non-collegiate communications – get another reviewer.

Bruce

Though article review requires some effort, a good reviewer provides constructive feedback to make a paper better in a variety of ways. Ideally, a reviewer helps authors with clarity in writing, so that any practitioner can understand the gist of what was done and identifies possible alternative explanations for what is stated. I believe that practitioners are singularly adept at these tasks since they encounter countless clinical conundrums and counterfactuals in their daily environment. Veterinarians are a clever and creative people; any practitioner who spends a bit of time reviewing RCVS Knowledge toolkits has a great deal to contribute to the mission of EBVM. The review process often illuminates knowledge gaps that I, as a reviewer, was unaware of, and increases my own interest in research in that topic domain.

Connie


It is normal to feel daunted – you find me an author who doesn’t have niggles and worries about what other people will think of something that you have painstakingly laboured over and which they are now going to expose to potential criticism. I can remember getting a review back on a paper that was so awful that it took me a week to read the second reviewer’s comments (who thought it was an ace paper!). The important thing to remember is that Veterinary Evidence reviewers all want to see one thing – you to publish and publish with a paper that will make you (and us) proud. I think it is really important to point out that it is VERY, VERY, VERY rare to get a paper accepted in a journal without any revisions first, so don’t be deflated when the decision comes back that changes need to be made (sometimes major ones). I always submit papers to journals expecting to have to make revisions and the reviewers will come back with some negative comments. If they didn’t I might question the thoroughness of the reviewer – I want them to find the stuff I have not phrased well, etc. It makes my paper look better when it finally is accepted!

Louise

I would say, the only way to gain experience and to learn is by doing. To give it a go. The process is straight-forward. Peer reviewers are not out there to put up unsurpassable barriers for you to publish, but to catch errors, improve your work, and make it a better paper that will benefit readers.

Jacqueline

What common mistakes do authors of Knowledge Summaries tend to make? What could authors look out for?


Not getting a ‘friendly’ peer review BEFORE submission. Having the first ‘round’ of review by a friendly colleague – this should include spelling, grammar and style – is far better than an irritated peer reviewer and can help to avoid the issues discussed earlier.

Bruce

For me, I think there are three issues that I see very commonly. Hell, I have been guilty of some of these myself! The first one is not closely enough addressing the PICO/research question. Don’t let the paper summaries focus on populations, interventions, comparisons or outcomes that are not relevant. It is okay to state, for example, that other interventions or outcomes were also investigated in this study but have been omitted here as they are not relevant to addressing the PICO. The reader has picked up your Knowledge Summary because they want the answer to that PICO specifically, so focus on just answering this. Otherwise you will make a busy practitioner turn away without reading it because the key information they need is not easy to find. You can always write another Knowledge Summary (or several) if you want to explore other aspects! The other issue is a lack of detail when it comes to reporting the outcome measures and what was found, and I find that this can be reflected in not reporting the findings with sufficient precision. Finally, it really helps to read the Knowledge Summary if the author numbers the relevant outcomes and then ensures that the findings that relate to the outcomes are also numbered (and match up).

Louise


Common mistakes I find tend to be that people forget to put enough statistics, such as p-values and confidence intervals to back up a paper’s main findings. I also have found people tend to have inherent bias towards the result they want to find, and it can be reflected in their summaries versus being fully objective on the level of evidence presented.

Jacqueline

What are your top tips for reviewing a Knowledge Summary? What general tips do you think a reviewer who is unfamiliar with a Knowledge Summary needs to know?

Read (and understand) the PICO question first. If the PICO and the summary don’t match, there is likely a fundamental misunderstanding of what a Knowledge Summary is, and further review is ‘editing’ only. Reject and let the authors know why.

Bruce


Knowledge Summaries are substantial contributions in evidence synthesis but are difficult to perform well, especially for a single author. I enjoy reviewing them but find that I often dedicate time and energy nearly equal to second authorship. One cannot find omissions in the literature search without performing an independent search; few vets, including specialist or specialist trainees, have been taught formal search strategies, and I find that papers are often missing, usually due to failure to include all relevant terms in the search. The RCVS Knowledge toolkit provides good information but I think that many Knowledge Summary authors would benefit from using the RCVS library staff to help with their search, at least during their first few Knowledge Summary projects. Additionally, study classification is a frequent area of confusion and our literature excels in muddying the waters. Often, studies which are labelled only as ‘retrospective’ may contain a cohort, case-control, or prevalence (cross-sectional) analysis (or some mix of those for different outcomes), whilst a number of studies named as ‘prospective clinical trials’ are simply descriptive case series. Much of the toolkit, as well as most EBVM resources, is derived from the human EBM, which is a bit more straightforward.

Connie


Be kind, honest and fair. A Knowledge Summary author is unlikely to be a hardened academic used to getting harsh reviewer feedback. They might be, but it may just as easily be the veterinary nurse that you spayed a cat with last week, or a keen vet student that undertook extra-mural studies with you last summer. Try to make sure that any feedback that is given lets the author know, not only what needs improving, but how. Balance it out with the good. It can be really tempting to focus on just pointing out the aspects that need improving, but try to also let the author know what aspects you liked or thought they presented particularly well, etc. We don’t want their first Knowledge Summary to also be their last.

Louise

My top tips:

  • Follow the template given. Look at the format: did they follow what was asked?
  • Compare to other published papers that are similar in context. Is the level of detail the same?
  • Ask the editor if you have any concerns, as they are experienced in looking at papers and can help direct you.
Jacqueline


Interested in reviewing for Veterinary Evidence? Contact us.

The benefits of peer review: Q&A with Veterinary Evidence authors

/0 Comments/in /by

Peer Review Week is an annual international event celebrating the essential role peer review plays in maintaining scientific quality.

For some authors, their submission to Veterinary Evidence is their first submission to any journal. So, for Peer Review Week 2019, we decided to ask three published authors to allay any pre-submission nerves by giving authors and readers an insight and overview of what it’s like to go through the peer-review process and why it’s important.

You can also read our Q&A with reviewers.

Liz Barter BVSc (Hons)

I graduated from Sydney University in 2012, and then started in mixed practice at Macleay Valley Veterinary Services and Macleay Valley Equine Reproductive Services. To further my interest in equine reproduction I moved to Newmarket in the UK, working at Rossdales as a stud assistant, then later at Twemlows Stud in Shropshire.

Mike Steele BSc (Hons) BVSc MRCVS

I have many years of experience in dairy and cattle consulting, practice, and teaching in the global animal health industry. I recently started my own consultancy, bringing scientific evidence to practical solutions so that customers can be reassured that learning, improvement projects and product development all have strong science foundations.

Clare Knottenbelt BVSc MSc DSAM MRCVS

After graduating from Bristol in 1994 and working for a year in mixed practice, I was the PetSavers Resident in small animal internal medicine at the Royal (Dick) School of Veterinary Studies. I obtained an MSc by Research in Feline Transfusion Medicine and the RCVS Diploma in Small Animal Medicine in 1999. I was Clinical Director of the University of Glasgow’s Small Animal Hospital for six years and worked within the University’s Oncology service for 11 years, before leaving to establish Hawk and Dove.


How did peer review help your paper?

Working in private practice it is often seen that there is little opportunity for scientific publication. My practice supervisors at the time were not publishing and I was unsure how to get involved with researching and investigating within the scientific literature, outside of taking a university position. Whilst I was daunted at the prospect of writing a peer-reviewed article and unsure of the support I would receive at the practice, RCVS Knowledge was both supportive and helpful at every stage. During the peer-review process, I felt my opinion on the subject matter was considered and I was encouraged to investigate and produce material of a high standard.

Liz

I get the most satisfaction by being able to share my clinical bottom line with fellow practitioners so that they don’t have to spend their time looking for the same answer. They help their patients more quickly and, hopefully, with more confidence in their decision. If peer review had not occurred, I feel that my conclusions would not hold as much weight with my audience and would be open to more questioning by the reader.

In summary, peer review, for me, is reassurance that my conclusions are sound and the interpretation of my appraisal has been sufficiently thorough.

When the final document is prepared and the review complete, the feelings from the knowledge that the information is as good as it can be is extremely reassuring.

What has been my experience of peer review? So far, after four submissions, it has only been positive. The reviewers have asked relevant questions and not been overly critical. They have thoroughly reviewed my paper and clearly spent a lot of time thinking about what I have written. Comments have been objective and are never personal, which may help the first-time writer feel more confident in their own submissions.

Mike

Peer review helps you spot the small errors that creep in when you are too close to something – it’s great to have an external person read the paper through and help it make sense. Also, I had never done a Knowledge Summary before, and, whilst there are guidelines, it was great to find out how much I needed to put in each section. So peer review meant there was more work to do, but it made the finished paper better in the end.

How did I feel going into the process? A little apprehensive. Submitting a paper feels like you are exposing yourself in the same way you do for a lecture or speech, but as the reviewers don’t know you, you can’t rely on your personality to get through!

Peer review remains daunting. In my experience, at least one of the reviewers usually likes what I have done, so it’s easier to take the criticism. However, even though I have been through it many times, I still feel a huge sense of disappointment when a paper needs revising, even if it’s only minor changes. I think that’s probably natural given the amount of time and work you spend on a paper.

Clare

What do you feel are the pros and cons to peer review?

The positives of the Knowledge Summaries being peer reviewed is that all opinions can be discussed. This can also prepare you for feedback post-publication. I did not have any negative experiences of the peer-review process. I would prepare potential authors, however, that the process takes time, with two reviewers to read and make comments for several edits. I am eternally grateful for the help and contributions that the reviewers made to create the final manuscript.

Liz

For me, the pros outweigh the cons.

The review process ensures that at least two other people have looked at what you have written and been a part of the construction of the paper. They have considered your views and looked with a critical eye at how you have conveyed the statistical information from the literature provided to them. This means that the final submission is not only your own conclusion: two experts in the relevant field have already agreed that your opinion is fit for purpose. This adds a lot of value to your clinical bottom line.

The cons are mainly in timing: it can be months before your review can go from writing to publishing. The advantage that evidence-based veterinary medicine has over textbooks is that it is a lot more up to date. If the process takes a long time, then it has to be accepted that the published version is x-number of months old. Depending on the subject area, this may be unsustainable. Another con to peer review is that it is only as good as the peers that construct the review. For this, we must trust the editing team that they have targeted their opinion leader with precision.

Mike


The main pros are that it improves papers by flagging stuff that needs amending and rewording. The major con, however, is that it has resulted in many papers that never get seen. For me, this is really sad. There is a huge amount of imperfect research out there that no-one ever sees. When I spoke to university colleagues, almost everyone had at least one unpublished paper. Whilst some of these might be flawed, I believe that knowledge is a good thing and even a flawed study could drive the future direction of research into a particular problem. There have been many medical papers over the years that have been retracted due to fraudulent peer review and other failings, but once they are out its almost impossible to change people’s belief in the findings.  At university we learn to critically review papers so we should be able to read a paper and see some of the flaws for ourselves, but if it’s never seen then that’s not going to be possible.

Clare

What would you say to authors who may feel daunted by the process?


I find writing an incredibly daunting process whether it is for a client handout or veterinary report. I also see the huge benefits of contributing towards the literature for your own professional development and those that read the finished product. Knowledge Summaries have allowed me to gain insight into literature reviews and have since allowed me the courage to delve further into scientific publication whilst still working in private practice.

Liz

Emotions of going through peer review are, of course, personal, and this is a process in itself. Firstly, while writing, I am constantly thinking of how the reviewers will look at my document. When appraising the papers, knowing that a peer will be checking my interpretations makes me feel compelled to consider the fine detail on methodology and statistical review. While writing the summaries and conclusions, I am more likely to base every statement on objective data from the literature. When submitting, I feel somewhat daunted and apprehensive, knowing that everything I have written will be strictly criticised. This is, emotionally, the hardest part.

When the reviewed email returns, I hastily go through the comments and begin to plan out how I can best answer any queries. Often, the comments are based around finding more detail from the papers: clarifying sample population numbers or intervention groups. Most of these are easily found from the original papers. Corrections in English and grammar are always well received, as a reader can very quickly make an opinion on the validity of a conclusion if the grammar is not correct!

In terms of timing, peer review is by far the most time-consuming part of the process. The Veterinary Evidence team have to firstly find a list of people that are capable of reviewing a submission in the relevant field of research. Then they need to find someone who is willing to review in the timeframe required. This is an understandably difficult task and patience must be exercised: when you submit a Knowledge Summary, you are excited and relieved that the writing is over, but now you must wait. Again, I would encourage the prospective writer to focus on the end goal. The value of review far outweighs the swift publication of flawed data for which the writer will be considered responsible!

Mike

We all feel a little daunted, so you are not alone. However, it’s important that the right sort of research gets out there and people hear about it, so if you believe in the value of what you want to publish, don’t give up. Sometimes the reviewer is wrong – after all you are the one who is closest to the paper, so stand firm if you need to!

Clare

Should authors also review papers? Why/why not?

As an author, I enjoy reading other articles. I also recognise the role specialists have in their subject matter and highly value their contribution to the peer-review process.

Liz

Absolutely. By going through the peer-review process, you learn a lot of things: how to write more objectively, how to appraise the evidence, and how to focus on the important points from your findings. If the busy authors can fit it into their schedules, then they should offer their services as reviewers. By taking the decision to write a Knowledge Summary, you are automatically signing up to a process where you look at scientific papers in a critical way, and consider statistical interpretation an important way of conveying the science into practice. In every paper I have published, I have become a go-to person in that scientific field: you can be sure that not many others have done this, which makes you useful to the practical community.

I cannot think of a good reason not to do so!

One anecdote: I published a review article on the evidence behind sand and recycled compost bedding for dairy cows. This was quite a large one, with over 50 papers referenced. The reviewers were thorough and very helpful in my interpretation of the overall summary. It was extremely satisfying when I heard from one of my colleagues in practice three months after publication: he said that one of his farmers was considering replacing his sand system with mattress bedding. This was contrary to all scientific evidence; outcomes for his cows would potentially be detrimental in terms of mastitis and lameness, opening up increased risks for poorer welfare and productivity. He told me that it was my Knowledge Summary that managed to convince the producer to continue with sand, seeing the full value of that medium. Knowing I had made a difference directly to the cows and to a fellow colleague made the whole process worth it.

Mike

I believe that authors should also review papers, and vice versa. However, it’s important that you don’t hold grudges from your experiences. One problem with authors reviewing is that they may come to a paper looking for holes if they feel resentment after their paper is rejected. Turning this the other way round, authors should be able to appreciate the huge amount of work and effort that has gone into a paper and show respect for that in their review. Obviously there are some papers that are not worthy, but as a profession we should be looking to support the authors so that they can publish relevant and useful research on a regular basis. Discussing your idea with others in your field is a good idea before you start writing anything!

Clare

Interested in writing for Veterinary Evidence? Contact us.

Peer review: what is it and why do we use it?

/0 Comments/in /by

Peer Review Week is an annual international event celebrating the essential role peer review plays in maintaining scientific quality.

As the theme for this year is ‘quality in peer review’, Veterinary Evidence‘s Editorial and Production Assistant, Jennifer Morris, explains the importance of peer review to our journal.

You can also read her blog on the pros and cons of different types of peer review.

You’ve written your manuscript, you’ve submitted it to Veterinary Evidence’s online journal submission system, and the Editor-in-chief has approved the paper for the submission process; but what happens next?

For some authors, especially those who have not submitted a paper to a journal before, the next step may be unclear. For an editor, it is simple: peer review. But what is peer review, and why is it important?

In their Global State of Peer Review 2018 publication, Publons defined peer review as “the process of subjecting an author’s scholarly work, research, or ideas to the scrutiny of others who are experts in the same field”. All manuscripts submitted to Veterinary Evidence (VE) undergo this process. This is when the editor, having read your manuscript, finds and invites people who have suitable expertise in the field pertaining to your paper’s subject matter. You may be thinking ‘is that all?’, but believe me when I say this is a more substantive task than it appears to be.

What do we look for in peer reviewers?

Things we have to consider when finding someone to peer review are:

  • Bias and conflicts of interest – There is a section in our submissions template that asks the authors to ‘provide details of suggested reviewers’. This can be very useful and cut down on the time it takes to find reviewers. However, more often than not, these suggestions come with a conflict of interest (COI). This exists when the editor believes that the relationship between the author and the named individual may affect the reviewer’s judgement. Reviewers can ‘also have inherent bias due to their own research interests’, as stated in the International Journal of Surgery: Oncology (IJS: Oncology); therefore, relying on authors whose papers are related to or written on the same topic can also be a minefield.
  • Expertise in the field – This is by far the most important thing to consider. VE aims to provide the veterinary community with relevant, up-to-date, evidence-based content, to lend their expertise to the development of veterinary medicine. Our associate editors are vital in this area, as they are able to suggest experts that they have worked with or know of who can help review our journal. Using search engines, such as PubMed, enables us to find academics interested in the same field as you by searching already published papers. The RCVS Find a Vet resource also helps to find those currently practising.
  • Reviewing experience – Has a person reviewed for VE and/or other veterinary journals before? We are more likely to invite people who have reviewed past papers for us. Not only are they familiar with our process, but this also provides us with a more consistent reviewing method. However, this is something that VE is considering moving away from in line with the recent calls for journals to allow Early Career Researchers (ECRs) to review. This is becoming a hot topic in the world of peer review, as allowing less experienced academic researchers to review will not only help cultivate and grow the next generation of peer reviewers, but according to Publons writer Dr Gary McDowell, will also help minimise the “‘ghostwriting’ of peer review reports – the submission of someone else’s work to an editor under the name of the invited peer reviewer”. VE works with our associate editors to find the best reviewers for a paper, and we also encourage experienced reviewers to mentor those less experienced ECRs to help teach them what is expected of a peer reviewer and how they can contribute to evidence-based veterinary medicine (EBVM).
  • Contactable – Finding current contact details can be another hurdle all by itself, particularly with the introduction of the General Data Protection Regulation (GDPR). Imagine this: you’ve done your research and you’ve found the perfect person to peer review, but they do not have an email address online, or their affiliated institution is reluctant to give it out to you. Now you have to start all over again.
  • Availability – Finally, it is important to always bear in mind the reviewer’s availability. Peer review is essentially comprised of hard-working individuals who give their free time and knowledge to do us – the editors, and you, the author – a favour. Rarely are there any incentives given by journals to review. At least reviewing for VE can count towards your CPD.

Roadblocks can present themselves at any moment, and that is before we have even invited anyone to review. VE has, on occasion, invited up to nine reviewers before we have received the minimum two needed to proceed with a submission. The peer review platform Publons noted that “the reviewer invitation acceptance rate for certain journals has dropped dramatically in recent years”.

What are we asking reviewers to do?

So what do we ask our reviewers to look for when assessing a paper? The comments the peer reviewers make will be sent to the author, so it is important for the reviewer to bear this in mind when providing feedback. We ask that reviewers:

  • use language that is ‘sensitive, objective and unbiased’
  • provide fair assessments of the strengths and weaknesses of the paper
  • only reflect upon the scientific content of the paper; any errors in spelling or grammar will be picked up by the editor at the copy-editing stage
  • provide clear recommendations, with explanations as to why they are making those suggestions
  • flag any suspicions of plagiarism to the editorial team
  • provide sound, critical advice that they would be happy to receive themselves.

Reviews are in – what happens next?

Once we have secured a minimum of two reviews for a paper, the reviews are sent to the paper’s associate editor for a recommendation. This recommendation can be one of the following:

Accept Submission*Your paper can proceed to editing as it has been accepted for publication
Revisions RequiredYour paper needs revisions and you should use the comments provided by the reviewers to make your amendments
Decline Submission*Your paper is not at an appropriate publishable standard for VE and has been declined. You are free to submit the paper elsewhere
*These decisions have to be confirmed/approved by the Editor-in-chief.

Currently, of all the papers submitted to VE this year, 30% have been accepted for publication. It is common for papers to go through two, even three rounds of review before acceptance for publication. Of these published papers 57% were accepted after round two and 43% after round three. These stages of revisions are key to ensuring the paper is the best that it can be. IJS: Oncology says reviewing is “an invaluable tool for authors since it allows them to produce a more polished and rigorous piece of work”. It also ensures that the information we publish is as relevant and up to date as it possibly can be, and gives your paper the needed attention that an editor with over 50 papers to process just cannot give.

In 2012, a study was conducted by C. R. Lamb and C. A. Adams at the RVC looking into the acceptance rates for manuscripts submitted to peer-reviewed veterinary journals. The theory behind it was that ‘knowing how many manuscripts are rejected might help stimulate efforts to submit better quality manuscripts’. It was found that “on average, 3% submitted manuscripts were accepted without revision, 44% […] were accepted after revision, 4% […] were withdrawn by authors, 46% […] were rejected outright and 3% […] were still pending at the end of the study”. 20% of manuscripts required minor revisions, and major revisions were needed for 36%. Of these papers, all manuscripts requiring minor revisions, and 67% manuscripts requiring major revisions, were eventually accepted for publication. This goes to show that authors should not be put off if they receive that ‘Revisions Required’ email. It is a natural part of journal submission and authors should expect to have to revise their work. In fact, it is a good sign and inevitably gets you closer to your goal of being a published author.

As a reviewer, you are helping to advance science within your subject area by providing crucial feedback to authors. Your review is vital to improving the quality and scientific standard of papers submitted to and published in VE, therefore you are playing a very real part in progressing your area of research. If you are interested in reading more and becoming a reviewer, please see our reviewer guidelines.