PRISMA-P is a reporting guideline for developing the protocol. It should be used in conjunction with a systematic review and/or meta-analysis reporting guideline (e.g., PRISMA). However, other systematic review and/or meta-analysis reporting guidelines can also support the development of your protocol (e.g., PRISMA, NIRO, ROSES).
It is important to know where you are headed, so a good first step is becoming familiar with each stage (e.g., tab in this guide, section in a handbook) of the systematic review before drafting the protocol.
Defining the protocol at the start of your review is one of the mechanisms used to reduce risk of bias. Think of the protocol as a map to your review journey - outlining the path you'll take before you begin will reduce the need for ad hoc decision-making mid-review. Ad hoc decision making is an invitation for bias!
An essential step in the early development of a systematic review is the development of a review protocol. A protocol pre-defines the objectives and methods of the systematic review which allows transparency of the process which in turns allows the reader to see how the findings and recommendations were arrived at. It must be done prior to conducting the systematic review as it is important in restricting the presence of reporting bias. The protocol is a completely separate document to the systematic review report.
Without review protocols, how can we be assured that decisions made during the research process aren’t arbitrary, or that the decision to include/exclude studies/data in a review aren’t made in light of knowledge about individual study findings. Given their costly time and resource requirements, we simply cannot afford wasted efforts when it comes to systematic reviews.
In the context of systematic reviews and meta-analyses, a protocol is a document that presents an explicit scientific “road map” of a planned, uninitiated systematic review. The protocol details the rational and planned methodological and analytical approach of the review.
Arguably one of the most important functions of systematic review protocols is their role as a documentation of planned review methods, outcomes, and analyses that can be compared with completed reviews to detect whether unintended and undocumented changes were made. Bias related to selective reporting of outcomes (that is, when reporting is related to the statistical significance or direction of effect estimate) is a problem in clinical research. When reviewers selectively choose which information to include in a report based on the direction and significance of findings, they risk biasing the evidence base on which healthcare decisions and policies are made.
Although this excerpt mentions clinical research, research in any field is susceptible to selective reporting bias. The development of a protocol before beginning your review will help to hold your team accountable to what you set out to do. It is one of several measures taken to reduce the risk of biasing results from a systematic review and/or meta-analysis.
The PRISMA-P elaboration and explanation document details the items that should be addressed in a systematic review and/or meta-analysis protocol. These items are addressed throughout this library guide, as applicable.
There are several approaches to piloting your review protocol. Regardless of how it's done, it is a vital part of the process. Piloting helps a team identify and address issues early on so that these issues don't cause a more serious problem (like having to redo a whole stage of the review) down the road.
Your team may choose to pilot all of the stages of the review, from search to synthesis, before finalizing and registering the protocol. This process is described in Long, 2014 "Routine piloting in systematic reviews--a modified approach?"
At a minimum, your team should pilot each stage as you go through the review itself. For example, during the eligibility screening stage, start by having the two reviewers review only 5-10% of the unique references from your search. Calculate the interrater reliability. A low score indicates a lack of consistency between reviewers, which may mean the eligibility criteria needs clarification or to be refined. The pilot concludes when reviewers are screening articles with relative consistency and confidence in applying eligibility criteria.
One of the first steps in your systematic review and/or meta-analysis journey was to identify in-progress reviews. Registering your protocol allows others to know that your review is underway. Registering will also help to keep your team accountable by having a public record of what you aim to accomplish and how.
Some journals use a pre-registration peer-review model, meaning you can receive feedback before starting the review by first registering your protocol. In some cases, if a journal publishes your protocol and your team does not deviate beyond reason, the journal will also publish the final systematic review and/or meta-analysis.
Submit for approval
No approval required
Deviation from the protocolDeviation from the protocol is expected! But your team should only deviate from the protocol when it is necessary.
Deviations mid-review can be avoided by piloting first. If you are already mid-review, but need to deviate from your protocol, be sure to document all changes (and justification for those changes) thoroughly, and include a report (PRISMA-P recommends a chart or table format) of all deviations in the final published protocol.
The protocol can be a peer reviewed publication in and of itself, for example, as is with the Aetiology and outcome of non-traumatic coma in African children: protocol for a systematic review and meta-analysis. It contains all of the sections (e.g., background, methods, funding) required for a reader to understand the purpose of the review and the expected conduct of the team.
Protocols that have undergone peer review should also be accompanied by documentation of this process, including links to other relevant documents, as is the case in this example (below).
