Stage 5. Verify the uncertainties

Each indicative question (evidence uncertainty) needs to be verified as a true uncertainty before it can go forward for prioritisation. For example, some of these questions may have already been addressed by research without all patients or clinicians being aware of this.

To check that an indicative question is a true uncertainty, a search needs to be undertaken for relevant and reliable systematic reviews and guidelines that might address the uncertainty. Large-scale registry data may be available for the condition, which can be reviewed. Other types of evidence may be considered if the Steering Group agrees that this is relevant and appropriate to the topic area in question.

Each PSP must agree and publish a Question Verification Form to describe and account for their evidence checking. The PSP Steering Group must discuss and agree an appropriate strategy for checking all relevant evidence for the condition or setting to which the PSP relates. A requirement of the JLA PSP process is that the PSP's Question Verification Form is published on the JLA website. This should be published once the Steering Group and JLA Adviser have signed it off. A template Question Verification Form is available from the Templates and useful documents section of the JLA website.

PSPs will already have agreed and noted in their PSP Protocol who will be responsible for verifying uncertainties.

It is suggested that at the same time as verifying the indicative questions, uncertainties from research recommendations are recorded and added to the list of indicative questions for prioritisation. Where research recommendations duplicate a question that has come from the survey, this can be taken as an indication of that question being unanswered.

For each verified uncertainty, the source of verification should be cited, e.g. the systematic review identified as being relevant.

As a minimum, it is recommended that PSPs check the resources listed below to confirm whether a submission is an uncertainty:

• The Cochrane Database of Systematic Reviews
• NICE guidelines
• SIGN clinical guidelines
• Relevant Royal Colleges' guidance.

When checking an uncertainty against a systematic review, the review needs to be relevant, up-to-date, and reliable. The JLA recommends that an up-to-date systematic review is less than three years old. The sources listed above can be assumed to be reliable and meet methodological standards. If a PSP decides to look beyond these, it will need the knowledge and expertise to be able to assess the reliability of its sources. This can be measured by seeing if the authors follow a published methodology for undertaking the review, and if the methodology has made provision for managing bias. When looking at guidelines, the author needs to have made efforts to identify all relevant and reliable trials or systematic reviews. Reliability can be further ascertained from the confidence intervals around the main outcomes, enabling an informed reader to make an informed decision about the result. Narrative reviews, which do not give details or numerical results, may fail the requirements of relevance and reliability.

The Steering Group may wish to discuss how it wishes to approach grading of reliability of evidence. There are guidelines and ideas about this online, including at www.casp-uk.net/ https://methods.cochrane.org/gradeing/ or https://www.gradeworkinggroup.org/.

The JLA recommends that the evidence search is pragmatic and proportionate. A PSP should be confident that the questions it puts forward for prioritisation are broadly unanswered and should demonstrate clearly and accountably in the Question Verification Form how it came to that conclusion. If a PSP finds that the number of systematic reviews that are relevant to its area is unmanageable within its budget and timeframe, it may decide to create limitations. It should be transparent about these. The size of the evidence base is something to be aware of when setting out the scope for the PSP.

Finally, some apparent uncertainties can in fact be resolved with reference to existing research evidence, i.e. they are 'answered questions' and not uncertainties. If a question can be answered with existing information but this is not known, it suggests that information is not being communicated effectively to those who need it. These findings may usefully inform future awareness-raising exercises and education programmes. The JLA recommends strongly that PSPs keep a record of these 'answerable questions' and deals with them separately from the 'true uncertainties' considered during the research priority setting process. The JLA suggests incorporating this commitment into the PSP's Protocol.

The Psoriasis PSP created two videos where health professionals talked about the questions received by the PSP which had already been answered by research. You can find these on the PSP page.