In the hierarchy of evidence, randomized controlled trials are the gold standard. While seeking expert opinion will never compare in terms of scientific rigor, it can be a valuable option in cases of limited, insufficient, or conflicting information. This is, of course, assuming that a consensus is achieved between a group of experts in a structured and systematic way – which is where Delphi panels come in.
The idea behind the Delphi panel is to gain consensus through controlled feedback from a group of subject experts. Panelists – who generally participate with some level of anonymity towards one another – are presented with several (usually three) iterative rounds of surveys, with the ultimate aim of reaching a target expert consensus by the end of the exercise. What is a Delphi panel and how is it carried out?
Conducting surveys and reaching a consensus
Participants respond to each of the statements using Likert scale responses (i.e., completely agree to completely disagree), with the range of statements and responses systematically reduced in each round. Statements can be eliminated in the case of, for instance, too great a disparity in the range of responses, or adapted based on participant feedback to reach a higher rate of consensus.
At the conclusion of the Delphi exercise, researchers should be left with a smaller set of statements on which a high or complete level of expert consensus has been reached. It should be noted that a statement to which all experts disagree could be just as important to report as one to which they all agree. These findings can then be applied in a variety of ways.
Key considerations in developing a Delphi panel:
-
- Number of opinions collected
- Expertise of the invited panelists
- Methodology of collecting opinions, including development of survey content.
Traditional vs modified Delphi panel
In the traditional Delphi panel format, panelists can freely provide ideas, opinions, or information in response to open-ended questions presented in round one. Managing the resulting range of responses in the second round tends to be a complicated and lengthy process, potentially resulting in a high rate of drop out. The modified Delphi panel tackles these issues by replacing the open-ended questions of the former with a set of statements to which panelists are asked to agree or disagree. The statements are typically developed through collaborative input from a separate group of experts (e.g., a steering committee). They are then used in each survey round, with the option of conducting the first round as a 1:1 interview – allowing panelists to elaborate on the reasons for their responses could be valuable when adapting survey content to reach a higher rate of consensus.
How can Delphi panels be used?
Delphi panels have several potential use cases:
-
- Generating clinical guidelines
- Establishing the basis of other research projects
- Acting as a methodology, for instance in a utility study
- Contributing to health policy development.
Delphi panels can be hugely valuable in tackling issues on which opinions tend to diverge, by providing a robust and systematic way of aligning expert viewpoints. At Vitaccess, we have in-house experience in the design and implementation of Delphi panels for a range of research purposes. To find out more about our capabilities, get in touch today at info@vitaccess.com.
By Sally Vincent and Fatemeh Amini
