Evidence reviews for selective laser trabeculoplasty in ocular hypertension or chronic open-angle glaucoma adult patients

1.1.4.1. Included studies

A systematic search was carried out to identify randomised controlled trials (RCTs) and systematic reviews of RCTs, which found 1,320 references (see Appendix L for the literature search strategy). Evidence from the original guideline (1 RCT) and evidence identified from systematic reviews (1 RCT) was also reviewed. In total, 1,322 references were identified for screening at title and abstract level with 1,298 excluded at this level. Full texts were ordered to be screened for 24 references.

In total 5 RCTs were included based on their relevance to the review protocol (Appendix A). The LiGHT trial was reported in 2 references, therefore 6 references were included in total. The clinical evidence study selection is presented as a PRISMA diagram in Appendix C.

See section 1.1.13 References – included studies for a list of included references.

1.1.4.2. Excluded studies

See Appendix J for a list of excluded studies with reasons for exclusion.

Comparison: 360° SLT vs eye drops

Table 3

Outcome: Intraocular pressure.

RR: relative risk; MD: mean difference

Table 4

Outcomes: Visual field progression; optic disc progression.

Table 5

Outcome: Quality of life.

Table 6

Outcome: Adverse events.

Table 7

Outcome: Treatment adherence.

Table 8

Outcome: Treatment discontinuation.

Comparison: 360° SLT vs latanoprost

Table 9

Outcomes: Intraocular pressure; adverse events.

Comparison: 180° SLT vs latanoprost

Table 10

Outcomes: Intraocular pressure; adverse events.

Comparison: 90° SLT vs latanoprost

Table 11

Outcomes: Intraocular pressure; adverse events.

Comparison: SLT (degrees not specified) vs latanoprost

Table 12

Outcome: Intraocular pressure.

See Appendix F for full GRADE tables.

1.1.7.1. Included studies

A single search was performed to identify published economic evidence (see Appendix B). The search retrieved 597 studies. Based on title and abstract screening, 578 of the studies could confidently be excluded. Eighteen studies were excluded following the full-text review. There was also a Health Technology Assessment (HTA) identified from citation searching that was linked to the included study from the review. Thus, two studies were included from the existing literature, both reporting different results from the same original study.

1.1.7.2. Excluded studies

See Appendix J for a list of references for excluded studies, with reasons for exclusion.

1.1.11.1. The outcomes that matter most

The committee agreed that the key outcome for adults with OHT or COAG was visual field progression which, in the long-term, could affect people’s vision. Intraocular pressure was considered to be a relevant surrogate outcome because lowering intraocular pressure could prevent the risk of optic nerve damage and vision loss. This means that if the IOP is high (≥24 mmHg) and left untreated, the high IOP is likely to damage the optic nerve and it could also cause vision loss. Other relevant outcomes were health-related quality of life, adverse events, optic disc progression, treatment adherence and treatment discontinuation. The committee discussed that the number of eye drop treatments might have an effect on people’s quality of life and on their treatment adherence which was taken into account when developing recommendations. The number of eye drop treatments was not an outcome in the protocol but it was considered important for its potential effect on people’s quality of life and treatment adherence. Most of the outcomes listed in the review protocol were reported by the included studies apart from glaucomatous visual field loss, normal visual field to visual field defect, vision loss, optic nerve head damage, and normal or suspicious-to- abnormal optic nerve head.

1.1.11.2. The quality of the evidence

Overall, the quality of the evidence was from high to very low with the main reasons for downgrading being due to imprecision of the evidence, risk of bias, and indirectness. In some of the evidence, imprecision was considered to be serious or very serious with the 95% confidence intervals crossing one or two ends of the defined minimally important differences (MIDs) thresholds. Risk of bias for some of the included RCTs was due to lack of information on allocation concealment, lack of reporting on statistical methods to estimate treatment effect (intention-to-treat analysis or per-protocol analysis), and lack of reporting that protocols were pre-registered where the pre-specific analysis plan would be reported. There were differences in comparators and follow-up times between included RCTs which prevented meta-analysis to be carried out.

The review protocol states that studies including people who have previously received first line treatment for OHT or COAG would be excluded. Two studies (Katz 2012 and Nagar 2005) were identified but it was unclear if the studies included people who were not treatment naïve. The committee highlighted that these studies provide context but that they should be downgraded for indirectness.

Three studies included in this review were conducted in the UK (Gazzard 2019, Nagar 2005 and Nagar 2009), 1 study was conducted in the US (Katz 2012) and 1 study was conducted in China (Lai 2004). Authors of Lai 2004 highlighted that Asian eyes have more pigmented trabecular meshwork. This means that the laser energy required and the clinical response might be different from eyes with lightly pigmented trabecular meshwork. The committee highlighted that this study is relevant to a subset of the UK population, but it’s not directly applicable to the UK general population.

RCTs were the main study designs included in this review. The committee noted that while RCTs are useful, these trials often include people who are highly motivated and who are provided extensive support, resulting in high adherence to treatment. The committee highlighted that in practice, it can be difficult to get patients to adhere to eye drops. Patient adherence to eye drops may have been overestimated in the included studies as this outcome was self-reported.

The committee highlighted that there was a lack of long-term evidence on progression of glaucomatous visual field defect and progression of optic nerve head damage (only 1 RCT reported both outcomes at 36 months follow-up). They also noted that patients care more about vision outcomes compared with other outcomes such as IOP. The committee discussed the importance of investigating these outcomes at longer follow-up times (≥3, 5 and 10 years) to know how effective SLT is at long-term. This evidence could help to target interventions which could prevent progression. Therefore, a research recommendation was developed to cover this gap in the evidence.

1.1.11.3. Benefits and harms

High quality evidence showed that there was no meaningful difference between SLT and eye drops in achieving the target IOP (either 20% or 30% IOP reduction), no meaningful difference in the change of health-related quality of life overtime, no meaningful difference in the risk of total adverse events, and no meaningful difference in treatment adherence.

Further evidence was identified from the LiGHT trial that showed people who were given eye drops as first line treatment, used more eye drops and required the use of more than 1 eye drop medication at 12 months, compared with people who were given SLT as first line treatment (see Table 18 in Appendix M). Cost effectiveness evidence further showed that first-line treatment with 360° SLT was more effective and less costly compared with eye drops. This evidence also showed that SLT resulted in a larger period without eye drops, or with fewer eyedrops, and slightly slower estimated progression rates for glaucoma, which improved quality of life. Based on this evidence (see section 1.1.11.4 cost effectiveness and resource use for further details), the committee agreed that 360° SLT could be a treatment option for OHT and COAG.

However, evidence did show that that there were transient adverse events associated with SLT such as transient discomfort, blurred vision, photophobia and hyperaemia. It was also noted that there are rare complications associated with SLT such as corneal failure. Based on their clinical expertise, the committee noted that some people with OHT or COAG might choose to have SLT to be free of having eye drops and that it was important to clarify that they might need to receive eye drops in the future if IOP is not successfully reduced after the SLT procedure. Additionally, people might also need further SLT if the effect of the first SLT procedure reduces over time, which is an important factor to take into consideration when choosing treatment options.

Furthermore, evidence was mainly identified for newly diagnosed OHT or newly diagnosed primary open-angle glaucoma (POAG) which is defined in the 2017 version of the guideline as COAG in the absence of any other ocular, systemic or pharmacological cause and accompanied by elevated intraocular pressure. In the evidence, people with secondary glaucoma associated with pigment dispersion syndrome were excluded from the LiGHT trial (Gazzard 2019 HTA).

The committee agreed that SLT is an appropriate treatment option for people with OHT and COAG but in practice people can be newly diagnosed but have pigment dispersion syndrome. In this population, SLT would not be appropriate. There was no evidence on the use of SLT in people with pigment dispersion syndrome and the committee agreed that eye drop treatment is more suitable for people with pigment dispersion syndrome. Based on the evidence and clinical understanding, the committee noted that SLT as first line treatment was not appropriate for people with pigment dispersion syndrome. Additionally, the LiGHT trial included a small number of people with pseudoexfoliation. Based on their clinical understanding, the committee did not think it was appropriate to exclude pseudoexfoliation from the recommendations but as the evidence based was relatively small, they opted to not explicitly highlight the condition in the recommendation.

Based on the new evidence, the committee agreed that 360° SLT should be offered as first-line treatment to people with newly diagnosed OHT with IOP of 24 mmHg or more (and if they are at risk of visual impairment within their lifetime) or COAG, however this should exclude cases associated with pigment dispersion syndrome. For people with newly diagnosed OHT, a threshold of 24 mmHg or more was identified based on existing NICE guidance. Additionally, in the LiGHT trial, baseline characteristics showed that mean IOP in the SLT arm was 24.5 (SD 5.2) and 24.4 (SD 5.0) in the eye drops arm.

To aid decision making, the committee further stated that information should be provided on the possibility of needing eye drops treatment after SLT, the time that SLT takes to improve IOP, the SLT specific side effects and complications including the type and duration, and that they might need further SLT treatment at a later date. The committee highlighted that when a generic prostaglandin analogue (PGA) is given as interim treatment to people waiting for an SLT procedure, it is important to follow recommendations on reassessment to use clinical judgement regarding IOP control and risk of progression.

The committee further highlighted that in general, treatment to reduce IOP has to work for at least 6 months to be considered successful, however this can also be based on clinician discretion. In the case of SLT procedures, there may be an initial reduction in the IOP level, but over time this level may begin to increase. This can occur at any time, meaning that retreatment with SLT may be required. The committee highlighted that, recommendations on repeating SLT were required as re-treatment with SLT is usually done in practice.

In the LiGHT trial, patients who were not at target IOP after a single SLT received another treatment of 360° SLT at the same energy setting, with re-evaluation after 2 weeks. SLT was also repeated in Katz 2012, where participants in the laser arm were offered repeat 180° SLT. However, it should be noted that this study (Katz 2012) was downgraded for indirectness as it was unclear if the study included people who were not treatment naive (see section 1.1.11.2 for more details).

Based on their clinical expertise and applicability of evidence to current practice, the committee opted to follow the treatment protocol highlighted in the LiGHT trial. Based on these factors the committee further recommended that a second 360° SLT could be considered for people with OHT and COAG if the effect of an initial successful SLT has subsequently reduced over time. This means that the IOP level has gone up and clinicians need to decide if there is risk of progression of COAG or conversion of OHT to COAG. The second SLT should be given at the discretion of the treating consultant ophthalmologist. Based on their clinical experience, the committee further noted that any effect from SLT might be reduced after repeating the procedure more than 2 or 3 times.

The committee agreed that some people might prefer not to have SLT or that this procedure might not be suitable for some people. The 2017 guideline recommended prostaglandin analogue (PGA) eye drops for OHT or COAG. Therefore, they amended this to reflect the new 2022 recommendations on using SLT. The amended recommendation offers a generic PGA to these people as an alternative first-line option instead of SLT. As previously noted, in people with pigment dispersion syndrome, SLT was not considered to be an appropriate treatment. Therefore, eye drops were recommended as first line treatment for this population. It was also recommended that healthcare professionals should demonstrate correct eye drop instillation technique and observe the person using the correct technique when eye drops are first prescribed.

The committee noted that the first line use of SLT to treat OHT or COAG might lead to a significant change in practice that requires better organisation of care and the establishment of a multidisciplinary team. The committee also noted that larger centres may see more referrals, resulting in an increase in the number of clinics per week. The committee highlighted that, although the increase should not be significant, any increase means there will be a change to the organisation of care. They also discussed the safety of the SLT procedure and agreed that healthcare professionals should discuss with the responsible consultant ophthalmologist the decision to offer SLT and how it will be performed. This means that healthcare professionals such as specialty doctors, associate specialists, specialist nurses, optometrists and allied health professionals can perform SLT with support from a consultant ophthalmologist. The committee also wanted to make clear that if SLT is suitable for a person, that person should be referred to a consultant ophthalmologist. Based on this, the committee updated an existing recommendation to state that people should be referred to a consultant ophthalmologist for consideration of a definitive diagnosis and formulation of a management plan if they are suitable for SLT.

The committee also noted that healthcare professionals who provide SLT should be given support and have relevant training on the suitability and safety of the procedure, including the benefits and risks. They should also be trained on discussing these points and patient consent with patients and their family members or carers. A similar approach was taken in the LiGHT trial where training was given to all treating surgeons before recruitment and at least 1 laser treatment was observed by the chief investigator, who was a consultant ophthalmic surgeon. Based on these discussions new recommendations were added to provide further clarification on organisation of care.

1.1.11.4. Cost effectiveness and resource use

The committee discussed the published cost-utility evidence relating to selective laser trabeculoplasty (SLT) compared with pharmacological treatment (eyedrops) in ocular hypertension and open-angle glaucoma. This included a published study and a health technology appraisal (HTA) that were both based on the same clinical trial (LiGHT) and both these studies were assessed to be directly relevant to the review question. The difference between the study and the HTA was the time horizon; the study had a three-year time horizon (to match that of the LiGHT trial) whereas the HTA had a lifetime time horizon and extrapolated the data beyond the time horizon of the trial.

The evidence from both analyses showed that SLT dominates eye drops for both a 3-year time horizon and a lifetime time horizon (that is, SLT is associated with both lower costs and better outcomes). The committee noted that the pathway modelled as the comparator (multiple lines of eye drops followed by surgery if needed) was not current treatment within the NHS, as SLT is an option later in the eye drops pathway in the previous NICE guidance. However, the committee felt this was not a significant limitation and the comparison of SLT and eye drops as a first-line treatment was valid, and therefore that the study and HTA still showed that it would be cost effective to move SLT to the beginning of the treatment pathway. They also noted the LiGHT trial found no meaningful differences in the effectiveness of SLT between people with ocular hypertension and people with open-angle glaucoma, and were therefore confident the findings applied to both populations, as long as they met the inclusion criteria for the LiGHT trial.

The committee noted that there was only a small difference in the quality of life between the two comparators and therefore the main reason that SLT was the dominant option was because the SLT arm was less costly. These lower costs were primarily driven by the finding that using SLT means people are likely to need to use significantly fewer eye drops, which reduces costs both of the medicines themselves, and also appointments to monitor and modify treatment. The committee noted that SLT may need to be repeated and that was included in the analyses (with approximately 15% of people in the SLT arm having a second procedure within the first year) and therefore that gave the committee more confidence in the result, as it reflected their expectations of how the treatment would be used in practice.

The committee acknowledged that there are some patients who may prefer using eye drops because it feels like they are actively doing something to improve their eyes and may make them feel more in control of their condition. However, the committee felt that more patients would prefer to not have to use eye drops. The committee also felt that it was important to be aware of patients who find eye drops difficult to use, for example if the patient also has dementia or arthritis, this can affect adherence and therefore effect the improvements the patients are able to achieve. The committee felt that in these cases some patients would require a carer to come in to administer the eye drops that would increase the cost of the eye drops arm. Therefore, this group are likely to benefit even more from SLT.

The sensitivity analyses from both the study and the HTA varied the parameters of the analysis and for each analysis the probability of SLT being cost effective was over 90%. The committee felt that this was strong evidence in support of SLT as a first line treatment. The committee discussed having SLT or eye drops as equal options as first line treatment. However, the committee felt that given the cost-effectiveness evidence, and their expectation that a significant majority of patients would prefer SLT if it were available, it was important to rank SLT higher than eye drops and therefore SLT should be the first line treatment. The committee felt that using SLT is becoming more common in practice and with the clinical and cost effectiveness data there was strong evidence to move SLT to become the first line treatment for glaucoma.

1.1.11.5. Other factors the committee took into account

The 2017 update of the guideline included recommendations for people with suspected COAG. The recommendation stated that generic PGA should be offered to people with suspected COAG and IOP of 24 mmHg or more. The committee flagged that this recommendation could cause confusion amongst clinicians as the IOP level stated is the same as the level recommended to treat OHT. The committee further stated that people with suspected glaucoma would not be treated unless there were clear clinical grounds, for example, if they developed OHT or COAG. Therefore, the committee agreed to remove that recommendation from the guideline.

The committee also highlighted that some people with suspected COAG may still be at risk of visual impairment within their lifetime, for example, in some people clinicians may be concerned with the appearance of the optic disc but may not find signs of visual defect but the person may have a strong family history of glaucoma. As there is a risk of visual impairment, such patients may require treatment at clinician’s discretion. Based on this understanding, the committee amended the 2017 recommendation to state that treatment should not be offered to people with suspected COAG and IOP less than 24 mmHg, unless they are at risk of visual impairment within their lifetime.

Additionally, the 2017 update included a recommendation on treatment adherence and checking the eye drop instillation technique. If adherence and eye drop instillation were satisfactory, a medicine from another therapeutic class, topical medicine from a different class, laser trabeculoplasty or surgery with pharmacological augmentation (MMC) could be offered. The 2017 update also included a recommendation on offering surgery with pharmacological augmentation (MMC) as indicated to people with COAG who are at risk of progressing to sight loss despite treatment.

The committee noted that these recommendations required further clarity as there were three important messages being conveyed across the two recommendations. Based on this understanding, the committee amended the 2017 recommendations and split them into 3 separate recommendations. The first recommendation highlights that clinician should check treatment adherence and eye drop instillation technique in people with COAG whose IOP has not been reduced sufficiently to prevent the risk of progression to sight loss, despite pharmacological treatment with a generic PGA.

The second recommendation highlights the treatment options that can be offered to people in whom eye drop instillation technique is satisfactory and IOP has not been reduced. As the evidence identified in the current review focused specifically on SLT, the committee noted it was important that all recommendations in the guideline are in line with the evidence and new recommendations. As SLT is the type of laser trabeculoplasty currently used in clinical practice this recommendation was also updated to specifically highlight SLT as a treatment option for people in whom adherence and eye drop instillation technique are satisfactory.

The third recommendation considers SLT or glaucoma surgery with pharmacological augmentation (MMC) as indicated to people with COAG who are at risk of progressing to sight loss despite treatment with medicines from 2 therapeutic classes. The committee highlighted that the purpose of this recommendation is to discourage polypharmacy with patients being given additional drug therapies and for healthcare professionals to consider SLT and glaucoma surgery as more favourable outcomes before considering further medical treatment.

It should also be noted that the committee suggested to change the term ‘surgery’ to ‘glaucoma surgery’ because the term ‘surgery’ is more general, and it can include other types of eye surgery which are not glaucoma surgery. This change was also made across the guideline.

The 2017 update included a recommendation which stated that clinicians could consider offering people with COAG who cannot tolerate treatment either a medication from another therapeutic class or preservative-free eye drops. After trying medications from 2 therapeutic classes, it was recommended to consider surgery with pharmacological augmentation (MMC) as indicated or laser trabeculoplasty. The committee amended this recommendation to add further clarity by stating that a medication from another therapeutic class or preservative-free eyedrops should be offered to people with COAG who cannot tolerate pharmacological treatment. The committee further added that SLT or surgery should be considered after trying medications from 2 therapeutic classes, in order to be consistent with the new recommendations.

The 2017 update also included recommendations for people with COAG who have undergone surgery, but IOP has not reduced sufficiently to prevent the risk of progression to sight loss. Laser trabeculoplasty or cyclodiode treatment were a suggested treatment option for the group. The committee amended this recommendation to state that SLT can be a treatment option instead of laser trabeculoplasty.

No evidence was identified in people with advanced COAG as the LiGHT trial excluded people with advanced COAG. Therefore, no specific recommendations were developed. However, the 2017 update, included a recommendation that stated in people with advanced COAG surgery could be offered with pharmacological augmentation. The committee noted that there are instances where surgery might not be suitable (for example due to systemic comorbidities). The 2017 update also included a recommendation stating that for people with COAG who prefer not to have surgery or for whom surgery is not suitable pharmacological treatment, laser trabeculoplasty or cyclodiode laser treatment could be offered. The committee amended this recommendation to explicitly state that SLT can be a treatment option instead of laser trabeculoplasty.

Recommendations for people with OHT have been amended to be in line with the new recommendation which offers SLT as first-line treatment. In particular, SLT was added to the recommendation to refer people whose IOP cannot be reduced sufficiently with pharmacological treatment to a consultant ophthalmologist.

The committee identified older people (aged over 70 years) as an important subgroup. Older people, including people with cognitive or physical impairment (for example arthritis), people with learning disabilities and people with dementia might find it difficult to administer eye drops or may require assistance from carers in receiving IOP-lowering eye drops to manage their OHT or COAG. In these populations, adherence to medication is a concern. It was also highlighted that IOP-lowering eye drops might not be the preferred treatment to manage OHT or COAG during pregnancy or breastfeeding because of the side effects that these treatments could have on women and their children (manufacturers advise to avoid use during pregnancy and breastfeeding).

The new recommendations allow SLT to be considered as a treatment option in these groups. This can be beneficial for older people, people with cognitive or physical impairment (for example arthritis), people with learning disabilities and people with dementia as this can lead to people needing to use significantly fewer eyedrops. Polypharmacy is also a concern, especially in older people therefore a reduction in the need for eyedrops can potentially result in fewer medications being used in this population.

The committee acknowledged that there might be waiting times for SLT procedures, and this is why they recommended to offer a generic PGA as interim treatment for people who are waiting for an SLT procedure. The committee also highlighted that SLT procedures should be prioritise to women who are pregnant/breastfeeding because of the side effects that IOP-lowering eye drops could have on women and their children if those were used.

The committee acknowledged that late presentation of glaucoma (usually in the form of advanced glaucoma) might be associated with people who are of black African or black Caribbean family background and with greater individual and area level deprivation. Late presentation of glaucoma might be driven by clinical variation and by variations in healthcare seeking behaviours and healthcare inequalities in referrals or diagnosis. Evidence included in the review, excluded people with advanced glaucoma, therefore specific recommendations could not be drafted for this population. However, the committee agreed that new recommendations were unlikely to have an impact on late presentation of glaucoma because surgery is the main treatment option for advanced glaucoma and this is stated in existing recommendations. As ethnicity is an important risk factor for glaucoma, the committee identified it as an important subgroup in the new research recommendation.

Clinical/public health searches

Data synthesis for intervention studies

Where possible, meta-analyses were conducted to combine the results of quantitative studies for each outcome. When there were 2 treatment alternatives, pairwise meta-analysis was used to compare interventions.

Pairwise meta-analysis

Pairwise meta-analyses were performed in Cochrane Review Manager V5.3. A pooled relative risk was calculated for dichotomous outcomes (using the Mantel–Haenszel method) reporting numbers of people having an event, and a pooled incidence rate ratio was calculated for dichotomous outcomes reporting total numbers of events. Both relative and absolute risks were presented, with absolute risks calculated by applying the relative risk to the risk in the comparator arm of the meta-analysis (calculated as the total number events in the comparator arms of studies in the meta-analysis divided by the total number of participants in the comparator arms of studies in the meta-analysis).

A pooled mean difference was calculated for continuous outcomes (using the inverse variance method) when the same scale was used to measure an outcome across different studies.

For continuous outcomes analysed as mean differences, change from baseline values were used in the meta-analysis if they were accompanied by a measure of spread (for example standard deviation). Where change from baseline (accompanied by a measure of spread) were not reported, the corresponding values at the timepoint of interest were used. If only a subset of trials reported change from baseline data, final timepoint values were combined with change from baseline values to produce summary estimates of effect. If some studies only reported data as a change from baseline, analysis was done on these data, and for studies where only baseline and final time point values were available, change from baseline standard deviations were estimated, assuming a correlation coefficient derived from studies reporting both baseline and endpoint data, or if no such studies were available, assuming a correlation of 0.5 as a conservative estimate (Follman et al., 1992; Fu et al., 2013).

Random effects models were fitted when there was significant between-study heterogeneity in methodology, population, intervention or comparator was identified by the reviewer in advance of data analysis. This decision was made and recorded before any data analysis was undertaken.

For all other syntheses, fixed- and random-effects models were fitted, with the presented analysis dependent on the degree of heterogeneity in the assembled evidence. Fixed-effects models were the preferred choice to report, but in situations where the assumption of a shared mean for fixed-effects model were clearly not met, even after appropriate prespecified subgroup analyses were conducted, random-effects results are presented. Fixed-effects models were deemed to be inappropriate if there was significant statistical heterogeneity in the meta-analysis, defined as I²≥50%.

However, in cases where the results from individual pre-specified subgroup analyses were less heterogeneous (with I² < 50%) the results from these subgroups were reported using fixed effects models. This may have led to situations where pooled results were reported from random-effects models and subgroup results were reported from fixed-effects models. Where sufficient studies were available, meta-regression was considered to explore the effect of study level covariates.

Intervention studies (relative effect estimates)

RCTs were quality assessed using the Cochrane Risk of Bias Tool. Evidence on each outcome for each individual study was classified into one of the following groups:

• Low risk of bias – The true effect size for the study is likely to be close to the estimated effect size.
• Moderate risk of bias – There is a possibility the true effect size for the study is substantially different to the estimated effect size.
• High risk of bias – It is likely the true effect size for the study is substantially different to the estimated effect size.

Each individual study was also classified into one of three groups for directness, based on if there were concerns about the population, intervention, comparator and/or outcomes in the study and how directly these variables could address the specified review question. Studies were rated as follows:

• Direct – No important deviations from the protocol in population, intervention, comparator and/or outcomes.
• Partially indirect – Important deviations from the protocol in one of the following areas: population, intervention, comparator and/or outcomes.
• Indirect – Important deviations from the protocol in at least two of the following areas: population, intervention, comparator and/or outcomes.