Increasing uptake of hepatitis A, hepatitis B and human papillomavirus (HPV) vaccinations in gay, bisexual and other men who have sex with men

Cover of Increasing uptake of hepatitis A, hepatitis B and human papillomavirus (HPV) vaccinations in gay, bisexual and other men who have sex with men

Increasing uptake of hepatitis A, hepatitis B and human papillomavirus (HPV) vaccinations in gay, bisexual and other men who have sex with men

Reducing sexually transmitted infections (STIs)

Evidence review F

NICE Guideline, No. 221

London: National Institute for Health and Care Excellence (NICE); 2022 Jun.

ISBN-13: 978-1-4731-4612-9

1. Effective and cost-effective interventions to increase uptake of vaccinations in men who have sex with men

1.1. Review question

What interventions are effective and cost effective at increasing uptake of hepatitis A, hepatitis B or human papillomavirus (HPV) vaccination in gay, bisexual and other men who have sex with men (MSM)?^a

1.1.1. Introduction

Sexually transmitted infections (STIs) include a range of clinical syndromes that can be acquired and transmitted through sexual activity and may be caused by various types of pathogens, including bacteria, fungi, viruses, and parasites. It can affect personal wellbeing, mental health and relationships and can also lead to serious health problems including pelvic inflammatory disease, ectopic pregnancy or infertility. Preventive interventions can reduce the spread of infection and avoid complications and consequences.

From 1^st April 2018 there has been a national HPV vaccination programme for men who have sex with men. This offers the vaccine to those aged up to and including 45-years-old through Specialist Sexual Health Services (SSHS) and/or HIV clinics. A pilot conducted in 2016/17 and using data from the GUMCAD and HARS reporting systems found first dose uptake was 45.5% (3.4% were offered and declined the vaccine, and 50.9% had no vaccination code), though this was expected to be an underestimate of true uptake due to variations in data recording. Uptake rates following the full introduction of the programme are not yet available for England, but a similar programme in Scotland had an uptake of 63.7% among eligible MSM attending sexual health clinics in 2017/18.

Since June 2017, it has been recommended that all men who have sex with men attending HIV, GUM or sexual health clinics should be opportunistically offered vaccination against hepatitis A. There are currently no published national data on uptake rates for this vaccination.

1.1.2. Summary of the protocol

Table 1

PICO inclusion criteria.

1.1.3. Methods and process

This evidence review was developed using the methods and process described in Developing NICE guidelines: the manual. Methods specific to this review question are described in the review protocol in Appendix A and the methods document.

Declarations of interest were recorded according to NICE’s conflicts of interest policy.

1.1.4. Identification of public health evidence

The effectiveness and qualitative reviews were carried out using a single literature search (Appendix B). 858 references were identified for title and abstract screening and 81 quantitative papers were ordered for full-text review. Of these, 3 RCTs met the inclusion criteria for the effectiveness review, as outlined in the review protocol. 78 studies were excluded. See Appendix C for a PRISMA flow diagram of the study selection process.

1.1.4.1. Included studies

Of the 3 RCTs included for the effectiveness review, 1 study was conducted in the Netherlands and two in the USA. 1 study reported on hepatitis B vaccination uptake and the other two studies reported on HPV vaccination uptake. See Table 2 for included study details for the effectiveness review.

1.1.4.2. Excluded studies

The full list of excluded studies and reasons for exclusion are in Appendix J.

Table 2

Summary of studies included in the effectiveness evidence review.

See Appendix D for full evidence tables.

1.1.5. Summary of the effectiveness evidence

Table 3

Summary of findings table.

See appendix F for full GRADE Tables.

1.1.6. Economic evidence

A search for relevant economic studies was undertaken, using the strategy in appendix B and applying a cost-effectiveness filter. 146 references were identified from this literature search; all of which were excluded during title and abstract screening. As such, no economic studies were included to inform this review question.

1.1.7. Economic model

No economic modelling was undertaken for this review question.

2. Barriers and facilitators for increasing uptake of vaccines in men who have sex with men

2.1. Review question

What are the barriers to, and facilitators for, increasing uptake of hepatitis A, hepatitis B or human papillomavirus (HPV) vaccinations in men who have sex with men?

2.1.1. Introduction

Current practice recommends that men who have sex with men attending HIV, GUM or sexual health clinics should be opportunistically offered vaccination against hepatitis A and there is a national HPV vaccination programme for men who have sex with men aged up-to and including 45 years attending Specialist Sexual Health Services (SSHS) and/or HIV clinics. Data on vaccine uptake rates are not available but strategies for improving uptake are important as vaccinations against hepatitis A, hepatitis B and HPV are important STI prevention strategies. The purpose of this review is to establish the barriers to, and facilitators for, vaccine uptake in MSM.

2.1.2. Summary of the protocol

Table 4

PICO inclusion criteria.

For the full review protocol see appendix A.

2.1.3. Methods and process

Declarations of interest were recorded according to NICE’s conflicts of interest policy.

2.1.4. Identification of qualitative evidence

The effectiveness and qualitative reviews were carried out using a single literature search (Appendix B). 858 references were identified for title and abstract screening and 18 qualitative papers were ordered for full-text review. Of these, 9 qualitative studies reported in 10 papers met the inclusion criteria for the qualitative review, as outlined in the review protocol. 8 studies were excluded.

2.1.4.1. Included studies

Of the 9 qualitative studies included in the qualitative review, 1 study was conducted in Canada, 2 in the United Kingdom, and 6 in the USA. All studies reported on the barriers to, facilitators for, or acceptability of HPV vaccine. Studies reported individual, provider and system level barriers, facilitators or acceptability. See Table 5 for included study details.

2.1.4.2. Excluded studies

The full list of excluded studies and reasons for exclusion are in Appendix J.

Table 5

Summary of studies included in the qualitative evidence review.

See Appendix D for full evidence tables

2.1.5. Summary of the qualitative evidence

2.1.5.1. Summary of themes and sub-themes

Iterative aggregation of codes generated two main themes and 10 sub-themes. These are outlined in Table 6.

Table 6

Summary of themes and sub-themes.

2.1.5.2. Summary of qualitative findings

The qualitative findings for the barriers to and facilitators for HPV vaccination in MSM are presented in Table 7. Full CERQual tables are presented in Appendix G.

Table 7

Summary of qualitative findings.

3. Integration and discussion of the evidence

3.1. Mixed methods integration

Are the results/findings from individual syntheses supportive or contradictory?

The effectiveness evidence showed that text-messaging based interventions designed to increase vaccine uptake were effective at increasing HPV vaccine initiation MSM. Similarly, an intervention focusing on implementation intention formation (prompting participants to consider ways to implement their intentions to get vaccinated by thinking about when, where and how they would obtain hepatitis B vaccine) was effective at increasing hepatitis B vaccine initiation in MSM. The text-messaging based interventions were not effective in supporting HPV vaccine completion and there were no studies that examined hepatitis B vaccine completion.

The qualitative evidence generated themes that showed a lack of information about HPV, HPV-related disease, and the HPV vaccine was a barrier to vaccination in MSM. This supports findings from the effectiveness review because the interventions that were effective at increasing vaccination initiation included information-based components that educated people about HPV and HPV-related disease.

Does the qualitative evidence explain why the intervention is/is not effective?

The qualitative evidence highlighted lack of knowledge about HPV as a barrier to vaccination, particularly relating to the widespread perception that HPV is something that only affects cisgender women. It also showed vaccine hesitancy, uncertainty about vaccine effectiveness and concerns about side effects were barriers to vaccination. Interventions that provided information about HPV, the vaccine, how it works, and potential side effects were effective in improving vaccine initiation, which suggests that the interventions were effective because they filled knowledge gaps or allayed fears about the vaccine that were acting as barriers to initial vaccine uptake.

The qualitative evidence suggested that some of the barriers to vaccination completion were related to the need for multiple doses, the long time intervals between doses, and the potential inconvenience of having to schedule appointments around other commitments. The effectiveness evidence showed that vaccine completion rates were generally very low in both the intervention and control arms (range 1.3% to 11%) and the text messaging based interventions were not effective in improving HPV vaccine course completion. The interventions may not have been effective because they focused more on information, motivation and behaviour needs rather than addressing issues relating to scheduling and appointment flexibility. Nevertheless, the interventions did include regular text or email reminders which still did not appear to be effective at facilitating vaccine course completion.

Does the qualitative evidence explain differences in the direction and size of effect across the included quantitative studies?

Quantitative findings for interventions to increase both HPV vaccine initiation and completion had relatively wide confidence intervals, indicating some uncertainty in the estimate of effect. This uncertainty may be partly explained by the qualitative findings because they identified a range of different barriers to, and facilitators for, vaccination. Different people may require different support from interventions to overcome personal barriers and encourage them to obtain vaccination, so digital interventions that focused primarily on information, motivation and behavioural needs may be effective for some, but not all people, contributing to the wide confidence intervals seen in the quantitative evidence.

Which aspects of the quantitative evidence were/were not explored in the qualitative studies?

The quantitative evidence included interventions that focused on intention formation implementation and how to move people from considering vaccination to making a specific plan to obtain it. The qualitative evidence looked more broadly at person- and system-level factors that may act as barriers or facilitators to vaccination, rather than individual approaches to supporting people in making behavioural changes that encouraged vaccination. The quantitative evidence also assessed the effectiveness of text- and email-based interventions while the qualitative evidence only considered app-based approaches for booking appointments and creating a reminder system and did not explore the potential barriers and facilitators for text messaging-based interventions.

Which aspects of the qualitative evidence were/were not tested in the quantitative studies?

The qualitative evidence highlighted themes relating to health care professionals and their role in supporting people to obtain vaccination. Specific themes included the importance of HCPs being open, accepting, and non-judgemental; that HCPs were considered to be the most trusted source of information about vaccination; and that recommendations from HCPs were seen as something that would substantially influence people’s decision to obtain the vaccine. Findings from the qualitative evidence also showed that people wanted vaccinations to be offered during other routine interactions with HCPs such as general sexual health checks or STI testing. The quantitative evidence did not examine the role of health care practitioners and did not evaluate the efficacy of offering vaccinations during other clinical interactions.

3.2. The committee's discussion and interpretation of the evidence

The qualitative and quantitative reviews are presented as a combined discussion.

3.2.1. The outcomes that matter most

The committee discussed the evidence and agreed that vaccine uptake is the most important outcome, but also emphasised the importance of vaccine course completion. They agreed that both vaccine initiation and vaccine course completion should be considered together.

The committee discussed and agreed on the importance of considering the barriers to or facilitators for vaccine uptake and completion, described by those who are likely to benefit from having vaccines, that may have substantial impact on the effectiveness of any intervention that aims to increase vaccine uptake.

The committee considered it important that gay, bisexual and other men who have sex with men are aware of, and able to easily access, HPV, Hepatitis A and Hepatitis B vaccinations. Particularly in relation to HPV they noted that many people in this group will not have been included in the school’s vaccination programme that has now been expanded to include boys. In future this group will have been offered vaccination at school, but currently this group have not been included as the vaccination programme did not initially include boys.

3.2.2. The quality of the evidence

Quantitative evidence

The committee discussed the lack of evidence on vaccine uptake in gay, bisexual and other men who have sex with men, with only 3 studies identified: two for HPV and one for Hepatitis B vaccination. They noted that there were some methodological concerns about these included papers because of the lack of detail provided about the randomisation methods used and on allocation concealment methods. They further noted that the studies had not specified the number of vaccine doses offered.

The committee discussed the evidence and noted that the interventions had a positive impact on HPV vaccine initiation but not on HPV vaccine completion. It was also noted that the finding for HPV vaccine completion had a very wide confidence interval, suggesting a high degree of imprecision for this outcome. The committee agreed that the evidence was useful in drafting recommendations on vaccine initiation and agreed that while there was a lack of evidence for interventions to encourage vaccine completion, it was important to also make recommendations about this because people need to have both doses of the vaccine to be fully protected. This absence of evidence prompted the committee to make a qualitative research recommendation on the barriers to HPV vaccine course completion and how people think they might be encouraged to complete it.

The committee highlighted that while the 7- and 9-month follow ups described in the two trials fit with the recommended HPV vaccine course of doses at 0, 1 and 4 months, in practice doses are often more widely spaced and, in some clinics, at least a year is allowed to complete the full course. The committee therefore considered that 7- and 9-month follow-up periods were a relatively short timeframe for people to complete the vaccine doses and this may have contributed to the lack of effect for this outcome.

The committee discussed the evidence showing an effect of motivational information and implementation intention formation activities on improving Hepatitis B vaccine uptake, though it was not clear from the study what the vaccination schedule was or whether participants completed their course. As noted previously, the committee considered vaccine completion to be important so this was acknowledged as a limitation of the applicability of this evidence. The committee also noted that drop out was 25% higher in the intervention group than the control group, although attrition analyses found no significant differences between participants in the experimental and control groups, or between completers and those lost to follow-up, indicating that attrition was not selective. The committee discussed the limited evidence for specific interventions that aim to improve vaccine uptake. They considered that there was not sufficient evidence to specifically recommend any of the included interventions, but they recognised the importance of including vaccination information within healthcare and specifically sexual health services that gay, bisexual and other men who have sex with men use. They noted that even though the evidence alone was insufficient, with the broader experience and expertise of the committee members, they were able to make recommendations about the importance of opportunistically providing information on and discussing vaccination with gay, bisexual and other men who have sex with men during routine or other healthcare appointments.

The committee noted the NICE guidance on behaviour change: individual approaches and noted that the recommendations about behaviour change techniques that could be effective may be useful when thinking about approaches to improving vaccine uptake.

Qualitative evidence

Evidence addressing barriers to and facilitators for increasing uptake of vaccination in gay, bisexual and other men who have sex with men was only identified for HPV vaccination. Nonetheless, the committee discussed and agreed that many of the identified themes for both barriers and facilitators identified for HPV vaccination for gay, bisexual and other men who have sex with men are also applicable to the other vaccines. These include barriers such as those around the approachability of healthcare staff for gay, bisexual and other men who have sex with men, the need for clear information about vaccines and the possible provision of vaccines as part of other healthcare visits. The committee discussed the identified barriers including a general lack of knowledge about HPV and HPV related diseases and vaccines; and concerns about vaccine effectiveness and potential side effects. Additional barriers included possible stigma or concerns about having discussions with healthcare professionals and the impact of previous negative experiences of this; and some potential difficulties with the scheduling of vaccines. The committee also discussed the identified facilitators including increasing knowledge and awareness of the benefits of getting vaccines, the influence of the views of healthcare providers on vaccination decisions, the possibility of vaccinations being offered during routine sexual health or other healthcare visits.. The Committee agreed that the themes identified were similar to those that they had expected and reflected their expertise and clinical experience.

The committee discussed the qualitative evidence in combination with the findings from the quantitative review and noted that the qualitative evidence supported the importance of discussing vaccination opportunistically. They also discussed the evidence that noted that there can be a misconception that HPV only has relevance for cisgender females and that the link with male cancers may not be widely known. The committee considered that this misconception also supports the recommendation for opportunistic discussion as it may be that gay, bisexual and other men who have sex with men do not know that the vaccination is relevant for them.

The evidence identified that there are ongoing concerns for gay, bisexual and other men who have sex with men around the perceived approachability of healthcare staff and this affects whether or not they are comfortable discussing their sexual history. The committee further discussed and agreed that gay, bisexual and other men who have sex with men need to feel confident discussing their healthcare needs, concerns, and sexual history with healthcare professionals without any apprehension about a negative reaction or stigma. The committee agreed that healthcare settings need to adopt approaches that will signal to gay, bisexual and other men who have sex with men that healthcare providers will support and facilitate these important discussions. They discussed programmes such as the ‘You’re Welcome’ quality criteria, which lays out key principles that help health services to be young people friendly, and considered how similar approaches could be used for gay, bisexual and other men who have sex with men to provide assurance to those using the services that it is designed with their needs in mind.

The committee discussed accessibility of vaccination appointments and the possibility of combining this with other healthcare visits. The qualitative evidence suggested that ensuring flexibility about when vaccines are given and reducing the number of visits by combining vaccines or combining vaccination with other healthcare visits may improve uptake. The committee agreed to recommend that services consider ways they can do this. The committee noted that one study identified the possible use of mobile apps for booking appointments and reminding people about them. They agreed that this is already current practice and approaches like this are already in use.

3.2.3. Benefits and harms

The committee agreed that improving vaccine uptake and completion in gay, bisexual and other men who have sex with men is beneficial. They noted that there is a widespread misconception that the HPV vaccine is only important for cisgender women and that the link with male cancers is not known, so correcting this misconception is important. The committee also recognised that there are people who consider vaccines harmful, and that many gay, bisexual and other men who have sex with men do not consider vaccination important, so providing clear information about the benefits and harms of HPV, Hepatitis A and Hepatitis B vaccines is important. The quantitative and qualitative evidence did not identify any harms of HPV and hepatitis B vaccinations.

3.2.4. Cost effectiveness and resource use

No economic evidence was identified for this review question. Nevertheless, the committee discussed the potential cost-effectiveness and resource impact of the recommendations made. They agreed that the recommendations on providing information and opportunistically assessing eligibility for vaccination should reflect current practice, and therefore did not consider that there would be a substantial resource impact associated with implementing them.

Combining vaccinations alongside other routine healthcare (either care around sexual health or more general healthcare) would be expected to increase rates of vaccination and be a more efficient and therefore less costly method of providing vaccination, as the person does not need an additional separate appointment solely for the purpose of delivering the vaccination. The committee noted such an approach would not be suitable for all vaccine eligible men but would be a cost-effective approach for those it can be delivered to.

It was also noted that because these vaccines had already been assessed as being cost-effective by the Joint Committee on Vaccination and Immunisation, an increase in the number of people being vaccinated should also be cost-effective.

3.3. Recommendations supported by this evidence review

This evidence review supports recommendations 1.4.1 to 1.4.4 and the research recommendation on barriers to vaccination course completion.

3.4. References – included studies

3.4.1. Effectiveness

Bass Michael, Gerend Mary A., Madkins Krystal et al (2021) Evaluation of a Text Messaging-Based Human Papillomavirus Vaccination Intervention for Young Sexual Minority Men: Results from a Pilot Randomized Controlled Trial. Annals of behavioral medicine: a publication of the Society of Behavioral Medicine 55(4): 321–332 [PMC free article: PMC8025080] [PubMed: 32914838]
Reiter Paul L, Katz Mira L, Bauermeister Jose A et al (2018) Increasing Human Papillomavirus Vaccination Among Young Gay and Bisexual Men: A Randomized Pilot Trial of the Outsmart HPV Intervention. LGBT health 5(5): 325–329 [PMC free article: PMC6034390] [PubMed: 29979642]
Vet Raymond; de Wit John B F; Das Enny (2014) The role of implementation intention formation in promoting hepatitis B vaccination uptake among men who have sex with men. International journal of STD & AIDS 25(2): 122–9 [PubMed: 24216031]

3.4.2. Qualitative

Apaydin Kaan Z, Fontenot Holly B, Shtasel Derri et al (2018) Facilitators of and barriers to HPV vaccination among sexual and gender minority patients at a Boston community health center. Vaccine 36(26): 3868–3875 [PMC free article: PMC5990434] [PubMed: 29778516]
Fontenot Holly B, Fantasia Heidi C, Vetters Ralph et al (2016) Increasing HPV vaccination and eliminating barriers: Recommendations from young men who have sex with men. Vaccine 34(50): 6209–6216 [PubMed: 27838067]
Gerend M.A., Madkins K., Crosby S. et al (2019) A Qualitative Analysis of Young Sexual Minority Men's Perspectives on Human Papillomavirus Vaccination. LGBT health 6(7): 350–356 [PMC free article: PMC6797075] [PubMed: 31556791]
Grace Daniel, Gaspar Mark, Paquette Rachelle et al (2018) HIV-positive gay men's knowledge and perceptions of Human Papillomavirus (HPV) and HPV vaccination: A qualitative study. PloS one 13(11): e0207953 [PMC free article: PMC6264470] [PubMed: 30496221]
Jaiswal Jessica, LoSchiavo Caleb, Maiolatesi Anthony et al (2020) Misinformation, Gendered Perceptions, and Low Healthcare Provider Communication Around HPV and the HPV Vaccine Among Young Sexual Minority Men in New York City: The P18 Cohort Study. Journal of community health 45(4): 702–711 [PMC free article: PMC7774381] [PubMed: 32016677]
Kesten J.M., Flannagan C., Ruane-Mcateer E. et al (2019) Mixed-methods study in England and Northern Ireland to understand young men who have sex with men's knowledge and attitudes towards human papillomavirus vaccination. BMJ Open 9(5): e025070 [PMC free article: PMC6530382] [PubMed: 31092645]
Koskan Alexis M and Fernandez-Pineda Madeline (2018) Human Papillomavirus Vaccine Awareness Among HIV-Positive Gay and Bisexual Men: A Qualitative Study. LGBT health 5(2): 145–149 [PMC free article: PMC5833245] [PubMed: 29412771]
Koskan Alexis M and Fernandez-Pineda Madeline (2018) Anal Cancer Prevention Perspectives Among Foreign-Born Latino HIV-Infected Gay and Bisexual Men. Cancer control: journal of the Moffitt Cancer Center 25(1): 1073274818780368 [PMC free article: PMC6028166] [PubMed: 29925247]
Nadarzynski Tom, Smith Helen, Richardson Daniel et al (2017) Perceptions of HPV and attitudes towards HPV vaccination amongst men who have sex with men: A qualitative analysis. British journal of health psychology 22(2): 345–361 [PubMed: 28191723]
Wheldon C.W., Daley E.M., Buhi E.R. et al (2017) HPV vaccine decision-making among young men who have sex with men. Health Education Journal 76(1): 52–65

Appendices

Appendix D. Effectiveness evidence

D.1. Quantitative evidence

Download PDF (343K)

D.2. Qualitative evidence

Download PDF (490K)

Appendix E. Forest plots

Download PDF (105K)

Appendix F. GRADE tables

F.1. Human papillomavirus vaccination initiation

Download PDF (106K)

F.2. Human papillomavirus vaccination completion

Download PDF (106K)

F.3. Hepatitis B vaccination uptake

Download PDF (99K)

Appendix G. GRADE CERQual Tables

G.1. CERQual: Barriers to increasing uptake of HPV vaccinations in gay, bisexual and other men who have sex with men (MSM)

Download PDF (149K)

G.2. CERQual: Facilitators for increasing uptake of HPV vaccinations in gay, bisexual and other men who have sex with men (MSM)

Download PDF (143K)

Appendix H. Economic evidence study selection

Download PDF (98K)

Appendix I. Economic evidence tables

No economic evidence was identified for this review question

Appendix J. Health economic model

No economic modelling was undertaken for this review question

Appendix K. Excluded studies

Table

Appendix L. Research recommendations – full details

L.1. Research recommendation

What are the barriers to completing the full course of hepatitis A and B or HPV vaccinations and how do people think they might be encouraged to complete it?

L.2. Why this is important

The committee discussed the lack of evidence for interventions to facilitate vaccine completion for gay, bisexual and other men who have sex with men and emphasised the importance of people having all doses of the vaccine to be fully protected. They agreed that both the quantitative and qualitative evidence focused largely on vaccine initiation so there was an evidence gap relating to vaccine completion. They considered that understanding the barriers to vaccine course completion would help to support eligible gay, bisexual and other men who have sex with men to have all vaccine doses to obtain full protection.

L.3. Rationale for research recommendation

Download PDF (131K)

Footnotes

a: Throughout this review, the term men who have sex with men (MSM) is used to refer to gay, bisexual and other men who have sex with men

Final version

Evidence reviews underpinning recommendations 1.4.1 to 1.4.5 and research recommendations in the NICE guideline

National Institute for Health and Care Excellence

Disclaimer: The recommendations in this guideline represent the view of NICE, arrived at after careful consideration of the evidence available. When exercising their judgement, professionals are expected to take this guideline fully into account, alongside the individual needs, preferences and values of their patients or service users. The recommendations in this guideline are not mandatory and the guideline does not override the responsibility of healthcare professionals to make decisions appropriate to the circumstances of the individual patient, in consultation with the patient and/or their carer or guardian.

Local commissioners and/or providers have a responsibility to enable the guideline to be applied when individual health professionals and their patients or service users wish to use it. They should do so in the context of local and national priorities for funding and developing services, and in light of their duties to have due regard to the need to eliminate unlawful discrimination, to advance equality of opportunity and to reduce health inequalities. Nothing in this guideline should be interpreted in a way that would be inconsistent with compliance with those duties.

NICE guidelines cover health and care in England. Decisions on how they apply in other UK countries are made by ministers in the Welsh Government, Scottish Government, and Northern Ireland Executive. All NICE guidance is subject to regular review and may be updated or withdrawn.

Bookshelf ID: NBK589711PMID: 36947645

Table 1PICO inclusion criteria

Eligibility criteria	Content
Population	Gay, bisexual and other men who have sex with men (MSM) from age 16
Interventions	Interventions with the primary aim of increasing uptake of hepatitis A and B and HPV vaccinations in MSM such as: targeted mass media campaigns education for example peer led education recommendations from for example health care practitioner internet based interventions for example social media interventions designed to increase recall / adherence / vaccination schedule completion interventions that aim to reduce any difficulties with access or expand access
Comparator	No intervention Usual care Comparator as defined by the paper
Outcomes	Primary outcomes: Changes in uptake of hepatitis A and B vaccination in MSM Changes in uptake of HPV vaccination in MSM Uptake of initial and subsequent vaccinations Secondary outcomes: Safety or adverse effects Health related quality of life

Table 2Summary of studies included in the effectiveness evidence review

Study

Setting

Population

Intervention

Comparator

Outcome(s)

Bass (2021)

USA

Gay, bisexual and other men who have sex with men (GBMSM) aged 18–25 years

n = 150

Txt2protect: a text-messaging based HPV vaccination intervention. Information Motivation Behaviour (IMB) informed messages focused primarily on HPV vaccination, with brief mention of other sexual health practices (e.g. condom use, HIV testing).

Daily text messages for first 3 weeks then monthly text messages for the remaining 8 months

Attention control: text messages focusing on a variety of sexual health practices; only brief mention of HPV vaccination.

HPV vaccine initiation

HPV vaccine completion

Reiter (2018)

USA

Young gay and bisexual men (YGBM) aged 18–25 years

n = 150

The Outsmart HPV intervention consisting of two components:

(a) population-targeted individually tailored content about HPV, HPV-related disease, and the HPV vaccine;

(b) monthly HPV vaccination reminders sent via email and/or text message

Standard information about HPV and the HPV vaccine

HPV vaccine initiation

HPV vaccine completion

Vet (2014)

The Netherlands

Men who have sex with men aged 13 to 88 years (mean = 32.6 years; SD = 12.4)

n = 616

The intervention was based on implementation intention formation: participants were prompted to consider and respond to questions about when, where and how they would make an appointment to obtain vaccination against HBV. Also received information about local sites offering HBV vaccination.

General information providing contact details of Public Health Services offering hepatitis B vaccination

Hepatitis B vaccination uptake

Table 3Summary of findings table

Interventions to increase vaccine uptake vs. control

Outcomes	Illustrative comparative risks^* (95% CI)		Relative effect (95% CI)	No of Participants (studies)	Quality of the evidence (GRADE)	Comments
Outcomes	Assumed risk Control	Corresponding risk Intervention	Relative effect (95% CI)	No of Participants (studies)	Quality of the evidence (GRADE)	Comments

HPV vaccine initiation Follow-up: up to 9 months	160 per 1000	307 per 1000 (203 to 466)	RR 1.92 (1.27 to 2.91)	298 (2 studies¹^,²)	⨁⨁⊝⊝ low ³ ^, ⁴

HPV vaccine completion Follow-up: up to 9 months	20 per 1000	65 per 1000 (18 to 234)	RR 3.26 (0.91 to 11.72)	298 (2 studies¹^,²)	⨁⊝⊝⊝ very low ³ ^, ⁴ ^, ⁵

Hepatitis B vaccine uptake	90 per 1000	212 per 1000 (118 to 380)	RR 2.37 (1.32 to 4.24)	300 (1 study⁶)	⨁⨁⊝⊝ moderate ⁷

*: The basis for the assumed risk (e.g. the median control group risk across studies) is provided in footnotes. The corresponding risk (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI).

: CI: Confidence interval; RR: Risk ratio;

: GRADE Working Group grades of evidence

: High quality: Further research is very unlikely to change our confidence in the estimate of effect.

: Moderate quality: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate.

: Low quality: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate.

: Very low quality: We are very uncertain about the estimate.

1: Bass 2021

2: Reiter 2018

3: Downgraded once for some concerns of bias due to no information about allocation concealment for Bass 2021, and no information on randomisation and allocation concealment for Reiter 2018

4: US studies

5: Downgraded once as 95%CI crosses line of no effect and 1 MID

6: Vet 2012

7: Downgraded once for some concerns of bias due to no information on randomisation or allocation concealment

Table 4PICO inclusion criteria

Eligibility criteria	Content
Population	Men who have sex with men (MSM) from age 16
Interventions	Barriers to, and facilitators for, increasing uptake of Hepatitis A, Hepatitis B and HPV vaccinations This will include interventions or strategies identified in the effectiveness review, but is not restricted to these
Comparator	Not applicable
Outcomes	Barriers to and facilitators for increasing vaccine uptake at individual and system level. These may include: Affective attitude Acceptability Preferences Burden Value system

Table 5Summary of studies included in the qualitative evidence review

Study	Design and analysis	Country	Setting	Population and number included in study	Objective	CASP Risk of Bias
Apaydin (2018)	Focus groups Thematic content analysis	USA	Urban community health centre specialising in care for sexual and gender minority patients	Vaccine eligible sexual and gender minority youth, including trans and cisgender men and women. 53% were gay cisgender men. 80% reported HPV vaccine initiation and 73% reported HPV vaccine completion. n = 15	To identify and understand patient-, provider- and system-level barriers to and facilitators for HPV vaccination among sexual and gender minority patients	Moderate risk of bias (limited information about recruitment strategy and data analysis)
Fontenot (2016)	Focus groups and individual interviews Content analysis	USA	Private conference rooms at health and community centres	Young MSM 18–26 years. 58.8% reported HPV vaccine initiation and 35.3% reported HPV vaccine completion. n = 34	To elicit YMSM’s beliefs about HPV and the HPV vaccine and to describe perceived barriers and facilitators of vaccine initiation and completion.	Moderate risk of bias (Insufficient detail on how codes were developed; interrater reliability or agreement between coders not reported)
Gerend (2019)	Semi-structured interviews Analysis method unclear	USA	Unclear; assumed community settings	Young MSM 18–26 years. Participants were vaccinated (n=9) and unvaccinated (n=20) n = 29	To identify young sexual minority men’s perspectives on HPV vaccination, using the Information, Motivation and Behavioural Skills model as a framework	Moderate risk of bias (researcher and participant relationship not adequately considered and limited discussion of ethical considerations)
Grace (2018)	Semi-structured interviews Grounded theory	Canada	Private meeting rooms at the University of Toronto	HIV positive gay men. 24% had received the HPV vaccine. n = 25	To explore HPV vaccination barriers and hesitancy among HIV-positive gay men by examining participants’ narrative accounts of their knowledge, experiences and perceptions related to HPV and HPV vaccination.	Moderate risk of bias (researcher and participant relationship not adequately considered; process of researcher agreement not described and interrater reliability not reported)
Jaiswal (2020)	Semi-structured interviews Multi-step approach to identify salient themes	USA	Unclear	Young sexual minority men aged 24–27 years. n = 38	To elucidate the nature and depth of (a) HPV and HPV vaccine knowledge and (b) provider communication about HPV vaccine, in a diverse sample of young urban sexual minority men; and to understand barriers and facilitators to vaccination, the degree of vaccine literacy, and sources of vaccine knowledge in SMM.	Moderate risk of bias (researcher and participant relationship not adequately addressed; no critical examination of researcher’s own role or the potential for bias, and no discussion of the credibility of findings or inter-rater reliability.
Kesten (2019)	Focus groups Thematic analysis	UK	LGBTQ organisational settings and a University student’s union	Young men who have sex with men aged 16–24 years. 22% had received the HPV vaccine n = 17	To understand young MSM’s knowledge and attitudes towards HPV vaccination	Low risk of bias
Koskan (2018)	Semi-structured interviews Content analysis	USA	Settings of the participants’ choice (usually fast food restaurants)	HIV positive gay and bisexual men aged 18–60 years. 13% had received the first dose of the vaccine n = 15	To explore HIV-positive gay and bisexual men’s understanding of HPV and the HPV vaccine as well as preferences for future health promotion.	Low risk of bias
Nadarzynski (2017)	Focus groups and individual interviews Framework analysis	UK	Community-based LGBT venues and organisations	MSM aged 16–40 years. n = 33	To explore men who have sex with men’s’ perceptions of HPV and HPV vaccination prior to the introduction of the vaccination programme in the UK	Moderate risk of bias (researcher and participant relationship not adequately considered and some lack of rigour in data analysis)
Wheldon (2017)	Semi-structured interviews 5-step analysis process	USA	Participants recruited from Student Pride groups and sexual networking apps. Interviews conducted in person or via telephone.	HIV positive young MSM aged 18–26 years. 23% reported receiving at least one dose of the HPV vaccine. n = 22	To (1) describe salient beliefs related to HPV vaccination among young MSM (2) determine factors that underlie these beliefs; and (3) describe a model for HPV vaccine decision-making.	Moderate risk of bias (authors report the results are exploratory as they are based on a small and sociodemographically limited sample)

Table 6Summary of themes and sub-themes

Major theme	Sub-themes
Barriers to HPV vaccination	Lack of knowledge about HPV, HPV-related diseases, and the HPV vaccine
	Questions over vaccine effectiveness and potential side effects
	Vaccination series, including the timing of doses
	Possible stigma
	Clinical settings
Facilitators for HPV vaccination	Awareness of vaccines and the health benefits of getting vaccinated
	Interactions with health care practitioners (HCPs)
	Support from friends and family
	Making vaccination part of other clinical interactions
	Mobile applications

Table 7Summary of qualitative findings

Summary of review finding	Studies contributing to the review finding	Illustrative quotes	CERQual assessment	Explanation of GRADE-CERQual assessment
Barriers to HPV vaccination uptake
Lack of knowledge about HPV, HPV-related disease, and the HPV vaccine Participants lacked knowledge about HPV in general, transmission in MSM, and the connection between HPV and male associated cancers. Participants often confused HPV and other STIs. There was a pervasive lack of knowledge of HPV vaccination and misunderstandings about efficacy. There was a widespread perception that only cisgender women are vulnerable to HPV and that the vaccine was predominantly or exclusively for women.	Apaydin 2018 Fontenot 2016 Gerend 2019 Grace 2018 Jaiswal 2020 Kesten 2019 Koskan 2018 Nadarzynki 2017	“I’ve heard it doesn’t affect men, but they can transmit it. I don’t know if that’s true” “I didn’t know like it affected guys at all’ “My own ignorance was that HPV was something that really only affected women and cervical cancer. I didn’t hear it as much as something that was affecting men”	Moderate confidence	Downgraded because of moderate concerns regarding methodological limitations
Questions over vaccine effectiveness and potential side effects Participants wanted clear information about the vaccine and any possible side effects. They questioned vaccine effectiveness.	Fontenot 2016 Gerend 2019 Koskan 2018 Nadarzynki 2017 Wheldon 2017	“I think like with all vaccines there are probably side effects” “I mean I don’t know, I mean I’m not too fond of the guinea pig thing, so I don’t know, it depends on the side effects, I actually [inaudible] that. So I would have to see—weigh my options and see what my side effects are”	Moderate confidence	Downgraded because of moderate concerns regarding methodological limitations
Vaccination series, including timing of doses Participants discussed barriers to vaccine completion due to long time intervals between doses, multiple doses, and the inconvenience of work conflicting with clinic hours.	Apaydin 2018 Fontenot 2016 Kesten 2019	“... you know, going back and coming back, and going back [for 3 doses], it’s too much work” “It is hard to remember, keep track of all the shots”	Low confidence	Downgraded because of moderate concerns regarding methodological limitations and adequacy of data
Possible stigma Participants described stigmas related to being gay, gay health, acceptance by HCPs, and STIs. They were concerned that they would be labelled as promiscuous upon receiving vaccination. Participants described the importance of being able to discuss sexual activity with healthcare professionals. There was some concern about people being singled out by their sexuality when offering the vaccine	Fontenot 2016 Gerend 2019 Koskan 2018 Nadarzynki 2017 Wheldon 2017	“If there’s another virus, like HPV, it’s going to be strongly linked to gay men community again. I don’t think it’s a good thing for people because it will strengthen the gay label to this specific disease. I don’t think people will like it. Since they just got rid of HIV labels and they don’t want another stigma again” “...some people will consider somebody very promiscuous. ‘Oh, you’re getting a vaccine because you’re sleeping with multiple people,’ and there’s just a stigma associated with that”	Moderate confidence	Downgraded because of moderate concerns regarding methodological limitations
Clinical settings Most participants suggested sexual health clinics were the most suitable setting to reach MSM as the openness and non-judgemental attitudes of staff in these settings may be reassuring. Others suggested GP practices may be more suitable as some young men do not access specialist sexual health services. Some noted the difficulty of discussing their sexual health with their GPs. Participants also noted that MSM who do not identify as gay may not benefit from the vaccine if it was only targeted to gay or bisexual men.	Gerend 2019 Nadarzynki 2017 Wheldon 2017	“Well, it was offered right there while I was getting the physical done. So I didn’t even have to make a special trip or anything” “I feel like he judges me. I feel like if I had a provider or somebody who is a little more open-minded ... my doctor is a staunch Republican, white dude who is like 65 and I’m sitting there like a gay little Puerto Rican kid, and you know, it is just always awkward when I go to my doctor. We come from opposite ends of the earth”	Low confidence	Downgraded because of moderate concerns regarding methodological limitations and minor concerns regarding adequacy of data
Facilitators for increasing HPV vaccination uptake
Awareness of vaccines and the health benefits of getting vaccinated Participants acknowledged the physical and psychological health benefits of HPV vaccination, including reducing their anxieties around the risk of anal cancer. Participants were also enthusiastic about the ability to protect both themselves and their partner(s). They discussed the need to raise public awareness and suggested that HPV education should be widespread and more inclusive of all sexes, particularly in the sexual health education curriculum in schools. Participants believed that better understanding of the benefits and side effects of the vaccine would encourage uptake.	Apaydin 2018 Fontenot 2016 Gerend 2019 Grace 2018 Kesten 2019 Koskan 2018 Nadarzynki 2017 Wheldon 2017	“Yeah like I care about my health, but I also care about other people’s health too and I don’t want anyone else to get infected or have to go through with something like that” “I would be less susceptible to anal cancer at least from HPV” “If there was more education about it [HPV vaccine], if the vaccine gets known, then it won’t be as taboo”	Moderate confidence	Downgraded because of moderate concerns regarding methodological limitations
Interactions with health care practitioners (HCPs) Participants perceived healthcare providers and doctors to be the most trusted source of information, and their opinions as well as recommendations would substantially influence their decision to obtain the vaccine	Fontenot 2016 Gerend 2019 Grace 2018 Kesten 2019 Koskan 2018 Nadarzynki 2017 Wheldon 2017	“I think I’d be more likely to accept it if it were offered than I would actively request it. I think because if it was, if it was recommended to you it would be coming from a trusted source” “It was the doctor’s recommendation. I honestly wouldn’t have thought about it had he not recommended it”	Moderate confidence	Downgraded because of moderate concerns regarding methodological limitations
Support from friends and family Participants highlighted that most people in their lives would be supportive of their decision to get vaccinated. Participants with unsupportive referents typically mentioned their parents (notably their father). Awareness and knowledge about the HPV vaccine was primarily limited to female friends and siblings.	Apaydin 2018 Fontenot 2016	“Maybe my dad. Because he’s just ignorant with regard to sexuality and vaccines and stuff like that. He’s kind of a anti-government conspiracies person, so I don’t really have a good relationship with him” “I know [about HPV vaccine] because my sister got it”	Very low confidence	Downgraded because of moderate concerns regarding methodological limitations and serious concerns about adequacy
Making HPV vaccination part of other clinical interactions Participants described wanting to combine HPV vaccination with other types of visits like annual physical examinations, general sexual health screening or other STI tests.	Fontenot 2016 Gerend 2019 Nadarzynki 2017	“Well, it was offered right there while I was getting the physical done. So I didn’t even have to make a special trip or anything” “If they start routinely testing for this at GUM clinics, and you’re negative and not carrying it, then it should be suggested to you at the same point [like] they would suggest a hepatitis A and C vaccine”	Low confidence	Downgraded because of moderate concerns regarding methodological limitations and adequacy of data
Mobile applications Participants suggested the use of mobile applications for booking appointments and creating a reminder system. They felt that flexibility in scheduling and app-based reminder systems would facilitate 3-dose completion	Fontenot, 2016	“People are uncomfortable having to make phone calls. . . it’s a lot easier to just do something on your phone, like an appointment confirmation, so that you can go in without having to talk [to someone] or feel uncomfortable disclosing things [on the phone]”	Very low confidence	Downgraded because of moderate concerns regarding methodological limitations and very serious concerns regarding adequacy

Study	Reason
Abara, Winston E, Qaseem, Amir, Schillie, Sarah et al (2017) Hepatitis B Vaccination, Screening, and Linkage to Care: Best Practice Advice From the American College of Physicians and the Centers for Disease Control and Prevention. Annals of internal medicine 167(11): 794–804 [PubMed: 29159414]	- Literature review. References checked for eligible studies
Agenor, M., Murchison, G.R., Chen, J.T. et al (2019) Impact of the Affordable Care Act on human papillomavirus vaccination initiation among lesbian, bisexual, and heterosexual U.S. women. Health services research [PMC free article: PMC6980951] [PubMed: 31709542]	- Surveillance data. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Alberts, C.J., Boyd, A., Bruisten, S.M. et al (2019) Hepatitis A incidence, seroprevalence, and vaccination decision among MSM in Amsterdam, the Netherlands. Vaccine 37(21): 2849–2856 [PubMed: 30992222]	- Surveillance data. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Allen-Leigh, B., Rivera-Rivera, L., Yunes-Diaz, E. et al (2019) Uptake of the HPV vaccine among people with and without HIV, cisgender and transgender women and men who have sex with men and with women at two sexual health clinics in Mexico City. Human Vaccines and Immunotherapeutics [PMC free article: PMC7227620] [PubMed: 31657665]	- Cross-sectional study. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Anderson, Jonathan S, Hoy, Jennifer, Hillman, Richard et al (2009) A randomized, placebo-controlled, dose-escalation study to determine the safety, tolerability, and immunogenicity of an HPV-16 therapeutic vaccine in HIV-positive participants with oncogenic HPV infection of the anus. Journal of acquired immune deficiency syndromes (1999) 52(3): 371–81 [PubMed: 19661810]	- RCT. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Apaydin, Kaan Z, Fontenot, Holly B, Shtasel, Derri L et al (2018) Primary care provider practices and perceptions regarding HPV vaccination and anal cancer screening at a Boston community health center. Journal of Community Health: The Publication for Health Promotion and Disease Prevention 43(4): 792–801 [PMC free article: PMC6033675] [PubMed: 29480339]	- Survey to evaluate primary care providers’ HPV vaccination and anal cancer screening practices and perceptions. Data not usable
Bhagey, A, Foster, K, Ralph, S et al (2018) High prevalence of anti-hepatitis A IgG in a cohort of UK HIV-negative men who have sex with men: implications for local hepatitis A vaccine policy. International journal of STD & AIDS 29(10): 1007–1010 [PubMed: 29743003]	- Cohort. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Bonafide, K.E. and Vanable, P.A. (2015) Male human papillomavirus vaccine acceptance is enhanced by a brief intervention that emphasizes both male-specific vaccine benefits and altruistic motives. Sexually Transmitted Diseases 42(2): 76–80 [PubMed: 25585065]	- Cross-sectional study on perceptions of HPV and acceptability of vaccine
Budenz, A., Klassen, A., Leader, A. et al (2019) HPV vaccine, Twitter, and gay, bisexual and other men who have sex with men. Health promotion international [PubMed: 31006017]	- Cross-sectional study. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Burrell, S., Vodstrcil, L.A., Fairley, C.K. et al (2019) Hepatitis A vaccine uptake among men who have sex with men from a time-limited vaccination programme in Melbourne in 2018. Sexually Transmitted Infections [PubMed: 31346067]	- Retrospective cohort study. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Calin, Ruxandra, Massari, Veronique, Pialoux, Gilles et al (2020) Acceptability of on-site rapid HIV/HBV/HCV testing and HBV vaccination among three at-risk populations in distinct community-healthcare outreach centres: the ANRS-SHS 154 CUBE study. BMC infectious diseases 20(1): 851 [PMC free article: PMC7670674] [PubMed: 33198672]	- Cohort. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Charlton, Brittany M, Reisner, Sari L, Agenor, Madina et al (2017) Sexual Orientation Disparities in Human Papillomavirus Vaccination in a Longitudinal Cohort of U.S. Males and Females. LGBT health 4(3): 202–209 [PMC free article: PMC5485217] [PubMed: 28467238]	- Prospective cohort. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM. Mixed population
Checchi, M., Mesher, D., McCall, M. et al (2019) HPV vaccination of gay, bisexual and other men who have sex with men in sexual health and HIV clinics in England: Vaccination uptake and attendances during the pilot phase. Sexually Transmitted Infections 95(8): 608–613 [PubMed: 31028226]	- Surveillance data. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Cummings, Teresa, Kasting, Monica L, Rosenberger, Joshua G et al (2015) Catching Up or Missing Out? Human Papillomavirus Vaccine Acceptability Among 18- to 26-Year-old Men Who Have Sex With Men in a US National Sample. Sexually transmitted diseases 42(11): 601–6 [PubMed: 26462183]	- Cross-sectional study on HPV vaccine acceptability
Edmiston, E.K., Donald, C.A., Sattler, A.R. et al (2016) Opportunities and Gaps in Primary Care Preventative Health Services for Transgender Patients: A Systematic Review. Transgender Health 1(1): 216–230 [PMC free article: PMC5367473] [PubMed: 28861536]	- Systematic review. References checked for eligible studies
Farfour, Eric, Lesprit, Philippe, Chan Hew Wai, Aurelie et al (2018) Acute hepatitis A breakthrough in MSM in Paris area: implementation of targeted hepatitis A virus vaccine in a context of vaccine shortage. AIDS (London, England) 32(4): 531–532 [PubMed: 29381561]	- Editorial. References checked for eligible studies
FitzGerald, S.M.; Savage, E.B.; Hegarty, J.M. (2014) The Human Papillomavirus: Men’s Attitudes and Beliefs Toward the HPV Vaccination and Condom Use in Cancer Prevention. Journal of Men’s Health 11(3): 121–129	- Qualitative. Not on MSM
Fontenot, H.B., Rosenberger, J.G., McNair, K.T. et al (2019) Perspectives and preferences for a mobile health tool designed to facilitate HPV vaccination among young men who have sex with men. Human Vaccines and Immunotherapeutics 15(78): 1815–1823 [PMC free article: PMC6746522] [PubMed: 30625049]	- Qualitative. Not on barriers and facilitators
Fontenot, Holly B, White, Bradley Patrick, Rosenberger, Joshua G et al (2020) Mobile App Strategy to Facilitate Human Papillomavirus Vaccination Among Young Men Who Have Sex With Men: Pilot Intervention Study. Journal of medical Internet research 22(11): e22878 [PMC free article: PMC7673982] [PubMed: 33146621]	- Cohort. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Forster, A.S. and Gilson, R. (2019) Challenges to optimising uptake and delivery of a HPV vaccination programme for men who have sex with men. Human Vaccines and Immunotherapeutics 15(78): 1541–1543 [PMC free article: PMC6746470] [PubMed: 30570380]	- Editorial. References checked for eligible studies
Galea, Jerome T, Monsour, Emmi, Nurena, Cesar R et al (2017) HPV vaccine knowledge and acceptability among Peruvian men who have sex with men and transgender women: A pilot, qualitative study. PloS one 12(2): e0172964 [PMC free article: PMC5330512] [PubMed: 28245234]	- Non-OECD country
Gerend, Mary A, Madkins, Krystal, Phillips, Gregory 2nd et al (2016) Predictors of Human Papillomavirus Vaccination Among Young Men Who Have Sex With Men. Sexually transmitted diseases 43(3): 185–91 [PMC free article: PMC4748724] [PubMed: 26859806]	- Cross-sectional study on predictors of HPV vaccine
Gilbert, L K, Levandowski, B A, Scanlon, K E et al (2010) A comparison of hepatitis A and hepatitis B measures among vaccinated and susceptible online men who have sex with men. International journal of STD & AIDS 21(6): 400–5 [PubMed: 20606219]	- Surveillance data. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Gilbert, Paul A; Brewer, Noel T; Reiter, Paul L (2011) Association of human papillomavirus-related knowledge, attitudes, and beliefs with HIV status: a national study of gay men. Journal of lower genital tract disease 15(2): 83–8 [PMC free article: PMC4024215] [PubMed: 21169868]	- Cross-sectional study on knowledge, attitudes and beliefs
Gilbert, Paul, Brewer, Noel T, Reiter, Paul L et al (2011) HPV vaccine acceptability in heterosexual, gay, and bisexual men. American journal of men’s health 5(4): 297–305 [PMC free article: PMC3083462] [PubMed: 20798149]	- Cross-sectional study on HPV vaccine acceptability
Giuliani, Massimo, Vescio, Maria Fenicia, Dona, Maria Gabriella et al (2016) Perceptions of Human Papillomavirus (HPV) infection and acceptability of HPV vaccine among men attending a sexual health clinic differ according to sexual orientation. Human vaccines & immunotherapeutics 12(6): 1542–50 [PMC free article: PMC4964721] [PubMed: 26752151]	- Cross-sectional study on perceptions of HPV and acceptability of vaccine
Gorbach, Pamina M, Cook, Ryan, Gratzer, Beau et al (2017) Human Papillomavirus Vaccination Among Young Men Who Have Sex With Men and Transgender Women in 2 US Cities, 2012–2014. Sexually transmitted diseases 44(7): 436–441 [PMC free article: PMC5553567] [PubMed: 28608795]	- Cross-sectional study. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Grace, D., Gaspar, M., Rosenes, R. et al (2019) Economic barriers, evidentiary gaps, and ethical conundrums: A qualitative study of physicians’ challenges recommending HPV vaccination to older gay, bisexual, and other men who have sex with men. International Journal for Equity in Health 18(1): 159 [PMC free article: PMC6798383] [PubMed: 31623613]	- Study on cost implications of recommending HPV vaccination. Not addressing UK context
Groene, E.A., Mohammed, I., Horvath, K. et al (2019) Online media scans: Applying systematic review techniques to assess statewide human papillomavirus vaccination activities. Journal of Public Health Research 8(2): 1623 [PMC free article: PMC6747022] [PubMed: 31572697]	- Literature review. References checked for eligible studies
Gutierrez, Baudelio Jr, Leung, Anthony, Jones, Kevin Trimell et al (2013) Acceptability of the human papillomavirus vaccine among urban adolescent males. American journal of men’s health 7(1): 27–36 [PubMed: 22892147]	- Mixed smaple of MSM and heterosexual men. Themes were not reported separately by group (although quotes for each theme were provided separately)
Halkitis, Perry N, Valera, Pamela, LoSchiavo, Caleb E et al (2019) Human Papillomavirus Vaccination and Infection in Young Sexual Minority Men: The P18 Cohort Study. AIDS patient care and STDs 33(4): 149–156 [PMC free article: PMC6459271] [PubMed: 30932696]	- Cohort. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Hernandez, Brenda Y, Wilkens, Lynne R, Thompson, Pamela J et al (2010) Acceptability of prophylactic human papillomavirus vaccination among adult men. Human vaccines 6(6): 467–75 [PMC free article: PMC3010761] [PubMed: 20671442]	- Prospective cohort. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM. Mixed population
Hidalgo-Tenorio, Carmen, Ramirez-Taboada, Jessica, Gil-Anguita, Concepcion et al (2017) Safety and immunogenicity of the quadrivalent human papillomavirus (qHPV) vaccine in HIV-positive Spanish men who have sex with men (MSM). AIDS research and therapy 14: 34 [PMC free article: PMC5516305] [PubMed: 28720147]	- RCT. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Hoefer, Lea, Tsikis, Savas, Bethimoutis, George et al (2018) HPV vaccine acceptability in high-risk Greek men. Human vaccines & immunotherapeutics 14(1): 134–139 [PMC free article: PMC5791582] [PubMed: 28937851]	- Cross-sectional study on perceptions of HPV and acceptability of vaccine
Hoover, Karen W, Butler, Mary, Workowski, Kimberly A et al (2012) Low rates of hepatitis screening and vaccination of HIV-infected MSM in HIV clinics. Sexually transmitted diseases 39(5): 349–53 [PubMed: 22504597]	- Surveillance data. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Houlihan, Catherine F, Larke, Natasha L, Watson-Jones, Deborah et al (2012) Human papillomavirus infection and increased risk of HIV acquisition. A systematic review and meta-analysis. AIDS (London, England) 26(17): 2211–22 [PMC free article: PMC3831022] [PubMed: 22874522]	- Systematic review. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM. References checked for eligible studies
Jones, Jeff, Parrish, Adam, Collins, Tom et al (2016) HPV vaccine acceptance among a sample of Southern men who have sex with men: A comparison of younger and older men. Journal of Gay & Lesbian Social Services: The Quarterly Journal of Community & Clinical Practice 28(3): 245–254	- Survey on vaccine acceptability. Not qualitative
Juvet, Lene K., Saeterdal, Ingvil, Couto, Elisabeth et al (2015) No title provided.	- Editorial. References checked for eligible studies
King, Eleanor M, Oomeer, Soonita, Gilson, Richard et al (2016) Oral Human Papillomavirus Infection in Men Who Have Sex with Men: A Systematic Review and Meta-Analysis. PloS one 11(7): e0157976 [PMC free article: PMC4934925] [PubMed: 27384050]	- Systematic review. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM. References checked for eligible studies
Kourkounti, Sofia, Paparizos, Vassilios, Leuow, Kirsten et al (2015) Adherence to hepatitis A virus vaccination in HIV-infected men who have sex with men. International journal of STD & AIDS 26(12): 852–6 [PubMed: 25411352]	- Retrospective cohort study. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Lau, Joseph T F, Wang, Zixin, Kim, Jean H et al (2013) Acceptability of HPV vaccines and associations with perceptions related to HPV and HPV vaccines among men who have sex with men in Hong Kong. PloS one 8(2): e57204 [PMC free article: PMC3579800] [PubMed: 23451188]	- Non-OECD country
Lee Mortensen, Gitte and Larsen, Helle K (2010) Quality of life of homosexual males with genital warts: a qualitative study. BMC research notes 3: 280 [PMC free article: PMC2989980] [PubMed: 21050431]	- Qualitative. Not on barriers and facilitators
Lee, S.-J., Newman, P.A., Duan, N. et al (2014) Development of an HIV vaccine attitudes scale to predict HIV vaccine acceptability among vulnerable populations: L.A. VOICES. Vaccine 32(39): 5013–5018 [PMC free article: PMC4137321] [PubMed: 25045817]	- Surveillance data. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Looker, Katharine J, Ronn, Minttu M, Brock, Patrick M et al (2018) Evidence of synergistic relationships between HIV and Human Papillomavirus (HPV): systematic reviews and meta-analyses of longitudinal studies of HPV acquisition and clearance by HIV status, and of HIV acquisition by HPV status. Journal of the International AIDS Society 21(6): e25110 [PMC free article: PMC5989783] [PubMed: 29873885]	- Systematic review. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM. References checked for eligible studies
Lott, Breanne E., Okusanya, Babasola O., Anderson, Elizabeth J. et al (2020) Interventions to increase uptake of Human Papillomavirus (HPV) vaccination in minority populations: A systematic review. Preventive Medicine Reports 19: 101163 [PMC free article: PMC7372149] [PubMed: 32714778]	- Literature review. References checked for eligible studies
Mangen, M-J J, Stibbe, H, Urbanus, A et al (2017) Targeted outreach hepatitis B vaccination program in high-risk adults: The fundamental challenge of the last mile. Vaccine 35(24): 3215–3221 [PubMed: 28483198]	- Study on cost analysis
Marra, E, Alberts, C J, Zimet, G D et al (2016) HPV vaccination intention among male clients of a large STI outpatient clinic in Amsterdam, the Netherlands. Papillomavirus research (Amsterdam, Netherlands) 2: 178–184 [PMC free article: PMC5886895] [PubMed: 29074179]	- Cross-sectional study. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
McGrath, Launcelot, Fairley, Christopher K, Cleere, Eoin F et al (2019) Human papillomavirus vaccine uptake among young gay and bisexual men who have sex with men with a time-limited targeted vaccination programme through sexual health clinics in Melbourne in 2017. Sexually transmitted infections 95(3): 181–186 [PubMed: 30409917]	- Retrospective cohort study. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
McIver, R., Dyda, A., Knight, V. et al (2015) Hepatitis B screening and vaccination: How does a Sexual Health service measure up?. Sexual Health 12(5): 458–459 [PubMed: 26145389]	- Editorial. References checked for eligible studies
McIver, Ruthy, Dyda, Amalie, McNulty, Anna M et al (2016) Text message reminders do not improve hepatitis B vaccination rates in an Australian sexual health setting. Journal of the American Medical Informatics Association : JAMIA 23(e1): e88–92 [PMC free article: PMC4954625] [PubMed: 26499103]	- Before and after study design on text message reminders to improve Hep B vaccination. Not a controlled trial
McRee, Annie-Laurie, Reiter, Paul L, Chantala, Kim et al (2010) Does framing human papillomavirus vaccine as preventing cancer in men increase vaccine acceptability?. Cancer epidemiology, biomarkers & prevention : a publication of the American Association for Cancer Research, cosponsored by the American Society of Preventive Oncology 19(8): 1937–44 [PMC free article: PMC2919615] [PubMed: 20647398]	- Survey on vaccine acceptability. Not qualitative
McRee, Annie-Laurie, Shoben, Abigail, Bauermeister, Jose A et al (2018) Outsmart HPV: Acceptability and short-term effects of a web-based HPV vaccination intervention for young adult gay and bisexual men. Vaccine 36(52): 8158–8164 [PMC free article: PMC6039283] [PubMed: 29331245]	- RCT. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Meites, Elissa, Markowitz, Lauri E, Paz-Bailey, Gabriela et al (2014) HPV vaccine coverage among men who have sex with men - National HIV Behavioral Surveillance System, United States, 2011. Vaccine 32(48): 6356–9 [PMC free article: PMC4677785] [PubMed: 25258097]	- Cross-sectional study. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Metheny, Nicholas and Stephenson, Rob (2016) Disclosure of Sexual Orientation and Uptake of HIV Testing and Hepatitis Vaccination for Rural Men Who Have Sex With Men. Annals of family medicine 14(2): 155–8 [PMC free article: PMC4781519] [PubMed: 26951591]	- Surveillance data. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Nadarzynski, Tom, Miller, Danny, Frost, Miles et al (2021) Vaccine acceptability, uptake and completion amongst men who have sex with men: A systematic review, meta-analysis and theoretical framework. Vaccine [PubMed: 34034949]	- Literature review. References checked for eligible studies
Nadarzynski, Tom, Smith, Helen E, Richardson, Daniel et al (2015) Sexual healthcare professionals’ views on HPV vaccination for men in the UK. British journal of cancer 113(11): 1599–601 [PMC free article: PMC4705902] [PubMed: 26575602]	- Survey on health professionals’ views. Not qualitative
Nadarzynski, Tom, Smith, Helen, Richardson, Daniel et al (2018) Men who have sex with men who do not access sexual health clinics nor disclose sexual orientation are unlikely to receive the HPV vaccine in the UK. Vaccine 36(33): 5065–5070 [PubMed: 30037419]	- Cross-sectional study. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Nadarzynski, Tom, Smith, Helen, Richardson, Daniel et al (2014) Human papillomavirus and vaccine-related perceptions among men who have sex with men: a systematic review. Sexually transmitted infections 90(7): 515–23 [PubMed: 24787367]	- Systematic review. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM. References checked for eligible studies
Oliver, Sara E, Hoots, Brooke E, Paz-Bailey, Gabriela et al (2017) Increasing Human Papillomavirus Vaccine Coverage Among Men Who Have Sex With Men-National HIV Behavioral Surveillance, United States, 2014. Journal of acquired immune deficiency syndromes (1999) 75suppl3: 370–s374 [PMC free article: PMC5761322] [PubMed: 28604441]	- Surveillance data. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Onyeabor, Onyekachi S, Martin, Nicolle, Orish, Verner N et al (2015) Awareness of Human Papillomavirus Vaccine Among Adolescent African American Males Who Have Sex with Males: a Pilot Study. Journal of racial and ethnic health disparities 2(3): 290–4 [PubMed: 26863459]	- Qualitative themes not reported
Ou, Lihong and Youngstedt, Shawn D. (2020) The Role of Vaccination Interventions to Promote HPV Vaccine Uptake Rates in a College-Aged Population: a Systematic Review. Journal of cancer education : the official journal of the American Association for Cancer Education [PMC free article: PMC8986718] [PubMed: 32564253]	- Literature review. References checked for eligible studies
Palefsky, Joel M, Giuliano, Anna R, Goldstone, Stephen et al (2011) HPV vaccine against anal HPV infection and anal intraepithelial neoplasia. The New England journal of medicine 365(17): 1576–85 [PubMed: 22029979]	- RCT. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Pollock, K.G., Wallace, L.A., Wrigglesworth, S. et al (2019) HPV vaccine uptake in men who have sex with men in Scotland. Vaccine 37(37): 5513–5514 [PubMed: 30545714]	- Editorial. References checked for eligible studies
Rank, Claudia, Gilbert, Mark, Ogilvie, Gina et al (2012) Acceptability of human papillomavirus vaccination and sexual experience prior to disclosure to health care providers among men who have sex with men in Vancouver, Canada: implications for targeted vaccination programs. Vaccine 30(39): 5755–60 [PubMed: 22796376]	- Survey on vaccine acceptability. Not qualitative
Reiter, P.L., McRee, A.-L., Katz, M.L. et al (2015) Human papillomavirus vaccination among young adult gay and bisexual men in the United States. American Journal of Public Health 105(1): 96–102 [PMC free article: PMC4265907] [PubMed: 25393178]	- Survey on vaccine acceptability. Not qualitative
Reiter, Paul L, Brewer, Noel T, McRee, Annie-Laurie et al (2010) Acceptability of HPV vaccine among a national sample of gay and bisexual men. Sexually transmitted diseases 37(3): 197–203 [PMC free article: PMC4018212] [PubMed: 20118831]	- Survey on vaccine acceptability. Not qualitative
Reiter, Paul L, Katz, Mira L, Bauermeister, Jose A et al (2017) Recruiting Young Gay and Bisexual Men for a Human Papillomavirus Vaccination Intervention Through Social Media: The Effects of Advertisement Content. JMIR public health and surveillance 3(2): e33 [PMC free article: PMC5473946] [PubMed: 28576758]	- Editorial. Main study included
Sadlier, C, Lynam, A, O’Dea, S et al (2016) HPV vaccine acceptability in HIV-infected and HIV negative men who have sex with men (MSM) in Ireland. Human vaccines & immunotherapeutics 12(6): 1536–41 [PMC free article: PMC4964722] [PubMed: 27153289]	- Survey on vaccine acceptability. Not qualitative
Said, A. and Jou, J.H. (2014) Hepatitis B vaccination and screening awareness in primary care practitioners. Hepatitis Research and Treatment 2014: 373212 [PMC free article: PMC3963216] [PubMed: 24729872]	- Survey on health professionals’ views. Not qualitative
Sauvageau, Chantal and Dufour-Turbis, Christine (2016) HPV vaccination for MSM: Synthesis of the evidence and recommendations from the Quebec Immunization Committee. Human vaccines & immunotherapeutics 12(6): 1560–5 [PMC free article: PMC4964716] [PubMed: 26554731]	- Literature review. References checked for eligible studies
Segura, Marcela, Bautista, Christian T, Marone, Ruben et al (2010) HIV/STI co-infections, syphilis incidence, and hepatitis B vaccination: the Buenos Aires cohort of men who have sex with men. AIDS care 22(12): 1459–65 [PubMed: 21154033]	- Non-OECD country
Seto, Katherine, Marra, Fawziah, Raymakers, Adam et al (2012) The cost effectiveness of human papillomavirus vaccines: a systematic review. Drugs 72(5): 715–43 [PubMed: 22413761]	- Systematic review. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM. References checked for eligible studies
Siconolfi, Daniel E; Halkitis, Perry N; Rogers, Meighan E (2009) Hepatitis vaccination and infection among gay, bisexual, and other men who have sex with men who attend gyms in New York City. American journal of men’s health 3(2): 141–9 [PubMed: 19477727]	- Cross-sectional study. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Solomon, Marc M, Schechter, Mauro, Liu, Albert Y et al (2016) The Safety of Tenofovir-Emtricitabine for HIV Pre-Exposure Prophylaxis (PrEP) in Individuals With Active Hepatitis B. Journal of acquired immune deficiency syndromes (1999) 71(3): 281–6 [PMC free article: PMC4752387] [PubMed: 26413853]	- RCT. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Storholm, Erik David, Fisher, Dennis G, Reynolds, Grace L et al (2010) Hepatitis vaccination of men who have sex with men at gay pride events. Prevention Science 11(2): 219–227 [PMC free article: PMC2858271] [PubMed: 20049541]	- Prospective cohort. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM. Mixed population
Tsachouridou, O., Georgiou, A., Naoum, S. et al (2019) Factors associated with poor adherence to vaccination against hepatitis viruses, streptococcus pneumoniae and seasonal influenza in HIV-infected adults. Human Vaccines and Immunotherapeutics 15(2): 295–304 [PMC free article: PMC6422462] [PubMed: 30111224]	- Before and after study design on adherence. Not controlled trial
Vet, R; de Wit, J B F; Das, E (2011) The efficacy of social role models to increase motivation to obtain vaccination against hepatitis B among men who have sex with men. Health education research 26(2): 192–200 [PubMed: 21106651]	- RCT. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Vet, Raymond; de Wit, John Bf; Das, Enny (2017) Factors associated with hepatitis B vaccination among men who have sex with men: a systematic review of published research. International journal of STD & AIDS 28(6): 534–542 [PubMed: 26503555]	- Systematic review. References checked for eligible studies
Wheldon, Christopher W (2016) HPV vaccine decision-making among male sexual minorities: An integrative theoretical framework for vaccine promotion. Dissertation Abstracts International: Section B: The Sciences and Engineering 76(9be): no-specified	- Dissertation. References checked for eligible studies
Wheldon, Christopher W, Daley, Ellen M, Buhi, Eric R et al (2011) Health beliefs and attitudes associated with HPV vaccine intention among young gay and bisexual men in the Southeastern United States. Vaccine 29(45): 8060–5 [PubMed: 21864615]	- Cross-sectional study. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Wheldon, Christopher W, Sutton, Steven K, Fontenot, Holly B et al (2018) Physician Communication Practices as a Barrier to Risk-Based HPV Vaccine Uptake Among Men Who Have Sex with Men. Journal of cancer education : the official journal of the American Association for Cancer Education 33(5): 1126–1131 [PMC free article: PMC7771357] [PubMed: 28456947]	- Cross-sectional study. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Wielding, Sally; Ma, Andy Hy; Clutterbuck, Daniel J (2016) An audit of hepatitis B vaccination for men who have sex with men before and after sexual health service integration. International journal of STD & AIDS 27(10): 898–900 [PubMed: 26738517]	- Audit. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Xavier, Martinez-Gomez, Curran, A., Campins, M. et al (2019) Multidisciplinary, evidence-based consensus guidelines for human papillomavirus (HPV) vaccination in highrisk populations, Spain, 2016. Eurosurveillance 24(7): 1700857 [PMC free article: PMC6381660] [PubMed: 30782268]	- Guideline on the effects of HPV vaccination on high risk groups
Zhang, S X, Shoptaw, S, Reback, C J et al (2018) Cost-effective way to reduce stimulant-abuse among gay/bisexual men and transgender women: a randomized clinical trial with a cost comparison. Public health 154: 151–160 [PMC free article: PMC5745270] [PubMed: 29245022]	- RCT. Not on interventions to increase uptake of Hepatitis A, Hepatitis B and HPV vaccinations in MSM
Zou, Huachun, Grulich, Andrew E, Cornall, Alyssa M et al (2014) How very young men who have sex with men view vaccination against human papillomavirus. Vaccine 32(31): 3936–41 [PubMed: 24852719]	- Survey on vaccine acceptability. Not qualitative

Increasing uptake of hepatitis A, hepatitis B and human papillomavirus (HPV) vaccinations in gay, bisexual and other men who have sex with men

1. Effective and cost-effective interventions to increase uptake of vaccinations in men who have sex with men

1.1. Review question

1.1.1. Introduction

1.1.2. Summary of the protocol

1.1.3. Methods and process

1.1.4. Identification of public health evidence

1.1.4.1. Included studies

1.1.4.2. Excluded studies

1.1.5. Summary of the effectiveness evidence

1.1.6. Economic evidence

1.1.7. Economic model

2. Barriers and facilitators for increasing uptake of vaccines in men who have sex with men

2.1. Review question

2.1.1. Introduction

2.1.2. Summary of the protocol

2.1.3. Methods and process

2.1.4. Identification of qualitative evidence

2.1.4.1. Included studies

2.1.4.2. Excluded studies

2.1.5. Summary of the qualitative evidence

2.1.5.1. Summary of themes and sub-themes

2.1.5.2. Summary of qualitative findings

3. Integration and discussion of the evidence

3.1. Mixed methods integration

Are the results/findings from individual syntheses supportive or contradictory?

Does the qualitative evidence explain why the intervention is/is not effective?

Does the qualitative evidence explain differences in the direction and size of effect across the included quantitative studies?

Which aspects of the quantitative evidence were/were not explored in the qualitative studies?

Which aspects of the qualitative evidence were/were not tested in the quantitative studies?

3.2. The committee's discussion and interpretation of the evidence

3.2.1. The outcomes that matter most

3.2.2. The quality of the evidence

Quantitative evidence

Qualitative evidence

3.2.3. Benefits and harms

3.2.4. Cost effectiveness and resource use

3.3. Recommendations supported by this evidence review

3.4. References – included studies

3.4.1. Effectiveness

3.4.2. Qualitative

Appendices

Appendix A. Review protocols

Appendix B. Literature search strategies

Appendix C. Public health evidence study selection

Appendix D. Effectiveness evidence

D.1. Quantitative evidence

D.2. Qualitative evidence

Appendix E. Forest plots

Appendix F. GRADE tables

F.1. Human papillomavirus vaccination initiation

F.2. Human papillomavirus vaccination completion

F.3. Hepatitis B vaccination uptake

Appendix G. GRADE CERQual Tables

G.1. CERQual: Barriers to increasing uptake of HPV vaccinations in gay, bisexual and other men who have sex with men (MSM)

G.2. CERQual: Facilitators for increasing uptake of HPV vaccinations in gay, bisexual and other men who have sex with men (MSM)

Appendix H. Economic evidence study selection

Appendix I. Economic evidence tables

Appendix J. Health economic model

Appendix K. Excluded studies

Appendix L. Research recommendations – full details

L.1. Research recommendation

L.2. Why this is important

L.3. Rationale for research recommendation

Footnotes

Table 1PICO inclusion criteria

Table 2Summary of studies included in the effectiveness evidence review

Table 3Summary of findings table

Table 4PICO inclusion criteria

Table 5Summary of studies included in the qualitative evidence review

Table 6Summary of themes and sub-themes

Table 7Summary of qualitative findings