Measuring experience

National Guideline Alliance (UK)

Measuring experience

Babies, children and young people’s experience of healthcare

Evidence review L

NICE Guideline, No. 204

Authors

National Guideline Alliance (UK).

London: National Institute for Health and Care Excellence (NICE); 2021 Aug.

ISBN-13: 978-1-4731-4231-2

Measuring experience

Review question

How can the experience of babies, children and young people be measured so as to improve their experience of healthcare?

Introduction

In order to provide a good experience of healthcare, and to continually improve that experience, healthcare services need to review the experiences of those using the services, and act on this feedback to make changes to their services. Babies, children and young people may have different needs, experiences and perceptions of healthcare services compared to adults and it is important to capture these, and not rely on feedback from an adult-only population. In addition, babies, children and young people may have specific needs with respect to the methods used to obtain this feedback.

The aim of this question is to determine the best ways to measure the healthcare experience of babies, children and young people.

Summary of the protocol

See Table 1 for a summary of the Population, Intervention, Comparison and Outcome (PICO) characteristics of this review.

Table 1

Summary of the protocol (PICO table).

For further details see the review protocol in appendix A.

Methods and process

This evidence review was developed using the methods and process described in Developing NICE guidelines: the manual. Methods for this review question are described in the review protocol in appendix A and the methods supplement.

Clinical evidence

Included studies

This was a quantitative review with the aim of:

Determining if there is evidence to support the use of a particular method of collecting feedback on the healthcare experience of babies, children and young people.

A systematic review of the literature was conducted. One study (Horn 2010), a cluster-randomised controlled trial (RCT) was included in this review. This study compared different formats of questionnaire administration in 3 Child and Adolescent Mental Health Service (CAMHS) teams.

The included study is summarised in Table 2.

See the literature search strategy in appendix B and study selection flow chart in appendix C.

Excluded studies

Studies not included in this review are listed, and reasons for their exclusion are provided in appendix K.

Summary of studies included in the evidence review

A summary of the study included in this review is presented in Table 2.

Table 2

Summary of included study.

See the full evidence tables in appendix D. No meta-analysis was conducted (and so there are no forest plots in appendix E).

Summary of the evidence

Evidence was found for 1 of the pre-defined critical outcomes set out in the protocol: response rate. No evidence was found for acceptability to respondent, mode effect or time taken to complete survey.

The included cluster-RCT compared response rates sent out 6 months after use of the Child and Adolescent Mental Health Service (CAMHS) for two questionnaires, the Strength and Difficulties Questionnaire (SDQ) and the Experience of Service Questionnaire (ESQ), which were sent during a 3-month baseline period and a 3-month intervention period. Interventions included a control intervention, which consisted of general administration improvements (centralisation of questionnaire administration, quality improvements to questionnaires, covering letters and information sheets, and increased presentation of feedback in patient waiting areas), and two additive interventions, which consisted of the general improvements plus a postal reminder only or both postal and telephone reminder. After adjustment for the cluster-RCT design assuming conservative intra-class correlations (ICCs), there were no significant differences between any of the three CAMHS team in response rates during the 3-month intervention period.

Quality assessment of studies included in the evidence review

See the evidence profiles in appendix F.

Evidence from reference groups and focus groups

The children and young people’s reference groups and focus groups provided additional evidence for this review. A summary of the findings is presented in Table 3.

Table 3

Summary of the evidence from reference groups and focus groups.

See the full evidence summary in appendix M.

Evidence from national surveys

The grey literature review of national surveys provided additional evidence for this review. A summary of the findings is presented in Table 4.

Table 4

Summary of the evidence from national surveys.

See the full evidence summary in appendix N.

Economic evidence

Included studies

One economic study was identified which was relevant to this question (Horn 2010).

A single economic search was undertaken for all topics included in the scope of this guideline. See supplementary material 6 for details.

Excluded studies

Economic studies not included in this review are listed, and reasons for their exclusion are provided in appendix K.

Summary of studies included in the economic evidence review

Horn 2010 was a cost-effectiveness study conducted in the UK. The economic evaluation was conducted alongside an RCT (N = 268). The study compared three strategies: mailing only, mailing plus postal reminder, and mailing plus postal reminder plus telephone reminder. The study population comprised families who had used CAMHS. The study took a narrow NHS perspective and included only costs associated with administration, calls, stamps, and business reply envelopes. The time horizon was 4 weeks. The study reported outcomes in terms of cost per returned completed questionnaire. The study found that mailing plus postal reminder was extendedly dominated by other strategies and the incremental cost-effectiveness ratio of mailing plus postal reminder plus telephone reminder (versus mailing only) was £10.52 per additional completed and returned questionnaire.

See the economic evidence tables in appendix H and economic evidence profiles in appendix I.

Economic model

No economic modelling was undertaken for this review because the committee agreed that other topics were higher priorities for economic evaluation.

The committee’s discussion of the evidence

Interpreting the evidence

The outcomes that matter most

The aim of this review was to identify what is the best method to measure babies, children and young people’s experience of healthcare, and therefore acceptability to the respondent and response rate were prioritised as critical outcomes by the committee. Acceptability is paramount to ensure healthcare services are not using experience measuring tools that children or young people find difficult to complete, or do not want to complete. Survey response rate is linked to acceptability but is also a critical outcome in its own right. The use of measurement tools that have a high response rate is likely to lead to the most representative and informative data.

The committee also agreed that measurements relating to the mode effect of a survey question (that is, data accuracy and data equivalence), and the time taken to complete the survey were important outcomes to capture. Mode effect was considered an important aspect that could considerably affect the reliability of an experience measuring tool when transferred between different formats. Time taken to complete the survey was also considered an important outcome because long surveys could result in respondent fatigue, and respondents tend to provide incomplete or less accurate responses the further through the questionnaire they get. Additionally, there may be a time factor to be considered for the healthcare practitioner who is implementing the questionnaire.

The quality of the evidence

The quality of included studies was assessed using GRADE methodology. Evidence was considered very low quality. There were concerns regarding the risk of bias, namely in the lack of blinding (both possible assessment and measurement bias) and indirectness (specific population of children and young people attending CAMHS). Importantly, families were sent two types of questionnaire – a health outcomes questionnaire and an experience questionnaire. A ‘return’ was counted if either of these was returned, and no further information was provided regarding the proportions of returned questionnaire types. Additionally, very serious imprecision was found in the estimate of effects for all comparisons. This was due to the adjusted sample size, which was small, of the included cluster RCT.

It should also be noted that the 6-month post assessment questionnaire was sent to families of CAMHS attendees. There is no information provided about who received the correspondence, nor who answered the questionnaire (for example, whether it was the young person or another family member).

No effectiveness data was found for acceptability to respondent, mode effect, or time taken to complete the survey.

Benefits and harms

The committee noted that the included study (Horn 2010) suggested that telephone follow-up may increase response rates to a postal survey compared to the control group, and compared to a postal reminder, but that due to the limitations with this study, such as the uncertainty over who had actually completed the surveys, the small study size, and the overall very low quality of the study, they were unable to use it as a basis for specific recommendations.

In addition to the evidence from the systematic review there was also evidence from the reference and focus groups and from the national surveys of children and young people’s experience. The 11-14 years old reference group had suggested a variety of methods to collect feedback from children and young people including face-to face, using a variety of computer-based methods, audio surveys, token voting boxes and by having surveys delivered by other young people. The reference group thought it was best to carry out experience surveys while children or young people were still receiving care, and not leave it until later. The reference group also provided suggestions for the questions that should be included on experience surveys and these included numerical rankings, closed questions and open questions. The young people thought the surveys should be easy, smooth, positive, simple and quick.

The evidence from the national surveys of children and young people’s experience also found that young people were keen to provide feedback (positive and negative) on healthcare experience. There were a variety of views on the timing for collecting feedback but most young people thought it should be at regular intervals during treatment, or at the end of treatment. As with the reference groups there were suggestions on the methods and content of the feedback – with most support for open questions or qualitative questions, and for questions covering many aspects of healthcare provision such as interactions with healthcare professionals, involvement in decision-making, the environment, food, privacy, and entertainment and social activities. There was also evidence that children and young people wanted to be told how their feedback had been implemented.

There were also a number of comments on complaint systems, with young people reporting that they were difficult to access, there were barriers to making complaints, concerns that no action was taken, or that complaints could lead to repercussions, and the committee therefore made a specific recommendation on the provision of accessible complaints systems for children and young people.

Based on the evidence from the reference and focus groups and the review of national surveys, as well as their knowledge and expertise, the committee agreed that it was possible to make some good practice recommendations. They agreed that it was good practice to collect feedback from children and young people, and the parents or carers of babies and young children, and that while some NHS organisations already had these systems in place, making a recommendation to this effect would encourage all organisations to do this. The committee also noted the importance of the United Nations Convention on the Rights of the Child which states that children have the right to ‘express their views, feelings and wishes in all matters relating to them, and to have their views considered and taken seriously.’ They agreed that this recommendation was therefore in accordance with this convention.

The committee did not feel that the evidence was strong enough to recommend one method for collecting feedback over another, but agreed that there were some common principles that could be applied, such as the need to develop the assessment tools in conjunction with babies, children and young people to make sure they were acceptable, to adapt tools to allow those with disabilities or communication difficulties to provide feedback, to ensure that feedback was obtained at an appropriate time, to ensure feedback was obtained from a representative population (for example children from under-represented groups, parents and carers to represent babies), and to use techniques to maximize response rates. The committee discussed how to identify these under-represented groups but were aware of pro-active methods that could be used such as outreach work to engage and ask opinions from people who are not accessing services, targeting economically deprived areas, using index of multiple deprivation for schools and home addresses, and using snowball sampling.

The committee drew on the evidence for review question 5.1 which had identified that children and young people want feedback on their input, and the effect it has had on the design of services. The committee agreed that this would also be likely to be true about information collected about healthcare experience, and therefore they made a recommendation that information should be fed back to children and young people and parents or carers of babies and young children about the actions that had been taken.

The committee discussed if there were any potential harms from their recommendations. They identified that the need to provide feedback, especially if follow-up mechanisms were used to improve response rates, may lead to children and young people feeling under pressure to complete surveys at times in their life where they are dealing with health issues. The committee were also concerned that it was difficult to obtain responses from a truly representative sample (for example those with no fixed abode, those who had communication difficulties) and that this might lead to the results of surveys not being reflective of the whole population of babies, children and young people.

Due to the lack of published evidence available for this review and the fact that there are no comparative studies comparing different methods of measuring healthcare experience, the committee made a research recommendation.

Cost effectiveness and resource use

Evidence from a published cost-effectiveness study showed that, if complete and accurate telephone contact details were available, mailing plus postal reminder plus telephone reminder was potentially a cost-effective way to increase response rates compared to mailing only and mailing plus postal reminder. However, it was difficult to judge the cost-effectiveness as the cost per quality-adjusted life year (QALY) could not be estimated from the information provided in the publication. Also, this evidence was based on a single small UK study with potentially severe limitations and the committee could not draw any firm conclusions from this.

The committee discussed the fact that administering any kind of method to measure experience, and then to analyse the results, would have associated costs, although this would be likely to be similar across the different methods. Feedback data is routinely collected across most of the health service and therefore the recommendations do not represent a change in practice. Actions taken in response to this feedback may also have associated costs, however, this would apply to all methods. The committee also discussed the fact that sometimes feedback from service-users suggested ways that costs could be reduced, or could point out things that did not need to be done and would potentially result in the cost savings to the health service.

Other factors the committee took into account

The committee discussed the lack of evidence available for this review, and were aware that there was research on different methods of measuring the experience of babies, children and young people, but these were non-comparative studies and so had not met the protocol criteria for inclusion in the review.

The committee were also aware of an ongoing study comparing the administration of the National Inpatient Survey via online and paper formats, but it was unlikely this study would be published before the publication of the guideline.

The committee were aware that the right to complain about healthcare services was contained with the NHS constitution, including how complaints should be handled, and therefore they did not make detailed recommendations about complaint systems.

Recommendations supported by this evidence review

This evidence review supports recommendations 1.7.5 to 1.7.9 and the research recommendation on measuring experience of healthcare.

References

Horn 2010
Horn, R., Jones, S., Warren, K., The Cost-Effectiveness of Postal and Telephone Methodologies in Increasing Routine Outcome Measurement Response Rates in CAMHS, Child and Adolescent Mental Health, 15(1), 60–63, 2010. [PubMed: 32847211]

Appendices

Appendix E. Forest plots

Forest plots for review question: How can the experience of babies, children and young people be measured so as to improve their experience of healthcare?

No meta-analysis was conducted for this review question and so there are no forest plots.

Appendix F. GRADE tables

GRADE tables for review question: How can the experience of babies, children and young people be measured so as to improve their experience of healthcare? (PDF, 314K)

Appendix G. Economic evidence study selection

Economic evidence study selection for review question: How can the experience of babies, children and young people be measured so as to improve their experience of healthcare?

One global search was conducted for this review question. See supplementary material 6 for further information

Appendix H. Economic evidence tables

Economic evidence tables for review question: How can the experience of babies, children and young people be measured so as to improve their experience of healthcare? (PDF, 371K)

Appendix I. Economic evidence profiles

Economic evidence profiles for review question: How can the experience of babies, children and young people be measured so as to improve their experience of healthcare? (PDF, 301K)

Appendix J. Economic analysis

Economic evidence analysis for review question: How can the experience of babies, children and young people be measured so as to improve their experience of healthcare?

No economic analysis was conducted for this review question.

Appendix K. Excluded studies

Excluded studies for review question: How can the experience of babies, children and young people be measured so as to improve their experience of healthcare?

Clinical studies

Table 13

Excluded studies and reasons for their exclusion.

Economic studies

Table 14

Excluded studies and reasons for their exclusion.

Appendix L. Research recommendations

Research recommendations for review question: How can the experience of babies, children and young people be measured so as to improve their experience of healthcare? (PDF, 321K)

Appendix M. Evidence from reference groups and focus groups

Reference group and focus group evidence for review question: How can the experience of babies, children and young people be measured so as to improve their experience of healthcare? (PDF, 235K)

Appendix N. Evidence from national surveys

Evidence from national surveys for review question: How can the experience of babies, children and young people be measured so as to improve their experience of healthcare? (PDF, 382K)

Final

Evidence reviews underpinning recommendations 1.7.5 to 1.7.9 and research recommendations in the NICE guideline

These evidence reviews were developed by the National Guideline Alliance which is a part of the Royal College of Obstetricians and Gynaecologists

Disclaimer: The recommendations in this guideline represent the view of NICE, arrived at after careful consideration of the evidence available. When exercising their judgement, professionals are expected to take this guideline fully into account, alongside the individual needs, preferences and values of their patients or service users. The recommendations in this guideline are not mandatory and the guideline does not override the responsibility of healthcare professionals to make decisions appropriate to the circumstances of the individual patient, in consultation with the patient and/or their carer or guardian.

Local commissioners and/or providers have a responsibility to enable the guideline to be applied when individual health professionals and their patients or service users wish to use it. They should do so in the context of local and national priorities for funding and developing services, and in light of their duties to have due regard to the need to eliminate unlawful discrimination, to advance equality of opportunity and to reduce health inequalities. Nothing in this guideline should be interpreted in a way that would be inconsistent with compliance with those duties.

NICE guidelines cover health and care in England. Decisions on how they apply in other UK countries are made by ministers in the Welsh Government, Scottish Government, and Northern Ireland Executive. All NICE guidance is subject to regular review and may be updated or withdrawn.

Bookshelf ID: NBK574986PMID: 34730909

Table 1Summary of the protocol (PICO table)

Population	People <18 years-old who have experience of healthcare Studies that use the responses of parents or carers as proxies for their child will be included only if they are responding on behalf of their child or charge, and The baby or child of the parent or carer is under 5 years-old, or There is a clear rationale provided as to why the study is using parents’ or carers’ views on healthcare as proxies for their child.
Intervention	Any survey, questionnaire or other means of assessment designed to measure the healthcare experiences of babies, children and young people
Comparison	Same survey, questionnaire, or other means of assessment in different setting, format, or at different time Different survey, questionnaire, or other means of assessment
Outcomes	Critical Acceptability to respondent Response rate Important Mode effect: phenomenon when a particular survey administration mode causes different data to be collected. Data accuracy (proportion of number of errors to amount of missing data) Data equivalence (proportion of missing data to total possible data) Time taken to complete survey

Table 2Summary of included study

Study

Population

Intervention

Comparison

Outcomes

Horn 2010

Study design

Cluster-RCT

N (clusters) = 3 CAMHS teams randomised to 3 different treatment arms, including outcomes from N=178 children and young people, out of N = 268 eligible to participate

Characteristics

Age of children and young people: not reported

Gender of children and young people: not reported

Postal reminder

Same as control plus a postal reminder letter after 2 weeks to non-responders

Telephone reminder

Same as postal reminder intervention plus telephone call 2 weeks after postal reminder to non-responders

Control

General improvements to the administration process of the initial 6 month post-assessment questionnaire

Response rate

: CAMHS: Child and Adolescent Mental Health Service; N: number; RCT: randomised controlled trial

Table 3Summary of the evidence from reference groups and focus groups

Age groups	11-14 years
Areas covered	Methods to obtain feedback Questions to include
Illustrative quotes	Methods to obtain feedback ‘Some text is too talky/intimidating’ ‘Have surveys on iPads – cuts down on writing – some people can’t write/press buttons – need audio/speech options’ ‘Surveys delivered by people same age = easier to communicate – cuts out jargon. Better communication’ Questions to include ‘Do you think you are getting enough care?’ ‘Did you like your treatment?’ ‘What could we do differently?’

Table 4Summary of the evidence from national surveys

National surveys	Association for Young People’s Health. Young people’s views on involvement and feedback in healthcare 2014 Care Quality Commission. Children and young people’s inpatient and day case survey 2018 Opinion Matters. Declare your care survey 2018 Word of Mouth Research and Point of Care Foundation. An options appraisal for obtaining feedback on the experiences of children and young people with cancer 2018
Areas covered	Complaints Concerns Importance of using patient experience information
Key findings	More than half of the young people reported that they had wanted to make a complaint, but they did not do it. Of the ones who did it, half reported that nothing had happened as a result Reasons for not making a complaint included: not knowing who the person to raise it with was, not wanting to be seen as a trouble-maker, or not thinking it would make a difference Factors that would encourage young people to make a complaint included: receiving regular feedback on actions taken; a more open culture encouraging feedback; knowing which staff or services to raise it with Young people felt that it was very important for the local services to collect information about the experiences of patients they treated

Table 13Excluded studies and reasons for their exclusion

Study	Reason for Exclusion
Abrol, E., Groszmann, M., Pitman, A., Hough, R., Taylor, R. M., Aref-Adib, G., Exploring the digital technology preferences of teenagers and young adults (TYA) with cancer and survivors: a cross-sectional service evaluation questionnaire, Journal of cancer survivorship : research and practice, 11, 670–682, 2017 [PMC free article: PMC5671543] [PubMed: 28634734]	Study design not in PICO - Cross-sectional
AlSaud, A. M., Taddese, H. B., Filippidis, F. T., Trends and correlates of the public’s perception of healthcare systems in the European Union: a multilevel analysis of Eurobarometer survey data from 2009 to 2013, BMJ open, 8, e018178, 2018 [PMC free article: PMC5780735] [PubMed: 29317413]	Study design not in PICO - Cross-sectional
Amari, E., Vandebeek, C., Montgomery, C. J., Skarsgard, E., Ansermino, J. M., Telephone and web-based pediatric day surgery questionnaires, International Journal of Health Care Quality Assurance, 23, 339–351, 2010 [PubMed: 20535905]	Country: Canada
Ambresin, A. E., Bennett, K., Patton, G. C., Sanci, L. A., Sawyer, S. M., Assessment of youth-friendly health care: A systematic review of indicators drawn from young people’s perspectives, Journal of Adolescent Health, 52, 670–681, 2013 [PubMed: 23701887]	Outcomes of included studies not in PICO. Included studies checked for inclusion.
Anonymous,, Development of more focused questionnaires improves results, Nursing children and young people, 28, 13, 2016 [PubMed: 27266743]	Editorial article
Barber, S., Bekker, H., Marti, J., Pavitt, S., Khambay, B., Meads, D., Development of a Discrete-Choice Experiment (DCE) to Elicit Adolescent and Parent Preferences for Hypodontia Treatment, Patient, 12, 137–148, 2019 [PMC free article: PMC6335368] [PubMed: 30367434]	Study design not in PICO - Cross-sectional
Bennett, K. E., Ambresin, A. E., Patton, G. C., Sawyer, S. M., Development of the ‘adolescent friendly hospital survey’, Turk Pediatri Arsivi, 2), 97, 2013	Conference abstract
Beresford, B., Clarke, S., Greco, V., Referrers’ use and views of specialist mental health services for deaf children and young people in England, Journal of Mental Health, 19, 193–201, 2010 [PubMed: 20433327]	Population not in PICO - Healthcare professionals
Bikker, A. P., Fitzpatrick, B., Murphy, D., Mercer, S. W., Measuring empathic, person-centred communication in primary care nurses: validity and reliability of the Consultation and Relational Empathy (CARE) Measure, BMC family practice, 16, 149, 2015 [PMC free article: PMC4619021] [PubMed: 26493072]	Study design not in PICO - No comparison
Blasche, G., Marktl, W., Eisenwort, B., Skolka, A., Pichlhofer, O., The treatment experience questionnaire: development and validation of a questionnaire assessing the individual’s emotional, perceptual, and cognitive reactions to alternative, physical, and dental treatments, Forschende Komplementarmedizin (2006), 20, 205–12, 2013 [PubMed: 23860022]	Study design not in PICO - No comparison
Boss, R. D., Kinsman, H. I., Donohue, P. K., Health-related quality of life for infants in the neonatal intensive care unit, Journal of Perinatology, 32, 901–906, 2012 [PubMed: 22743406]	Narrative review. Included studies checked for inclusion.
Bowling, A., Rowe, G., McKee, M., Patients’ experiences of their healthcare in relation to their expectations and satisfaction: a population survey, Journal of the Royal Society of Medicine, 106, 143–9, 2013 [PMC free article: PMC3618164] [PubMed: 23564898]	Population not in PICO - Participants not under 18, or data not presented separately for target population
Bravery, K., Cabrera, A. M., Gibson, F., Harding, V., Martins, A., Oldreive, N., Polly, S., Jaffrey, M., Usability testing of a digital child experience measure for children with cancer, Pediatric Blood and Cancer, 63 (Supplement 3), S223, 2016	Conference abstract
Bravery, K., Cabrera, A., Harding, V., Martins, A., Oldreive, N., Smithson, D., Snowdon, P., Sweet, R., Gibson, F., Development of a child experience measure for children with cancer, Pediatric Blood and Cancer, 62 (Supplement 4), S345–S346, 2015	Conference abstract
Brown, A., Ford, T., Deighton, J., Wolpert, M., Satisfaction in child and adolescent mental health services: translating users’ feedback into measurement, Administration and policy in mental health, 41, 434–446, 2014 [PubMed: 22829193]	Study design not in PICO - No comparison
Brown, J., Aladangady, N., Measuring the quality of care: using patient experience trackers in a neonatal unit, Nursing Times, 106, 10–11, 2010 [PubMed: 20334015]	Study design not in PICO - No comparison
Burger, S. A., Tallett, A., Maconochie, I., Pall, K., Children’s care pathway and parental experiences following use of NHS 111, a non-emergency medical helpline in England, International Journal for Quality in Health Care, 28 (Supplement 1), 31, 2016	Conference abstract
Butt, M. L., Pinelli, J., Boyle, M. H., Thomas, H., Hunsberger, M., Saigal, S., Lee, D. S., Fanning, J. K., Austin, P., Development and Evaluation of an Instrument to Measure Parental Satisfaction With Quality of Care in Neonatal Follow-Up, Journal of Developmental and Behavioral Pediatrics, 30, 57–65, 2009 [PubMed: 19194325]	Outcome not in PICO - Validity of new measurement
Cahill, Paul, O’Reilly, Ken, Carr, Alan, Dooley, Barbara, Stratton, Peter, Validation of a 28-item version of the Systemic Clinical Outcome and Routine Evaluation in an Irish context: The score-28, Journal of Family Therapy, 32, 210–231, 2010	Intervention not in PICO - Measurement of progress in therapy, not healthcare experience
Canaway, A. G., Frew, E. J., Measuring preference-based quality of life in children aged 6-7 years: a comparison of the performance of the CHU-9D and EQ-5D-Y--the WAVES pilot study, Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation, 22, 173–183, 2013 [PubMed: 22302618]	Intervention not in PICO - Measurement of health-related quality of life, not healthcare experience
Chakravorty, S., Tallett, A., John, W., Using a new patient feedback survey to explore experiences of living with Sickle Cell Disease in the UK, British Journal of Haematology, 176 (Supplement 1), 15, 2017	Conference abstract
Chakravorty, S., Tallett, A., Sathyamoorthy, G., James, J., Using a new patient feedback survey to explore experiences of living with sickle cell disease in the UK, International Journal for Quality in Health Care, 28 (Supplement 1), 60–61, 2016	Conference abstract
Challenor, R., Perry, R., Measuring patient satisfaction: three years’ data and experience of using a validated patient questionnaire, International Journal of STD and AIDS, 26, 667–671, 2015 [PubMed: 25122579]	Study design not in PICO - Cross-sectional
Chow, M. Y., Morrow, A. M., Cooper Robbins, S. C., Leask, J., Condition-specific quality of life questionnaires for caregivers of children with pediatric conditions: a systematic review, Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation, 22, 2183–2200, 2013 [PubMed: 23292299]	Intervention of included studies not in PICO. Included studies checked for inclusion.
Cohen, W., Wynne, D. M., Parent and Child Responses to the Pediatric Voice-Related Quality-of-Life Questionnaire, Journal of Voice, 29, 299–303, 2015 [PubMed: 25619466]	Intervention not in PICO - Measurement of health related quality of life, not healthcare experience.
Cole, J. A., Gillespie, P., Smith, S. M., Byrne, M., Murphy, A. W., Cupples, M. E., Using postal questionnaires to evaluate physical activity and diet behaviour change: case study exploring implications of valid responder characteristics in interpreting intervention outcomes, BMC research notes, 7, 725, 2014 [PMC free article: PMC4203940] [PubMed: 25315148]	Outcome not in PICO - All related to physical activity
Dall’Oglio, I., Mascolo, R., Gawronski, O., Tiozzo, E., Portanova, A., Ragni, A., Alvaro, R., Rocco, G., Latour, J. M., A systematic review of instruments for assessing parent satisfaction with family-centred care in neonatal intensive care units, Acta Paediatrica, International Journal of Paediatrics, 107, 391–402, 2018 [PubMed: 29239021]	Population of included studies not in PICO. Included studies checked for inclusion.
Davies, F. C., Clancy, M., Surveying child patients in emergency care is possible, Emergency Medicine Journal, 31, 262, 2014 [PubMed: 23851035]	Commentary article
Davis, E., Young, D., Gilson, K. M., Swift, E., Chan, J., Gibbs, L., Tonmukayakul, U., Reddihough, D., Williams, K., A Rights-Based Approach for Service Providers to Measure the Quality of Life of Children with a Disability, Value in Health, 21, 1419–1427, 2018 [PubMed: 30502786]	Outcomes of included studies not in PICO. Included studies checked for inclusion.
Day, C., Michelson, D., Hassan, I., Child and adolescent service experience (ChASE): measuring service quality and therapeutic process, The British journal of clinical psychology / the British Psychological Society, 50, 452–464, 2011 [PubMed: 22003953]	Study design not in PICO - Cross-sectional
Day, L. A., Brice, P., Development and initial validation of a questionnaire to measure hearing parents’ perceptions of health care professionals’ advice, Journal of Deaf Studies and Deaf Education, 18, 123–137, 2013 [PubMed: 23197316]	Population not in PICO - Parental views
De Stefani, A., Bruno, G., Irlandese, G., Barone, M., Costa, G., Gracco, A., Oral health-related quality of life in children using the child perception questionnaire CPQ11-14: a review, European Archives of Paediatric Dentistry: Official Journal of the European Academy of Paediatric DentistryEur Arch Paediatr Dent, 14, 14, 2019 [PubMed: 30762210]	Population of included studies not in PICO. Included studies checked for inclusion.
Desai, A. D., Popalisky, J., Simon, T. D., Mangione-Smith, R. M., The effectiveness of family-centered transition processes from hospital settings to home: A review of the literature, Hospital Pediatrics, 5, 219–231, 2015 [PubMed: 25832977]	Outcomes of included studies not in PICO. Included studies checked for inclusion.
Down, C., Waldron, B., Maini, R., Williams, F. L. R., Brown, A., Notghi, L., Martin, K., Chin, R., Basu, H., Kirkpatrick, M., Ferrie, C., Whitehouse, W. P., Dunkley, C., Epilepsy12-United Kingdom collaborative clinical audit of health care for children and young people with suspected epileptic seizures, Archives of disease in childhood, 3), A118–A119, 2015	Conference abstract
Dunkley, C., Waldron, B., Maini, R., Williams, F., Brown, A., Ranmal, R., Flower, D., Colaco, F., Bowyer, K., Notghi, L., Martin, K., Chin, R., Basu, H., Paton, J., Kirkpatrick, M., Ferrie, C., Whitehouse, W. P., Epilepsy12 - United Kingdom collaborative clinical audit of health care for children and young people with suspected epileptic seizures, Epilepsia, 5), 16, 2012	Conference abstract
Espinel, A. G., Shah, R. K., McCormick, M. E., Krakovitz, P. R., Boss, E. F., Patient satisfaction in pediatric surgical care: A systematic review, Otolaryngology - Head and Neck Surgery (United States), 150, 739–749, 2014 [PubMed: 24671459]	Intervention of included studies not in PICO. Included studies checked for inclusion.
Espinel, A. G., Shah, R. K., McCormick, M. E., Krakovitz, P. R., Boss, E. F., Patient satisfaction in pediatric surgical care: A systematic review, Otolaryngology - Head and Neck Surgery (United States), 150, 739–749, 2014 [PubMed: 24671459]	Outcomes of included studies not in PICO. Included studies checked for inclusion.
Espinel, A. G., Shah, R., McCormick, M. E., Boss, E. F., Patient satisfaction as a quality indicator in pediatric surgical care: A systematic review, Otolaryngology - Head and Neck Surgery (United States), 1), P34, 2013	Outcomes of included studies not in PICO. Included studies checked for inclusion.
Evans, J., Rose, D., Flach, C., Csipke, E., Glossop, H., McCrone, P., Craig, T., Wykes, T., VOICE: developing a new measure of service users’ perceptions of inpatient care, using a participatory methodology, Journal of Mental Health, 21, 57–71, 2012 [PMC free article: PMC4018995] [PubMed: 22257131]	Outcome not in PICO - Validity of new measurement
Foster, T., Maillardet, V., Surveying young patients, Emergency Medicine Journal, 27, 221–223, 2010 [PubMed: 20304894]	Outcome not in PICO - Validity of new measurement
Glenny, A. M., Worthington, H. V., Milsom, K. M., Rooney, E., Tickle, M., Strategies for maximizing consent rates for child dental health surveys: a randomised controlled trial, BMC medical research methodology, 13, 108, 2013 [PMC free article: PMC3846683] [PubMed: 24006895]	Outcome not in PICO - Consent for survey participation
Gore, C., Griffin, R., Rothenberg, T., Tallett, A., Hopwood, B., Sizmur, S., O’Keeffe, C., Warner, J. O., New patient-reported experience measure for children with allergic disease: development, validation and results from integrated care, Archives of Disease in Childhood, 101, 935–43, 2016 [PubMed: 27484970]	Outcome not in PICO - Validity of new measurement
Gore, C., Rothenberg, T., Griffin, R., Tallett, A., O’Keeffe, C., Warner, J. O., Patient reported experience measures - Development and validation of an allergic disease specific tool for children and young people, Clinical and Experimental Allergy, 43 (12), 1467, 2013	Conference abstract
Griffin, R., Gore, C., Rothenberg, T., Tallett, A., O’Keeffe, C., Makrinioti, C. N., Warner, J. O., Allergy-specific patient reported experience measures - The views of children, young people and their families on their care, Clinical and Experimental Allergy, 43 (12), 1434, 2013	Conference abstract
Gurung, G., Richardson, A., Wyeth, E., Edmonds, L., Derrett, S., Child/youth, family and public engagement in paediatric services in high-income countries: A systematic scoping review, Health expectations : an international journal of public participation in health care and health policy, 23, 261–273, 2020 [PMC free article: PMC7104655] [PubMed: 31981295]	Scoping review. Included studies checked for inclusion.
Hargreaves, D. S., Viner, R. M., McDonagh, J. E., What do young people value in health services? Validation of the you’re welcome quality criteria against data from 2 national inpatient surveys in england, Archives of disease in childhood, 1), A77, 2012	Conference abstract
Harley, C., Adams, J., Booth, L., Selby, P., Brown, J., Velikova, G., Patient experiences of continuity of cancer care: Development of a new medical care questionnaire (MCQ) for oncology outpatients, Value in Health, 12, 1180–1186, 2009 [PubMed: 19659706]	Outcome not in PICO - Validity of new measurement
Hindmarsh, P., Wright, A., Diabetes Patient Experience Project with Children, Young People and Parents: Developing a standard toolkit to guide regional paediatric diabetes networks in the ongoing collection of quantitative and qualitative patient experience feedback from children and young people as well as parental opinion, Diabetic medicine, 1), 87, 2012	Conference abstract
Holley, S., Knibb, R., Latter, S., Liossi, C., Mitchell, F., Radley, R., Roberts, G., Development and validation of the Adolescent Asthma Self-Efficacy Questionnaire (AASEQ), The European respiratory journal., 02, 2019 [PubMed: 31048348]	Outcome is validity of new measurement
Homa, K., Sabadosa, K. A., Nelson, E. C., Rogers, W. H., Marshall, B. C., Development and validation of a cystic fibrosis patient and family member experience of care survey, Quality Management in Health Care, 22, 100–16, 2013 [PubMed: 23542365]	Outcome not in PICO - Validity of new measurement
Hopwood, B., Lloyd, K., Tallett, A., Chow, C., Warner, J., Developing an allergy specific patient reported experience measure (PREM), Archives of Disease in Childhood, 96, A48, 2011	Conference abstract
Hopwood, B., Tallett, A., Little voice: giving young patients a say, Nursing times, 107, 18–20, 2011 [PubMed: 22239046]	Study design not in PICO - Cross-sectional
Hutchings, A., Grosse Frie, K., Neuburger, J., van der Meulen, J., Black, N., Late response to patient-reported outcome questionnaires after surgery was associated with worse outcome, Journal of clinical epidemiology, 66, 218–25, 2013 [PubMed: 23200295]	Population not in PICO - Participants not under 18, or data not presented separately for target population
Ishaque, S., Roberts, R., Karnon, J., Salter, A., Thomas, D., Adaptation/content validation of measure yourself medical outcomes profile (MYMOP) questionnaire for 7-11 year old children, Quality of Life Research, 27 (Supplement 1), S114–S115, 2018	Conference abstract
Jenkins, P. J., Sng, S., Brooksbank, K., Brooksbank, A. J., Socioeconomic deprivation and age are barriers to the online collection of patient reported outcome measures in orthopaedic patients, Annals of the Royal College of Surgeons of England, 98, 40–44, 2016 [PMC free article: PMC5234380] [PubMed: 26688398]	Population not in PICO - Participants not under 18, or data not presented separately for target population
Jensen, H. I., Ammentorp, J., Kofoed, P. E., Assessment of health care by children and adolescents depends on when they respond to the questionnaire, International Journal for Quality in Health Care, 22, 259–265, 2010 [PubMed: 20427521]	International (passed through UK only filter)
Johal, A., Fleming, P. S., Al Jawad, F. A., A prospective longitudinal controlled assessment of pain experience and oral health-related quality of life in adolescents undergoing fixed appliance treatment, Orthodontics & craniofacial research, 17, 178–186, 2014 [PubMed: 24703180]	Intervention not in PICO - Fixed appliance treatment
John, Mary, Jeffries, Fiona W., Acuna-Rivera, Marcela, Warren, Fiona, Simonds, Laura M., Development of measures to assess personal recovery in young people treated in specialist mental health services, Clinical psychology & psychotherapy, 22, 513–524, 2015 [PubMed: 24913148]	Intervention not in PICO - Designed to assess perception of mental health progress rather than healthcare experiences
Joy, R., Smith, D. P., Mannion, J. F., Using an electronic tablet to survey patient satisfaction in an adolescent transitional diabetes clinic at York, UK, Pediatric Diabetes, 19), 132–133, 2014	Poster presentation
Kappesser, J., de Laffolie, J., Faas, D., Ehrhardt, H., Hermann, C., Comparison of two neonatal pain assessment tools (Children and Infant’s Postoperative Pain Scale and the Neonatal Facial Coding System-Revised) and their relations to clinicians’ intuitive pain estimates, European Journal of Pain (United Kingdom), 23, 708–718, 2019 [PubMed: 30407684]	Intervention not in PICO - Measuring clinicians ability to discern pain in neonates
Kemp, K. A., Chan, N., McCormack, B., Douglas-England, K., Drivers of Inpatient Hospital Experience Using the HCAHPS Survey in a Canadian Setting, Health Services Research, 50, 982–97, 2015 [PMC free article: PMC4545343] [PubMed: 25483921]	Comparisons of included studies not in PICO. Included studies checked for inclusion.
Khangah, H. A., Jannati, A., Imani, A., Assessment of healthcare providers’ performance considering three healthcare quality indicators: A review article, Shiraz E Medical Journal, 17 (4-5) (no pagination), 2016	Population of included studies not in PICO. Included studies checked for inclusion.
Kjell, O. N. E., Kjell, K., Garcia, D., Sikstrom, S., Semantic measures: Using natural language processing to measure, differentiate, and describe psychological constructs, Psychological methods, 24, 92–115, 2019 [PubMed: 29963879]	Study designs of included studies not in PICO. Included studies checked for inclusion.
Latour, J. M., Measuring parent satisfaction with nursing care, Archives of Disease in Childhood, 99, A16, 2014	Conference abstract
Lorenc, A., Robinson, N., Community use of traditional and complementary healthcare approaches (TCA) for children-Comparing internet and face to face survey methods, European Journal of Integrative Medicine, 2 (4), 229, 2010	Conference abstract
MacKenzie, H., Thavaneswaran, A., Chandran, V., Gladman, D. D., Patient-reported outcome in psoriatic arthritis: a comparison of Web-based versus paper-completed questionnaires, Journal of Rheumatology, 38, 2619–24, 2011 [PubMed: 22045844]	Population not in PICO - Participants not under 18, or data not presented separately for target population
Maguire, L., Aventin, A., Lohan, M., Clarke, M., What do young people really understand when completing questionnaires? Lessons learnt from developing a questionnaire to measure behavioural outcomes in a sexual health trial, Trials. Conference: 3rd International Clinical Trials Methodology Conference. United Kingdom, 16, 2015	Poster presentation
Maini, Rishma, Kirkpatrick, Martin, McCafferty, Aileen, Dunkley, Colin, Ogston, Simon, Williams, Fiona, Evaluation of a questionnaire to measure parent/carer and child/young person experience of NHS epilepsy services, Seizure, 63, 71–78, 2018 [PubMed: 30445294]	Study design not in PICO - Cross-sectional
Marshman, Z., Eddaiki, A., Bekker, H. L., Benson, P. E., Development and evaluation of a patient decision aid for young people and parents considering fixed orthodontic appliances, Journal of Orthodontics, 43, 276–287, 2016 [PubMed: 27748643]	Intervention not in PICO - Questionnaire measuring shared-decision making, not healthcare experience
McMurray, J., McNeil, H., Lafortune, C., Black, S., Prorok, J., Stolee, P., Measuring patients’ experience of rehabilitation services across the care continuum. Part I: A systematic review of the literature, Archives of Physical Medicine and Rehabilitation, 97, 104–120, 2016 [PubMed: 26299752]	Population of included studies not in PICO. Included studies checked for inclusion.
McMurray, J., McNeil, H., Lafortune, C., Black, S., Prorok, J., Stolee, P., Measuring patients’ experience of rehabilitation services across the care continuum. Part I: A systematic review of the literature, Archives of Physical Medicine and Rehabilitation, 97, 104–120, 2016 [PubMed: 26299752]	Population of included studies not in PICO. Included studies checked for inclusion.
McNair, A., Drage, K. J., Ireland, A. J., Sandy, J. R., Williams, A. C., Piloting a patient-based questionnaire to assess patient satisfaction with the process of orthodontic treatment, Angle Orthodontist, 79, 759–765, 2009 [PubMed: 19537857]	Outcome not in PICO - Validity of new measurement
Morgan, A. J., Rapee, R. M., Bayer, J. K., Increasing response rates to follow-up questionnaires in health intervention research: randomized controlled trial of a gift card prize incentive, Clinical trials (london, england), 14, 381–386, 2017 [PubMed: 28393632]	Population not in PICO - Parental views
Morley, D., Dummett, S., Kelly, L., Dawson, J., Fitzpatrick, R., Jenkinson, C., Pretesting the Oxford participation and activities questionnaire: Results from an expert review, Movement Disorders, 1), S419, 2015	Conference abstract
Noyes, J., Edwards, R. T., EQ-5D for the assessment of health-related quality of life and resource allocation in children: A systematic methodological review, Value in health, 14, 1117–1129, 2011 [PubMed: 22152182]	Intervention of included studies not in PICO. Included studies checked for inclusion.
O’Cathain, A., Knowles, E., Nicholl, J., Measuring patients’ experiences and views of the emergency and urgent care system: psychometric testing of the urgent care system questionnaire, BMJ Quality & Safety, 20, 134–40, 2011 [PubMed: 21209128]	Population not in PICO - Participants not under 18, or data not presented separately for target population
Oldham, G., Sidhu-Bevan, H., Wray, J., Using patient-reported experience measures as quality improvement tools in a specialist children’s hospital, Archives of disease in childhood, 102 (Supplement 1), A24–A25, 2017	Conference abstract
Patalay, P., Hayes, D., Deighton, J., Wolpert, M., A Comparison of Paper and Computer Administered Strengths and Difficulties Questionnaire, Journal of Psychopathology and Behavioral Assessment, 38, 242–250, 2016	Intervention not in PICO - Measurement of mental health only, not healthcare experience
Patalay, Praveetha, Deighton, Jessica, Fonagy, Peter, Wolpert, Miranda, Equivalence of paper and computer formats of a child self-report mental health measure, European Journal of Psychological Assessment, 31, 54–61, 2015	Intervention not in PICO - Measurement of mental health only, not healthcare experience
Perry, Sarah, Carpenter, Simon, Preliminary development and piloting of a user-generated routine outcome measure in a children and young people’s counselling service, Counselling & Psychotherapy Research, 16, 171–182, 2016	Study design not in PICO - Cross-sectional
Quigley, D. D., Palimaru, A., Lerner, C., Hays, R. D., A review of best practices for monitoring and improving inpatient pediatric patient experiences, Hospital Pediatrics, 10, 277–285, 2020 [PubMed: 32071119]	Interventions of included studies not in PICO. Included studies checked for inclusion.
Rahi, J. S., Tadi, V., Keeley, S., Lewando-Hundt, G., Capturing children and young people’s perspectives to identify the content for a novel vision-related quality of life instrument, Ophthalmology, 118, 819–824, 2011 [PubMed: 21126769]	Study design not in PICO - Qualitative study
Ramaswami, U., Stull, D. E., Parini, R., Pintos-Morell, G., Whybra, C., Kalkum, G., Rohrbach, M., Raluy-Callado, M., Beck, M., Chen, W. H., Wiklund, I., Measuring patient experiences in Fabry disease: validation of the Fabry-specific Pediatric Health and Pain Questionnaire (FPHPQ), Health and quality of life outcomes, 10 (no pagination), 2012 [PMC free article: PMC3487958] [PubMed: 22992222]	Intervention not in PICO - Measurement of specific disease symptoms only, not healthcare experience
Richards, S. H., Campbell, J. L., Dickens, A., Does the method of administration influence the UK GMC patient questionnaire ratings?, Primary health care research & development, 12, 68–78, 2011 [PubMed: 21426616]	Study design not in PICO - Cross-sectional
Richardson, P., George, B., Doyle, A., Kelly, S., Kisler, J., How should we listen to the children? Developing a child reporting assessment questionnaire in a tertiary spasticity clinic, Developmental medicine and child neurology, 2), 54, 2013	Conference abstract
Riley, A. R., Walker, B. L., Hall, T. A., Development and initial validation of a measure of parents’ preferences for behavioral counseling in primary care, Families, systems & health : the journal of collaborative family healthcare, 2020 [PMC free article: PMC7292737] [PubMed: 32297758]	Country: USA
Robertson, S., Pryde, K., Evans, K., Patient involvement in quality improvement: Is it time we let children, young people and families take the lead?, Archives of Disease in Childhood: Education and Practice Edition, 99, 23–27, 2014 [PubMed: 24216190]	Outcomes of included studies not in PICO. Included studies checked for inclusion.
Rosenberg, A. R., Bona, K., Wharton, C. M., Bradford, M., Shaffer, M. L., Wolfe, J., Baker, K. S., Adolescent and Young Adult Patient Engagement and Participation in Survey-Based Research: A Report From the “Resilience in Adolescents and Young Adults With Cancer” Study, Pediatric Blood and Cancer, 63, 734–736, 2016 [PMC free article: PMC4755794] [PubMed: 26681427]	Population not in PICO - Participants not under 18, or data not presented separately for target population
Ryninks, K. E., Burren, C. P., Garratt, V. L., Developing a patient reported outcome and experience measure for a specialised paediatric service, Archives of disease in childhood, 1), A101, 2014	Conference abstract
Sadler-Williams, E., Wang, L., Carmichael, S., McSkimming, P., Patient perceptions of IMPs: An international perspective, Pharmaceutical Engineering, 36, 50–58, 2016	Study design not in PICO - No comparison
Sadlo, A., Altevers, J., Peplies, J., Kaltz, B., Classen, M., Bauer, A., Koletzko, S., Timmer, A., Measuring satisfaction with health care in young persons with inflammatory bowel disease--an instrument development and validation study, BMC health services research, 14, 97, 2014 [PMC free article: PMC3946022] [PubMed: 24581043]	Outcome not in PICO - Validity of new measurement
Sakonidou, S., Andrzejewska, I., Webbe, J., Modi, N., Bell, D., Gale, C., Interventions to improve quantitative measures of parent satisfaction in neonatal care: A systematic review, BMJ Paediatrics Open, 4, e000613, 2020 [PMC free article: PMC7073789] [PubMed: 32201743]	Outcomes of included studies not in PICO. Included studies checked for inclusion.
Sanford, K., Rivers, A. S., Braun, T. L., Schultz, K. P., Buchanan, E. P., Medical Consultation Experience Questionnaire: Assessing perceived alliance and experienced confusion during medical consultations, Psychological AssessmentPsychol Assess, 30, 1499–1511, 2018 [PubMed: 29878815]	Population not in PICO - Parental views
Sizmur, S., Graham, C., Walsh, J., Influence of patients’ age and sex and the mode of administration on results from the NHS friends and family test of patient experience, Journal of Health Services Research and Policy, 20, 5–10, 2015 [PubMed: 24973979]	Study design not in PICO - Cross-sectional
Sousa, V. E. C., Dunn Lopez, K., Towards Usable E-Health. A Systematic Review of Usability Questionnaires, Applied clinical informatics, 8, 470–490, 2017 [PMC free article: PMC6241759] [PubMed: 28487932]	Outcomes of included studies not in PICO. Included studies checked for inclusion.
Starr, K., McPherson, G., Forrest, M., Cotton, S. C., SMS text pre-notification and delivery of reminder e-mails to increase response rates to postal questionnaires in the SUSPEND trial: a factorial design, randomised controlled trial, Trials, 16, 295, 2015 [PMC free article: PMC4494692] [PubMed: 26152519]	Population not in PICO - Participants not under 18, or data not presented separately for target population
Tallett, A., Hopwood, B., Using a child-friendly survey to obtain feedback about the hospital experience of young inpatients, Archives of Disease in Childhood: Education and Practice Edition, 1), A67, 2013	Conference abstract
Weston, R. L., Hopwood, B., Harding, J., Sizmur, S., Ross, J. D. C., Development of a validated patient satisfaction survey for sexual health clinic attendees, International Journal of STD and AIDS, 21, 584–590, 2010 [PubMed: 20975093]	Population not under 18, or unable to calculate proportions
Wiebe, Natalie, Fiest, Kirsten M., Dykeman, Jonathan, Liu, Xiaorong, Jette, Nathalie, Patten, Scott, Wiebe, Samuel, Patient satisfaction with care in epilepsy: How much do we know?, Epilepsia, 55, 448–455, 2014 [PubMed: 24502616]	Interventions of included studies not in PICO. Included studies checked for inclusion.
Willems, D. C. M., Joore, M. A., Nieman, F. H. M., Severens, J. L., Wouters, E. F. M., Hendriks, J. J. E., Using EQ-5D in children with asthma, rheumatic disorders, diabetes, and speech/language and/or hearing disorders, International Journal of Technology Assessment in Health Care, 25, 391–399, 2009 [PubMed: 19619359]	Intervention not in PICO - Measurement of health status only, not healthcare experience
Wolpert, M., Cheng, H., Deighton, J., Measurement issues: Review of four patient reported outcome measures: SDQ, RCADS, C/ORS and GBO - their strengths and limitations for clinical use and service evaluation, Child and Adolescent Mental Health, 20, 63–70, 2015 [PubMed: 32680325]	Intervention not in PICO - Measurement of health status only, not healthcare experience
Wray, J., Hobden, S., Knibbs, S., Oldham, G., Hearing the voices of children and young people to develop and test a patient-reported experience measure in a specialist paediatric setting, Archives of disease in childhood, 103, 272–279, 2018 [PubMed: 28903950]	Study design not in PICO - No comparison

Table 14Excluded studies and reasons for their exclusion

Study	Reason for exclusion
Cohen, W., Wynne, D. M., Parent and Child Responses to the Pediatric Voice-Related Quality-of-Life Questionnaire, Journal of Voice, 29, 299–303, 2015 [PubMed: 25619466]	Intervention does not fit the inclusion criteria: Adapting an adult related Quality of Life questionnaire
Hamilton, M. P., Hetrick, S. E., Mihalopoulos, C., Baker, D., Browne, V., Chanen, A. M., Pennell, K., Purcell, R., Stavely, H., McGorry, P. D., Identifying attributes of care that may improve cost-effectiveness in the youth mental health service system, Medical Journal of Australia, 207, S27–S37, 2017 [PubMed: 29129184]	Intervention does not fit the inclusion criteria: Examining different types of therapeutic treatments for mental health problems
Knapp, M., Evers, S., Health economic aspects of child and adolescent mental health, Psychiatrische Praxis, 38, 2011	This is a conference abstract and not a fully published study
Montgomery, S., Hassan, M., Kusel, J., Economic orphans? The prevalence of child-specific utilities in nice appraisals for paediatric indications, Value in Health, 17 (7), A326–A327, 2014 [PubMed: 27200547]	This is a conference abstract and not a fully published study
Noyes, J., Edwards, R. T., Hastings, R. P., Hain, R., Totsika, V., Bennett, V., Hobson, L., Davies, G. R., Humphreys, C., Devins, M., Spencer, L. H., Lewis, M., Evidence-based planning and costing palliative care services for children: novel multi-method epidemiological and economic exemplar, BMC Palliative Care, 12, 18, 2013 [PMC free article: PMC3651264] [PubMed: 23617814]	Intervention does not fit the inclusion criteria: Not using Health Economics to compare different palliative care options
Ougrin, D., Corrigall, R., Poole, J., Zundel, T., Sarhane, M., Slater, V., Stahl, D., Reavey, P., Byford, S., Heslin, M., Ivens, J., Crommelin, M., Abdulla, Z., Hayes, D., Middleton, K., Nnadi, B., Taylor, E., Comparison of effectiveness and cost-effectiveness of an intensive community supported discharge service versus treatment as usual for adolescents with psychiatric emergencies: a randomised controlled trial, The Lancet Psychiatry, 5, 477–485, 2018 [PMC free article: PMC5994473] [PubMed: 29731412]	Intervention does not fit the inclusion criteria: Treatments for mental health
Rennie, L., Porteous, T., Ryan, M., Preferences for managing symptoms of differing severity: a discrete choice experiment, Value in Health, 15, 1069–76, 2012 [PubMed: 23244809]	Intervention does not fit the inclusion criteria: Population only adults
Revill, P., Ryan, P., McNamara, A., Normand, C., A cost and outcomes analysis of alternative models of care for young children with severe disabilities in Ireland, Alter, 7, 260–274, 2013	Intervention does not fit the inclusion criteria: Parental opinion of care
Thorrington, D., Eames, K., Measuring health utilities in children and adolescents: A systematic review of the literature, PLoS ONE, 10 (8) (no pagination), 2015 [PMC free article: PMC4537138] [PubMed: 26275302]	Intervention does not fit the inclusion criteria: Evaluating the ways of measuring BCYP health states not BCYP experience of healthcare
Trepel, D., Ali, S., Health-related quality of life measures in economic evaluations of child and adolescence mental health interventions: A systematic review, Journal of Mental Health Policy and Economics, 1), S22, 2012	This is a conference abstract and not a fully published study
Wray, J., Oldham, G., Using patient reported experience measures (PREMs) as quality improvement tools in paediatric cardiothoracic services: Making it happen, European journal of pediatrics, 175 (11), 1520–1521, 2016	This is a conference abstract and not a fully published study
Yu, A. P., Ben-Hamadi, R., Wu, E. Q., Kaltenboeck, A., Bergman, R., Xie, J., Blum, S., Erder, M. H., Impact of initiation timing of SSRI or SNRI on depressed adolescent healthcare utilization and costs, Journal of Medical Economics, 14, 508–515, 2011 [PubMed: 21692605]	Intervention does not fit the inclusion criteria: Timing of treating Major Depressive Disorder not Babies, Children and Young people’s experience of healthcare

Measuring experience

Authors

Measuring experience

Review question

Introduction

Summary of the protocol

Table 1

Methods and process

Clinical evidence

Included studies

Excluded studies

Summary of studies included in the evidence review

Table 2

Summary of the evidence

Quality assessment of studies included in the evidence review

Evidence from reference groups and focus groups

Table 3

Evidence from national surveys

Table 4

Economic evidence

Included studies

Excluded studies

Summary of studies included in the economic evidence review

Economic model

The committee’s discussion of the evidence

Interpreting the evidence

The outcomes that matter most

The quality of the evidence

Benefits and harms

Cost effectiveness and resource use

Other factors the committee took into account

Recommendations supported by this evidence review

References

Appendices

Appendix A. Review protocol

Appendix B. Literature search strategies

Appendix C. Clinical evidence study selection

Appendix D. Clinical evidence tables

Appendix E. Forest plots

Forest plots for review question: How can the experience of babies, children and young people be measured so as to improve their experience of healthcare?

Appendix F. GRADE tables

Appendix G. Economic evidence study selection

Economic evidence study selection for review question: How can the experience of babies, children and young people be measured so as to improve their experience of healthcare?

Appendix H. Economic evidence tables

Appendix I. Economic evidence profiles

Appendix J. Economic analysis

Economic evidence analysis for review question: How can the experience of babies, children and young people be measured so as to improve their experience of healthcare?

Appendix K. Excluded studies

Excluded studies for review question: How can the experience of babies, children and young people be measured so as to improve their experience of healthcare?

Clinical studies

Table 13

Economic studies

Table 14

Appendix L. Research recommendations

Appendix M. Evidence from reference groups and focus groups

Appendix N. Evidence from national surveys

Table 1Summary of the protocol (PICO table)

Table 2Summary of included study

Table 3Summary of the evidence from reference groups and focus groups

Table 4Summary of the evidence from national surveys

Table 13Excluded studies and reasons for their exclusion

Table 14Excluded studies and reasons for their exclusion