THE SPASTICITY-RELATED QUALITY OF LIFE 6-DIMENSIONS INSTRUMENT IN UPPER-LIMB SPASTICITY: Part II A FIRST PSYCHOMETRIC EVALUATION

Lynne Turner-Stokes, DM, FRCP1,2, Klemens Fheodoroff, MD3, Jorge Jacinto, MD4, Jeremy Lambert, PhD5, Christine de la Loge, MSc5, Françoise Calvi-Gries, MSc6, John Whalen, BSc MBA7, Andreas Lysandropoulos, MD8, Pascal Maisonobe, MSc6 and Stephen Ashford, PhD FCSP1,2,9

From the 1Department of Palliative Care, Policy and Rehabilitation, Cicely Saunders Institute, Florence Nightingale Faculty of Nursing, Midwifery and Palliative Care, King’s College London, 2Regional Hyper-acute Rehabilitation Unit, Northwick Park Hospital, London, UK, 3Neurorehabilitation, Gailtal-Klinik, Hermagor, Austria, 4Centro de Medicina de Reabilitaçãode Alcoitão, Serviço de Reabilitação de adultos 3, Estoril, Portugal, 5Patient-Centred Outcomes, ICON plc, Lyon, 6Ipsen Pharma, Boulogne-Billancourt, France, 7Ipsen Biopharm Ltd, Slough, UK, 8Ipsen, Cambridge, MA, USA and 9Centre for Nursing, Midwifery and Allied Health Research, University College London Hospitals, London, UK

Objective: Psychometric evaluation of the Spasticity-
related Quality of Life 6-Dimensions instrument (SQoL-6D).

Design: A clinimetric evaluation conducted in a
multicentre, prospective, longitudinal cohort study at 8 UK sites.

Patients: Adult patients (n = 104) undergoing focal treatment of upper-limb spasticity.

Methods: The SQoL-6D was administered in the clinic at enrolment and at 8 weeks, then 1–4 days later at home to assess test-retest reliability.

Results: The SQoL-6D demonstrated adequate construct validity and unidimensionality of the scale, allowing the calculation of a Total score. Cronbach’s alpha (0.74) supported the internal consistency
reliability, while the intraclass correlation coefficient supported test-retest reliability (0.82).
Correlation coefficients with established instruments supported convergent validity, while significant differences between known-groups (of differing clinical
severity) in SQoL-6D Total score confirmed its sensitivity to both cross-sectional and longitudinal differences.

Conclusion: The SQoL-6D is a promising new measure
to assess health status for patients with upper-limb spasticity of any aetiology. Further investigation and exploration of the allocation of weights to convert the SQoL-6D to a health-related quality of life utility index, are required.

Key words: psychometrics; upper extremity; central nervous system diseases; muscle spasticity; quality of life

Accepted Oct 7, 2021; Epub ahead of print Nov 2, 2021

J Rehabil Med 2021; 53: jrm002XX

Correspondence address: Lynne Turner-Stokes, Regional Rehabilitation Unit, Northwick Park Hospital, Watford Road, Harrow, Middlesex, HA1 3UJ, UK. E-mail: lynne.turner-stokes@doctors.org.uk

lay abstract

In upper-limb spasticity, the use of the arm and hand become restricted due to “tight” muscles. Although questionnaires to assess a person’s quality of life have been developed, they do not specifically assess the impact of upper-limb spasticity. To address this deficiency, the Spasticity-related Quality of Life 6-Dimensions (SQoL-6D) questionnaire was developed (see also the companion “development” paper available in this issue).
To test that the SQoL-6D works as expected, it was completed by 104 people living with spasticity and compared with other questionnaires they completed. The results show that the SQoL-6D questions are relevant to people with spasticity. The answers were reliable when the SQoL-6D was repeated, and differences in the burden of spasticity between people grouped by the
severity of their condition were captured accurately. The SQoL-6D is a promising new measure to assess aspects of quality of life in people living with spasticity.

INTRODUCTION

Spasticity and spastic dystonia impact quality of life (QoL) and health status in terms of functional impairment, reduced activities of daily living, and restricted societal participation (1–3). Upper-limb spasticity (ULS) can occur after a stroke or other acquired brain injury (4).

The benefits of recommended treatments for ULS, such as botulinum toxin A (BoNT-A), are demonstrable at the level of impairment and daily activities (5, 6); however, the impact of treatment on health-related QoL (HRQoL) is more difficult to understand. This is due to the interaction of spasticity with other features of neurological disability, such as motor weakness and cognitive problems. Condition-specific HRQoL questionnaires are potentially more responsive to change in symptoms and more clinically useful than generic measures (7, 8); however, no condition-specific tool is available for ULS of any aetiology.

To fulfil the need for a health status measure that is sensitive to the diverse burden of patient experience in ULS and the changes in symptoms following treatment, the Spasticity-related Quality of Life 6-Dimensions instrument (SQoL-6D) was developed. This instrument might also be used in the future for evaluation of treatments for ULS. The initial development of the SQoL-6D is described in a companion article in this issue, with results supportive of the responsiveness of SQoL-6D (i.e. its ability to detect change over time) following treatment.

We present here the first formal psychometric evaluation (internal consistency, test-retest reliability, construct and concurrent validity) of the SQoL-6D as a measure of health status using a traditional psychometric approach based on classical test theory (9). The results are presented according to the framework described by Terwee et al. in 2007 for reporting the psychometric properties of health-related measurement tools, which was developed into the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) checklist (10). Our COSMIN self-assessment is shown in Appendix SI. The SQoL-6D is freely available online with a user guide, at https://www.kcl.ac.uk/cicelysaunders/resources/tools/the-spasticity-related-quality-of-life-tool-(sqol-6d).

METHODS

Study design

This multicentre, prospective, longitudinal, clinimetric evaluation, cohort study was conducted at 8 UK sites between May 2018 and October 2019 (NCT03442660). Adults (aged ≥18 years) with an understanding of English and a comprehension of the SQoL-6D questions and fulfilment of protocol requirements (as judged by the investigator), who were receiving focal treatment for ULS as part of their routine clinical management, were enrolled independently from therapeutic decisions. All participants provided informed consent.

The SQoL-6D was self-administered at enrolment (Visit 1) and at 8 weeks (±2 weeks; Visit 2/follow-up) in the clinic, by which time a maximum treatment effect was expected. Patients were asked to repeat the self-assessment at 1–4 days after Visit 2 at home, in order to assess test-retest reliability.

Intervention

This study was set in the context of a cohort with clinical interventions for spasticity, and therefore was a “non-clinical trial of investigational medicinal products”. Investigators chose healthcare strategies in accordance with participant needs and routine clinical practice; assessment tools were collected outside of normal clinical practice. As the performance of the SQoL-6D
in real-life clinical practice was to be assessed regardless of intervention, full details of interventions were not collected.

Other measures

Data collection at Visit 1 included assessment by clinicians of the severity of functional impairment using a ULS-adapted Neurological Impairment Scale (ULS-NIS) (11). The severity grade for the affected arm was used in the analysis (or the right arm if both arms were affected). Patients completed a participant experience questionnaire for SQoL-6D to assess its acceptability (time taken, ease of completion, relevance), and any questions a patient thought should be added.

Other standardised measures were recorded on Visits 1 and 2, as follows:

Patient disposition

Overall, 104 patients were enrolled across 8 sites (3–22 patients per site, representing a convenience sample). The full analysis set (FAS; n = 98) population was all adults who completed Visit 1 and the “test-retest” population (n = 78) comprising those from the FAS who had completed Visit 2 and the retest assessment within 1–4 days after Visit 2 (Fig. 1).

Subgroups of patients with different aetiology of ULS (vascular disease (stroke): infarction or haemorrhage; non-vascular disease: trauma, hypoxia, inflammatory infective, tumour, degenerative, congenital, other) were also assessed.

Missing data were minimised through use of electronic data capture whenever possible, which did not allow patients to skip items or save incomplete entries. No imputations were made for missing data. Statistical analyses were performed using Statistical Analysis System (SAS®) version 9.4 (SAS Institute Inc., Cary, NC, USA).

Statistical analysis

The analyses were all pre-planned. All statistical tests were 2-sided at the 5% significance level, unless otherwise stated, and were not conducted for inferential purposes, but as a guide to interpretation. Although both parametric and non-parametric tests were conducted, after review of score distribution and
normality tests, parametric tests are presented for summary scores and non-parametric tests for dimension scores.

As responsiveness of the SQoL-6D was of key interest, calculation of sample size was based on this parameter, as
described in the companion SQoL-6D development paper in this issue. Briefly, 87 patients were required to detect a statistically significant change (paired t-test at 5%; power of 80%) and the target for recruitment was set at 100 patients, allowing for a drop-out rate of up to 13%.

As the SQoL-6D is based on a formative model, it was not expected to be unidimensional, but its factor structure and dimensionality were explored nevertheless. Internal consistency was assessed using Cronbach’s alpha raw and standardised coefficients for the whole scale and removing one item at a time. Unidimensionality of the SQoL-6D construct was assessed through principal component analysis (PCA) and interpretation was based on the eigenvalues over 1, the reduction in their amplitude and the magnitude of the factor loadings.

Item-convergent validity was assessed by calculating the Pearson and Spearman’s correlation coefficients between each item and a score combining all other items. To demonstrate adequate item convergent validity, correlation coefficients between each item score and the corresponding SQoL-6D Total-item score (calculated without the corresponding item) were expected to be at least 0.40.

Test-retest reliability was assessed using the intraclass correlation coefficient (ICC) and linear weighted kappa coefficients between SQoL-6Ds completed at Visit 2 and the retest assessment. The difference between the test and retest dimension and Total scores were compared using a paired t-test (Wilcoxon signed-rank test) to assess for consistency and any systematic bias between the 2 assessments.

In the absence of an accepted gold standard, the convergent and known-groups validity of the SQoL-6D was assessed by exploring the relationship between SQoL-6D scores and severity of spasticity presentation at Visits 1 and 2, using different disease severity criteria. Convergent validity was assessed using Pearson and Spearman’s correlation coefficients between SQoL-6D individual dimension and Total scores and the various scores available, and derived from ULS-NIS, MAS and ArmA questionnaires with higher item scores associated with a worse condition. The MAS and ArmA have been specifically developed and validated for use in patients with ULS (12, 13). Low-to-moderate positive and negative correlations were expected with the relevant items of the SQoL-6D, and with the Total score, respectively. The known-groups validity was tested by examining differences in mean scores between patients with differing severity of weakness (Table I).

RESULTS

Demographic and disease characteristics are shown in
Table II. Patients with vascular aetiology had a significantly shorter duration of spasticity than the non-vascular group, although the overall severity grade of impairment (ULS-NIS score, total range 0–21) was similar (mean (standard deviation; SD) 12.7 (3.7) vs 12.4 (3.2), respectively). The aetiology of spasticity for the non-vascular group (n = 25) was mainly trauma (n = 10, 40%), congenital (n = 5, 20%) or degenerative (n = 3, 12%). The study population demographics are broadly similar to those of other published large international cohorts with ULS (16, 17).

Acceptability

Of the 92 patients who completed the SQoL-6D completion experience form, 75% stated that it took less than 10 min to complete and all patients thought the questions were relevant. Approximately one-quarter (27%) of patients expressed some difficulty completing the questionnaire, perhaps due to spasticity interfering with the hand selecting answers. Eighty-seven patients (95%) had no further questions to add.

Distribution of SQoL-6D scores

At the dimension level, a ceiling effect was seen for “Using the affected limb”, where more than half of patients in the FAS (53%) had the highest score, indicating worst condition, at Visit 1, and no further worsening could be captured for these patients. This ceiling effect remained at Visit 2 with 40% of patients at the highest score.

A floor effect was seen at Visit 2 for the dimension “Involuntary movements or spasms”, where 43% of patients had the lowest score, indicating they had no more involuntary movement and could not improve further on this dimension. No other significant floor or ceiling effects were seen in other dimensions and, most importantly, no floor or ceiling effect was observed for the SQoL-6D Total score. Only one patient had the lowest possible Total score at Visit 1.

Internal consistency, factor structure and dimensionality

Internal consistency was assessed for the FAS (n = 98, no missing data) at Visit 1. Cronbach’s alpha coefficient for the Total score was 0.74, supporting acceptable internal consistency reliability (a value of 0.70 or above was considered to reflect adequate internal consistency reliability) (18). When one dimension in turn was removed, the Cronbach’s alpha decreased to below 0.70 for 3 of 6 items, indicating reduced reliability when excluding 1 item (Table III).

PCA presenting the 2 factors having an eigenvalue of greater than 1, explained 45%, and 18% of the variance, respectively, leading to a cumulative proportion of 62% (Table IV). The first factor can be interpreted as a ULS severity factor with all 6 items contributing positively and substantially (factor loadings ranging from 0.51 to 0.80) to its construction. The second factor is driven positively by item 2 (involuntary movements) and, to a lesser degree, by item 1 (pain/discomfort) and negatively by items 5, 3 and 6. The reduction in eigenvalues from first to second factor (2.68 and 1.05, respectively) and the large factor loadings for all 6 dimensions on the first factor support the unidimensionality of the SQoL-6D and thus the appropriateness of calculating a Total score.

Correlations between each SQoL-6D dimension and the summary score calculated without the considered item ranged from –0.33 to –0.61, suggesting consistency between items and an absence of redundancy (Table V).

Test-retest reliability

The observed mean duration between follow-up and retest was 1.8 days (range 0–5 days). Dates and times of assessments were checked, leading to exclusion of one patient who completed both Visit 2 and the retest questionnaire within 2 h. Interpretation of ICC values was based on published work by Koo et al., where values less than 0.5 were defined as “poor”, between 0.5 and 0.75 as “’moderate”, between 0.75 and 0.9 as “good”’, and greater than 0.90 as “excellent”’ reliability (19). The Total score indicated good test-retest reliability, while, at the dimension level, ICCs indicated moderate to good reliability (Table VI). Similarly, weighted kappa values reflected fair to good test-retest reliability. There was a small, but significant, worsening of the Total score between the 2 assessments (mean –5.6 (95% CI –8.5, –2.7); effect size = –0.3). At item level, this reflected a statistically significantly poorer QoL for 3 of the items (“involuntary movements”, “range of movement” and “caring for the affected limb”) when rated at home.

Convergent and known-groups validity

As the SQoL-6D is designed to be a measure of spasticity-related health status, we did not expect to find a marked difference in scores between groups defined by the level of cognitive impairment. However, we expected to find evidence of poorer health status in those with more severe spasticity (as measured by MAS) and with more severe difficulty with caring for and using the affected limb (as measured by ArmA). Low-to-moderate correlations were anticipated between SQoL-6D scores and scores derived from the ULS-NIS, the MAS, the EQ-5D-5L and the ArmA. At enrolment, correlation coefficients in the FAS (n = 98) were moderate to strong, as expected (Table VII). The highest correlations were generally seen between scores measuring similar concepts. Correlations involving the SQoL-6D Total score were of medium to large amplitude, the highest being with the EQ-5D-5L index score (ρ = 0.77; p < 0.0001). These results provide evidence of convergent validity of the SQoL-6D.

The SQoL-6D Total score for known-groups based on the ULS-NIS, MAS and ArmA scores showed, on average, that patients with severe spasticity, motor impairment and poorer function in the affected upper limb tended to have higher dimension mean scores reflecting higher severity in all SQoL-6D dimensions, compared with those without severe symptoms (Table VIII). This translates into significantly lower total mean score when transformed to a total HRQoL score on a range of 0–100, supporting the known-groups validity of the SQoL-6D.

A summary of key results is presented in Fig. 2.

DISCUSSION

This first psychometric evaluation of the SQoL-6D in a UK multicentre cohort of patients with ULS demonstrates acceptable reliability and validity of the SQoL-6D Total score. In addition, responsiveness to clinical change was supported in analyses presented in the companion SQoL-6D development paper in this issue. Indeed, the results of this study were, in general, as expected and provided statistics that were beyond desired thresholds. The SQoL-6D demonstrated adequate construct validity and unidimensionality, allowing the calculation of a Total summary score. The calculated Cronbach’s alpha value of 0.74 supported the internal consistency reliability, and ICC supported the test-retest reliability (ICC=0.82).

The small, but significant, worsening of the Total score between the 2 assessments in test-retest, at item level, reflected a poorer health status for 3 of the items (“involuntary movements”, “range of movement” and “caring for the affected limb”) when rated at home, although the other items remained stable. While the 2 questionnaires were self-administered with the same instructions, it is possible that the change of environment between clinic- and home-based administrations may have introduced an element of systematic bias in the patients’ perception of health with respect to these 3 domains.

Correlation coefficients with established instruments supported the convergent validity, while significant differences in SQoL-6D Total score between known-groups confirmed the sensitivity of the Total score to both cross-sectional and longitudinal differences. The latter was further supported by the effect sizes. As the SQoL-6D is designed to be a measure of spasticity-related health status, marked differences in known-groups scores were expected, with evidence of poorer health expected in those with more severe spasticity and with difficulty caring for and using the affected limb.

Despite the fact the SQoL-6D was developed on a formative model, given the diverse nature of the goal areas on which it is based, it showed greater unidimensionality than expected. As a result of this unidimensionality, rotational and other confirmatory factor analyses were not deemed to be required. Further exploration has to be undertaken to determine whether these findings are reproducible in a multi-national population.

Ideally, measurement of changes in disease burden should be reported directly from patients, and the utility of these changes should be based on public preferences (20). The findings presented here provide encouraging preliminary evidence for the usefulness of the SQoL-6D as a measure of health status in patients with ULS. In terms of discriminative validity, the SQoL-6D appeared to be more sensitive to change than the EQ-5D-5L Visual Analogue Scale or index scores, and, in future, it may provide more accurate economic evaluations of treatments for ULS in combination with generic health utility tools (health utility index, SF-6D, EQ-5D-5L) (21–23). The next stage would be to develop the SQoL-6D into an economic evaluation tool by developing utility weights for the different items using either a direct approach using a choice-based method, such as the standard gamble (24) or time trade-off (TTO) (25), or an indirect approach by mapping from a disease-specific HRQoL instrument on to the utility algorithm of a generic instrument, such as the EuroQoL-5 dimensions (EQ-5D).

As part of this study, a preliminary exploration of TTO was undertaken with 250 healthy volunteers; however, they found it very difficult to assign weights to items, having not experienced spasticity. This was particularly the case for the impact of spasticity over and above other elements of neurological impairment, such as motor paralysis. TTO of the SQoL-6D is currently under exploration amongst a population of patients with ULS; the results will be published separately in due course.

Strengths and limitations

This study has a number of strengths and weaknesses.

Strengths include the multicentre sample, and the fact that the study was conducted in real-life clinical practice, both of which help to support the generalisability of the findings. Although the sample size of just under 100 patients was fairly modest, the study was adequately powered for its intended purpose.

Weaknesses include the fact that the study was confined to just one country and, so far, only traditional psychometric techniques based on classical test theory have been applied. Future analyses should explore the performance of the SQoL-6D in a multi-national study that includes patients from other English-speaking countries (e.g. the US and Australia) and the application of modern psychometric methodologies based on item response theory, such as Rasch analysis. The subsequent roll-out and translation to other more disabled or non-English-speaking populations is a further stage in development, which will need to be addressed in future research.

In conclusion, the SQoL-6D is a promising new tool to assess key components of health status at a personal level for patients with ULS. Further investigation in wider settings, as well as further development to explore the allocation of weights to convert the SQoL-6D to a HRQoL utility index, are warranted.

ACKNOWLEDGEMENTS

The authors thank all patients involved in the study, as well as their caregivers, care team, investigators and research staff in participating institutions. The authors would like to acknowledge in particular the local site investigators, who were as follows:

The authors would also like to thank Tanguy Vilcot and Mickael Henry-Szatkowski (ICON plc) for their support in running the statistical analyses, Aasif Motala (ICON plc) for his support with the data collection and site monitoring, Benoit Arnould (ICON plc) for his scientific input during the study, and Professor Turner-Stokes who prepared the first draft of this paper. The authors also thank Nicola Winstone, DPhil, and
Germanicus Hansa-Wilkinson, MSc, of Ashfield MedComms, UK, an Ashfield Health company, for providing medical writing and editorial support, which was sponsored by Ipsen in accordance with Good Publication Practice guidelines.

Disclosures. LTS has a specific interest in outcomes evaluation and has published on the use of Goal Attainment Scaling in this context, as well as standardised measures, such as the
GAS-eous, NIS, Arm and Leg Activity measures (ArmA and LegA). These tools are freely available. She has received honoraria and travel grants from Ipsen and Merz. She has no personal financial interest in any of the material mentioned in this article. KF has a specific interest in ICF-based outcomes evaluation goal setting. He has received unrestricted research grants from Ipsen and has received honoraria from Allergan (now AbbVie), Ipsen and Merz. He has no personal financial interest in any of the material mentioned in this article. JJ has received honoraria for services such as scientific advisor, clinical researcher, trainer and speaker from Allergan (now AbbVie), Ipsen and Merz. JL is an ICON plc employee and a paid consultant to Ipsen. CDLL is a paid consultant to ICON plc. FCG, JW, AL and PM are employees of Ipsen. SA has a specific interest in outcomes evaluation and has published on the use of Goal Attainment Scaling in this context, as well as standardised measures such as the Arm and Leg Activity measures (ArmA and LegA). These tools are freely available. He has received 2 unrestricted research grants from Ipsen and has received honoraria from Allergan (now AbbVie), Ipsen, Merz and Nutricia. He has no personal financial interest in any of the material mentioned in this article.

Data sharing. Ipsen will share aggregated data that underlie the results reported in this article with qualified researchers who provide a valid research question, and subject to an appropriate data sharing agreement. Study documents, such as the study protocol and clinical study report, are not always available. Proposals should be submitted to DataSharing@Ipsen.com and will be assessed by a scientific review board. Data are available beginning 6 months and ending 5 years after publication; after this time, only raw data may be available.

Ethics. This study was an interventional, non-clinical trial of an investigational medicinal product (non-CTIMP) and therefore fell outside the scope of the European Union (EU) Directive 2001/20/EC and the EU Directive 2005/28/EC. The study was defined as “interventional”, not because of the use of an investigational product, but because of data collection procedures imposing multiple questionnaire completion and 1 on-site visit that were not necessarily part of routine clinical practice. Before initiating the study, ethics permission was granted by the UK Health Research Authority IRAS number: 232862.

All patients provided written informed consent prior to participating in any study-related activities.

Funding and management. This study was sponsored by Ipsen. Statistical analysis was performed by an external CRO (ICON plc) on behalf of Ipsen Pharma. London North West University Healthcare Trust (Northwick Park Hospital), UK, and King’s College London, UK, contributed to time spent by the lead author in preparation of the manuscript and the last author (SA) in supporting approvals and oversite of data collection.

REFERENCES

  1. Barnes M, Kocer S, Murie Fernandez M, Balcaitiene J, Fheodoroff K. An international survey of patients living with spasticity. Disabil Rehabil 2017; 39: 1428–1434.
  2. Ghai A, Garg N, Hooda S, Gupta T. Spasticity – pathogenesis, prevention and treatment strategies. Saudi J Anaesth 2013; 7: 453–460.
  3. Pandyan AD, Gregoric M, Barnes MP, Wood D, Van Wijck F, Burridge J, et al. Spasticity: clinical perceptions, neurological realities and meaningful measurement. Disabil Rehabil 2005; 27: 2–6.
  4. Angulo-Parker FJ, Adkinson JM. Common etiologies of upper extremity spasticity. Hand Clin 2018; 34: 437–443.
  5. Childers MK, Brashear A, Jozefczyk P, Reding M, Alexander D, Good D, et al. Dose-dependent response to intramuscular botulinum toxin type A for upper-limb spasticity in patients after a stroke. Arch Phys Med Rehabil 2004; 85: 1063–1069.
  6. McCrory P, Turner-Stokes L, Baguley IJ, De Graaff S, Katrak P, Sandanam J, et al. Botulinum toxin A for treatment of upper limb spasticity following stroke: a multi-centre randomized placebo-controlled study of the effects on quality of life and other person-centred outcomes. J Rehabil Med 2009; 41: 536–544.
  7. Wiebe S, Guyatt G, Weaver B, Matijevic S, Sidwell C. Comparative responsiveness of generic and specific quality-of-life instruments. J Clin Epidemiol 2003; 56: 52–60.
  8. Wilson IB, Cleary PD. Linking clinical variables with health-related quality of life. A conceptual model of patient outcomes. JAMA 1995; 273: 59–65.
  9. Crocker L, Algina J. Introduction to classical and modern test theory. New York: Holt, Rinehart, and Winston; 1986.
  10. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007; 60: 34–42.
  11. Turner-Stokes L, Thu A, Williams H, Casey R, Rose H, Siegert RJ. The Neurological Impairment Scale: reliability and validity as a predictor of functional outcome in neurorehabilitation. Disabil Rehabil 2014; 36: 23–31.
  12. Bohannon RW, Smith MB. Interrater reliability of a modified Ashworth scale of muscle spasticity. Phys Ther 1987; 67: 206–207.
  13. Ashford S, Slade M, Turner-Stokes L. Conceptualisation and development of the arm activity measure (ArmA) for assessment of activity in the hemiparetic arm. Disabil Rehabil 2013; 35: 1513–1518.
  14. Devlin NJ, Shah KK, Feng Y, Mulhern B, van Hout B. Valuing health-related quality of life: An EQ-5D-5L value set for England. Health Econ 2018; 27: 7–22.
  15. Turner-Stokes L, Ashford SA. The GAS-eous tool. 2013. Available from: http://www.kcl.ac.uk/lsm/research/divisions/cicelysaunders/attachments/Tools-GASeous-GASeous-tool.pdf Accessed 18th October 2021.
  16. Turner-Stokes L, Fheodoroff K, Jacinto J, Maisonobe P. Results from the Upper Limb International Spasticity Study-II (ULIS-II): a large, international, prospective cohort study investigating practice and goal attainment following treatment with botulinum toxin A in real-life clinical management. BMJ Open 2013; 3: e002771.
  17. Turner-Stokes L, Fheodoroff K, Jacinto J, Maisonobe P, Ashford S. ULIS (Upper Limb International Spasticity), a 10-year Odyssey: an international, multicentric, longitudinal cohort of person-centered spasticity management in real-life practice. J Int Soc Phys Rehabil Med 2019; 2: 138–150.
  18. Nunnally JC. Psychometric theory. New York: McGraw-Hill; 1978.
  19. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15: 155–163.
  20. Torrance GW, Thomas WH, Sackett DL. A utility maximization model for evaluation of health care programs. Health Serv Res 1972; 7: 118–133.
  21. Horsman J, Furlong W, Feeny D, Torrance G. The Health Utilities Index (HUI): concepts, measurement properties and applications. Health Qual Life Outcomes 2003; 1: 54.
  22. Brazier J, Roberts J, Deverill M. The estimation of a preference-based measure of health from the SF-36. J Health Econ 2002; 21: 271–292.
  23. Payakachat N, Ali MM, Tilford JM. Can the EQ-5D detect meaningful change? A systematic review. Pharmacoeconomics 2015; 33: 1137–1154.
  24. von Neumann J, Morgenstern O. Theory of games and economic behavior. New York: John Wiley, 1953.
  25. Arnesen T, Trommald M. Are QALYs based on time trade-off comparable? – a systematic review of TTO methodologies. Health Econ 2005; 14: 39–53.