The multiple sclerosis impact scale (MSIS-29) is a reliable and sensitive measure
http://www.100md.com
《神经病学神经外科学杂志》
Department of Neurology, University College Dublin and St Vincent’s University Hospital, Dublin, Ireland
Correspondence to:
Dr C McGuigan
St Vincent’s University Hospital, Elm Park, Dublin 4, Ireland; cmcguigan@doctors.org.uk
ABSTRACT
Objective: The aim of this study was to assess the psychometric properties of the Multiple Sclerosis Impact Scale (MSIS-29) for patients in the community and in a hospital setting.
Methods: During an epidemiological study, 172 people with multiple sclerosis (MS) were examined and completed the MSIS-29, the London Handicap Scale, and Beck’s Depression Inventory; disability was assessed by the Kurtzke Expanded Disability Status Score (EDSS) and the Multiple Sclerosis Functional Composite. At the hospital neurology clinic, 102 MS patients completed the MSIS-29 and EDSS assessments were performed; 41 of these patients had repeat evaluations six months later. The psychometric properties of the MSIS-29 were examined.
Results: In the 172 community and the 102 hospital patients the psychometric properties of the MSIS-29 were satisfactory, with high convergent and low divergent validity. It was significantly responsive to change in the contexts of self-reported change (p<0.034) and EDSS worsening (p<0.001). The MSIS-29 physical score did not change over time when the EDSS was stable, and increased significantly in proportion to EDSS deterioration (p = 0.014).
Conclusions: The psychometric properties of the MSIS-29 are acceptable; it is a valuable outcome measure in intervention studies of patients with MS.
Keywords: multiple sclerosis; Multiple Sclerosis Impact Scale (MSIS-29); outcome measures; psychometric properties
Abbreviations: MS, multiple sclerosis; MSIS, Multiple Sclerosis Impact Scale; EDSS, Expanded Disability Status Score; MSFC, Multiple Sclerosis Functional Composite; FAMS, Functional Assessment of Multiple Sclerosis; MSQLI, Multiple Sclerosis Quality of Life Inventory; UKNDS, United Kingdom Neurological Disability Scale; MSQOL, Multiple Sclerosis Quality of Life; HRQOL, Health Related Quality of Life; LHS, London Handicap Score; BDI, Beck’s Depression Inventory
The assessment by patients of their own state of well-being and of the limitations imposed by a condition is increasingly recognised as a valid part of the assessment of therapeutic interventions in chronic diseases. Two instruments commonly used in outcome assessment and administered by physicians are the Kurtzke Expanded Disability Status Scale (EDSS)1 and the Multiple Sclerosis Functional Composite (MSFC).2 The EDSS, developed before the acceptance of psychometric methods of scale development,3 has a number of problems, including: an ordinal scale; rater variability; poor reliability; insensitivity to change at certain levels; and an emphasis on mobility status.4,6 The more recently introduced measure, the Multiple Sclerosis Functional Composite (MSFC), has been shown to be a sensitive and responsive instrument in the combined assessment of cognitive, upper limb, and gait disorder.2,7,8
Comprehensive self-reported measures of disease impact, both generic and specific to the disease, have been widely assessed in multiple sclerosis (MS). The Short Form Health Survey Scale has been frequently used in the MS population, but it does not evaluate many symptoms specific to MS, and the high frequency of floor and ceiling effects indicates a lack of responsiveness in this disease.9,11 We evaluated measures specific to the disease, including: the Functional Assessment of MS (FAMS)12; the Multiple Sclerosis Quality of Life Inventory (MSQLI)13; the UK Neurological Disability Scale (UKNDS)14; the MS Quality of Life Score (MSQOL-54)15; and the Health Related Quality of Life in MS Measure (HRQOL-MS).16 Although useful in varying degrees, none of these measures used scientifically based psychometric methods of scale construction; the first such validated measure of disease impact specific to the disease and rated by the patient, the Multiple Sclerosis Impact Scale (MSIS-29), was published in 2001.17 The MSIS-29 consists of 29 questions of which 20 address the physical impact component and nine assess the psychological impact; a combined score can be generated, or both components can be reported separately. The reliability and validity of the MSIS-29 were assessed by a postal survey of members of the United Kingdom MS society, and its responsiveness was evaluated in a patient population undergoing treatment.17 The validity of the measure in persons undergoing rehabilitation in MS has also been assessed.18
The MSIS-29 has not been independently validated in other MS populations. It is the aim of this paper to examine the reliability, validity, and responsiveness of the MSIS-29 in MS patients both in a community setting and attending the hospital neurology outpatient department.
PARTICIPANTS AND METHODS
Two groups of subjects were studied.
Individuals with MS in the community
People with clinically definite or probable MS (Poser criteria)19 were identified during the course of an epidemiological study in Counties Wexford and Donegal in the Republic of Ireland. A total of 366 cases were ascertained in both counties (126 in Wexford and 240 in Donegal), and of these 71 (56.3%) in Wexford and 101 (42.1%) in Donegal were interviewed and agreed to take part in the study. There were no demographic differences between study participants and non-participants.
Cross-sectional study
Kurtzke EDSS scores and MSFC scores were assessed by C McG. Subjects were also asked to complete the MSIS-29; the London Handicap scale (LHS), a generic measure of handicap20; and Beck’s Depression Inventory (BDI-II), a validated, generic measure of depression.21
Test-retest reliability
The 71 participants enrolled in the study from County Wexford were asked to complete a second MSIS-29 form six months after the first assessment, by postal survey. They were also asked whether their physical condition had improved, remained stable, or deteriorated since the first assessment. This information was used in the determination of item test-retest reliability.
Individuals with MS attending outpatients
Cross-sectional assessment
Of patients attending the neurology outpatient department with clinically definite or probable MS (Poser criteria),19 102 were seen by both authors and completed MSIS-29 questionnaires; the Kurtzke EDSS scores were also assessed.
Responsiveness
Of these patients, 55 were prospectively followed over six months; at each visit they were asked to complete an MSIS-29 questionnaire and the EDSS was assessed. In 41 cases an MSIS-29 questionnaire and a Kurtzke EDSS score were obtained at both time points. The EDSS was assessed by both authors without knowledge of the MSIS-29 score; the authors had trained in EDSS assessment together to improve inter-rater reliability. On the basis of the difference in EDSS scores between the two time points (and before the MSIS-29 was scored), the 41 patients were assigned to one of two categories: either "Changed" (if the two EDSS scores differed by 1.0 or more), or "Static" (if the difference between the two scores was not more than 0.5 points). The ethics committee of St. Vincent’s University Hospital, Dublin, granted approval for the study.
Statistical methods
The groups were assessed using standard psychometric techniques including:3,22
data quality: percentage missing data and percentage computable scores
scaling assumptions: item mean scores and standard deviations and item to total correlations
acceptability as determined by score range, mean scores, floor/ceiling effects, and skewness
reliability: Cronbach’s alpha was estimated for all samples. The standard error of measurement was used to calculate the 95% confidence interval for individual scores. Item test-retest reliability was assessed in the Wexford population who completed a second MSIS-29 by postal survey, and also in the "Static" group of hospital patients whose Kurtzke EDSS scores had not changed over a six month period
validity: the convergent validity of the MSIS-29 physical scale was tested against the Kurtzke EDSS scale, the MSFC, and the LHS in the community population, and against the Kurtzke EDSS scale only in the outpatient population. The divergent validity of the MSIS-29 physical scale was examined against the MSIS-29 psychological scale and the BDI-II. The convergent validity of the MSIS-29 psychological scale was assessed in the community participants only against the BDI-II, and the divergent validity was measured against the EDSS and the MSIS-29 physical scale. A Spearman’s rank correlation coefficient was used to test the strength of the correlations. Rasch analysis was used to examine correlations that were greater than expected23
responsiveness: the responsiveness of the MSIS-29 scale was evaluated by comparing the mean scores at the assessments at the two time points in the "Changed" subgroup of the 41 hospital outpatients. A Wilcoxin Signed Rank test was used to assess the significance of the mean differences between groups. Effect sizes (mean change score / standard deviation of Time 1 scores) were calculated for the "Changed" and "Static" groups.24
RESULTS
Populations
The demographic details of the 172 participants from the epidemiological study and the 102 hospital attendees are given in table 1. The former group is more representative of the community MS population, with a wider range of disability and a more realistic proportion of primary progressive MS patients.
Table 1 Participant characteristics for each population
MSIS-29 psychometric properties
Data quality
The percentage missing data for the MSIS-29 was very low at 0–2.8%. Scores were computable for all participants (table 2).
Table 2 Psychometric properties of the MSIS-29 physical and psychological scales in each population
Scaling assumptions
Items within each sample had similar means and standard deviations. Frequency results for each item were well distributed. Item-total correlations were acceptable but consistently less for the MSIS-29 psychological score in comparison with the physical score (table 2).
Acceptability
Floor/ceiling effects were very low for all samples. Score ranges spanned from 86.1% to 98.68% of the possible range. The mean scores were near the scale midpoints and were not notably skewed (table 2).
Reliability
All estimates of reliability using Cronbach’s alpha were in excess of the recommended 0.80. The 95% confidence intervals for the MSIS-29 physical scales were narrower in all groups compared with the MSIS-29 psychological scale, indicating the expected increase in precision due to the number of items (table 2).
Validity
The convergent validity of the MSIS-29 physical scale with other measures of physical impairment is indicated by the high correlations found with the Kurtzke EDSS (0.704), the LHS (0.843), and the MSFC (0.577). In a test of divergent validity, the correlation of the MSIS-29 physical scale and the BDI-II at 0.399 is higher than would have been expected. Rasch analysis, a statistical method using interval level measurement, was applied to the MSIS-29 results of patients scoring between 0 and19 on the BDI-II scale (minimally to mildly depressed), and between 20 and 60 (moderately to severely depressed). The analysis confirmed that the MSIS-29 score items were operating similarly in the two populations, and therefore were not unduly influenced by mood. The greater than expected correlation is likely to highlight the problems associated with applying parametric analyses to interval measure scores.
The MSIS-29 psychological scale scores correlated highly with the BDI-II, as predicted (0.799). The divergent validity of the MSIS-29 psychological impact scale was confirmed when assessed against the EDSS (0.095) and the MSFC (0.020). The correlation between the MSIS-29 physical and psychological scales was 0.438, showing, as expected, that they measure similar but distinct constructs.
Test-retest reliability
Community postal survey. In the community study, 58 of 71 (81%) patients who
"Static" hospital group. There were 24 patients with unchanged EDSS scores. Their mean MSIS-29 physical score at the first assessment was 27.1 compared with 29.1 at the second assessment. There was no significant change in their mean MSIS-29 psychological scores from first to second assessments.
Responsiveness
The responsiveness of the MSIS-29 was measured for the 17 hospital attendees who were assessed as having worsened or improved by one point or more in the EDSS between the two time points. The mean change in the EDSS was 1.35 (range 1–2.5). The mean difference in the MSIS-29 physical impact score from first to second examination was 18.16 (Z = -3.259, p = 0.001); the mean difference in the MSIS-29 psychological impact scores was 12.25 (Z = -1.967, p = 0.049). Table 3 shows the mean MSIS-29 physical and psychological scores at both time points, together with the range in scores and mean differences for both the "Static" and the "Changed" groups according to change in the EDSS scores. The effect size for the change in the MSIS-29 physical score in the "Static" group was 0.43 compared with 1.25 for the "Changed" group, further supporting the responsiveness to change of measure. The moderate effect size noted in the "Static" group is likely to represent the lack of fine sensitivity of the EDSS scale in detecting minimal change in MS disability. The differences in the mean MSIS-29 physical score over the six month period when plotted against change in the EDSS (Figure 1) indicates that the relationship between these measures is highly correlated (MSIS-29 physical change v EDSS change, R2 = 0.9, p = 0.014).
Table 3 Mean scores, difference in mean scores and median scores with interquartile range for each scale at both time points displayed according to change in the Kurtzke EDSS score
Figure 1 The difference in the means of the MSIS-29 physical scores for the 41 patients enrolled in the responsiveness study plotted against the observed change in the Kurtzke EDSS score.
DISCUSSION
The objective of this study was to assess the psychometric properties of the MSIS-29 physical and psychological scales in two different populations with MS in Ireland, and to determine their effectiveness as an outcome measure in population based studies and clinical trials. The MSIS-29 is an easy instrument to administer, taking approximately five to 10 minutes to complete. The questionnaire was acceptable to patients, and no particular areas of concern were raised with regard to the content or the phrasing of the individual items.
The psychometric analysis revealed the consistency of the results observed for both community and outpatient based populations; the results are similar to the published results of the reference populations.17,18 The scale is of use in the full range of impairments, disabilities, and handicaps seen in the MS population (other than dementia). The correlations of the MSIS-29 physical scale with MSFC and the EDSS, and of the psychological scale with the BDI-II, indicate that the measures identify the appropriate impact status. Initial assessment suggested a higher than expected correlation between the MSIS-29 physical scale and the BDI-II; further analysis by Rasch techniques showed that the physical impact scale operated similarly in depressed and in euthymic patients, allowing it to be used with confidence irrespective of the patient’s affect.
The results of the test-retest analysis and of the study of responsiveness to change suggest that the MSIS-29 is not only of use in cross-sectional studies but may also be used longitudinally to monitor disease progression. The MSIS-29 was responsive for both change perceived by participants in the community sample, and change as assessed by the EDSS in the outpatient sample. However, change perceived by participants and change rated by observers may not be concordant, and measuring both in the same cohort would help to establish further the sensitivity to change of the MSIS-29. The results are therefore consistent with a recent report of the relationship between another self-assessed measure of disability, the Guy’s Neurological Disability Scale, and with the Kurtzke EDSS scale, strengthening the case for the use of self-assessed measures of disease impact in future clinical trials.25
Further validation by other MS centres is required, particularly in the longitudinal assessment of the MSIS-29 to confirm or deny its responsiveness in comparison with recognised instruments such as the EDSS and the MSFC. The scale offers a promising, scientifically developed, self-reported measure of both the physical and the psychological impacts of MS, for use in clinical trials, population based studies, and continuing evaluation of individual patients.
REFERENCES
Kurtzke JF. Rating neurological impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology 1983;33:1444–52.
Cutter GR, Baier ML, Rudick RA, et al. Development of a multiple sclerosis functional composite as a clinical trial outcome measure. Brain 1999;122:871–82.
Nunnally JC, Bernstein IH. Psychometric theory, 3rd edn. New York: McGraw-Hill, 1994.
Hobart J, Freeman J, Thompson A. Kurtzke scales revisited: the application of psychometric methods to clinical intuition. Brain 2000;123:1027–40.
Sharrack B, Hughes RA, Soudain S, et al. The psychometric properties of clinical rating scales used in multiple sclerosis. Brain 1999;122:141–59.
Thompson A, Hobart JC. Multiple sclerosis: assessment of disability and disability scales. J Neurol 1998;245:189–96.
Kalkers NF, de Groot V, Lazeron RHC, et al. MS functional composite: relation to disease phenotype and disability strata. Neurology 2000;54:1233–9.
Cohen JA, Fischer JS, Bolibrush DM, et al. Intrarater and interrater variability of the MS functional composite outcome measure. Neurology 2000;54:802–6.
Vickrey BG, Hays RD, Harooni R, et al. A health-related quality of life measure for multiple sclerosis. Qual Life Res 1995;4:187–206.
Hobart J, Freeman J, Lamping D, et al. The SF-36 in multiple sclerosis: why basic assumptions must be tested. J Neurol Neurosurg Psychiatry 2001;71:363–70.
Peto V, Jenkinson C, Fitzpatrick R, et al. The development and validation of a short measure of functioning and well being for individuals with Parkinson’s disease. Qual Life Res 1995;4:241–8.
Cella DF, Dineen K, Arnason B, et al. Validation of the functional assessment of multiple sclerosis quality of life instrument. Neurology 1996;47:129–39.
LaRocca NG, Ritvo PG, Miller DM, et al. ‘Quality of life’ assessment in multiple sclerosis clinical trials: current status and strategies for improving multiple sclerosis clinical trial design. In: Goodkin DE, Rudick RA, eds. Multiple sclerosis: advances in clinical trial design, treatment and future perspectives. London: Springer-Verlag, 1996:145–60.
Sharrack B, Hughes RAC. The Guy’s Neurological Disability Scale (GNDS): a new disability measure for multiple sclerosis. Mult Scler 1999;5:223–33.
Vickrey BG, Hays RD, Harooni R, et al. A health-related quality of life measure for multiple sclerosis. Qual Life Res 1995;4:187–206.
Pfennings LE, Van der Ploeg HM, Cohen L, et al. A health-related quality of life questionnaire for multiple sclerosis patients. Acta Neurol Scand 1999;100:148–55.
Hobart J, Lamping D, Fitzpatrick R, et al. The multiple sclerosis impact scale (MSIS-29). A new patient-based outcome measure. Brain 2001;124:962–73.
Riazi A, Hobart J, Lamping D, et al. Multiple sclerosis impact scale (MSIS-29): reliability and validity in hospital based samples. J Neurol Neurosurg Psychiatry 2002;73:701–4.
Poser CM, Paty DW, Scheinberg L, et al. New diagnostic criteria for multiple sclerosis: guidelines for research protocols. Ann Neurol 1983;13:227–31.
Harwood RH, Rogers A, Dickinson E, et al. Measuring handicap: the London handicap scale, a new outcome measure for chronic disease. Qual Health Care 1994;3 (1) :11–16.
Robinson BE, Kelley L. Concurrent validity of the Beck Depression Inventory as a measure of depression. Psychol Rep 1996;79:929–30.
Staquet MJ, Hays RD, Fayers PM. Quality of life assessment in clinical trials: methods and practice, 1st edn. Oxford: Oxford University Press, 1998.
Rasch G. Probabilistic models for some intelligence and attainment tests. Chicago: University of Chicago Press, 1960.
Kaziz LE, Anderson JJ, Meenan RF. Effect sizes for determining changes in health status. Med Care 1989;27:S178–89.
Hoogervorst ELJ, Eikelenboom MJ, Uitehaag BMJ, et al. One year changes in disability in multiple sclerosis: neurological examination compared with patient self report. J Neurol Neurosurg Psychiatry 2003;74:439–42.(C McGuigan and M Hutchins)
Correspondence to:
Dr C McGuigan
St Vincent’s University Hospital, Elm Park, Dublin 4, Ireland; cmcguigan@doctors.org.uk
ABSTRACT
Objective: The aim of this study was to assess the psychometric properties of the Multiple Sclerosis Impact Scale (MSIS-29) for patients in the community and in a hospital setting.
Methods: During an epidemiological study, 172 people with multiple sclerosis (MS) were examined and completed the MSIS-29, the London Handicap Scale, and Beck’s Depression Inventory; disability was assessed by the Kurtzke Expanded Disability Status Score (EDSS) and the Multiple Sclerosis Functional Composite. At the hospital neurology clinic, 102 MS patients completed the MSIS-29 and EDSS assessments were performed; 41 of these patients had repeat evaluations six months later. The psychometric properties of the MSIS-29 were examined.
Results: In the 172 community and the 102 hospital patients the psychometric properties of the MSIS-29 were satisfactory, with high convergent and low divergent validity. It was significantly responsive to change in the contexts of self-reported change (p<0.034) and EDSS worsening (p<0.001). The MSIS-29 physical score did not change over time when the EDSS was stable, and increased significantly in proportion to EDSS deterioration (p = 0.014).
Conclusions: The psychometric properties of the MSIS-29 are acceptable; it is a valuable outcome measure in intervention studies of patients with MS.
Keywords: multiple sclerosis; Multiple Sclerosis Impact Scale (MSIS-29); outcome measures; psychometric properties
Abbreviations: MS, multiple sclerosis; MSIS, Multiple Sclerosis Impact Scale; EDSS, Expanded Disability Status Score; MSFC, Multiple Sclerosis Functional Composite; FAMS, Functional Assessment of Multiple Sclerosis; MSQLI, Multiple Sclerosis Quality of Life Inventory; UKNDS, United Kingdom Neurological Disability Scale; MSQOL, Multiple Sclerosis Quality of Life; HRQOL, Health Related Quality of Life; LHS, London Handicap Score; BDI, Beck’s Depression Inventory
The assessment by patients of their own state of well-being and of the limitations imposed by a condition is increasingly recognised as a valid part of the assessment of therapeutic interventions in chronic diseases. Two instruments commonly used in outcome assessment and administered by physicians are the Kurtzke Expanded Disability Status Scale (EDSS)1 and the Multiple Sclerosis Functional Composite (MSFC).2 The EDSS, developed before the acceptance of psychometric methods of scale development,3 has a number of problems, including: an ordinal scale; rater variability; poor reliability; insensitivity to change at certain levels; and an emphasis on mobility status.4,6 The more recently introduced measure, the Multiple Sclerosis Functional Composite (MSFC), has been shown to be a sensitive and responsive instrument in the combined assessment of cognitive, upper limb, and gait disorder.2,7,8
Comprehensive self-reported measures of disease impact, both generic and specific to the disease, have been widely assessed in multiple sclerosis (MS). The Short Form Health Survey Scale has been frequently used in the MS population, but it does not evaluate many symptoms specific to MS, and the high frequency of floor and ceiling effects indicates a lack of responsiveness in this disease.9,11 We evaluated measures specific to the disease, including: the Functional Assessment of MS (FAMS)12; the Multiple Sclerosis Quality of Life Inventory (MSQLI)13; the UK Neurological Disability Scale (UKNDS)14; the MS Quality of Life Score (MSQOL-54)15; and the Health Related Quality of Life in MS Measure (HRQOL-MS).16 Although useful in varying degrees, none of these measures used scientifically based psychometric methods of scale construction; the first such validated measure of disease impact specific to the disease and rated by the patient, the Multiple Sclerosis Impact Scale (MSIS-29), was published in 2001.17 The MSIS-29 consists of 29 questions of which 20 address the physical impact component and nine assess the psychological impact; a combined score can be generated, or both components can be reported separately. The reliability and validity of the MSIS-29 were assessed by a postal survey of members of the United Kingdom MS society, and its responsiveness was evaluated in a patient population undergoing treatment.17 The validity of the measure in persons undergoing rehabilitation in MS has also been assessed.18
The MSIS-29 has not been independently validated in other MS populations. It is the aim of this paper to examine the reliability, validity, and responsiveness of the MSIS-29 in MS patients both in a community setting and attending the hospital neurology outpatient department.
PARTICIPANTS AND METHODS
Two groups of subjects were studied.
Individuals with MS in the community
People with clinically definite or probable MS (Poser criteria)19 were identified during the course of an epidemiological study in Counties Wexford and Donegal in the Republic of Ireland. A total of 366 cases were ascertained in both counties (126 in Wexford and 240 in Donegal), and of these 71 (56.3%) in Wexford and 101 (42.1%) in Donegal were interviewed and agreed to take part in the study. There were no demographic differences between study participants and non-participants.
Cross-sectional study
Kurtzke EDSS scores and MSFC scores were assessed by C McG. Subjects were also asked to complete the MSIS-29; the London Handicap scale (LHS), a generic measure of handicap20; and Beck’s Depression Inventory (BDI-II), a validated, generic measure of depression.21
Test-retest reliability
The 71 participants enrolled in the study from County Wexford were asked to complete a second MSIS-29 form six months after the first assessment, by postal survey. They were also asked whether their physical condition had improved, remained stable, or deteriorated since the first assessment. This information was used in the determination of item test-retest reliability.
Individuals with MS attending outpatients
Cross-sectional assessment
Of patients attending the neurology outpatient department with clinically definite or probable MS (Poser criteria),19 102 were seen by both authors and completed MSIS-29 questionnaires; the Kurtzke EDSS scores were also assessed.
Responsiveness
Of these patients, 55 were prospectively followed over six months; at each visit they were asked to complete an MSIS-29 questionnaire and the EDSS was assessed. In 41 cases an MSIS-29 questionnaire and a Kurtzke EDSS score were obtained at both time points. The EDSS was assessed by both authors without knowledge of the MSIS-29 score; the authors had trained in EDSS assessment together to improve inter-rater reliability. On the basis of the difference in EDSS scores between the two time points (and before the MSIS-29 was scored), the 41 patients were assigned to one of two categories: either "Changed" (if the two EDSS scores differed by 1.0 or more), or "Static" (if the difference between the two scores was not more than 0.5 points). The ethics committee of St. Vincent’s University Hospital, Dublin, granted approval for the study.
Statistical methods
The groups were assessed using standard psychometric techniques including:3,22
data quality: percentage missing data and percentage computable scores
scaling assumptions: item mean scores and standard deviations and item to total correlations
acceptability as determined by score range, mean scores, floor/ceiling effects, and skewness
reliability: Cronbach’s alpha was estimated for all samples. The standard error of measurement was used to calculate the 95% confidence interval for individual scores. Item test-retest reliability was assessed in the Wexford population who completed a second MSIS-29 by postal survey, and also in the "Static" group of hospital patients whose Kurtzke EDSS scores had not changed over a six month period
validity: the convergent validity of the MSIS-29 physical scale was tested against the Kurtzke EDSS scale, the MSFC, and the LHS in the community population, and against the Kurtzke EDSS scale only in the outpatient population. The divergent validity of the MSIS-29 physical scale was examined against the MSIS-29 psychological scale and the BDI-II. The convergent validity of the MSIS-29 psychological scale was assessed in the community participants only against the BDI-II, and the divergent validity was measured against the EDSS and the MSIS-29 physical scale. A Spearman’s rank correlation coefficient was used to test the strength of the correlations. Rasch analysis was used to examine correlations that were greater than expected23
responsiveness: the responsiveness of the MSIS-29 scale was evaluated by comparing the mean scores at the assessments at the two time points in the "Changed" subgroup of the 41 hospital outpatients. A Wilcoxin Signed Rank test was used to assess the significance of the mean differences between groups. Effect sizes (mean change score / standard deviation of Time 1 scores) were calculated for the "Changed" and "Static" groups.24
RESULTS
Populations
The demographic details of the 172 participants from the epidemiological study and the 102 hospital attendees are given in table 1. The former group is more representative of the community MS population, with a wider range of disability and a more realistic proportion of primary progressive MS patients.
Table 1 Participant characteristics for each population
MSIS-29 psychometric properties
Data quality
The percentage missing data for the MSIS-29 was very low at 0–2.8%. Scores were computable for all participants (table 2).
Table 2 Psychometric properties of the MSIS-29 physical and psychological scales in each population
Scaling assumptions
Items within each sample had similar means and standard deviations. Frequency results for each item were well distributed. Item-total correlations were acceptable but consistently less for the MSIS-29 psychological score in comparison with the physical score (table 2).
Acceptability
Floor/ceiling effects were very low for all samples. Score ranges spanned from 86.1% to 98.68% of the possible range. The mean scores were near the scale midpoints and were not notably skewed (table 2).
Reliability
All estimates of reliability using Cronbach’s alpha were in excess of the recommended 0.80. The 95% confidence intervals for the MSIS-29 physical scales were narrower in all groups compared with the MSIS-29 psychological scale, indicating the expected increase in precision due to the number of items (table 2).
Validity
The convergent validity of the MSIS-29 physical scale with other measures of physical impairment is indicated by the high correlations found with the Kurtzke EDSS (0.704), the LHS (0.843), and the MSFC (0.577). In a test of divergent validity, the correlation of the MSIS-29 physical scale and the BDI-II at 0.399 is higher than would have been expected. Rasch analysis, a statistical method using interval level measurement, was applied to the MSIS-29 results of patients scoring between 0 and19 on the BDI-II scale (minimally to mildly depressed), and between 20 and 60 (moderately to severely depressed). The analysis confirmed that the MSIS-29 score items were operating similarly in the two populations, and therefore were not unduly influenced by mood. The greater than expected correlation is likely to highlight the problems associated with applying parametric analyses to interval measure scores.
The MSIS-29 psychological scale scores correlated highly with the BDI-II, as predicted (0.799). The divergent validity of the MSIS-29 psychological impact scale was confirmed when assessed against the EDSS (0.095) and the MSFC (0.020). The correlation between the MSIS-29 physical and psychological scales was 0.438, showing, as expected, that they measure similar but distinct constructs.
Test-retest reliability
Community postal survey. In the community study, 58 of 71 (81%) patients who
"Static" hospital group. There were 24 patients with unchanged EDSS scores. Their mean MSIS-29 physical score at the first assessment was 27.1 compared with 29.1 at the second assessment. There was no significant change in their mean MSIS-29 psychological scores from first to second assessments.
Responsiveness
The responsiveness of the MSIS-29 was measured for the 17 hospital attendees who were assessed as having worsened or improved by one point or more in the EDSS between the two time points. The mean change in the EDSS was 1.35 (range 1–2.5). The mean difference in the MSIS-29 physical impact score from first to second examination was 18.16 (Z = -3.259, p = 0.001); the mean difference in the MSIS-29 psychological impact scores was 12.25 (Z = -1.967, p = 0.049). Table 3 shows the mean MSIS-29 physical and psychological scores at both time points, together with the range in scores and mean differences for both the "Static" and the "Changed" groups according to change in the EDSS scores. The effect size for the change in the MSIS-29 physical score in the "Static" group was 0.43 compared with 1.25 for the "Changed" group, further supporting the responsiveness to change of measure. The moderate effect size noted in the "Static" group is likely to represent the lack of fine sensitivity of the EDSS scale in detecting minimal change in MS disability. The differences in the mean MSIS-29 physical score over the six month period when plotted against change in the EDSS (Figure 1) indicates that the relationship between these measures is highly correlated (MSIS-29 physical change v EDSS change, R2 = 0.9, p = 0.014).
Table 3 Mean scores, difference in mean scores and median scores with interquartile range for each scale at both time points displayed according to change in the Kurtzke EDSS score
Figure 1 The difference in the means of the MSIS-29 physical scores for the 41 patients enrolled in the responsiveness study plotted against the observed change in the Kurtzke EDSS score.
DISCUSSION
The objective of this study was to assess the psychometric properties of the MSIS-29 physical and psychological scales in two different populations with MS in Ireland, and to determine their effectiveness as an outcome measure in population based studies and clinical trials. The MSIS-29 is an easy instrument to administer, taking approximately five to 10 minutes to complete. The questionnaire was acceptable to patients, and no particular areas of concern were raised with regard to the content or the phrasing of the individual items.
The psychometric analysis revealed the consistency of the results observed for both community and outpatient based populations; the results are similar to the published results of the reference populations.17,18 The scale is of use in the full range of impairments, disabilities, and handicaps seen in the MS population (other than dementia). The correlations of the MSIS-29 physical scale with MSFC and the EDSS, and of the psychological scale with the BDI-II, indicate that the measures identify the appropriate impact status. Initial assessment suggested a higher than expected correlation between the MSIS-29 physical scale and the BDI-II; further analysis by Rasch techniques showed that the physical impact scale operated similarly in depressed and in euthymic patients, allowing it to be used with confidence irrespective of the patient’s affect.
The results of the test-retest analysis and of the study of responsiveness to change suggest that the MSIS-29 is not only of use in cross-sectional studies but may also be used longitudinally to monitor disease progression. The MSIS-29 was responsive for both change perceived by participants in the community sample, and change as assessed by the EDSS in the outpatient sample. However, change perceived by participants and change rated by observers may not be concordant, and measuring both in the same cohort would help to establish further the sensitivity to change of the MSIS-29. The results are therefore consistent with a recent report of the relationship between another self-assessed measure of disability, the Guy’s Neurological Disability Scale, and with the Kurtzke EDSS scale, strengthening the case for the use of self-assessed measures of disease impact in future clinical trials.25
Further validation by other MS centres is required, particularly in the longitudinal assessment of the MSIS-29 to confirm or deny its responsiveness in comparison with recognised instruments such as the EDSS and the MSFC. The scale offers a promising, scientifically developed, self-reported measure of both the physical and the psychological impacts of MS, for use in clinical trials, population based studies, and continuing evaluation of individual patients.
REFERENCES
Kurtzke JF. Rating neurological impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology 1983;33:1444–52.
Cutter GR, Baier ML, Rudick RA, et al. Development of a multiple sclerosis functional composite as a clinical trial outcome measure. Brain 1999;122:871–82.
Nunnally JC, Bernstein IH. Psychometric theory, 3rd edn. New York: McGraw-Hill, 1994.
Hobart J, Freeman J, Thompson A. Kurtzke scales revisited: the application of psychometric methods to clinical intuition. Brain 2000;123:1027–40.
Sharrack B, Hughes RA, Soudain S, et al. The psychometric properties of clinical rating scales used in multiple sclerosis. Brain 1999;122:141–59.
Thompson A, Hobart JC. Multiple sclerosis: assessment of disability and disability scales. J Neurol 1998;245:189–96.
Kalkers NF, de Groot V, Lazeron RHC, et al. MS functional composite: relation to disease phenotype and disability strata. Neurology 2000;54:1233–9.
Cohen JA, Fischer JS, Bolibrush DM, et al. Intrarater and interrater variability of the MS functional composite outcome measure. Neurology 2000;54:802–6.
Vickrey BG, Hays RD, Harooni R, et al. A health-related quality of life measure for multiple sclerosis. Qual Life Res 1995;4:187–206.
Hobart J, Freeman J, Lamping D, et al. The SF-36 in multiple sclerosis: why basic assumptions must be tested. J Neurol Neurosurg Psychiatry 2001;71:363–70.
Peto V, Jenkinson C, Fitzpatrick R, et al. The development and validation of a short measure of functioning and well being for individuals with Parkinson’s disease. Qual Life Res 1995;4:241–8.
Cella DF, Dineen K, Arnason B, et al. Validation of the functional assessment of multiple sclerosis quality of life instrument. Neurology 1996;47:129–39.
LaRocca NG, Ritvo PG, Miller DM, et al. ‘Quality of life’ assessment in multiple sclerosis clinical trials: current status and strategies for improving multiple sclerosis clinical trial design. In: Goodkin DE, Rudick RA, eds. Multiple sclerosis: advances in clinical trial design, treatment and future perspectives. London: Springer-Verlag, 1996:145–60.
Sharrack B, Hughes RAC. The Guy’s Neurological Disability Scale (GNDS): a new disability measure for multiple sclerosis. Mult Scler 1999;5:223–33.
Vickrey BG, Hays RD, Harooni R, et al. A health-related quality of life measure for multiple sclerosis. Qual Life Res 1995;4:187–206.
Pfennings LE, Van der Ploeg HM, Cohen L, et al. A health-related quality of life questionnaire for multiple sclerosis patients. Acta Neurol Scand 1999;100:148–55.
Hobart J, Lamping D, Fitzpatrick R, et al. The multiple sclerosis impact scale (MSIS-29). A new patient-based outcome measure. Brain 2001;124:962–73.
Riazi A, Hobart J, Lamping D, et al. Multiple sclerosis impact scale (MSIS-29): reliability and validity in hospital based samples. J Neurol Neurosurg Psychiatry 2002;73:701–4.
Poser CM, Paty DW, Scheinberg L, et al. New diagnostic criteria for multiple sclerosis: guidelines for research protocols. Ann Neurol 1983;13:227–31.
Harwood RH, Rogers A, Dickinson E, et al. Measuring handicap: the London handicap scale, a new outcome measure for chronic disease. Qual Health Care 1994;3 (1) :11–16.
Robinson BE, Kelley L. Concurrent validity of the Beck Depression Inventory as a measure of depression. Psychol Rep 1996;79:929–30.
Staquet MJ, Hays RD, Fayers PM. Quality of life assessment in clinical trials: methods and practice, 1st edn. Oxford: Oxford University Press, 1998.
Rasch G. Probabilistic models for some intelligence and attainment tests. Chicago: University of Chicago Press, 1960.
Kaziz LE, Anderson JJ, Meenan RF. Effect sizes for determining changes in health status. Med Care 1989;27:S178–89.
Hoogervorst ELJ, Eikelenboom MJ, Uitehaag BMJ, et al. One year changes in disability in multiple sclerosis: neurological examination compared with patient self report. J Neurol Neurosurg Psychiatry 2003;74:439–42.(C McGuigan and M Hutchins)