Evaluating neurorehabilitation: lessons from routi
http://www.100md.com
神经科学杂志 2005年第5期
1 Institute of Neurology, London, UK
2 University of Plymouth, Plymouth, Devon, UK
3 Peninsula Medical School, Plymouth, Devon, UK
ABSTRACT
Background: Clinical databases are being used increasingly to assess outcomes in healthcare services to provide evidence of clinical effectiveness in routine clinical practice.
Objectives: To explore the benefits of a database for routine collection of clinical outcomes within an inpatient neurorehabilitation setting; determine the effectiveness of inpatient neurorehabilitation in a range of neurological conditions; and determine variables influencing change in functional outcome.
Methods: Over a nine year period, demographic and diagnostic characteristics were collected for the 1458 patients admitted consecutively to a neurorehabilitation unit. The level of function was measured on admission and discharge using the Barthel Index (BI) and Functional Independence Measure (FIM). Patient perception of rehabilitation benefit was evaluated using visual analogue scales (VAS).
Results: Of the 1413 patients (mean (SD) age 48 (14.8), range 16 to 87) whose length of stay was more than 10 days (mean 34 (24) range 10 to 184), 282 had stroke, 614 multiple sclerosis, 248 spinal cord injuries, 93 a neuromuscular condition, and 176 other brain pathology. Patients improved in functional ability as measured by both BI and the FIM motor subscale (effect sizes 0.93 to 1.44 and 1.01 to 1.48, respectively). VAS ratings demonstrated high levels of patient perceived benefit. Diagnosis, functional activity score on admission, and length of stay were significant predictors of functional gain, explaining 44% of the variability in the change scores.
Conclusions: Systematic collection, analysis, and interpretation of standardised clinical outcomes data are feasible within routine clinical practice, and provide evidence that inpatient rehabilitation is effective in improving functional level in neurologically impaired patients. These data complement those of clinical trials and are useful in informing and developing clinical and research practice.
Abbreviations: BI, Barthel Index; FIM, Functional Independence Measure; ICP, integrated care pathway; MS, multiple sclerosis; NRU, Neurorehabilitation Unit (of the National Hospital for Neurology and Neurosurgery); VAS, visual analogue scale(s)
Keywords: database; evaluation; neurorehabilitation; outcomes
Standardised clinical information, gathered routinely in a systematic and reliable manner and stored in easily accessible databases, is of potential value for informing clinical practice, management of services, and evaluative research.1,2 It allows care to be examined as it occurs in routine practice rather than in controlled circumstances thereby enhancing the generalisability of results.3 As a consequence clinical databases are being increasingly used within health care to:
examine outcome within organisations4,5
make comparisons between organisations6,7
complement the results of randomised controlled clinical trials8,9
gain a more detailed understanding of the characteristics of outcome measurement instruments.10
Our experience in the UK suggests that, although routine data collection within rehabilitation units is becoming more prevalent, many units do not have a structured way of collecting, storing, analysing, or systematically interpreting and disseminating the information collected. Consequently, in many cases, the data are underutilised. This is wasteful of resources, both in terms of the manpower and time used to collect the information and knowledge lost. Recently, a web based directory of clinical databases (www.lshtm.ac.uk/docdat) within the UK has been developed with the aims of enabling greater access and use of existing databases and enhancing their quality.11 The directory highlights the paucity of UK rehabilitation services using databases to collate information about clinical outcomes.
This article describes the introduction, development, and routine use of a clinical outcome database within a neurological inpatient rehabilitation unit over a nine year period. It details the clinical characteristics and outcome of 1413 participants of a structured multidisciplinary rehabilitation programme. From a clinical perspective this information provides indicators of levels of change for patients undergoing neurorehabilitation and examines how different variables impact on patterns of improvement. From a research perspective, it provides information that complements the results of previous clinical trials and highlights some of the complexities involved in the interpretation of routine data.
METHODS
The rehabilitation unit and programme
The Neurorehabilitation Unit (NRU) of the National Hospital for Neurology and Neurosurgery, London, UK, is an 18 bedded unit that specialises in intensive, individually tailored goal oriented rehabilitation of people with neurological disorders.4 Patients are referred from within the hospital, and directly from general practitioners and consultants from surrounding district and teaching hospitals; they are selected for admission following assessment by a multidisciplinary team. The criteria for selection are: patients with a neurological impairment who (a) are medically stable, (b) would benefit from integrated input from at least two disciplines other than medical and nursing staff, and (c) demonstrate the potential to actively participate in an intensive goal oriented rehabilitation programme.12 Goals of admission are broad ranging and include functional improvement, enhancement of coping strategies, and facilitation of "set-up" within the community for those with complex needs. More specifically the rehabilitation programme aims to address the individual’s problems identified on assessment. This programme, which has been described previously13,14 typically includes efforts to improve functional independence, mobility, bladder and bowel function, and communication, together with advice and education on work and leisure, tone and fatigue management, and strategies to compensate for cognitive dysfunction. It is supported and monitored by an integrated care pathway (ICP).15 An ICP is a structured multidisciplinary care plan, which details essential steps in the care of patients with a specific clinical problem, in this case neurological impairment.
Process of data collection
Clinical data
Clinical data, which includes diagnostic information, sex, age, length of stay, and admission and discharge destination, are collected by the doctor.
Outcome measures
The treating team jointly lists impairments, functional activity limitations (disabilities), and restriction in participation (handicap).16 Patient rated visual analogue scales (VAS) are scored at the end of the first week of admission and within 24 hours prior to discharge. In addition, in partnership with the patient, the team defines measurable short and long term goals that are regularly monitored by the ICP throughout the inpatient stay.
Limitations in functional activities of daily living
Limitations in functional activities of daily living are measured by the widely used Barthel Index (BI)17 and Functional Independence Measure (FIM).18 Both have proved psychometric properties of reliability, validity, and responsiveness within this12,19 and similar rehabilitation settings.18 At the NRU, the scales are scored by consensus following observation of the patient by the multidisciplinary team. Core members of this team include a nurse, a physiotherapist, and an occupational therapist. A psychologist, a social worker, and a speech and language therapist are involved when indicated.
Visual analogue scale ratings
VAS ratings are collected for:
the main problem as identified by the patient on admission
the benefit gained from inpatient rehabilitation.
These are self-rated by patients in the presence of their keyworkers. The keyworker provides assistance in understanding the method of scoring but does not give advice concerning the nominated problem or the rating of benefit. VAS are commonly used in routine clinical practice. Although they have recognised limitations,20 there is evidence that they provide clinically useful information and have reasonable psychometric properties in a range of settings.21
The keyworker coordinates the collection and completion of data and secretarial staff input the data into the electronic database. This process of data collection has remained essentially unchanged since the initial set-up of the database.
The database
The custom designed database was developed in close collaboration with a database expert and the multidisciplinary team throughout 1992–93 using a commercially available software package, Microsoft Access. The database is stored on an IBM computer, which is networked to the main hospital’s server.
The database stores basic demographic data (sex, age), diagnosis coded according to one of five groups (stroke, spinal cord syndrome/lesion, multiple sclerosis (MS), neuromuscular condition, other brain pathology), duration of stay, and the total and item scores for a range of outcome measures on admission and discharge. Free text data is minimised by the use of codes for all but one variable, the patient perceived main problem. Coded diagnostic information is supplemented with free text to capture subtleties of diagnostic information.
The study sample
The sample for the present study included all patients admitted to the rehabilitation unit between May 1993 and December 2002 whose length of stay was greater than 10 days. Patients with stays of shorter than 10 days tend to be those admitted for purposes of "set-up" or assessment, and hence were excluded.
Analyses
Quality control of data
Two consultant neurologists (AJT and EDP) reviewed all the diagnostic codes. In some cases this required a more detailed review of the medical notes, and subsequent reclassification. Quality control checks for the data set were undertaken to review missing and out-of-range values and consistency of data.
Analyses
Descriptive statistics were used for diagnostic and demographic variables. In determining the effectiveness of inpatient neurorehabilitation, in this paper we have restricted its analysis to the functional activity (BI and FIM motor subscale) and VAS data. Effect sizes were calculated for each measure22 to determine the magnitude of change between admission and discharge.
A general linear model analysis was undertaken to investigate the extent to which different explanatory variables (diagnostic group, age, sex, length of stay, BI scores, and FIM motor admission scores) could predict the functional gains made (as determined by the BI scores and FIM motor change scores) during the rehabilitation programme. This analysis, which combines factors (sex and diagnostic code) and explanatory variables (age, length of stay, initial BI and FIM motor scores), investigated the extent to which these different variables affected the mean BI and FIM motor change scores between admission and discharge from the rehabilitation unit.
An assumptions check was undertaken for the general linear model. Formal tests of normality and equality of variance were undertaken and, although significant, graphical appraisals showed that these were detecting very small departures from normality that were of no practical importance.
RESULTS
Between May 1993 and December 2002, 1458 consecutive patients were admitted to the NRU. Table 1 presents the characteristics of the 1413 patients whose length of stay was greater than 10 days. Complete diagnostic and demographic information was available for all patients. Patient with MS accounted for almost half of all admissions; of these, 16% were in the relapsing remitting phase of the disease, and 84% were in the progressive phase (of whom 83% were secondary progressive and 17% primary progressive). Stroke formed the next largest diagnostic group, comprising 20% of the total sample. It was observed that length of stay appeared to be related to diagnosis; patients with MS had the shortest (mean (SD) 23 (11.9) days, range 10–102) and stroke patients the longest (mean 51 (30.2) days, range 10–149) inpatient stay.
Cognitive and communication function, as measured by the FIM cognitive subscale scores improved in 46% of the total sample. The pattern of change differed according to diagnosis, for example of the 94 left sided stroke patients, 80% increased their score (mean (SD) unadjusted change score 4.3 (5.7), range –12 to 20; effect size 0.75) compared with 28% of patients with spinal cord injuries (mean (SD) unadjusted change 0.5 (3.2), range –11 to 20; effect size 0.15). As predicted the extent of change was directly related to the spread of admission scores, in particular the presence of a ceiling effect. For example, in the spinal cord sample where communication and cognitive problems would not be expected, the ceiling effect was very high at 58%. Conversely, the ceiling effect was 4.3% in the left sided stroke sample, in which cognitive and communication problems were more common.
To examine the data in more detail, we categorised patients according to their response to rehabilitation, as determined by the unadjusted BI change scores between admission and discharge. The categories were as follows:
good response: >8 point gain, n = 256
moderate response: 5–7 point gain, n = 280
minimal response: 2–4 point gain, n = 445
no change: –1 to 1 point change, n = 382
poor response: 2 point loss, n = 20.
Examination of the data revealed that 156 of the 382 patients (41%) allocated to the "no change" group, had a BI admission score of either 19 (n = 81) or 20 (n = 75). Thus, they were unable to improve their score by more than one point, making it impossible for them to be categorised in any other than the "no change" or "poor responders" group. Of these, only 31 patients who initially scored 19 points did not change their score. This means that, when using the BI as a measure of outcome, there was a potential underestimate of change for 125 patients (8.9% of the total sample). A further way of exploring this observation is by examining the frequency distribution of the discharge BI score for each of the subgroups, as categorised according to response to intervention. This shows that 30% of the "no change", 24% of the "minimum", 27% of the "moderate" and 31% of the "very good" responders scored a maximum of 20 points at discharge. Again, it is possible that the patients’ scores could have further improved had the scale range been larger.
Patient perceived main problem
Fifty four per cent of the total sample report mobility as their commonest "main problem", ranging from 44% to 61% across the different diagnostic groups. More specifically, walking was cited as the commonest mobility problem across the majority of diagnostic subgroups. Psychosocial issues such as home and leisure roles (range 0 to 4%), mood (range 0 to 1%), or cognition (range 0 to 1%) were rarely cited by patients as their main problem. Self-care activities (such as dressing, washing, feeding) were also rarely listed as the major problem (range 2 to 4%).
VAS scores
After adjusting the BI and FIM motor change scores for significant explanatory variables (los, admission BI, admission FIM motor subscale), the mean BI and FIM motor subscale change scores for the MS group were significantly smaller (p<0.05, 5% least significant difference comparison) and for the neuromuscular group were significantly larger (p<0.05) than each of the other diagnostic groups apart from the other brain pathology group. There was no significant difference in the mean BI or FIM motor subscale change scores for stroke, spinal cord or other brain pathology (table 5).
DISCUSSION
We have reported data obtained over a nine year period. This is the largest reported data set of inpatient neurological rehabilitation patients in the UK. It demonstrates that the systematic collection, analysis, and interpretation of standardised clinical outcomes data can be successfully incorporated into routine clinical practice within an inpatient neurological rehabilitation unit. This could act as a model for other rehabilitation units, thereby facilitating the sharing of data at a national level.
The methods for collecting the data are robust and reproducible, and the validity of the data is supported by a range of factors. Firstly, an annual comparison of demographic, diagnostic, and outcomes data demonstrated a consistency of results over the nine year period (data available on request from the authors). Secondly, features of the case mix are in line with clinical observations—more female patients with MS than male, and more male patients with stroke and spinal injuries than female. Similarly, the average age of the stroke group was higher than the MS group. Finally, the pattern of change is in keeping with clinical expectation: patients with progressively deteriorating conditions such as MS had the smallest change in physical function (effect size 0.93–1.01), and those with spontaneously recovering conditions, such as the neuromuscular group (in whom the majority had a diagnosis of Guillain–Barré syndrome), had the greatest change (effect size 1.44–1.48).
The first objective of this database study was to explore the benefits of using the database. Results from clinical databases are useful for examining, informing, and ultimately improving clinical practice based on empirical evidence. On a practical level they provide a focus for careful recording and monitoring of caseload, thus enabling the systematic accrual of information over longer periods of time about less common conditions whose details might otherwise become lost within the "paperwork system". This is particularly relevant in rehabilitation where the turnover is generally slow and the numbers admitted for specific diagnoses are relatively small. At a local level, assessment of the appropriateness of staffing levels and educational input can be facilitated by easy access to data such as the severity of disability of patients attending a unit and the patients’ perception of their main presenting problem. In our experience this has facilitated the development of a more performance oriented and accountable system of rehabilitation.
Databases are also beneficial for research purposes. For example, they can provide valuable objective data for determining sample size calculations for clinical trials. This has traditionally been difficult in neurorehabilitation where the populations studied are generally small and the turnover is slow. Our database of over nine years of data highlights considerable consistency in the change scores of the total population when annual comparisons are made, thus validating it as a source for determining sample size calculations. It also provides a quick and accurate method for assessing the feasibility of patient recruitment.
Our second objective was to determine the effectiveness (rather than efficacy) of inpatient neurorehabilitation in a range of neurological conditions. Our results provide objective evidence that, overall, patients improve significantly in functional ability. Furthermore, the VAS ratings of the patients’ perception of benefit demonstrate high levels of satisfaction. These findings support the increasing body of evidence from randomised clinical trials demonstrating the benefits of multidisciplinary inpatient rehabilitation for people with a broad range of neurological conditions.14,23 The findings contribute to the evidence base by providing important complementary information to aid our understanding of the generalisability of the results of clinical trials in a typical clinical setting.9,24
Evaluation of effectiveness by retrospective extraction of information from patients’ notes is widely recognised as being of questionable reliability because the relevant data are frequently hard to find, missing, or are inadequate in detail. A database of coded information, which is accessible on site, solves some of these problems by focusing the prospective collection of data to specific standardised information, which has been chosen for its relevance to clinical practice and the scientific rigour of the outcome measures used. Nevertheless, the quality of evidence gathered from database studies can be limited both in detail, comprehensiveness, and potential bias.3 It should not be used in isolation but in conjunction with rigorously designed clinical trials, such as randomised controlled clinical trials and prospective observational studies,24 to determine treatment effectiveness.
Deciding on relevant and meaningful outcomes to measure, and choosing validated outcome measures to evaluate them, is central to the usefulness of information gained. Many units, such as ours, use a generic disability measure such as the BI or the FIM as their primary method of evaluation. The results of the present study demonstrate that a limitation of the BI was the significant ceiling effect in our inpatient rehabilitation sample, with a potential underestimate of change for almost 10% of the total sample. These results provide empirical evidence to support the reason why many rehabilitation clinicians express dissatisfaction with the BI in its ability to detect change in patients who are less severely disabled. Such information is important in aiding clinicians to select appropriate outcomes and interpret the information generated.
The VAS results indicate that there is sometimes a disparity between changes recorded by objective functional measures and those reported by patients. This reflects the complexity of interpreting outcomes’ information and supports the view that functional measures alone do not reflect the totality of benefits gained from rehabilitation.25 Our results highlight the importance of seeking the patient’s perspective in determining where the focus and evaluation of rehabilitation should lie and in defining what constitutes a successful outcome. Qualitative work is needed to explore this further.
Our third objective was to determine variables that influence change in functional outcome. Like others in similar settings, our results demonstrate associations between functional gains and diagnosis, length of stay and functional ability at admission.26–28 It is generally agreed, however, that prediction of outcome, and the identification of factors within rehabilitation programmes that are responsible for outcome differences, is complex and requires continued research.29 This is highlighted by the fact that only 44% of the variance could be accounted for in our model.
Validity of conclusions drawn from database material depends upon the quality of data stored (completeness, reliability, and validity of the data). Over the past nine years we have learnt a number of lessons about how to optimise data quality. All patients are allocated a keyworker who coordinates and monitors the collection of data; this is facilitated by the use of an ICP. The process of outcome measurement is kept simple, short, and meaningful by choosing measures that are relatively quick and straightforward to score and which have proved psychometric properties. In our experience, long and complicated measures, however useful, are generally demotivating for staff, which inevitably impacts negatively on data completeness. Regular training about the purpose and process of outcome measurement, with specific reference to the measures used, is included within both the induction programme and regular multidisciplinary in-service educational sessions. Staff are reassurred that information will not be used out of context or to misrepresent either the patients’ needs or the outcome of intervention. Feedback of audit results of the database is provided at staff meetings. Regular feedback and training enhances the staff’s understanding of the data and their sense of ownership in how it is used, as well as maximising the reliability of the data collected. A person with dedicated time manages the database; validates, analyses, and interprets the data generated; and disseminates the results. Finally, clear and dynamic leadership is integral to the entire process. These are powerful motivating factors in optimising the quality of data collection as audit becomes integrated with clinical practice.
This database only provides local information pertaining to inpatient rehabilitation, just one stage in the rehabilitation process. In order to gain a comprehensive view, the total episode of care including the acute hospital, inpatient rehabilitation, and community follow up phase across a range of services needs to be examined. The current structure of service delivery, wherein each individual service within distinct geographical areas holds separate records of the patients’ progress, renders this virtually impossible in practical terms. Changes in terms of (a) "patient centred focus" of service delivery, (b) the use of electronic records, and (c) standardised assessment procedures may enable this in the future.
REFERENCES
Black N . High quality clinical databases: breaking down barriers. Lancet 1999;353:1205–6.
Black N . Using clinical databases in practice. Individualised prediction of survival for patients with cancer may be possible. BMJ 2003;326:2–3.
Motheral B, Brooks J, Clark MA, et al. A checklist for retrospective database studies—report of the ISPOR Task Force on retrospective databases. Value Health 2003;6:90–7.
Freeman JA, Playford ED, Thompson AJ. A neurological rehabilitation unit: audit of activity and outcome. J R Coll Phys Lond 1996;30:21–6.
Chan L, Houck P, Prela CM, et al. Using Medicare databases for outcomes research in rehabilitation medicine. Am J Phys Med Rehabil 2001;80:474–80.
Trabin T . The quality agenda accelerates: outcomes collaboratives emerge. Behav Healthcare Tomorrow 1997;April:11–13.
Johnston MV, Wood KD, Fiedler R. Characteristics of effective and efficient rehabilitation programs. Arch Phys Med Rehabil 2003;84:410–18.
Lewsey JD, Leyland AH, Murray GD, et al. Using routine data to complement and enhance the results of randomised controlled trials. Health Technol Assess 2000;4:1–54.
Padkin A, Rowan K, Black N. Using high quality clinical databases to complement the results of randomised controlled trials: the case of recombinant human activated protein C. BMJ 2001;323:923–6.
Walsh TL, Hanscom B, Lurie JD, et al. Is a condition-specific instrument for patients with low back pain/leg symptoms really necessary? The responsiveness of the Oswestry Disability Index, MODEMS, and the SF-36. Spine 2003;28:607–15.
Black N, Payne M. Improving the use of clinical databases. BMJ 2002;324:1194.
van der Putten JMF, Hobart JC, Freeman JA, et al. Measuring change in disability following inpatient rehabilitation: comparison of the responsivenesss of the Barthel Index and the Functional Independence Measure. J Neurol Neurosurg Psychiatry 1999;66:480–4.
Johnson J, Thompson AJ. Rehabilitation in a neuroscience centre: the role of expert assessment and selection. Br J Ther Rehabil 1996;3:303–8.
Freeman JA, Langdon DW, Hobart JC, et al. The impact of inpatient rehabilitation on progressive multiple sclerosis. Ann Neurol 1997;42:236–44.
Rossiter DA, Edmondson A, Al Shahi R, et al. Integrated care pathways in multiple sclerosis rehabilitation: completing the audit cycle. Mult Scler 1998;4:85–9.
World Health Organization International Classification of Functioning Disability and Health. Available from www.who.int/icf (accessed December 2003).
Mahoney FI, Barthel DW. Functional evaluation: the Barthel Index (BI). Md State Med J 1965;14:61–5.
Granger CV, Cotter AC, Hamilton BB, et al. Functional assessment scales: a study of persons after stroke. Arch Phys Med Rehabil 1993;74:133–8.
Hobart JC, Lamping DL, Freeman JA, et al. Evidence-based measurement: which disability scale for neurological rehabilitation? Neurology 2001;57:639–44.
Streiner DL, Norman GR. Health Measurement Scales: a practical guide to the development and use. Oxford: Oxford University Press, 1995:32–3.
Finch E, Brooks D, Stratford PW, et al. Physical Rehabilitation Outcome Measures: A Guide to Enhanced Clinical Decision Making. Ontario: Canadian Physiotherapy Association, 2002:244–5.
Cohen J .Statistical power analysis for the behavioral sciences, 2nd edn. Hillsdale, New Jersey: Lawrence Earlbaum Associates, 1988.
Stroke Unit Trialists’ Collaboration. Collaborative systematic review of the randomised trials of organised inpatient (stroke unit) care after stroke. BMJ 1997;314:1151–9.
Black N . Why we need observational studies to evaluate the effectiveness of health care. BMJ 1996;312:1215–18.
Edwards SGM, Playford ED, Hobart JC, et al. Comparison of physician outcome measures and patients’ perception of benefits of inpatient neurorehabilitation. BMJ 2002;324:1493.
Carey RG, Seibert JH. Who makes the most progress in inpatient rehabilitation? An analysis of functional gain. Arch Phys Med Rehabil 1988;69:337–43.
Langdon DW, Thompson AJ. Multiple sclerosis: a preliminary study of selected variables affecting rehabilitation outcome. Mult Scler 1999;5:94–100.
Johnston MV, Wood KD, Fiedler R. Characteristics of effective and efficient rehabilitation programs. Arch Phys Med Rehabil 2003;84:410–18.
Wade DT. Selection criteria for rehabilitation services. Clin Rehabil 2003;17:115–18.(J A Freeman,, J C Hobart,)
2 University of Plymouth, Plymouth, Devon, UK
3 Peninsula Medical School, Plymouth, Devon, UK
ABSTRACT
Background: Clinical databases are being used increasingly to assess outcomes in healthcare services to provide evidence of clinical effectiveness in routine clinical practice.
Objectives: To explore the benefits of a database for routine collection of clinical outcomes within an inpatient neurorehabilitation setting; determine the effectiveness of inpatient neurorehabilitation in a range of neurological conditions; and determine variables influencing change in functional outcome.
Methods: Over a nine year period, demographic and diagnostic characteristics were collected for the 1458 patients admitted consecutively to a neurorehabilitation unit. The level of function was measured on admission and discharge using the Barthel Index (BI) and Functional Independence Measure (FIM). Patient perception of rehabilitation benefit was evaluated using visual analogue scales (VAS).
Results: Of the 1413 patients (mean (SD) age 48 (14.8), range 16 to 87) whose length of stay was more than 10 days (mean 34 (24) range 10 to 184), 282 had stroke, 614 multiple sclerosis, 248 spinal cord injuries, 93 a neuromuscular condition, and 176 other brain pathology. Patients improved in functional ability as measured by both BI and the FIM motor subscale (effect sizes 0.93 to 1.44 and 1.01 to 1.48, respectively). VAS ratings demonstrated high levels of patient perceived benefit. Diagnosis, functional activity score on admission, and length of stay were significant predictors of functional gain, explaining 44% of the variability in the change scores.
Conclusions: Systematic collection, analysis, and interpretation of standardised clinical outcomes data are feasible within routine clinical practice, and provide evidence that inpatient rehabilitation is effective in improving functional level in neurologically impaired patients. These data complement those of clinical trials and are useful in informing and developing clinical and research practice.
Abbreviations: BI, Barthel Index; FIM, Functional Independence Measure; ICP, integrated care pathway; MS, multiple sclerosis; NRU, Neurorehabilitation Unit (of the National Hospital for Neurology and Neurosurgery); VAS, visual analogue scale(s)
Keywords: database; evaluation; neurorehabilitation; outcomes
Standardised clinical information, gathered routinely in a systematic and reliable manner and stored in easily accessible databases, is of potential value for informing clinical practice, management of services, and evaluative research.1,2 It allows care to be examined as it occurs in routine practice rather than in controlled circumstances thereby enhancing the generalisability of results.3 As a consequence clinical databases are being increasingly used within health care to:
examine outcome within organisations4,5
make comparisons between organisations6,7
complement the results of randomised controlled clinical trials8,9
gain a more detailed understanding of the characteristics of outcome measurement instruments.10
Our experience in the UK suggests that, although routine data collection within rehabilitation units is becoming more prevalent, many units do not have a structured way of collecting, storing, analysing, or systematically interpreting and disseminating the information collected. Consequently, in many cases, the data are underutilised. This is wasteful of resources, both in terms of the manpower and time used to collect the information and knowledge lost. Recently, a web based directory of clinical databases (www.lshtm.ac.uk/docdat) within the UK has been developed with the aims of enabling greater access and use of existing databases and enhancing their quality.11 The directory highlights the paucity of UK rehabilitation services using databases to collate information about clinical outcomes.
This article describes the introduction, development, and routine use of a clinical outcome database within a neurological inpatient rehabilitation unit over a nine year period. It details the clinical characteristics and outcome of 1413 participants of a structured multidisciplinary rehabilitation programme. From a clinical perspective this information provides indicators of levels of change for patients undergoing neurorehabilitation and examines how different variables impact on patterns of improvement. From a research perspective, it provides information that complements the results of previous clinical trials and highlights some of the complexities involved in the interpretation of routine data.
METHODS
The rehabilitation unit and programme
The Neurorehabilitation Unit (NRU) of the National Hospital for Neurology and Neurosurgery, London, UK, is an 18 bedded unit that specialises in intensive, individually tailored goal oriented rehabilitation of people with neurological disorders.4 Patients are referred from within the hospital, and directly from general practitioners and consultants from surrounding district and teaching hospitals; they are selected for admission following assessment by a multidisciplinary team. The criteria for selection are: patients with a neurological impairment who (a) are medically stable, (b) would benefit from integrated input from at least two disciplines other than medical and nursing staff, and (c) demonstrate the potential to actively participate in an intensive goal oriented rehabilitation programme.12 Goals of admission are broad ranging and include functional improvement, enhancement of coping strategies, and facilitation of "set-up" within the community for those with complex needs. More specifically the rehabilitation programme aims to address the individual’s problems identified on assessment. This programme, which has been described previously13,14 typically includes efforts to improve functional independence, mobility, bladder and bowel function, and communication, together with advice and education on work and leisure, tone and fatigue management, and strategies to compensate for cognitive dysfunction. It is supported and monitored by an integrated care pathway (ICP).15 An ICP is a structured multidisciplinary care plan, which details essential steps in the care of patients with a specific clinical problem, in this case neurological impairment.
Process of data collection
Clinical data
Clinical data, which includes diagnostic information, sex, age, length of stay, and admission and discharge destination, are collected by the doctor.
Outcome measures
The treating team jointly lists impairments, functional activity limitations (disabilities), and restriction in participation (handicap).16 Patient rated visual analogue scales (VAS) are scored at the end of the first week of admission and within 24 hours prior to discharge. In addition, in partnership with the patient, the team defines measurable short and long term goals that are regularly monitored by the ICP throughout the inpatient stay.
Limitations in functional activities of daily living
Limitations in functional activities of daily living are measured by the widely used Barthel Index (BI)17 and Functional Independence Measure (FIM).18 Both have proved psychometric properties of reliability, validity, and responsiveness within this12,19 and similar rehabilitation settings.18 At the NRU, the scales are scored by consensus following observation of the patient by the multidisciplinary team. Core members of this team include a nurse, a physiotherapist, and an occupational therapist. A psychologist, a social worker, and a speech and language therapist are involved when indicated.
Visual analogue scale ratings
VAS ratings are collected for:
the main problem as identified by the patient on admission
the benefit gained from inpatient rehabilitation.
These are self-rated by patients in the presence of their keyworkers. The keyworker provides assistance in understanding the method of scoring but does not give advice concerning the nominated problem or the rating of benefit. VAS are commonly used in routine clinical practice. Although they have recognised limitations,20 there is evidence that they provide clinically useful information and have reasonable psychometric properties in a range of settings.21
The keyworker coordinates the collection and completion of data and secretarial staff input the data into the electronic database. This process of data collection has remained essentially unchanged since the initial set-up of the database.
The database
The custom designed database was developed in close collaboration with a database expert and the multidisciplinary team throughout 1992–93 using a commercially available software package, Microsoft Access. The database is stored on an IBM computer, which is networked to the main hospital’s server.
The database stores basic demographic data (sex, age), diagnosis coded according to one of five groups (stroke, spinal cord syndrome/lesion, multiple sclerosis (MS), neuromuscular condition, other brain pathology), duration of stay, and the total and item scores for a range of outcome measures on admission and discharge. Free text data is minimised by the use of codes for all but one variable, the patient perceived main problem. Coded diagnostic information is supplemented with free text to capture subtleties of diagnostic information.
The study sample
The sample for the present study included all patients admitted to the rehabilitation unit between May 1993 and December 2002 whose length of stay was greater than 10 days. Patients with stays of shorter than 10 days tend to be those admitted for purposes of "set-up" or assessment, and hence were excluded.
Analyses
Quality control of data
Two consultant neurologists (AJT and EDP) reviewed all the diagnostic codes. In some cases this required a more detailed review of the medical notes, and subsequent reclassification. Quality control checks for the data set were undertaken to review missing and out-of-range values and consistency of data.
Analyses
Descriptive statistics were used for diagnostic and demographic variables. In determining the effectiveness of inpatient neurorehabilitation, in this paper we have restricted its analysis to the functional activity (BI and FIM motor subscale) and VAS data. Effect sizes were calculated for each measure22 to determine the magnitude of change between admission and discharge.
A general linear model analysis was undertaken to investigate the extent to which different explanatory variables (diagnostic group, age, sex, length of stay, BI scores, and FIM motor admission scores) could predict the functional gains made (as determined by the BI scores and FIM motor change scores) during the rehabilitation programme. This analysis, which combines factors (sex and diagnostic code) and explanatory variables (age, length of stay, initial BI and FIM motor scores), investigated the extent to which these different variables affected the mean BI and FIM motor change scores between admission and discharge from the rehabilitation unit.
An assumptions check was undertaken for the general linear model. Formal tests of normality and equality of variance were undertaken and, although significant, graphical appraisals showed that these were detecting very small departures from normality that were of no practical importance.
RESULTS
Between May 1993 and December 2002, 1458 consecutive patients were admitted to the NRU. Table 1 presents the characteristics of the 1413 patients whose length of stay was greater than 10 days. Complete diagnostic and demographic information was available for all patients. Patient with MS accounted for almost half of all admissions; of these, 16% were in the relapsing remitting phase of the disease, and 84% were in the progressive phase (of whom 83% were secondary progressive and 17% primary progressive). Stroke formed the next largest diagnostic group, comprising 20% of the total sample. It was observed that length of stay appeared to be related to diagnosis; patients with MS had the shortest (mean (SD) 23 (11.9) days, range 10–102) and stroke patients the longest (mean 51 (30.2) days, range 10–149) inpatient stay.
Cognitive and communication function, as measured by the FIM cognitive subscale scores improved in 46% of the total sample. The pattern of change differed according to diagnosis, for example of the 94 left sided stroke patients, 80% increased their score (mean (SD) unadjusted change score 4.3 (5.7), range –12 to 20; effect size 0.75) compared with 28% of patients with spinal cord injuries (mean (SD) unadjusted change 0.5 (3.2), range –11 to 20; effect size 0.15). As predicted the extent of change was directly related to the spread of admission scores, in particular the presence of a ceiling effect. For example, in the spinal cord sample where communication and cognitive problems would not be expected, the ceiling effect was very high at 58%. Conversely, the ceiling effect was 4.3% in the left sided stroke sample, in which cognitive and communication problems were more common.
To examine the data in more detail, we categorised patients according to their response to rehabilitation, as determined by the unadjusted BI change scores between admission and discharge. The categories were as follows:
good response: >8 point gain, n = 256
moderate response: 5–7 point gain, n = 280
minimal response: 2–4 point gain, n = 445
no change: –1 to 1 point change, n = 382
poor response: 2 point loss, n = 20.
Examination of the data revealed that 156 of the 382 patients (41%) allocated to the "no change" group, had a BI admission score of either 19 (n = 81) or 20 (n = 75). Thus, they were unable to improve their score by more than one point, making it impossible for them to be categorised in any other than the "no change" or "poor responders" group. Of these, only 31 patients who initially scored 19 points did not change their score. This means that, when using the BI as a measure of outcome, there was a potential underestimate of change for 125 patients (8.9% of the total sample). A further way of exploring this observation is by examining the frequency distribution of the discharge BI score for each of the subgroups, as categorised according to response to intervention. This shows that 30% of the "no change", 24% of the "minimum", 27% of the "moderate" and 31% of the "very good" responders scored a maximum of 20 points at discharge. Again, it is possible that the patients’ scores could have further improved had the scale range been larger.
Patient perceived main problem
Fifty four per cent of the total sample report mobility as their commonest "main problem", ranging from 44% to 61% across the different diagnostic groups. More specifically, walking was cited as the commonest mobility problem across the majority of diagnostic subgroups. Psychosocial issues such as home and leisure roles (range 0 to 4%), mood (range 0 to 1%), or cognition (range 0 to 1%) were rarely cited by patients as their main problem. Self-care activities (such as dressing, washing, feeding) were also rarely listed as the major problem (range 2 to 4%).
VAS scores
After adjusting the BI and FIM motor change scores for significant explanatory variables (los, admission BI, admission FIM motor subscale), the mean BI and FIM motor subscale change scores for the MS group were significantly smaller (p<0.05, 5% least significant difference comparison) and for the neuromuscular group were significantly larger (p<0.05) than each of the other diagnostic groups apart from the other brain pathology group. There was no significant difference in the mean BI or FIM motor subscale change scores for stroke, spinal cord or other brain pathology (table 5).
DISCUSSION
We have reported data obtained over a nine year period. This is the largest reported data set of inpatient neurological rehabilitation patients in the UK. It demonstrates that the systematic collection, analysis, and interpretation of standardised clinical outcomes data can be successfully incorporated into routine clinical practice within an inpatient neurological rehabilitation unit. This could act as a model for other rehabilitation units, thereby facilitating the sharing of data at a national level.
The methods for collecting the data are robust and reproducible, and the validity of the data is supported by a range of factors. Firstly, an annual comparison of demographic, diagnostic, and outcomes data demonstrated a consistency of results over the nine year period (data available on request from the authors). Secondly, features of the case mix are in line with clinical observations—more female patients with MS than male, and more male patients with stroke and spinal injuries than female. Similarly, the average age of the stroke group was higher than the MS group. Finally, the pattern of change is in keeping with clinical expectation: patients with progressively deteriorating conditions such as MS had the smallest change in physical function (effect size 0.93–1.01), and those with spontaneously recovering conditions, such as the neuromuscular group (in whom the majority had a diagnosis of Guillain–Barré syndrome), had the greatest change (effect size 1.44–1.48).
The first objective of this database study was to explore the benefits of using the database. Results from clinical databases are useful for examining, informing, and ultimately improving clinical practice based on empirical evidence. On a practical level they provide a focus for careful recording and monitoring of caseload, thus enabling the systematic accrual of information over longer periods of time about less common conditions whose details might otherwise become lost within the "paperwork system". This is particularly relevant in rehabilitation where the turnover is generally slow and the numbers admitted for specific diagnoses are relatively small. At a local level, assessment of the appropriateness of staffing levels and educational input can be facilitated by easy access to data such as the severity of disability of patients attending a unit and the patients’ perception of their main presenting problem. In our experience this has facilitated the development of a more performance oriented and accountable system of rehabilitation.
Databases are also beneficial for research purposes. For example, they can provide valuable objective data for determining sample size calculations for clinical trials. This has traditionally been difficult in neurorehabilitation where the populations studied are generally small and the turnover is slow. Our database of over nine years of data highlights considerable consistency in the change scores of the total population when annual comparisons are made, thus validating it as a source for determining sample size calculations. It also provides a quick and accurate method for assessing the feasibility of patient recruitment.
Our second objective was to determine the effectiveness (rather than efficacy) of inpatient neurorehabilitation in a range of neurological conditions. Our results provide objective evidence that, overall, patients improve significantly in functional ability. Furthermore, the VAS ratings of the patients’ perception of benefit demonstrate high levels of satisfaction. These findings support the increasing body of evidence from randomised clinical trials demonstrating the benefits of multidisciplinary inpatient rehabilitation for people with a broad range of neurological conditions.14,23 The findings contribute to the evidence base by providing important complementary information to aid our understanding of the generalisability of the results of clinical trials in a typical clinical setting.9,24
Evaluation of effectiveness by retrospective extraction of information from patients’ notes is widely recognised as being of questionable reliability because the relevant data are frequently hard to find, missing, or are inadequate in detail. A database of coded information, which is accessible on site, solves some of these problems by focusing the prospective collection of data to specific standardised information, which has been chosen for its relevance to clinical practice and the scientific rigour of the outcome measures used. Nevertheless, the quality of evidence gathered from database studies can be limited both in detail, comprehensiveness, and potential bias.3 It should not be used in isolation but in conjunction with rigorously designed clinical trials, such as randomised controlled clinical trials and prospective observational studies,24 to determine treatment effectiveness.
Deciding on relevant and meaningful outcomes to measure, and choosing validated outcome measures to evaluate them, is central to the usefulness of information gained. Many units, such as ours, use a generic disability measure such as the BI or the FIM as their primary method of evaluation. The results of the present study demonstrate that a limitation of the BI was the significant ceiling effect in our inpatient rehabilitation sample, with a potential underestimate of change for almost 10% of the total sample. These results provide empirical evidence to support the reason why many rehabilitation clinicians express dissatisfaction with the BI in its ability to detect change in patients who are less severely disabled. Such information is important in aiding clinicians to select appropriate outcomes and interpret the information generated.
The VAS results indicate that there is sometimes a disparity between changes recorded by objective functional measures and those reported by patients. This reflects the complexity of interpreting outcomes’ information and supports the view that functional measures alone do not reflect the totality of benefits gained from rehabilitation.25 Our results highlight the importance of seeking the patient’s perspective in determining where the focus and evaluation of rehabilitation should lie and in defining what constitutes a successful outcome. Qualitative work is needed to explore this further.
Our third objective was to determine variables that influence change in functional outcome. Like others in similar settings, our results demonstrate associations between functional gains and diagnosis, length of stay and functional ability at admission.26–28 It is generally agreed, however, that prediction of outcome, and the identification of factors within rehabilitation programmes that are responsible for outcome differences, is complex and requires continued research.29 This is highlighted by the fact that only 44% of the variance could be accounted for in our model.
Validity of conclusions drawn from database material depends upon the quality of data stored (completeness, reliability, and validity of the data). Over the past nine years we have learnt a number of lessons about how to optimise data quality. All patients are allocated a keyworker who coordinates and monitors the collection of data; this is facilitated by the use of an ICP. The process of outcome measurement is kept simple, short, and meaningful by choosing measures that are relatively quick and straightforward to score and which have proved psychometric properties. In our experience, long and complicated measures, however useful, are generally demotivating for staff, which inevitably impacts negatively on data completeness. Regular training about the purpose and process of outcome measurement, with specific reference to the measures used, is included within both the induction programme and regular multidisciplinary in-service educational sessions. Staff are reassurred that information will not be used out of context or to misrepresent either the patients’ needs or the outcome of intervention. Feedback of audit results of the database is provided at staff meetings. Regular feedback and training enhances the staff’s understanding of the data and their sense of ownership in how it is used, as well as maximising the reliability of the data collected. A person with dedicated time manages the database; validates, analyses, and interprets the data generated; and disseminates the results. Finally, clear and dynamic leadership is integral to the entire process. These are powerful motivating factors in optimising the quality of data collection as audit becomes integrated with clinical practice.
This database only provides local information pertaining to inpatient rehabilitation, just one stage in the rehabilitation process. In order to gain a comprehensive view, the total episode of care including the acute hospital, inpatient rehabilitation, and community follow up phase across a range of services needs to be examined. The current structure of service delivery, wherein each individual service within distinct geographical areas holds separate records of the patients’ progress, renders this virtually impossible in practical terms. Changes in terms of (a) "patient centred focus" of service delivery, (b) the use of electronic records, and (c) standardised assessment procedures may enable this in the future.
REFERENCES
Black N . High quality clinical databases: breaking down barriers. Lancet 1999;353:1205–6.
Black N . Using clinical databases in practice. Individualised prediction of survival for patients with cancer may be possible. BMJ 2003;326:2–3.
Motheral B, Brooks J, Clark MA, et al. A checklist for retrospective database studies—report of the ISPOR Task Force on retrospective databases. Value Health 2003;6:90–7.
Freeman JA, Playford ED, Thompson AJ. A neurological rehabilitation unit: audit of activity and outcome. J R Coll Phys Lond 1996;30:21–6.
Chan L, Houck P, Prela CM, et al. Using Medicare databases for outcomes research in rehabilitation medicine. Am J Phys Med Rehabil 2001;80:474–80.
Trabin T . The quality agenda accelerates: outcomes collaboratives emerge. Behav Healthcare Tomorrow 1997;April:11–13.
Johnston MV, Wood KD, Fiedler R. Characteristics of effective and efficient rehabilitation programs. Arch Phys Med Rehabil 2003;84:410–18.
Lewsey JD, Leyland AH, Murray GD, et al. Using routine data to complement and enhance the results of randomised controlled trials. Health Technol Assess 2000;4:1–54.
Padkin A, Rowan K, Black N. Using high quality clinical databases to complement the results of randomised controlled trials: the case of recombinant human activated protein C. BMJ 2001;323:923–6.
Walsh TL, Hanscom B, Lurie JD, et al. Is a condition-specific instrument for patients with low back pain/leg symptoms really necessary? The responsiveness of the Oswestry Disability Index, MODEMS, and the SF-36. Spine 2003;28:607–15.
Black N, Payne M. Improving the use of clinical databases. BMJ 2002;324:1194.
van der Putten JMF, Hobart JC, Freeman JA, et al. Measuring change in disability following inpatient rehabilitation: comparison of the responsivenesss of the Barthel Index and the Functional Independence Measure. J Neurol Neurosurg Psychiatry 1999;66:480–4.
Johnson J, Thompson AJ. Rehabilitation in a neuroscience centre: the role of expert assessment and selection. Br J Ther Rehabil 1996;3:303–8.
Freeman JA, Langdon DW, Hobart JC, et al. The impact of inpatient rehabilitation on progressive multiple sclerosis. Ann Neurol 1997;42:236–44.
Rossiter DA, Edmondson A, Al Shahi R, et al. Integrated care pathways in multiple sclerosis rehabilitation: completing the audit cycle. Mult Scler 1998;4:85–9.
World Health Organization International Classification of Functioning Disability and Health. Available from www.who.int/icf (accessed December 2003).
Mahoney FI, Barthel DW. Functional evaluation: the Barthel Index (BI). Md State Med J 1965;14:61–5.
Granger CV, Cotter AC, Hamilton BB, et al. Functional assessment scales: a study of persons after stroke. Arch Phys Med Rehabil 1993;74:133–8.
Hobart JC, Lamping DL, Freeman JA, et al. Evidence-based measurement: which disability scale for neurological rehabilitation? Neurology 2001;57:639–44.
Streiner DL, Norman GR. Health Measurement Scales: a practical guide to the development and use. Oxford: Oxford University Press, 1995:32–3.
Finch E, Brooks D, Stratford PW, et al. Physical Rehabilitation Outcome Measures: A Guide to Enhanced Clinical Decision Making. Ontario: Canadian Physiotherapy Association, 2002:244–5.
Cohen J .Statistical power analysis for the behavioral sciences, 2nd edn. Hillsdale, New Jersey: Lawrence Earlbaum Associates, 1988.
Stroke Unit Trialists’ Collaboration. Collaborative systematic review of the randomised trials of organised inpatient (stroke unit) care after stroke. BMJ 1997;314:1151–9.
Black N . Why we need observational studies to evaluate the effectiveness of health care. BMJ 1996;312:1215–18.
Edwards SGM, Playford ED, Hobart JC, et al. Comparison of physician outcome measures and patients’ perception of benefits of inpatient neurorehabilitation. BMJ 2002;324:1493.
Carey RG, Seibert JH. Who makes the most progress in inpatient rehabilitation? An analysis of functional gain. Arch Phys Med Rehabil 1988;69:337–43.
Langdon DW, Thompson AJ. Multiple sclerosis: a preliminary study of selected variables affecting rehabilitation outcome. Mult Scler 1999;5:94–100.
Johnston MV, Wood KD, Fiedler R. Characteristics of effective and efficient rehabilitation programs. Arch Phys Med Rehabil 2003;84:410–18.
Wade DT. Selection criteria for rehabilitation services. Clin Rehabil 2003;17:115–18.(J A Freeman,, J C Hobart,)