Assessment of an intervention to train teaching hospital care providers in quality management
http://www.100md.com
《健康质量安全杂志》
1 Qualitique et évaluation médicale Unit, Centre Hospitalier Universitaire, Grenoble, France
2 Centre de Recherche: Innovation Socio-Technique et Organisations industrielles Laboratory, Université Pierre Mendès-France, Grenoble, France
ABSTRACT
Background: Successful implementation of continuous quality improvement (CQI) programs in hospitals remains rare in all countries, making it necessary to experiment with implementation methods while considering the cultural factors of resistance to change.
Objective: To assess the impact of an educational intervention on involvement of clinical department staff in the quality process.
Setting: Twelve voluntary clinical departments (six experimental and six controls) in a French 2000-bed university hospital comprising 40 clinical departments.
Intervention: Three day training seminar to a group of 12–20 staff members from each department.
Design: Quasi-experimental post-test only design study with control group conducted 12 months after the intervention with a questionnaire completed in a face-to-face interview.
Subjects: 98 trained staff and 100 untrained staff from the six experimental departments and 100 staff from the six control departments.
Principal measurements: Declared knowledge of the CQI methods and participation in quality management activities.
Results: 286 people (96%) were involved in the study. More of the trained staff knew the CQI methods (62.4%) than staff in the control departments (16.5%) (adjusted odds ratio (ORa) = 10.6 (95% CI 4.97 to 22.62)). More trained staff also participated in quality improvement work groups than control department staff (76.3% v 14.0%; ORa = 27.4 (95% CI 11.6 to 64.4)). In the experimental departments the untrained staff’s knowledge of CQI methods and their participation in work groups did not differ from that of control department staff.
Conclusions: A continuing education intervention can involve care providers in CQI. Dissemination of knowledge from trained personnel to other staff members remains limited.
Keywords: quality of care; continuing education; continuous quality improvement; training
It is widely acknowledged that hospitals need to implement a quality improvement process. However, successful initiatives to implement continuous quality improvement (CQI) in large hospitals remain rare.1–4 Quality management is more easily implemented in hospitals with a participatory and flexible organizational culture directed towards the group and development, and in hospitals that actively encourage employees to take responsibility.3,5–9 These cultural features should be considered in the development of implementation plans, which should include actions aimed at supporting an organizational culture that favors quality improvement efforts.3,10 This involves changing the mental attitudes of professional staff and moving from a culture where quality is related to individual skills to a culture based on group performance and cooperation.10–15
Our team works within a public health medical structure in charge of implementing quality and risk management processes in a teaching hospital (box 1). We have conducted organization-wide improvement projects relating to topics such as the quality of hospitalization reports and drug prescriptions.16,17 These actions, founded on the traditional "guidelines–audit–feedback" model, did not succeed in modifying practices significantly and did not result in care providers actively participating in the targeted improvements.
Box 1 Quality and medical evaluation unit
Status: public health department medical unit of the Grenoble, France, teaching hospital.
Staff: three public health doctors; one quality engineer; two statisticians; one nurse; one secretary.
Missions: to develop medical practice evaluation procedures and to develop quality management in medical departments.
Activities: training, surveys, assessment, counselling, expertise.
This observation has led to a modification of our implementation strategy by giving greater importance to continuing education activity and to turning to the CQI model, which is based on participation of staff members in multidisciplinary work groups using problem solving tools.18–20
The program consists of setting up a multidisciplinary quality team of 8–15 people with two quality leaders (a doctor and a head nurse) in each voluntary medical department. Integrating the department into the program starts with a training seminar on quality management methods and tools delivered to a group of department staff.
The objective of this study was to evaluate the impact of this continuing education program on the representations, knowledge, and involvement in quality management activities of staff in six experimental departments.
METHODS
Context
The study was carried out in a French 2000-bed university hospital center employing 8000 people. A call for voluntary participation was made to all clinical department heads. Of the hospital’s 40 clinical departments, six of 12 departments which volunteered to participate were selected by the hospital’s quality committee for testing the intervention on criteria of feasibility, immediate availability, and balance between the types of service; the other six departments acted as controls.
Intervention
The intervention consisted of a training seminar divided into six sessions of 4 hours delivered over 3 days separated by 2–4 weeks. The training successively tackled problem identification, problem solving methods, process analysis, development of guidelines, evaluation of practices, and accreditation. The training seminar was addressed to a group of 12–20 volunteers from the various professions in the departments. Each session included a lecture and an interactive workshop in small groups (5–8 people). During workshops the participants were led to identify their department’s problems, define improvement priorities, analyse the causes of the problems, build solutions, plan their implementation, and evaluate the results. Teaching documents, lecture notes, and reference texts were given to the participants and were supplemented in each session by the report of the preceding workshop session.
Study design
We used a quasi-experimental post-test only design study with a control group, comparing six experimental departments with six control departments. The study was carried out in the year 2000, 12–18 months after the previous training course.
The data were collected by a questionnaire filled in during a face-to-face interview at the work place (see Appendix 1 available online at http://www.qshc.com/supplemental). The questionnaire was developed by a multi-field working group comprising physicians and sociologists, starting from a study of semi-directed interviews conducted with 12 staff members of the six experimental departments.
The questionnaire was composed of questions with a closed answer format and explored three fields: (1) quality representations, (2) knowledge and control of the quality management methods and tools, and (3) involvement of the staff in the quality process. In the experimental departments, open questions explored the dissemination of information vectors and knowledge relating to the quality process. The questionnaire took an estimated 15–20 minutes to complete.
Participants
The six experimental departments comprised two medical departments (nephrology and gastroenterology), two surgical departments (vascular and maxillofacial), and two intensive care departments. The six control departments were also distributed into two medical departments (pediatrics and internal medicine), two surgical departments (urology and otorhinolaryngology), and two intensive care departments.
The investigation involved 298 staff members: 98 trained staff members of the six experimental departments; 100 untrained staff members (those who did not participate in the CQI training course) randomly selected from the staff of the six experimental departments; and 100 untrained staff members randomly selected from the staff of the six control departments. The random selection was done with stratification on profession.
Data analysis
The answers given by members of the three staff groups were compared. The purpose of comparing the answers of the trained personnel with those of the control department staff was to identify the effects of the intervention on staff members’ representations, knowledge and practices. Comparison of the untrained staff in the experimental departments with the staff in the control departments enabled us to study the dissemination of quality management knowledge and activities in the departments involved in the intervention.
Estimations were provided with a 95% confidence interval. The answers were compared using the 2 test which was replaced by the exact Fisher method for an expected strength lower than 3. Multivariate analysis was carried out by logistic regression to measure the effect of the study group after adjustment for the other characteristics (odds ratio adjusted, ORa). The model’s adequacy to the data was checked by the Hosmer-Lemeshow test.21 The significance threshold retained for all tests was 0.05. Data analysis was carried out using Stata 6.0 software.
RESULTS
Participation in the study
Two hundred and eighty six people agreed to answer the questionnaire (96% response rate). Data analysis was based on the answers of 93 trained and 97 untrained staff members in the experimental departments and 96 staff members in the control departments (table 1). They were divided into 191 paramedical staff (67%, mainly nurses), 50 physicians (17%), and 45 other personnel (16%) occupying administrative or logistic positions (secretaries, food service, laundry and cleaning staff).
Quality representations
Trained staff members associated the word "quality" with the idea of organization of care more often than staff members of the control departments (36.6% v 26.1%) who associated it with the idea of care skills (43.5% v 26.9%; table 2).
The idea of compliance with standards was found more often in trained people than in controls (21.5% v 7.6%). The untrained staff in the experimental departments had an intermediate position on this question, which could reflect a displacement of the representations from being in contact with trained co-workers. The representation of quality also differed depending on profession. The idea of organization was dominant in physicians (48.0%) and in the administrative and logistic staff (53.3%), whereas the paramedics placed the idea of care skills (39.6%) and of human relations first (23.5%; p<0.01). Trained staff members adhered to the quality process insofar as they recognized that situations needing improvement exist in their departments (97.9%) and that the quality process is likely to change these situations (92.5%). They viewed the main obstacles to the quality process as investment in time (71.7%), querying working behaviors (63%), and fear of paperwork (42.4%). These questions did not bring out significant differences between the study groups. On the other hand, physicians differed from the other staff in that they were motivated by accreditation (42.0% v 22.9%; p<0.05) and more feared the paperwork related to the quality process (66.7% v 39.4%; p<0.01).
Level of self-reported knowledge
Eighty six percent of the trained staff stated that they had good knowledge of the quality process compared with 34.1% of the untrained staff members in the experimental departments and 33.3% of the controls (p<0.01). More trained staff than untrained staff members of the experimental departments and controls stated that they had mastered all quality methods and tools (table 3). Among the quality methods and tools, guideline writing appeared the most familiar to them (87%) while specific quality tools such as process analysis, the Ishikawa cause-effects diagram, and the Paretto diagram were the least familiar.
The untrained staff in the experimental departments did not differ from the staff in the control departments. In fact, they reported that they were informed of the quality process activities by the guideline binder (63.9%) and by informal discussions between colleagues (57.7%). Specific communication actions concerned only some of the staff: briefings (48.5%), memorandums (36.1%), and team meetings (34%).
Multivariate analysis showed that trained staff had significantly greater knowledge of CQI than staff in the control departments (ORa = 10.61 (95% CI 4.97 to 22.62), table 4). Knowledge of CQI did not differ between untrained staff in the experimental departments and staff in the control departments (ORa = 1.24 (95% CI 0.57 to 2.71)). It should be noted that this knowledge appeared better in the surgical departments and poorer in the administrative and logistic staff.
Involvement in the quality process
Eighty nine percent of trained staff stated that they were involved in the quality process compared with 48.9% of untrained staff in the experimental departments and 65.6% of the staff members in the control departments (p<0.01). More trained staff members stated that they were involved in every quality activity (table 3). The most frequent activity was guideline writing in both the experimental and control departments.
Multivariate analysis showed that trained staff participated significantly more in quality improvement work groups than control staff (ORa = 27.39 (95% CI 11.64 to 64.45), table 5). Participation in work groups was also significantly lower in intensive care units and among other staff.
DISCUSSION
The results of this study show that self-reported knowledge of quality methods and tools and involvement in quality management are higher in hospital staff after a continuing education intervention than in untrained staff. This positive result is probably related to two characteristics of the training seminar—organization into sequential sessions and working in interactive groups. Other studies have also shown that these characteristics result in behavior changes.22
Self-reported knowledge and practice acquisition related primarily to simple tools and methods which gradually penetrated the hospital culture preceding the intervention. Both in knowledge of the tools and in practical application, guideline writing was the dominant activity. The importance given to guideline writing appears paradoxical, firstly because the message delivered during the training course was focused on problem solving methods, and also because staff members expressed reservations about this activity which they considered time consuming and which some compared to excessive bureaucracy.23 This paradox raises questions; the fact that writing guidelines is both the most criticized and the most frequent activity means that something more may be involved in writing guidelines than the simple application of a quality improvement method.
The second result of this study is the poor dissemination of knowledge to untrained staff in the experimental departments. The quality representations of untrained staff were intermediate between those of trained staff and controls. On the other hand, the knowledge of methods and tools and the involvement in the activities of quality management of the untrained staff did not differ significantly from those of controls. These data suggest that the quality process is not disseminated within departments. This may risk creating a sort of caste system, with those involved in quality teams isolating themselves from the rest of the staff. The quality team members then become "prescribers" of organizational and professional practice changes imposed on the other staff. We identify this as a risk to the permanence of departmental quality processes.
Careful attention must be given to involve physicians, who occupy a central position in the care process and have a key role in team management and departmental organization.24,25 Thus, physicians must take on the leadership of quality management in the care structures and accompany the organizational changes involved.11,13,19 In our experiment physicians behaved as good learners, as in other studies showing that continuing education can lead physicians to be interested in quality management.26
The main limitation of this study lies in the selection of the participants. Although the controls were randomized, the trained personnel were volunteers and were not randomized. A self-selection bias cannot therefore be eliminated, which may have resulted in over-representation in the trained group of staff members who were already particularly interested in quality management.
Secondly, staff knowledge and involvement in quality management were evaluated from their statements and one cannot formally eliminate an information bias. However, the data are in agreement with those of another study in which we analysed quality management activities in the experimental departments 1 year after the intervention and checked the effective participation of the trained personnel in the work groups.23 We found that 82% of trained personnel took part in the quality work groups, which is very similar to the declared participation rate (76.3%) by the same people. This data crosscheck can only be made for the trained group but lends some credibility to the declared data.
Key messages
Successfully implemented quality management requires the involvement of care providers.
This involvement means introducing continuous quality improvement into the culture of the individuals and the organization—that is, moving from a culture where quality is related to individual skills to a culture based on group performance and cooperation.
A strategy of decentralized implementation, based on the training of a multiprofessional team in every department of the hospital, allows a flexible adaptation to specific conditions and facilitates staff adherence to change. Multiprofessional teamwork leads to cooperation and supports a group culture.
This type of training can modify the representations, knowledge and practices of quality management by the participants and lead to real involvement by care providers in improvement projects.
The acquisition of knowledge and practices relates preferentially to the simplest tools and methods: guideline writing appears to be the dominant activity although, paradoxically, healthcare professionals fear the paperwork associated with quality assurance.
The effects of training remain limited to the participants. Knowledge is not well disseminated to untrained staff so training programmes need to involve all staff members directly.
Thirdly, the relevance of the 12–16 month period between training and evaluation can be questioned. This time period is within the usual evaluation times of medical continuing education activities which vary from 1 to 24 months.22 It should be recognized, however, that the goal of the intervention was to install quality management practices over the long term in the departments and the permanence of the effects of the intervention will have to be checked again in the future.
Finally, we did not study the possible effect of the training course on the quality of patient care, which is the final objective of staff training. We do, however, intend to study the impact of the intervention on user satisfaction, measured with a standardized and validated instrument, by repeating measurements before and after the educational intervention.27
CONCLUSIONS
The strategy of quality management implementation that we have adopted is based on three characteristics: (1) educating healthcare providers, (2) creating multiprofessional groups for CQI, and (3) focusing on problem solving methods.28–30 This decentralized approach based on the voluntary involvement of care providers allows a flexible adaptation to specific conditions and facilitates staff adherence to change.6,31 We consider this approach, which respects the strongly segmented organization of the hospital, as a stage in CQI implementation which corresponds to the strategic option defined by Shortell et al5 as creating "quality pockets". We recognize, however, that extending change from motivated workgroups to a broader community remains a challenge.6
FOOTNOTES
This study has been supported by INSERM (Institut National de la Santé et de la Recherche Médicale), the CNRS (Centre National de la Recherche Scientifique), and the MIRE (Mission Recherche du Ministère de l’Emploi et de la Solidarité) program "Decisional processes and changes in health care systems".
REFERENCES
Ovretveit J. Total quality management in European health care. Int J Health Care Qual Assur 2000;13:74–9.
Shulkin DJ. Why quality improvement efforts in health care fail and what can be done about it. Am J Med Qual 2000;15:49–53.
Huq Z, Martin TN. Workforce cultural factors in TQM/CQI implementation in hospitals. Health Care Manage Rev 2000;25:80–93.
Counte MA, Meurer S. Issues in the assessment of continuous quality improvement implementation in health care organizations. Int J Qual Health Care 2001;13:1997–2007.
Shortell SM, O’Brien JL, Carman JM, et al. Assessing the impact of continuous quality improvement/total quality management: concept versus implementation. Health Serv Res 1995;30:377–401.
Fontaine A, Vinceneux P, Pauchet Traversat AP, et al. Toward quality improvement in a French hospital: structures and culture. Int J Qual Health Care 1997;9:177–81.
Lee S, Choi K-S, Kang H-Y, et al. Assessing the factors influencing continuous quality improvement implementation: experience in Korean hospitals. Int J Qual Health Care 2002;14:383–91.
Wakefield BJ, Blegen MA, Uden-Holman T, et al. Organizational culture, continuous quality improvement, and medication administration error reporting. Am J Med Qual 2001;16:128–34.
Parker VA, Wubbenhorts WH, Young GJ, et al. Implementing quality improvement in hospitals: the role of leadership and culture. Am J Med Qual 1999;14:64–9.
Weber V, Joshi MS. Effecting and leading change in health care organizations. Jt Comm J Qual Improv 2000;26:388–99.
Berwick DM, Nolan TW. Physicians as leaders in improving health care: a new series in Annals of Internal Medicine. Ann Intern Med 1998;128:289–92.
Scott T, Mannion R, Davies HTO, et al. Implementing culture change in health care: theory and practice. Int J Qual Health Care 2003;15:111–8.
Shine KI. Health care quality and how to achieve it. Acad Med 2002;77:91–9.
Berwick DM. Crossing the boundary: changing mental models in the service of improvement. Int J Qual Health Care 1998;10:435–41.
Wakefield DS, Wakefield BJ. Overcoming the barriers to implementation of TQM/CQI in hospitals: myths and realities. QRB Qual Rev Bull 1993;19:83–8.
Bertrand D, Franois P, Olive F, et al. A programme to improve the quality of discharge letters in a university hospital. Qual Health Care 1998;7:248.
Franois P, Bertrand D, Labarère J, et al. Evaluation of a program to improve the prescription-writing quality in a university hospital. Int J Health Care Qual Assur 2001;14:268–74.
Berwick DM, Enthoven A, Bunker JP. Quality management in the NHS: the doctor’s role—I. BMJ 1992;304:235–9.
Berwick DM, Enthoven A, Bunker JP. Quality management in the NHS: the doctor’s role—II. BMJ 1992;304:304–8.
Moss F, Garside P, Dawson S. Organisational change: the key to quality improvement. Qual Health Care 1998;7 (Suppl) :1–2.
Hosmer D, Lemeshow S. Applied logistic regression. 2nd ed. New York: John Wiley & Sons, 2000.
Davis D, Thomson O’Brien MA, Freemantle N, et al. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes JAMA 1999;282:867–74.
Franois P, Peyrin JC, Touboul M, et al. Evaluating implementation of quality management systems in a teaching hospital’s clinical departments. Int J Qual Health Care 2003;15:47–55.
Reinertsen JL. Physicians as leaders in the improvement of health care systems. Ann Intern Med 1998;128:833–8.
Weeks WB, Robinson JL, Brooks WB, et al. Using early clinical experiences to integrate quality-improvement learning into medical education. Acad Med 2000;75:81–4.
Saturno PJ. Training for quality management: report on a nationwide distance learning initiative for physicians in Spain. Int J Qual Health Care 1999;11:67–71.
Labarère J, Franois P, Auquier P, et al. Development of a French inpatient satisfaction questionnaire. Int J Qual Health Care 2001;13:99–108.
Alvanzo AH, Cohen GM, Nettleman M. Changing physician behavior: half-empty or half-full Clin Governance 2003;8:69–78.
Clemmer TP, Vicki JS, Berwick DM, et al. Cooperation: the foundation of improvement. Ann Intern Med 1998;128:1004–9.
Shortell SM, Levin DZ, O’Brien JL, et al. Assessing the evidence on CQI: is the glass half empty or half full Hosp Health Serv Admin 1995;40:4–24.
Chassin MR. Improving the quality of health care: what strategy works Bull NY Acad Med 1996;73:81–91.(P Franois, D Vinck, J Lab)
2 Centre de Recherche: Innovation Socio-Technique et Organisations industrielles Laboratory, Université Pierre Mendès-France, Grenoble, France
ABSTRACT
Background: Successful implementation of continuous quality improvement (CQI) programs in hospitals remains rare in all countries, making it necessary to experiment with implementation methods while considering the cultural factors of resistance to change.
Objective: To assess the impact of an educational intervention on involvement of clinical department staff in the quality process.
Setting: Twelve voluntary clinical departments (six experimental and six controls) in a French 2000-bed university hospital comprising 40 clinical departments.
Intervention: Three day training seminar to a group of 12–20 staff members from each department.
Design: Quasi-experimental post-test only design study with control group conducted 12 months after the intervention with a questionnaire completed in a face-to-face interview.
Subjects: 98 trained staff and 100 untrained staff from the six experimental departments and 100 staff from the six control departments.
Principal measurements: Declared knowledge of the CQI methods and participation in quality management activities.
Results: 286 people (96%) were involved in the study. More of the trained staff knew the CQI methods (62.4%) than staff in the control departments (16.5%) (adjusted odds ratio (ORa) = 10.6 (95% CI 4.97 to 22.62)). More trained staff also participated in quality improvement work groups than control department staff (76.3% v 14.0%; ORa = 27.4 (95% CI 11.6 to 64.4)). In the experimental departments the untrained staff’s knowledge of CQI methods and their participation in work groups did not differ from that of control department staff.
Conclusions: A continuing education intervention can involve care providers in CQI. Dissemination of knowledge from trained personnel to other staff members remains limited.
Keywords: quality of care; continuing education; continuous quality improvement; training
It is widely acknowledged that hospitals need to implement a quality improvement process. However, successful initiatives to implement continuous quality improvement (CQI) in large hospitals remain rare.1–4 Quality management is more easily implemented in hospitals with a participatory and flexible organizational culture directed towards the group and development, and in hospitals that actively encourage employees to take responsibility.3,5–9 These cultural features should be considered in the development of implementation plans, which should include actions aimed at supporting an organizational culture that favors quality improvement efforts.3,10 This involves changing the mental attitudes of professional staff and moving from a culture where quality is related to individual skills to a culture based on group performance and cooperation.10–15
Our team works within a public health medical structure in charge of implementing quality and risk management processes in a teaching hospital (box 1). We have conducted organization-wide improvement projects relating to topics such as the quality of hospitalization reports and drug prescriptions.16,17 These actions, founded on the traditional "guidelines–audit–feedback" model, did not succeed in modifying practices significantly and did not result in care providers actively participating in the targeted improvements.
Box 1 Quality and medical evaluation unit
Status: public health department medical unit of the Grenoble, France, teaching hospital.
Staff: three public health doctors; one quality engineer; two statisticians; one nurse; one secretary.
Missions: to develop medical practice evaluation procedures and to develop quality management in medical departments.
Activities: training, surveys, assessment, counselling, expertise.
This observation has led to a modification of our implementation strategy by giving greater importance to continuing education activity and to turning to the CQI model, which is based on participation of staff members in multidisciplinary work groups using problem solving tools.18–20
The program consists of setting up a multidisciplinary quality team of 8–15 people with two quality leaders (a doctor and a head nurse) in each voluntary medical department. Integrating the department into the program starts with a training seminar on quality management methods and tools delivered to a group of department staff.
The objective of this study was to evaluate the impact of this continuing education program on the representations, knowledge, and involvement in quality management activities of staff in six experimental departments.
METHODS
Context
The study was carried out in a French 2000-bed university hospital center employing 8000 people. A call for voluntary participation was made to all clinical department heads. Of the hospital’s 40 clinical departments, six of 12 departments which volunteered to participate were selected by the hospital’s quality committee for testing the intervention on criteria of feasibility, immediate availability, and balance between the types of service; the other six departments acted as controls.
Intervention
The intervention consisted of a training seminar divided into six sessions of 4 hours delivered over 3 days separated by 2–4 weeks. The training successively tackled problem identification, problem solving methods, process analysis, development of guidelines, evaluation of practices, and accreditation. The training seminar was addressed to a group of 12–20 volunteers from the various professions in the departments. Each session included a lecture and an interactive workshop in small groups (5–8 people). During workshops the participants were led to identify their department’s problems, define improvement priorities, analyse the causes of the problems, build solutions, plan their implementation, and evaluate the results. Teaching documents, lecture notes, and reference texts were given to the participants and were supplemented in each session by the report of the preceding workshop session.
Study design
We used a quasi-experimental post-test only design study with a control group, comparing six experimental departments with six control departments. The study was carried out in the year 2000, 12–18 months after the previous training course.
The data were collected by a questionnaire filled in during a face-to-face interview at the work place (see Appendix 1 available online at http://www.qshc.com/supplemental). The questionnaire was developed by a multi-field working group comprising physicians and sociologists, starting from a study of semi-directed interviews conducted with 12 staff members of the six experimental departments.
The questionnaire was composed of questions with a closed answer format and explored three fields: (1) quality representations, (2) knowledge and control of the quality management methods and tools, and (3) involvement of the staff in the quality process. In the experimental departments, open questions explored the dissemination of information vectors and knowledge relating to the quality process. The questionnaire took an estimated 15–20 minutes to complete.
Participants
The six experimental departments comprised two medical departments (nephrology and gastroenterology), two surgical departments (vascular and maxillofacial), and two intensive care departments. The six control departments were also distributed into two medical departments (pediatrics and internal medicine), two surgical departments (urology and otorhinolaryngology), and two intensive care departments.
The investigation involved 298 staff members: 98 trained staff members of the six experimental departments; 100 untrained staff members (those who did not participate in the CQI training course) randomly selected from the staff of the six experimental departments; and 100 untrained staff members randomly selected from the staff of the six control departments. The random selection was done with stratification on profession.
Data analysis
The answers given by members of the three staff groups were compared. The purpose of comparing the answers of the trained personnel with those of the control department staff was to identify the effects of the intervention on staff members’ representations, knowledge and practices. Comparison of the untrained staff in the experimental departments with the staff in the control departments enabled us to study the dissemination of quality management knowledge and activities in the departments involved in the intervention.
Estimations were provided with a 95% confidence interval. The answers were compared using the 2 test which was replaced by the exact Fisher method for an expected strength lower than 3. Multivariate analysis was carried out by logistic regression to measure the effect of the study group after adjustment for the other characteristics (odds ratio adjusted, ORa). The model’s adequacy to the data was checked by the Hosmer-Lemeshow test.21 The significance threshold retained for all tests was 0.05. Data analysis was carried out using Stata 6.0 software.
RESULTS
Participation in the study
Two hundred and eighty six people agreed to answer the questionnaire (96% response rate). Data analysis was based on the answers of 93 trained and 97 untrained staff members in the experimental departments and 96 staff members in the control departments (table 1). They were divided into 191 paramedical staff (67%, mainly nurses), 50 physicians (17%), and 45 other personnel (16%) occupying administrative or logistic positions (secretaries, food service, laundry and cleaning staff).
Quality representations
Trained staff members associated the word "quality" with the idea of organization of care more often than staff members of the control departments (36.6% v 26.1%) who associated it with the idea of care skills (43.5% v 26.9%; table 2).
The idea of compliance with standards was found more often in trained people than in controls (21.5% v 7.6%). The untrained staff in the experimental departments had an intermediate position on this question, which could reflect a displacement of the representations from being in contact with trained co-workers. The representation of quality also differed depending on profession. The idea of organization was dominant in physicians (48.0%) and in the administrative and logistic staff (53.3%), whereas the paramedics placed the idea of care skills (39.6%) and of human relations first (23.5%; p<0.01). Trained staff members adhered to the quality process insofar as they recognized that situations needing improvement exist in their departments (97.9%) and that the quality process is likely to change these situations (92.5%). They viewed the main obstacles to the quality process as investment in time (71.7%), querying working behaviors (63%), and fear of paperwork (42.4%). These questions did not bring out significant differences between the study groups. On the other hand, physicians differed from the other staff in that they were motivated by accreditation (42.0% v 22.9%; p<0.05) and more feared the paperwork related to the quality process (66.7% v 39.4%; p<0.01).
Level of self-reported knowledge
Eighty six percent of the trained staff stated that they had good knowledge of the quality process compared with 34.1% of the untrained staff members in the experimental departments and 33.3% of the controls (p<0.01). More trained staff than untrained staff members of the experimental departments and controls stated that they had mastered all quality methods and tools (table 3). Among the quality methods and tools, guideline writing appeared the most familiar to them (87%) while specific quality tools such as process analysis, the Ishikawa cause-effects diagram, and the Paretto diagram were the least familiar.
The untrained staff in the experimental departments did not differ from the staff in the control departments. In fact, they reported that they were informed of the quality process activities by the guideline binder (63.9%) and by informal discussions between colleagues (57.7%). Specific communication actions concerned only some of the staff: briefings (48.5%), memorandums (36.1%), and team meetings (34%).
Multivariate analysis showed that trained staff had significantly greater knowledge of CQI than staff in the control departments (ORa = 10.61 (95% CI 4.97 to 22.62), table 4). Knowledge of CQI did not differ between untrained staff in the experimental departments and staff in the control departments (ORa = 1.24 (95% CI 0.57 to 2.71)). It should be noted that this knowledge appeared better in the surgical departments and poorer in the administrative and logistic staff.
Involvement in the quality process
Eighty nine percent of trained staff stated that they were involved in the quality process compared with 48.9% of untrained staff in the experimental departments and 65.6% of the staff members in the control departments (p<0.01). More trained staff members stated that they were involved in every quality activity (table 3). The most frequent activity was guideline writing in both the experimental and control departments.
Multivariate analysis showed that trained staff participated significantly more in quality improvement work groups than control staff (ORa = 27.39 (95% CI 11.64 to 64.45), table 5). Participation in work groups was also significantly lower in intensive care units and among other staff.
DISCUSSION
The results of this study show that self-reported knowledge of quality methods and tools and involvement in quality management are higher in hospital staff after a continuing education intervention than in untrained staff. This positive result is probably related to two characteristics of the training seminar—organization into sequential sessions and working in interactive groups. Other studies have also shown that these characteristics result in behavior changes.22
Self-reported knowledge and practice acquisition related primarily to simple tools and methods which gradually penetrated the hospital culture preceding the intervention. Both in knowledge of the tools and in practical application, guideline writing was the dominant activity. The importance given to guideline writing appears paradoxical, firstly because the message delivered during the training course was focused on problem solving methods, and also because staff members expressed reservations about this activity which they considered time consuming and which some compared to excessive bureaucracy.23 This paradox raises questions; the fact that writing guidelines is both the most criticized and the most frequent activity means that something more may be involved in writing guidelines than the simple application of a quality improvement method.
The second result of this study is the poor dissemination of knowledge to untrained staff in the experimental departments. The quality representations of untrained staff were intermediate between those of trained staff and controls. On the other hand, the knowledge of methods and tools and the involvement in the activities of quality management of the untrained staff did not differ significantly from those of controls. These data suggest that the quality process is not disseminated within departments. This may risk creating a sort of caste system, with those involved in quality teams isolating themselves from the rest of the staff. The quality team members then become "prescribers" of organizational and professional practice changes imposed on the other staff. We identify this as a risk to the permanence of departmental quality processes.
Careful attention must be given to involve physicians, who occupy a central position in the care process and have a key role in team management and departmental organization.24,25 Thus, physicians must take on the leadership of quality management in the care structures and accompany the organizational changes involved.11,13,19 In our experiment physicians behaved as good learners, as in other studies showing that continuing education can lead physicians to be interested in quality management.26
The main limitation of this study lies in the selection of the participants. Although the controls were randomized, the trained personnel were volunteers and were not randomized. A self-selection bias cannot therefore be eliminated, which may have resulted in over-representation in the trained group of staff members who were already particularly interested in quality management.
Secondly, staff knowledge and involvement in quality management were evaluated from their statements and one cannot formally eliminate an information bias. However, the data are in agreement with those of another study in which we analysed quality management activities in the experimental departments 1 year after the intervention and checked the effective participation of the trained personnel in the work groups.23 We found that 82% of trained personnel took part in the quality work groups, which is very similar to the declared participation rate (76.3%) by the same people. This data crosscheck can only be made for the trained group but lends some credibility to the declared data.
Key messages
Successfully implemented quality management requires the involvement of care providers.
This involvement means introducing continuous quality improvement into the culture of the individuals and the organization—that is, moving from a culture where quality is related to individual skills to a culture based on group performance and cooperation.
A strategy of decentralized implementation, based on the training of a multiprofessional team in every department of the hospital, allows a flexible adaptation to specific conditions and facilitates staff adherence to change. Multiprofessional teamwork leads to cooperation and supports a group culture.
This type of training can modify the representations, knowledge and practices of quality management by the participants and lead to real involvement by care providers in improvement projects.
The acquisition of knowledge and practices relates preferentially to the simplest tools and methods: guideline writing appears to be the dominant activity although, paradoxically, healthcare professionals fear the paperwork associated with quality assurance.
The effects of training remain limited to the participants. Knowledge is not well disseminated to untrained staff so training programmes need to involve all staff members directly.
Thirdly, the relevance of the 12–16 month period between training and evaluation can be questioned. This time period is within the usual evaluation times of medical continuing education activities which vary from 1 to 24 months.22 It should be recognized, however, that the goal of the intervention was to install quality management practices over the long term in the departments and the permanence of the effects of the intervention will have to be checked again in the future.
Finally, we did not study the possible effect of the training course on the quality of patient care, which is the final objective of staff training. We do, however, intend to study the impact of the intervention on user satisfaction, measured with a standardized and validated instrument, by repeating measurements before and after the educational intervention.27
CONCLUSIONS
The strategy of quality management implementation that we have adopted is based on three characteristics: (1) educating healthcare providers, (2) creating multiprofessional groups for CQI, and (3) focusing on problem solving methods.28–30 This decentralized approach based on the voluntary involvement of care providers allows a flexible adaptation to specific conditions and facilitates staff adherence to change.6,31 We consider this approach, which respects the strongly segmented organization of the hospital, as a stage in CQI implementation which corresponds to the strategic option defined by Shortell et al5 as creating "quality pockets". We recognize, however, that extending change from motivated workgroups to a broader community remains a challenge.6
FOOTNOTES
This study has been supported by INSERM (Institut National de la Santé et de la Recherche Médicale), the CNRS (Centre National de la Recherche Scientifique), and the MIRE (Mission Recherche du Ministère de l’Emploi et de la Solidarité) program "Decisional processes and changes in health care systems".
REFERENCES
Ovretveit J. Total quality management in European health care. Int J Health Care Qual Assur 2000;13:74–9.
Shulkin DJ. Why quality improvement efforts in health care fail and what can be done about it. Am J Med Qual 2000;15:49–53.
Huq Z, Martin TN. Workforce cultural factors in TQM/CQI implementation in hospitals. Health Care Manage Rev 2000;25:80–93.
Counte MA, Meurer S. Issues in the assessment of continuous quality improvement implementation in health care organizations. Int J Qual Health Care 2001;13:1997–2007.
Shortell SM, O’Brien JL, Carman JM, et al. Assessing the impact of continuous quality improvement/total quality management: concept versus implementation. Health Serv Res 1995;30:377–401.
Fontaine A, Vinceneux P, Pauchet Traversat AP, et al. Toward quality improvement in a French hospital: structures and culture. Int J Qual Health Care 1997;9:177–81.
Lee S, Choi K-S, Kang H-Y, et al. Assessing the factors influencing continuous quality improvement implementation: experience in Korean hospitals. Int J Qual Health Care 2002;14:383–91.
Wakefield BJ, Blegen MA, Uden-Holman T, et al. Organizational culture, continuous quality improvement, and medication administration error reporting. Am J Med Qual 2001;16:128–34.
Parker VA, Wubbenhorts WH, Young GJ, et al. Implementing quality improvement in hospitals: the role of leadership and culture. Am J Med Qual 1999;14:64–9.
Weber V, Joshi MS. Effecting and leading change in health care organizations. Jt Comm J Qual Improv 2000;26:388–99.
Berwick DM, Nolan TW. Physicians as leaders in improving health care: a new series in Annals of Internal Medicine. Ann Intern Med 1998;128:289–92.
Scott T, Mannion R, Davies HTO, et al. Implementing culture change in health care: theory and practice. Int J Qual Health Care 2003;15:111–8.
Shine KI. Health care quality and how to achieve it. Acad Med 2002;77:91–9.
Berwick DM. Crossing the boundary: changing mental models in the service of improvement. Int J Qual Health Care 1998;10:435–41.
Wakefield DS, Wakefield BJ. Overcoming the barriers to implementation of TQM/CQI in hospitals: myths and realities. QRB Qual Rev Bull 1993;19:83–8.
Bertrand D, Franois P, Olive F, et al. A programme to improve the quality of discharge letters in a university hospital. Qual Health Care 1998;7:248.
Franois P, Bertrand D, Labarère J, et al. Evaluation of a program to improve the prescription-writing quality in a university hospital. Int J Health Care Qual Assur 2001;14:268–74.
Berwick DM, Enthoven A, Bunker JP. Quality management in the NHS: the doctor’s role—I. BMJ 1992;304:235–9.
Berwick DM, Enthoven A, Bunker JP. Quality management in the NHS: the doctor’s role—II. BMJ 1992;304:304–8.
Moss F, Garside P, Dawson S. Organisational change: the key to quality improvement. Qual Health Care 1998;7 (Suppl) :1–2.
Hosmer D, Lemeshow S. Applied logistic regression. 2nd ed. New York: John Wiley & Sons, 2000.
Davis D, Thomson O’Brien MA, Freemantle N, et al. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes JAMA 1999;282:867–74.
Franois P, Peyrin JC, Touboul M, et al. Evaluating implementation of quality management systems in a teaching hospital’s clinical departments. Int J Qual Health Care 2003;15:47–55.
Reinertsen JL. Physicians as leaders in the improvement of health care systems. Ann Intern Med 1998;128:833–8.
Weeks WB, Robinson JL, Brooks WB, et al. Using early clinical experiences to integrate quality-improvement learning into medical education. Acad Med 2000;75:81–4.
Saturno PJ. Training for quality management: report on a nationwide distance learning initiative for physicians in Spain. Int J Qual Health Care 1999;11:67–71.
Labarère J, Franois P, Auquier P, et al. Development of a French inpatient satisfaction questionnaire. Int J Qual Health Care 2001;13:99–108.
Alvanzo AH, Cohen GM, Nettleman M. Changing physician behavior: half-empty or half-full Clin Governance 2003;8:69–78.
Clemmer TP, Vicki JS, Berwick DM, et al. Cooperation: the foundation of improvement. Ann Intern Med 1998;128:1004–9.
Shortell SM, Levin DZ, O’Brien JL, et al. Assessing the evidence on CQI: is the glass half empty or half full Hosp Health Serv Admin 1995;40:4–24.
Chassin MR. Improving the quality of health care: what strategy works Bull NY Acad Med 1996;73:81–91.(P Franois, D Vinck, J Lab)