Patients, Practices, and Relationships: Challenges and Lessons Learned from the Kentucky Ambulatory Network (KAN) CaRESS Clinical Trial
http://www.100md.com
《家庭医学美国版》
Abstract
The Cardiovascular Risk Education and Social Support (CaRESS) study is a randomized controlled trial that evaluates a social support intervention toward reducing cardiovascular risk in type 2 diabetic patients. It involves multiple community-based practice sites from the Kentucky Ambulatory Network (KAN), which is a regional primary care practice-based research network (PBRN). CaRESS also implements multiple modes of data collection. The purpose of this methods article is to share lessons learned that might be useful to others developing or implementing complex studies that consent patients in PBRNs. Key points include building long-term relationships with the clinicians, adaptability when integrating into practice sites, adequate funding to support consistent data management and statistical support during all phases of the study, and creativity and perseverance for recruiting patients and practices while maintaining the integrity of the protocol.
In the fall of 2000, the Kentucky Ambulatory Network (KAN) received its first extramural infrastructure funding as a regional primary care practice-based research network (PBRN). Approximately 2 years later, the Agency for Healthcare Research and Quality (AHRQ) funded a 3-year R18 project for implementing the Cardiovascular Risk Education and Social Support (CaRESS) study through KAN. CaRESS is a clinical trial designed to evaluate the effects of mobilizing type 2 diabetic patients’ existing social support to reduce cardiovascular risk and to explicate the mechanisms of influence. The social support intervention itself is designed to be simple and practical in busy community-based primary care practices. However, evaluation of the intervention is complex.
We had anticipated and planned for many aspects of PBRN research, thanks in large part to lessons learned from others, such as through workshops conducted by the Federation of Practice-based Research Networks (FPBRN)1 and from PBRN research consultants. Nevertheless, we encountered a number of unexpected challenges as we embarked on this prospective randomized controlled trial in our new PBRN. By necessity, we adapted.
The purpose of this article is to share lessons learned that may be useful to others developing or implementing randomized controlled trials involving consented patients in primary care PBRNs. Its focus is PBRN issues, rather than all facets of the research process. Brief descriptions of our PBRN and the CaRESS study will provide context for the discussion.
Kentucky Ambulatory Network
KAN is a primary care PBRN. KAN seeks to enhance the ability of office-based clinicians to deliver high-quality primary health care to their patients through collaborative research conducted in primary care practices and through the translation of research into primary care practices. KAN emphasizes the prevention and management of common health problems in Kentucky, as well as the broader implications of these problems.
KAN is administered through the Department of Family and Community Medicine at the University of Kentucky in Lexington. Most of its member clinicians are family physicians located in central and eastern Kentucky, primarily in small private practices. KAN's membership also includes a broad array of health services researchers and other academicians at the University of Kentucky and the University of Louisville. As of December 2004, our network membership included 130 community-based clinicians in 68 practices, 36 university-based family medicine faculty members from both universities, all 7 family medicine residency programs in Kentucky, and 37 other faculty members from these institutions representing fields of expertise such as public health, gerontology, medical informatics, or health services research. KAN works closely with the Kentucky Academy of Family Physicians to promote participation in studies and network-related scientific assemblies. Among KAN community-based clinicians, approximately 80% are physicians (mostly family medicine) and 20% are nurse practitioners, physician assistants, and certified nurse midwives.
CaRESS Study Protocol
The CaRESS study has the overarching goal of reducing the incidence and severity of cardiovascular disease among high-risk patients treated in primary care settings, by improving their adherence to prescribed treatments. It is a randomized controlled trial focusing on adults with type 2 diabetes mellitus who also have uncontrolled hypertension or both hypertension and dyslipidemia. Its primary objective is to test the effectiveness of an innovative program designed to foster the involvement of a relative or close friend as a support person (SP) to improve the control of hypertension and/or dyslipidemia among these patients. The eligibility criteria for participation are that the patient be 21 years of age, have type 2 diabetes mellitus, and have a systolic blood pressure 130.
The intervention is designed for use in busy clinical practice settings. It involves the SP with the patient in a single patient education session delivered by a nurse, followed by quarterly newsletters. All patient participants must designate one person as a potential SP for the purpose of this program. Practices and their patients are randomly assigned to control versus intervention groups only after patients’ potential SPs are named. Randomization is done at the practice (clinic) level, and group assignment is fully masked until all patients at each site are enrolled. Control group patients receive the patient education session and the newsletters without the SP being included in the education. Intervention group patients bring their chosen SP with them to the patient education session (which includes special attention to the SP’s potential role in helping the patient), and the SP gets special quarterly newsletters. All patients and SPs are followed for 12 months after randomization (Figure 1).
The main outcomes compared across control and intervention groups are: systolic blood pressure, low-density lipoprotein cholesterol level, health-related quality of life, cost-effectiveness of the intervention, patient satisfaction, and hemoglobin A1C levels. The CaRESS study also evaluates the following potential cognitive-behavioral mechanisms of action for our intervention: adherence to prescribed treatment regimens, quality and degree of support person involvement, and health knowledge and beliefs. Data sources for the CaRESS study include questionnaires, blood pressure readings, blood test results, pharmacy records, and medical records.
We wished to do relatively intense probing of potential moderators of this social support intervention, but we were concerned that the attention inherent to these probes would act as a positive intervention, threatening our ability to see the effects of the simple intervention itself. Therefore we included a second intervention group in which participants have longer study visits and complete more questionnaires. Thus, at the time of consent, participants acknowledge the possibility of being assigned to the control group or one of 2 intervention groups.
The CaRESS study was designed to enroll 375 patient-participants at 15 community-based primary care practice sites. We planned to start data collection in a new practice every 4 weeks. The number of practice sites has since been expanded to accommodate fewer patients being recruited at some practices. Potential study participants are identified via billing data, sent a letter of invitation from their primary care physician, and then recruited by telephone. Recruitment bias is minimized by using randomly ordered lists of diabetic patients and by not recruiting during visits for medical care. All study visits are completed at the patient-participant’s primary care physician’s office (clinic). Research nurses and research assistants coordinate scheduling study visits and go to the practice site to collect all questionnaire-based data. Nurses and medical assistants employed at each practice are responsible for measuring blood pressure levels, performing or arranging phlebotomy, and sending blood specimens to our university core laboratory. These practice-based staff are trained and certified by our research nurses in the accurate measure of blood pressure using portable mercury sphygmomanometers supplied for the study. Each participating practice is reimbursed to offset costs of study-related efforts by their staff. Specifically, the office staff efforts include pulling the charts for our team to screen, receiving the participants through their waiting room, and collecting the blood tests and blood pressure readings. The practices provide space to conduct the chart review, patient education, and the patient visits, as well as to store the study notebooks and other materials.
In summary, CaRESS is a patient-oriented randomized interventional study that involves multiple modes of data collection including medical record review, blood tests, blood pressure readings, and surveys both in person and by telephone. Patients make all study visits to their usual primary care practice. These design features should enhance generalizability of our findings to common primary care practice environments. They also introduce some special challenges for conducting the study.
Practice Recruitment: Willing and Able
Practice recruitment is a critical step. The literature2 and our own experience suggest that many factors influence practice participation. As of October 2005, we had approached 23 practices for participation and had enrolled patients from 15 practices. Eight practices declined because of barriers we could not overcome: lack of staff and space for the study (2), insufficient number of diabetic patients (one), the parent company would have absorbed the financial compensation CaRESS provides practices for their assistance (2), and the parent company required greater compensation than budgeted or feasible (3). None expressed lack of interest. The following sections describe strategies we have found effective in recruiting practices and enabling them to participate.
An Engaging Project and Feasible Protocol
Through our PBRN work, we have found primary care practices to be very busy enterprises, usually operating with small financial margins. The practices almost universally perceive their clinician manpower and staffing levels to be just at, or below, their needs for serving their patients. We have never been told that a practice has extra personnel time on hand for special projects. Nevertheless, the clinicians we approached indicated that they perceived CaRESS as worth the time and effort their staff members and they would exert. Their feedback suggests that CaRESS is evaluating an intervention they value in primary care, an important component of PBRN research.3
In addition, clinician acceptance of the study is probably because of several aspects of study design and how we "sell" the study. We strive to make key points clear to each practice during our first and/or second conversations with clinicians and staff, emphasizing the following: (1) this patient-oriented study is designed to have the least possible impact on the practice’s flow and operations; (2) KAN staff do most of the work; (3) patient safety and satisfaction, as well as clinician and practice satisfaction, are very important to us (we would rather end a patient’s or practice’s participation than allow unresolved safety or satisfaction issues); (4) we take great care to avoid interference with the clinician-patient relationships, and we respect practice policies; (5) we strive to make blood test results useful to patients and clinicians; (6) we provide modest reimbursement to practices ($210 per patient enrolled) to mitigate practice costs of involvement; (7) we discuss the potential benefits and low risks for patients.
Physician-to-Physician Initiation and Preliminary Evaluation
A family physician-investigator (for CaRESS, usually the principal investigator) always talks with a physician leader at each potential participating practice to explain the study and earn their interest in the project. We think that it is important that the practice recruiter for our relatively long and involved study be a physician, because community physicians are more likely to take a call from another physician, and primary care physicians have high credibility in discussing the merits and feasibility of a clinical trial in office practice. We have had success with the physician investigator contacting the community physician by telephone, without any preceding letter. We ask the physician contacted at each practice to champion the project and identify a staff member as our main point of contact for future logistic arrangements with the practice. The physician "project champion" does not have to be the senior partner.
The initial discussion not only informs and persuades; it also begins evaluation of the feasibility of conducting the study in this particular practice. Does the practice see enough diabetic patients? Is there space for chart review and patient visits? Can staff help as needed? Will the physician try to include practice partners?
Adapt to the Clinicians’ Office Timetable
Part of the physician-to-physician communication, which may take more than one phone call, includes working out when CaRESS could begin in their practice. Many factors come into play, such as changes in clinicians or office staff, or moving to a new building. We learned early that practices could not commit to a start date several months in advance. Therefore, recruiting new practices to the CaRESS study is a continuous process. We may have certain practices waiting in the wings that are interested but not ready, and we work with their schedule and our schedule to find the month that works for them to start. Our research nurse coordinator plays a key role in arranging the actual start date for the study.
Evaluate Potential Institutional Barriers
Physician interest is critical, but not always sufficient. Not every practice has been able to participate even when a physician leader wanted to. Three practices were owned by an organization that charged a fee for study participation that greatly exceeded our budget. For 2 practices, reimbursement would have gone to its parent organization. We couldn’t justify the imposition without the practice receiving reimbursement, so we did not enroll those practices. The parent organization of another practice would not allow practice staff to do any work for the study during normal work hours, even though we would pay for the associated costs. Thereafter, we did not invite other regional practices affiliated with these health care organizations. For future practices and studies, we ascertain institutional affiliations as early as possible.
Consider Project Resource Limitations When Recruiting Practices
On the one hand, we would like to involve any practice interested in participating, up to the limit of our planned recruitment. On the other hand, our funding and staff are finite and centrally located, thus demanding efficiency in KAN staff travel in terms of distance and practice size. Starting up the study in a practice takes considerable KAN staff time, regardless of whether the practice has 100 or 1000 diabetic patients. As we have learned more about efficiencies related to actual patient recruitment levels and participant no-shows or rescheduling, we have reduced staff travel time by narrowing our preferred radius from approximately 50 miles down to 30 miles from our home base. This was necessary in part because the project team has needed to make more visits to the practices than we originally anticipated, for reasons such as patients missing or rescheduling visits, or practices needing more study materials quickly.
At the point of recruiting the practices, we ask the physicians about whether they think they would have enough eligible patients to warrant enrollment. We have found that physicians tend to overestimate the number of diabetic patients in their practice, possibly because diabetic patients have more frequent, and/or more complicated visits. Therefore, before finalizing a decision to conduct the study in a practice, we ask them to query their administrative (billing) data for the number of patients having a diabetes diagnosis (ICD 9 code 250.XX) within the last 2 years.
Lunch Meetings at the Practice
The "CaRESS lunch" meeting, wherein the physician PI and RN study coordinator visit the practice, is an important step in readying the practice for participation. This meeting occurs once a physician has expressed interest in participating, and includes as many of the practice’s clinicians and staff as possible, especially those designated by the physician whom we contacted. We always suggest that the practice manager attend (if there is one). KAN provides the lunch. The PI and the Research RN explain the purpose and conduct of the study, and try to show its value and feasibility. This physician-to-physician endorsement is important, especially as the PI may not have previously met or spoken with all the clinicians in the practice. In addition, the research nurse has a critical role in explaining exactly what the staff members will need to do; they want an honest and complete description of how CaRESS will fit into their practice. Typically, the PI and the research RN tour the clinic and meet various staff members. This meeting usually results in a final commitment to participate. It is a critical step in getting the practice and researchers to figure out together how to make CaRESS work well in this particular practice.
Relationships Build the Network and the Study Builds the Network
This entire recruitment process involves relationship building, not just for CaRESS but also for KAN. This defining feature of a PBRN, the long-term relationship with the clinicians, is the context for all our contact and decisions with the practices.4 We have found that PBRN project planning and funding need to include plenty of hands-on involvement of core network professionals for recruiting the practices. CaRESS has significant hands-on involvement of the principal investigator (in this case the network director). We do not restrict recruitment to current KAN members. In fact, we have found that an interesting study is the best "hook" for recruitment into KAN. Many of our participating practices have never participated in a controlled clinical trial before. We help them with the process every step of the way.
Logistic Constraints in Practice
Despite their interest, each practice has limits to what it can do. The main limiting resources are staff time and clinic space. We try to work flexibly around these limits.
Time and Space in the Practice Are Necessities
The times when the practice can accommodate our study are limited by the scheduling of their patients, providers and support staff. Typically, a practice will have one or more half-day blocks per week when the clinic space is not used at full capacity. Even on days when there is space, there may be insufficient slack in staff commitments to accommodate even our limited needs (ie, BP measurement and phlebotomy for 3 to 5 study patients per half day). We must balance the available space and practice staff against the threat of a protracted patient enrollment period that would threaten protocol integrity or our success at the next practice. At one point, we doubled our project team temporarily to fit as many visits as possible into the practice’s schedule.
In general, we are flexible. We show willingness to work in small or unusual spaces. Because we supply mercury portable sphygmomanometers, any private room will do for participant visits, including treatment rooms, clinician offices, conference rooms, and storage spaces. We do require secure storage space for study-related materials, but that has never been a problem.
Consider Practice Staff Availability
Practice (clinic) staff availability must be considered in evaluating feasibility, even though the project team is willing to do as much as possible. For this study, the office receptionist greets the study participant, and the office nurse takes blood pressure readings and performs phlebotomy. Although each patient participant only needs a few minutes of office staff time, those few minutes are a precious resource during a busy clinic day. The practice wants to treat all their patients well, regardless of whether the visit is for a study. Likewise, we want to protect the patients’ long-term relationships with the practice. We have explained that study visits occur in 3 "waves" over the year (baseline, 6 months, 12 months), that they occur only on a few days per week, and that the volume of study patient visits rarely exceeds one per hour. These explanations help to quell practice staff concerns. Nevertheless, if staff cannot be spared to facilitate the study, the practice should not be enrolled.
Patient Identification and Enrollment
Finding and enrolling eligible patients, lowering barriers to participation, and minimizing losses to follow-up are crucial to any clinical trial. As with practice enrollment, we find ourselves working to build and maintain relationships, protect the integrity of the study, and balance extravagant effort with feasibility at any given practice.
Ensure More Than Adequate Power (>80%, eg, 90%) in the Sample Size Calculations to Protect against Unanticipated Problems with Patient Recruitment or Retention
Our original projections were that we could enroll approximately 8% of all diabetic adult patients identified by the practice into the study. We originally planned to include 15 practices and enroll 25 patients at each. However, we found it more difficult than expected to identify enough eligible patients in each practice. Thus, we now anticipate including 24 practices, with variable numbers of patients enrolled per practice depending on the number of patients eligible and able to participate from each practice. The good news for patient care, but the bad news for the study, was that the main reason for lower-than-expected eligibility rates has been better-than-anticipated rates of well-controlled blood pressure (BP) based on chart review. Furthermore, among patients eligible based on chart review, 28% were screened out because their BP was not high enough at the first study visit. Table 1 illustrates the variation across practices in the percentages of diabetic patients that screened out at chart review and screened out at the first visit. It is this variation that makes it necessary to budget time and funding for the worst-case scenario in each practice.
In light of our experience that recruitment could vary markedly across practices, powers of 90% or higher are advisable to assist in compensating for this variability.
Screen and Recruit Many Patients
Because of the aforementioned lower-than-expected eligibility rates, we have learned to screen as many charts as we can at every practice, and recruit as many participants as we can from each practice. This strategy has only helped us minimize underrecruitment; we have yet to exceed the 25 patient participants we expected to recruit from each practice. Recruitment numbers are not dependent on practice size alone. The volume of charts that we can review also varies with characteristics of the practice and their records. The more organized and accessible the charts, the more we can screen per hour.
A few of our practices have electronic health records (EHRs); depending on its design, the EHR may or may not facilitate review. EHR chart review can be quicker because the project team does not have to spend time pulling and putting away charts. In addition, the EHR will speed up the review process if it has vital signs and labs located in one central place. However computer problems can slow or even stop the review process until problems are fixed. Moreover, project team access to the EHR is limited by the availability of computers in the office. For example, only one member of the team can conduct chart review unless the office has more than one computer available.
Our chart review activities are generally less imposing to practices with more space and/or less hectic daily operations. The number of participants that we can recruit from a practice depends not only on the number of potential eligible patients gleaned from chart review, but on how easy it is to reach them by telephone and letter, and on their willingness to enroll. In our network, patient populations vary significantly from practice to practice in the stability of their contact information and the chance that we can reach them. The proportion of potential participants reached by telephone who consent and complete the first study visit has ranged from 24% to 66% so far, with a mean of 47% across practices.
Recognize When to Close Enrollment and Move on to the Next Practice
For the reasons discussed above, as the project team prepares to begin CaRESS in a practice, we cannot predict with confidence how many charts will need to be reviewed to enroll 25 patients per practice. Because of our clustered design, which includes randomization at the practice level, we do not make the assignment of the practice to a condition until all participant patients from the practice complete the consent process. Therefore, enrolled patients at each practice must wait to proceed with subsequent study visits until all patients at that practice are enrolled. Our design and our budget call for enrollment at each practice to extend for no longer than 4 weeks. In some practices, we extended that timeline to recruit more participants from the practice. Now, however, we do not extend recruitment through another course of screening chart review and patient telephone calls even if we discover that the first round of recruitment comes up short. This preserves consistency across practices, helps us stay within our budget, and minimizes attrition of the first enrollees at a practice.
Patient Participation and Retention
Anticipate Challenges in Reaching Participants for Follow-up and Budget Accordingly
Barriers to participation and loss to follow-up are issues we continuously evaluate and attempt to address. In PBRN research, dropout rates and contributing factors vary across practices, as do their underlying causes. Examples of variable barriers to initial or continued participation are transportation to the primary care practice for study visits, competing demands for patients that complicate scheduling and appointment-keeping, stability of home address and telephone numbers, and social support itself.
We try to minimize dropout rates through vigorous follow-up efforts. We routinely send reminders before study visits, and make reminder phone calls. On nights and weekends we telephone participants not reached during business hours. To those we still cannot reach by phone, we send letters asking them to call us. We give each participant a refrigerator magnet at enrollment with our telephone number. We are also as flexible as possible in scheduling study visits, eg, a 6-month visit is better done at 7 months than not at all. However, the timeline of the study, individual practice-site restrictions, and finite study personnel resources force us to let certain participants go if our attempts fail.
So far, loss to follow-up has ranged from 0% to 40% over 12 months among the first 15 practices. The variability across practices means that the fewer sites involved in a pilot project, the more difficult it is to accurately plan the overall duration of recruitment or anticipate problems. Thus, our experience has taught us to build maximum flexibility into budgets and timelines, have contingency plans in case the best conceived estimates for recruitment and retention fall short, and aim for high statistical power when initially calculating the number of participants needed.
Simplify and Shorten Study Visits
Based on participants’ comments, we believe that the duration of study visits affects the attrition rate. Therefore, we felt compelled to rearrange which surveys are administered at which visits, to shorten the longest visits, and to make visit lengths more consistent. We have been able to do this without sacrificing internal validity because visits are grouped (3 visits at baseline, 2 at 6 months and 2 at 12 months). Despite this rearranging, we still have complaints about long study visits that force us to consider curtailing the number of survey instruments used.
Data Quality
Assuring high data quality in PBRN research has some special concerns when data are collected off-site, by non-university personnel, and at multiple sites involving multiple staff members that may change over time. The CaRESS project is particularly complex because it involves a variety of different types of data collection.
Standardize Clinical Routines That Include Data Collection and Monitor Them
We train all clinic nurses and medical assistants who might record blood pressure levels for the study, or perform or arrange collection of blood samples. The blood pressure measurement is based on current American Heart Association guidelines. At lunchtime sessions, our research nurses review these guidelines with the appropriate clinic staff, as well as demonstrate and practice the proper technique with them. Each involved nurse or medical assistant must pass a test demonstrating their ability to accurately measure blood pressure according to these guidelines. Training and testing is repeated after 6 months. The personnel that take the blood pressure readings initial their entries, so that we can verify their training status. In addition, all study-related blood pressure levels are measured using portable mercury sphygmomanometers that we loan to each practice.
The blood samples are drawn by office staff members who routinely perform phlebotomy as part of patient care. Then they appropriately label, package and ship the samples in approved containers to our university core laboratory for analysis. This requires excellent tracking of blood samples, some of which are drawn during patient visits when our project team is not present. Our university lab helped us create identification and tracking mechanisms for blood samples collected outside of the university clinic system, because this was our university lab director’s first experience with PBRN research. These mechanisms included identification of the patient as being in the CaRESS study; designation of this particular sample as being study-related; an alert to bill the study, not the patient, for the test; proper routing of the results to the PI; and subsequent forwarding of the results in a clinically useful time frame to the primary physician in the private practice. Our best solution to these challenges has been to help put reliable systems in place and then check them vigilantly. For example, the university lab assigned one particular staff member to track the study’s blood tests, and developed a system for tracking our tests. In addition, when we expect blood samples to arrive at the lab, we call the lab to confirm specimen receipt, and then monitor our logs to assure that we receive results when expected. When visiting the practice, we check the refrigerator to determine whether there are any samples waiting to be sent.
Control as Much of the Process as Possible
We require pharmacy records as one measure of prescription medication compliance, and both pharmacy and medical records for our cost-effectiveness analyses. Initially, practice office staff would ask patients to request their pharmacy records and bring them to the next paired visit staffed by our project team (in a week or 2). This proved to invite missing data because it depended on special action being taken by the patient. Now, we ask patients which pharmacies they have used and for their permission to request the pharmacy records. Then we request the records from the pharmacies. The new strategy has several benefits: less work for the patients, less work for the practice staff, and more control by us. We receive 98% of requested records, finding the mail order pharmacies most difficult because patients may not have correct information about how to contact them and they might not be found on the Internet.
Statistical Design and Data Entry
Budget Adequate Support for Continuous Expert Data Management and Statistical Consultation
We strongly recommend a close working relationship, and ongoing clear communication, with one or more statisticians in the design, implementation, data management, and analysis of a PBRN clinical trial. More than one statistician may be needed because of different areas of expertise and interest. For example, the issues of analyzing clustered data in the context of clinical trial design may be new territory for the project team and/or the grant reviewers. We planned for the CaRESS study to be as sound as possible in its design, including statistical power, controls for bias, and modeling of causal pathways. Primary care PBRN research tends to be less predictable or controllable than many statisticians are used to. For example, we cannot recruit the same number of patients in each practice, nor accurately estimate the number of patients we will enroll in each practice; and the practices are randomized without regard for number enrolled. We think this lack of predictability broadens and complicates the statistician’s role; he or she must not only help design the study, but also help guide it through unanticipated modifications while preserving statistical integrity. For example, we have had to recalculate the number of practices and patients needed on an ongoing basis based on enrollment and "the number of evaluable outcomes," that is, how many participants stay in the study. Therefore, the statistician’s involvement has been less compartmentalized than expected (eg, design study, supervise data entry and management, analyze data), with advice needed on an ongoing basis.
As part of overall study design, investigators would be wise to compile comprehensive pilot data regarding the anticipated average number of subjects per clinic that would ultimately be eligible for the study. This will permit more precise power calculations in the planning phase of the study. It is often the case, however, that such pilot data are not collected, which in turn can lead to adjustment of power/sample size estimation at some point (or points) over the course of the study. Statisticians involved in such research, along with the project managers, should be diligent in monitoring recruitment numbers throughout the duration of the study.
Human Subjects Protection (HSP) and the Institutional Review Board (IRB)
Although HSP and IRB processes vary widely across institutions and PBRNs, some of our experiences may offer take home messages useful to others. We would refer readers to recent articles about HSP and IRB issues in PBRN research.5,6
Project Team Prepares Applications
Because we are working with a variety of community-based practices, some were affiliated with other hospitals or practice organizations that needed to review and approve the CaRESS protocol. So far, this has only required extra planning and preparation time, and has not prevented us from working with any practice. In all cases we facilitate the IRB process as much as possible; we, not the practice, shoulder the burden. We allow enough time for the process. We would not change the protocol in any significant way, nor have we been asked to.
Proactive Communication and Collaborative Problem Solving with the IRB
In general, PBRNs present new challenges for IRBs. We have found it helpful to have ongoing, proactive communication and education with our university IRB about our network and our studies. We emphasize our respect for our IRB, as well as our commitment to participant safety and the ethical conduct of research. We do our homework with respect to regulations, propose what we think will work, and sometimes challenge our IRBs to interpret regulations and standards that they have not had to address in this context before. We remember that every IRB decision may set a precedent for our PBRN's future activities. For each protocol, we specify the roles of everyone involved (eg, who recruits participants), because their roles have implications for human subjects protection, human subjects protection training of key personnel, and regulations due to the Health Insurance Portability and Accountability Act (HIPAA). We carefully document completion of all human subjects protection training required. By doing all this from the inception of our PBRN, we have won a certain level of trust from our IRB.
Conclusion
Multisite randomized controlled trials (RCTs) in PBRNs that require patient consent are challenging. They require careful planning, nurturing of relationships, and adaptability. It is important to avoid overly ambitious timelines, and have some room for delays and unexpected barriers in the budget. Table 2 presents an overview and synthesis of strategies we have discussed. Such strategies can help realize the potential of PBRN RCTs to improve patient care through collaborative research.
Notes
Funding: This work was supported by a grant (5R18HS10887) from the Agency for Healthcare Research and Quality (US Department of Health and Human Services).
Conflict of interest: none declared.
This article is based on a presentation made at the 2005 Convocation of Practices, hosted by the American Academy of Family Physicians National Research Network and the Federation of Practice-based Research Networks, Colorado Springs, CO, March 2005.
Received for publication July 31, 2005. Revision received October 27, 2005. Accepted for publication October 31, 2005.
References
Peterson K, Staton EW, Calmbach WL. AAFP Convocation of Practices and Network: report from the 2005 annual meeting. J Am Board Fam Pract 2005; 18: 323–5.
Croughan M. Factors influencing physician participation in practice-based research network studies: a call for further research. J Fam Pract 2001; 50: 978–9.
Nutting PA, Beasley JW, Werner JJ. Practice-based research networks answer primary care questions. JAMA 1999; 281: 686–8.
Hickner J. Practice-based network research. In: Bass M, Dunn E, Norton P, Stewart M, Tudiver F, editors. Conducting research in the practice setting. Newbury Park (CA): Sage Publications; 1993. p. 126–39.
Wolf LE, Croughan M, Lo B. The challenges of IRB review and human subjects protections in practice-based research. Med Care 2002; 40: 521–9.
Wolf LE, Walden JF, Lo B. Human subjects issues and IRB review in practice-based research. Ann Fam Med 2005; 3 Suppl 1: S30–7.(Margaret M. Love, PhD, Ke)
The Cardiovascular Risk Education and Social Support (CaRESS) study is a randomized controlled trial that evaluates a social support intervention toward reducing cardiovascular risk in type 2 diabetic patients. It involves multiple community-based practice sites from the Kentucky Ambulatory Network (KAN), which is a regional primary care practice-based research network (PBRN). CaRESS also implements multiple modes of data collection. The purpose of this methods article is to share lessons learned that might be useful to others developing or implementing complex studies that consent patients in PBRNs. Key points include building long-term relationships with the clinicians, adaptability when integrating into practice sites, adequate funding to support consistent data management and statistical support during all phases of the study, and creativity and perseverance for recruiting patients and practices while maintaining the integrity of the protocol.
In the fall of 2000, the Kentucky Ambulatory Network (KAN) received its first extramural infrastructure funding as a regional primary care practice-based research network (PBRN). Approximately 2 years later, the Agency for Healthcare Research and Quality (AHRQ) funded a 3-year R18 project for implementing the Cardiovascular Risk Education and Social Support (CaRESS) study through KAN. CaRESS is a clinical trial designed to evaluate the effects of mobilizing type 2 diabetic patients’ existing social support to reduce cardiovascular risk and to explicate the mechanisms of influence. The social support intervention itself is designed to be simple and practical in busy community-based primary care practices. However, evaluation of the intervention is complex.
We had anticipated and planned for many aspects of PBRN research, thanks in large part to lessons learned from others, such as through workshops conducted by the Federation of Practice-based Research Networks (FPBRN)1 and from PBRN research consultants. Nevertheless, we encountered a number of unexpected challenges as we embarked on this prospective randomized controlled trial in our new PBRN. By necessity, we adapted.
The purpose of this article is to share lessons learned that may be useful to others developing or implementing randomized controlled trials involving consented patients in primary care PBRNs. Its focus is PBRN issues, rather than all facets of the research process. Brief descriptions of our PBRN and the CaRESS study will provide context for the discussion.
Kentucky Ambulatory Network
KAN is a primary care PBRN. KAN seeks to enhance the ability of office-based clinicians to deliver high-quality primary health care to their patients through collaborative research conducted in primary care practices and through the translation of research into primary care practices. KAN emphasizes the prevention and management of common health problems in Kentucky, as well as the broader implications of these problems.
KAN is administered through the Department of Family and Community Medicine at the University of Kentucky in Lexington. Most of its member clinicians are family physicians located in central and eastern Kentucky, primarily in small private practices. KAN's membership also includes a broad array of health services researchers and other academicians at the University of Kentucky and the University of Louisville. As of December 2004, our network membership included 130 community-based clinicians in 68 practices, 36 university-based family medicine faculty members from both universities, all 7 family medicine residency programs in Kentucky, and 37 other faculty members from these institutions representing fields of expertise such as public health, gerontology, medical informatics, or health services research. KAN works closely with the Kentucky Academy of Family Physicians to promote participation in studies and network-related scientific assemblies. Among KAN community-based clinicians, approximately 80% are physicians (mostly family medicine) and 20% are nurse practitioners, physician assistants, and certified nurse midwives.
CaRESS Study Protocol
The CaRESS study has the overarching goal of reducing the incidence and severity of cardiovascular disease among high-risk patients treated in primary care settings, by improving their adherence to prescribed treatments. It is a randomized controlled trial focusing on adults with type 2 diabetes mellitus who also have uncontrolled hypertension or both hypertension and dyslipidemia. Its primary objective is to test the effectiveness of an innovative program designed to foster the involvement of a relative or close friend as a support person (SP) to improve the control of hypertension and/or dyslipidemia among these patients. The eligibility criteria for participation are that the patient be 21 years of age, have type 2 diabetes mellitus, and have a systolic blood pressure 130.
The intervention is designed for use in busy clinical practice settings. It involves the SP with the patient in a single patient education session delivered by a nurse, followed by quarterly newsletters. All patient participants must designate one person as a potential SP for the purpose of this program. Practices and their patients are randomly assigned to control versus intervention groups only after patients’ potential SPs are named. Randomization is done at the practice (clinic) level, and group assignment is fully masked until all patients at each site are enrolled. Control group patients receive the patient education session and the newsletters without the SP being included in the education. Intervention group patients bring their chosen SP with them to the patient education session (which includes special attention to the SP’s potential role in helping the patient), and the SP gets special quarterly newsletters. All patients and SPs are followed for 12 months after randomization (Figure 1).
The main outcomes compared across control and intervention groups are: systolic blood pressure, low-density lipoprotein cholesterol level, health-related quality of life, cost-effectiveness of the intervention, patient satisfaction, and hemoglobin A1C levels. The CaRESS study also evaluates the following potential cognitive-behavioral mechanisms of action for our intervention: adherence to prescribed treatment regimens, quality and degree of support person involvement, and health knowledge and beliefs. Data sources for the CaRESS study include questionnaires, blood pressure readings, blood test results, pharmacy records, and medical records.
We wished to do relatively intense probing of potential moderators of this social support intervention, but we were concerned that the attention inherent to these probes would act as a positive intervention, threatening our ability to see the effects of the simple intervention itself. Therefore we included a second intervention group in which participants have longer study visits and complete more questionnaires. Thus, at the time of consent, participants acknowledge the possibility of being assigned to the control group or one of 2 intervention groups.
The CaRESS study was designed to enroll 375 patient-participants at 15 community-based primary care practice sites. We planned to start data collection in a new practice every 4 weeks. The number of practice sites has since been expanded to accommodate fewer patients being recruited at some practices. Potential study participants are identified via billing data, sent a letter of invitation from their primary care physician, and then recruited by telephone. Recruitment bias is minimized by using randomly ordered lists of diabetic patients and by not recruiting during visits for medical care. All study visits are completed at the patient-participant’s primary care physician’s office (clinic). Research nurses and research assistants coordinate scheduling study visits and go to the practice site to collect all questionnaire-based data. Nurses and medical assistants employed at each practice are responsible for measuring blood pressure levels, performing or arranging phlebotomy, and sending blood specimens to our university core laboratory. These practice-based staff are trained and certified by our research nurses in the accurate measure of blood pressure using portable mercury sphygmomanometers supplied for the study. Each participating practice is reimbursed to offset costs of study-related efforts by their staff. Specifically, the office staff efforts include pulling the charts for our team to screen, receiving the participants through their waiting room, and collecting the blood tests and blood pressure readings. The practices provide space to conduct the chart review, patient education, and the patient visits, as well as to store the study notebooks and other materials.
In summary, CaRESS is a patient-oriented randomized interventional study that involves multiple modes of data collection including medical record review, blood tests, blood pressure readings, and surveys both in person and by telephone. Patients make all study visits to their usual primary care practice. These design features should enhance generalizability of our findings to common primary care practice environments. They also introduce some special challenges for conducting the study.
Practice Recruitment: Willing and Able
Practice recruitment is a critical step. The literature2 and our own experience suggest that many factors influence practice participation. As of October 2005, we had approached 23 practices for participation and had enrolled patients from 15 practices. Eight practices declined because of barriers we could not overcome: lack of staff and space for the study (2), insufficient number of diabetic patients (one), the parent company would have absorbed the financial compensation CaRESS provides practices for their assistance (2), and the parent company required greater compensation than budgeted or feasible (3). None expressed lack of interest. The following sections describe strategies we have found effective in recruiting practices and enabling them to participate.
An Engaging Project and Feasible Protocol
Through our PBRN work, we have found primary care practices to be very busy enterprises, usually operating with small financial margins. The practices almost universally perceive their clinician manpower and staffing levels to be just at, or below, their needs for serving their patients. We have never been told that a practice has extra personnel time on hand for special projects. Nevertheless, the clinicians we approached indicated that they perceived CaRESS as worth the time and effort their staff members and they would exert. Their feedback suggests that CaRESS is evaluating an intervention they value in primary care, an important component of PBRN research.3
In addition, clinician acceptance of the study is probably because of several aspects of study design and how we "sell" the study. We strive to make key points clear to each practice during our first and/or second conversations with clinicians and staff, emphasizing the following: (1) this patient-oriented study is designed to have the least possible impact on the practice’s flow and operations; (2) KAN staff do most of the work; (3) patient safety and satisfaction, as well as clinician and practice satisfaction, are very important to us (we would rather end a patient’s or practice’s participation than allow unresolved safety or satisfaction issues); (4) we take great care to avoid interference with the clinician-patient relationships, and we respect practice policies; (5) we strive to make blood test results useful to patients and clinicians; (6) we provide modest reimbursement to practices ($210 per patient enrolled) to mitigate practice costs of involvement; (7) we discuss the potential benefits and low risks for patients.
Physician-to-Physician Initiation and Preliminary Evaluation
A family physician-investigator (for CaRESS, usually the principal investigator) always talks with a physician leader at each potential participating practice to explain the study and earn their interest in the project. We think that it is important that the practice recruiter for our relatively long and involved study be a physician, because community physicians are more likely to take a call from another physician, and primary care physicians have high credibility in discussing the merits and feasibility of a clinical trial in office practice. We have had success with the physician investigator contacting the community physician by telephone, without any preceding letter. We ask the physician contacted at each practice to champion the project and identify a staff member as our main point of contact for future logistic arrangements with the practice. The physician "project champion" does not have to be the senior partner.
The initial discussion not only informs and persuades; it also begins evaluation of the feasibility of conducting the study in this particular practice. Does the practice see enough diabetic patients? Is there space for chart review and patient visits? Can staff help as needed? Will the physician try to include practice partners?
Adapt to the Clinicians’ Office Timetable
Part of the physician-to-physician communication, which may take more than one phone call, includes working out when CaRESS could begin in their practice. Many factors come into play, such as changes in clinicians or office staff, or moving to a new building. We learned early that practices could not commit to a start date several months in advance. Therefore, recruiting new practices to the CaRESS study is a continuous process. We may have certain practices waiting in the wings that are interested but not ready, and we work with their schedule and our schedule to find the month that works for them to start. Our research nurse coordinator plays a key role in arranging the actual start date for the study.
Evaluate Potential Institutional Barriers
Physician interest is critical, but not always sufficient. Not every practice has been able to participate even when a physician leader wanted to. Three practices were owned by an organization that charged a fee for study participation that greatly exceeded our budget. For 2 practices, reimbursement would have gone to its parent organization. We couldn’t justify the imposition without the practice receiving reimbursement, so we did not enroll those practices. The parent organization of another practice would not allow practice staff to do any work for the study during normal work hours, even though we would pay for the associated costs. Thereafter, we did not invite other regional practices affiliated with these health care organizations. For future practices and studies, we ascertain institutional affiliations as early as possible.
Consider Project Resource Limitations When Recruiting Practices
On the one hand, we would like to involve any practice interested in participating, up to the limit of our planned recruitment. On the other hand, our funding and staff are finite and centrally located, thus demanding efficiency in KAN staff travel in terms of distance and practice size. Starting up the study in a practice takes considerable KAN staff time, regardless of whether the practice has 100 or 1000 diabetic patients. As we have learned more about efficiencies related to actual patient recruitment levels and participant no-shows or rescheduling, we have reduced staff travel time by narrowing our preferred radius from approximately 50 miles down to 30 miles from our home base. This was necessary in part because the project team has needed to make more visits to the practices than we originally anticipated, for reasons such as patients missing or rescheduling visits, or practices needing more study materials quickly.
At the point of recruiting the practices, we ask the physicians about whether they think they would have enough eligible patients to warrant enrollment. We have found that physicians tend to overestimate the number of diabetic patients in their practice, possibly because diabetic patients have more frequent, and/or more complicated visits. Therefore, before finalizing a decision to conduct the study in a practice, we ask them to query their administrative (billing) data for the number of patients having a diabetes diagnosis (ICD 9 code 250.XX) within the last 2 years.
Lunch Meetings at the Practice
The "CaRESS lunch" meeting, wherein the physician PI and RN study coordinator visit the practice, is an important step in readying the practice for participation. This meeting occurs once a physician has expressed interest in participating, and includes as many of the practice’s clinicians and staff as possible, especially those designated by the physician whom we contacted. We always suggest that the practice manager attend (if there is one). KAN provides the lunch. The PI and the Research RN explain the purpose and conduct of the study, and try to show its value and feasibility. This physician-to-physician endorsement is important, especially as the PI may not have previously met or spoken with all the clinicians in the practice. In addition, the research nurse has a critical role in explaining exactly what the staff members will need to do; they want an honest and complete description of how CaRESS will fit into their practice. Typically, the PI and the research RN tour the clinic and meet various staff members. This meeting usually results in a final commitment to participate. It is a critical step in getting the practice and researchers to figure out together how to make CaRESS work well in this particular practice.
Relationships Build the Network and the Study Builds the Network
This entire recruitment process involves relationship building, not just for CaRESS but also for KAN. This defining feature of a PBRN, the long-term relationship with the clinicians, is the context for all our contact and decisions with the practices.4 We have found that PBRN project planning and funding need to include plenty of hands-on involvement of core network professionals for recruiting the practices. CaRESS has significant hands-on involvement of the principal investigator (in this case the network director). We do not restrict recruitment to current KAN members. In fact, we have found that an interesting study is the best "hook" for recruitment into KAN. Many of our participating practices have never participated in a controlled clinical trial before. We help them with the process every step of the way.
Logistic Constraints in Practice
Despite their interest, each practice has limits to what it can do. The main limiting resources are staff time and clinic space. We try to work flexibly around these limits.
Time and Space in the Practice Are Necessities
The times when the practice can accommodate our study are limited by the scheduling of their patients, providers and support staff. Typically, a practice will have one or more half-day blocks per week when the clinic space is not used at full capacity. Even on days when there is space, there may be insufficient slack in staff commitments to accommodate even our limited needs (ie, BP measurement and phlebotomy for 3 to 5 study patients per half day). We must balance the available space and practice staff against the threat of a protracted patient enrollment period that would threaten protocol integrity or our success at the next practice. At one point, we doubled our project team temporarily to fit as many visits as possible into the practice’s schedule.
In general, we are flexible. We show willingness to work in small or unusual spaces. Because we supply mercury portable sphygmomanometers, any private room will do for participant visits, including treatment rooms, clinician offices, conference rooms, and storage spaces. We do require secure storage space for study-related materials, but that has never been a problem.
Consider Practice Staff Availability
Practice (clinic) staff availability must be considered in evaluating feasibility, even though the project team is willing to do as much as possible. For this study, the office receptionist greets the study participant, and the office nurse takes blood pressure readings and performs phlebotomy. Although each patient participant only needs a few minutes of office staff time, those few minutes are a precious resource during a busy clinic day. The practice wants to treat all their patients well, regardless of whether the visit is for a study. Likewise, we want to protect the patients’ long-term relationships with the practice. We have explained that study visits occur in 3 "waves" over the year (baseline, 6 months, 12 months), that they occur only on a few days per week, and that the volume of study patient visits rarely exceeds one per hour. These explanations help to quell practice staff concerns. Nevertheless, if staff cannot be spared to facilitate the study, the practice should not be enrolled.
Patient Identification and Enrollment
Finding and enrolling eligible patients, lowering barriers to participation, and minimizing losses to follow-up are crucial to any clinical trial. As with practice enrollment, we find ourselves working to build and maintain relationships, protect the integrity of the study, and balance extravagant effort with feasibility at any given practice.
Ensure More Than Adequate Power (>80%, eg, 90%) in the Sample Size Calculations to Protect against Unanticipated Problems with Patient Recruitment or Retention
Our original projections were that we could enroll approximately 8% of all diabetic adult patients identified by the practice into the study. We originally planned to include 15 practices and enroll 25 patients at each. However, we found it more difficult than expected to identify enough eligible patients in each practice. Thus, we now anticipate including 24 practices, with variable numbers of patients enrolled per practice depending on the number of patients eligible and able to participate from each practice. The good news for patient care, but the bad news for the study, was that the main reason for lower-than-expected eligibility rates has been better-than-anticipated rates of well-controlled blood pressure (BP) based on chart review. Furthermore, among patients eligible based on chart review, 28% were screened out because their BP was not high enough at the first study visit. Table 1 illustrates the variation across practices in the percentages of diabetic patients that screened out at chart review and screened out at the first visit. It is this variation that makes it necessary to budget time and funding for the worst-case scenario in each practice.
In light of our experience that recruitment could vary markedly across practices, powers of 90% or higher are advisable to assist in compensating for this variability.
Screen and Recruit Many Patients
Because of the aforementioned lower-than-expected eligibility rates, we have learned to screen as many charts as we can at every practice, and recruit as many participants as we can from each practice. This strategy has only helped us minimize underrecruitment; we have yet to exceed the 25 patient participants we expected to recruit from each practice. Recruitment numbers are not dependent on practice size alone. The volume of charts that we can review also varies with characteristics of the practice and their records. The more organized and accessible the charts, the more we can screen per hour.
A few of our practices have electronic health records (EHRs); depending on its design, the EHR may or may not facilitate review. EHR chart review can be quicker because the project team does not have to spend time pulling and putting away charts. In addition, the EHR will speed up the review process if it has vital signs and labs located in one central place. However computer problems can slow or even stop the review process until problems are fixed. Moreover, project team access to the EHR is limited by the availability of computers in the office. For example, only one member of the team can conduct chart review unless the office has more than one computer available.
Our chart review activities are generally less imposing to practices with more space and/or less hectic daily operations. The number of participants that we can recruit from a practice depends not only on the number of potential eligible patients gleaned from chart review, but on how easy it is to reach them by telephone and letter, and on their willingness to enroll. In our network, patient populations vary significantly from practice to practice in the stability of their contact information and the chance that we can reach them. The proportion of potential participants reached by telephone who consent and complete the first study visit has ranged from 24% to 66% so far, with a mean of 47% across practices.
Recognize When to Close Enrollment and Move on to the Next Practice
For the reasons discussed above, as the project team prepares to begin CaRESS in a practice, we cannot predict with confidence how many charts will need to be reviewed to enroll 25 patients per practice. Because of our clustered design, which includes randomization at the practice level, we do not make the assignment of the practice to a condition until all participant patients from the practice complete the consent process. Therefore, enrolled patients at each practice must wait to proceed with subsequent study visits until all patients at that practice are enrolled. Our design and our budget call for enrollment at each practice to extend for no longer than 4 weeks. In some practices, we extended that timeline to recruit more participants from the practice. Now, however, we do not extend recruitment through another course of screening chart review and patient telephone calls even if we discover that the first round of recruitment comes up short. This preserves consistency across practices, helps us stay within our budget, and minimizes attrition of the first enrollees at a practice.
Patient Participation and Retention
Anticipate Challenges in Reaching Participants for Follow-up and Budget Accordingly
Barriers to participation and loss to follow-up are issues we continuously evaluate and attempt to address. In PBRN research, dropout rates and contributing factors vary across practices, as do their underlying causes. Examples of variable barriers to initial or continued participation are transportation to the primary care practice for study visits, competing demands for patients that complicate scheduling and appointment-keeping, stability of home address and telephone numbers, and social support itself.
We try to minimize dropout rates through vigorous follow-up efforts. We routinely send reminders before study visits, and make reminder phone calls. On nights and weekends we telephone participants not reached during business hours. To those we still cannot reach by phone, we send letters asking them to call us. We give each participant a refrigerator magnet at enrollment with our telephone number. We are also as flexible as possible in scheduling study visits, eg, a 6-month visit is better done at 7 months than not at all. However, the timeline of the study, individual practice-site restrictions, and finite study personnel resources force us to let certain participants go if our attempts fail.
So far, loss to follow-up has ranged from 0% to 40% over 12 months among the first 15 practices. The variability across practices means that the fewer sites involved in a pilot project, the more difficult it is to accurately plan the overall duration of recruitment or anticipate problems. Thus, our experience has taught us to build maximum flexibility into budgets and timelines, have contingency plans in case the best conceived estimates for recruitment and retention fall short, and aim for high statistical power when initially calculating the number of participants needed.
Simplify and Shorten Study Visits
Based on participants’ comments, we believe that the duration of study visits affects the attrition rate. Therefore, we felt compelled to rearrange which surveys are administered at which visits, to shorten the longest visits, and to make visit lengths more consistent. We have been able to do this without sacrificing internal validity because visits are grouped (3 visits at baseline, 2 at 6 months and 2 at 12 months). Despite this rearranging, we still have complaints about long study visits that force us to consider curtailing the number of survey instruments used.
Data Quality
Assuring high data quality in PBRN research has some special concerns when data are collected off-site, by non-university personnel, and at multiple sites involving multiple staff members that may change over time. The CaRESS project is particularly complex because it involves a variety of different types of data collection.
Standardize Clinical Routines That Include Data Collection and Monitor Them
We train all clinic nurses and medical assistants who might record blood pressure levels for the study, or perform or arrange collection of blood samples. The blood pressure measurement is based on current American Heart Association guidelines. At lunchtime sessions, our research nurses review these guidelines with the appropriate clinic staff, as well as demonstrate and practice the proper technique with them. Each involved nurse or medical assistant must pass a test demonstrating their ability to accurately measure blood pressure according to these guidelines. Training and testing is repeated after 6 months. The personnel that take the blood pressure readings initial their entries, so that we can verify their training status. In addition, all study-related blood pressure levels are measured using portable mercury sphygmomanometers that we loan to each practice.
The blood samples are drawn by office staff members who routinely perform phlebotomy as part of patient care. Then they appropriately label, package and ship the samples in approved containers to our university core laboratory for analysis. This requires excellent tracking of blood samples, some of which are drawn during patient visits when our project team is not present. Our university lab helped us create identification and tracking mechanisms for blood samples collected outside of the university clinic system, because this was our university lab director’s first experience with PBRN research. These mechanisms included identification of the patient as being in the CaRESS study; designation of this particular sample as being study-related; an alert to bill the study, not the patient, for the test; proper routing of the results to the PI; and subsequent forwarding of the results in a clinically useful time frame to the primary physician in the private practice. Our best solution to these challenges has been to help put reliable systems in place and then check them vigilantly. For example, the university lab assigned one particular staff member to track the study’s blood tests, and developed a system for tracking our tests. In addition, when we expect blood samples to arrive at the lab, we call the lab to confirm specimen receipt, and then monitor our logs to assure that we receive results when expected. When visiting the practice, we check the refrigerator to determine whether there are any samples waiting to be sent.
Control as Much of the Process as Possible
We require pharmacy records as one measure of prescription medication compliance, and both pharmacy and medical records for our cost-effectiveness analyses. Initially, practice office staff would ask patients to request their pharmacy records and bring them to the next paired visit staffed by our project team (in a week or 2). This proved to invite missing data because it depended on special action being taken by the patient. Now, we ask patients which pharmacies they have used and for their permission to request the pharmacy records. Then we request the records from the pharmacies. The new strategy has several benefits: less work for the patients, less work for the practice staff, and more control by us. We receive 98% of requested records, finding the mail order pharmacies most difficult because patients may not have correct information about how to contact them and they might not be found on the Internet.
Statistical Design and Data Entry
Budget Adequate Support for Continuous Expert Data Management and Statistical Consultation
We strongly recommend a close working relationship, and ongoing clear communication, with one or more statisticians in the design, implementation, data management, and analysis of a PBRN clinical trial. More than one statistician may be needed because of different areas of expertise and interest. For example, the issues of analyzing clustered data in the context of clinical trial design may be new territory for the project team and/or the grant reviewers. We planned for the CaRESS study to be as sound as possible in its design, including statistical power, controls for bias, and modeling of causal pathways. Primary care PBRN research tends to be less predictable or controllable than many statisticians are used to. For example, we cannot recruit the same number of patients in each practice, nor accurately estimate the number of patients we will enroll in each practice; and the practices are randomized without regard for number enrolled. We think this lack of predictability broadens and complicates the statistician’s role; he or she must not only help design the study, but also help guide it through unanticipated modifications while preserving statistical integrity. For example, we have had to recalculate the number of practices and patients needed on an ongoing basis based on enrollment and "the number of evaluable outcomes," that is, how many participants stay in the study. Therefore, the statistician’s involvement has been less compartmentalized than expected (eg, design study, supervise data entry and management, analyze data), with advice needed on an ongoing basis.
As part of overall study design, investigators would be wise to compile comprehensive pilot data regarding the anticipated average number of subjects per clinic that would ultimately be eligible for the study. This will permit more precise power calculations in the planning phase of the study. It is often the case, however, that such pilot data are not collected, which in turn can lead to adjustment of power/sample size estimation at some point (or points) over the course of the study. Statisticians involved in such research, along with the project managers, should be diligent in monitoring recruitment numbers throughout the duration of the study.
Human Subjects Protection (HSP) and the Institutional Review Board (IRB)
Although HSP and IRB processes vary widely across institutions and PBRNs, some of our experiences may offer take home messages useful to others. We would refer readers to recent articles about HSP and IRB issues in PBRN research.5,6
Project Team Prepares Applications
Because we are working with a variety of community-based practices, some were affiliated with other hospitals or practice organizations that needed to review and approve the CaRESS protocol. So far, this has only required extra planning and preparation time, and has not prevented us from working with any practice. In all cases we facilitate the IRB process as much as possible; we, not the practice, shoulder the burden. We allow enough time for the process. We would not change the protocol in any significant way, nor have we been asked to.
Proactive Communication and Collaborative Problem Solving with the IRB
In general, PBRNs present new challenges for IRBs. We have found it helpful to have ongoing, proactive communication and education with our university IRB about our network and our studies. We emphasize our respect for our IRB, as well as our commitment to participant safety and the ethical conduct of research. We do our homework with respect to regulations, propose what we think will work, and sometimes challenge our IRBs to interpret regulations and standards that they have not had to address in this context before. We remember that every IRB decision may set a precedent for our PBRN's future activities. For each protocol, we specify the roles of everyone involved (eg, who recruits participants), because their roles have implications for human subjects protection, human subjects protection training of key personnel, and regulations due to the Health Insurance Portability and Accountability Act (HIPAA). We carefully document completion of all human subjects protection training required. By doing all this from the inception of our PBRN, we have won a certain level of trust from our IRB.
Conclusion
Multisite randomized controlled trials (RCTs) in PBRNs that require patient consent are challenging. They require careful planning, nurturing of relationships, and adaptability. It is important to avoid overly ambitious timelines, and have some room for delays and unexpected barriers in the budget. Table 2 presents an overview and synthesis of strategies we have discussed. Such strategies can help realize the potential of PBRN RCTs to improve patient care through collaborative research.
Notes
Funding: This work was supported by a grant (5R18HS10887) from the Agency for Healthcare Research and Quality (US Department of Health and Human Services).
Conflict of interest: none declared.
This article is based on a presentation made at the 2005 Convocation of Practices, hosted by the American Academy of Family Physicians National Research Network and the Federation of Practice-based Research Networks, Colorado Springs, CO, March 2005.
Received for publication July 31, 2005. Revision received October 27, 2005. Accepted for publication October 31, 2005.
References
Peterson K, Staton EW, Calmbach WL. AAFP Convocation of Practices and Network: report from the 2005 annual meeting. J Am Board Fam Pract 2005; 18: 323–5.
Croughan M. Factors influencing physician participation in practice-based research network studies: a call for further research. J Fam Pract 2001; 50: 978–9.
Nutting PA, Beasley JW, Werner JJ. Practice-based research networks answer primary care questions. JAMA 1999; 281: 686–8.
Hickner J. Practice-based network research. In: Bass M, Dunn E, Norton P, Stewart M, Tudiver F, editors. Conducting research in the practice setting. Newbury Park (CA): Sage Publications; 1993. p. 126–39.
Wolf LE, Croughan M, Lo B. The challenges of IRB review and human subjects protections in practice-based research. Med Care 2002; 40: 521–9.
Wolf LE, Walden JF, Lo B. Human subjects issues and IRB review in practice-based research. Ann Fam Med 2005; 3 Suppl 1: S30–7.(Margaret M. Love, PhD, Ke)