CHIP Shots: Association Between the State Children's Health Insurance Programs and Immunization Rates
http://www.100md.com
《小儿科》
Department of Economics, Baruch College, City University of New York, New York, New York
National Bureau of Economic Research, New York, New York
Department of Pediatrics, Albert Einstein College of Medicine, New York, New York
ABSTRACT
Objective. The Balanced Budget Act of 1997 established the State Children's Health Insurance Program (SCHIP), which makes health insurance available to children in near-poor families who are ineligible for Medicaid. SCHIP mandates that all state plans cover the cost and administration of childhood vaccines. Whether SCHIP has narrowed immunization coverage rates between near-poor and nonpoor children is unknown. The objective of this study was to use data from the National Immunization Survey from 1995 to 2002 to analyze changes in immunization coverage rates among poor, near-poor, and nonpoor children before and after implementation of SCHIP.
Methods. A prepost analysis was made of changes in immunization rates among poor, near-poor, and nonpoor children before and after implementation of SCHIP in all 50 states and 28 Immunization Action Plan areas from 1995 to 2002. All children in the National Immunization Survey for whom information on vaccinations was available from the respondents' shot cards and/or from the children's immunization providers (N = 264214) were studied. Up-to-date status for the 4:3:1 (4 doses of diphtheria-tetanus toxoids-pertussis vaccine/3 doses of poliovirus vaccine/1 dose of measles-mumps-rubella vaccine) and the 4:3:1:3:3 (4 doses of diphtheria-tetanus toxoids-pertussis vaccine/3 doses of poliovirus vaccine/1 dose of measles-mumps-rubella vaccine/3 doses of Haemophilus influenzae type B vaccine/3 doses of hepatitis B vaccine) series as well at the hepatitis B and varicella vaccines was measured.
Results. The probability that a poor or near-poor child was up to date for the 4:3:1:3:3 vaccine series increased 11 percentage points after implementation of SCHIP. However, we observed a similar increase for nonpoor children. The proportion of poor and near-poor children who were up to date for the varicella vaccine increased between 7 and 8 percentage points more than among nonpoor children after implementation of SCHIP. Relative increases among poor and near-poor children were greater in the 28 Immunization Action Plan areas, in states with high rates of uninsured children, and among Hispanics.
Conclusion. SCHIP seems not to be associated with changes in the up-to-date status of poor and near-poor children for the 4:3:1 and the 4:3:1:3:3 vaccine series. Vaccine coverage rates increased broadly among all income groups between 1995 and 2002.
Key Words: immunizations up-to-date SCHIP
Abbreviations: SCHIP, State Children's Health Insurance Program FPL, federal poverty level NIS, National Immunization Survey 4:3:1, 4 doses of diphtheria-tetanus toxoids-pertussis vaccine/3 doses of poliovirus vaccine/1 dose of measles-mumps-rubella vaccine 4:3:1:3:3, 4 doses of diphtheria-tetanus toxoids-pertussis vaccine/3 doses of poliovirus vaccine/1 dose of measles-mumps-rubella vaccine/3 doses of Haemophilus influenzae type B vaccine/3 doses of hepatitis B vaccine VFC, Vaccine for Children
The Balanced Budget Act of 1997 established the State Children's Health Insurance Program (SCHIP), which makes health insurance available to children in near-poor families who are ineligible for Medicaid. The motivation for SCHIP came from the recognized high rates of uninsured children among the working poor. Approximately 23% of children from families with incomes between 100% and 200% of the federal poverty level (FPL) and 27% of children in poverty lacked health insurance in 1996.1 A recent analysis indicated that the proportion of children without insurance for some part of the year may be as high as 40%.2 The $40 billion appropriated over 10 years for SCHIP represents the largest extension of publicly provided health insurance since the creation of the Medicaid program.
Efforts to evaluate SCHIP have largely focused on take-up rates.3,4 The few studies of outcomes pertain to the precursors of SCHIP in selected states and have focused primarily on changes in utilization.5,6 An evaluation of New York State's Child Health Plus program in 1991, a model for the current SCHIP, found that the program was associated with increased immunization rates, although it is unclear whether the relationship was causal.7 A principal methodologic consideration to be addressed by any analysis of the effect of SCHIP is the bias inherent in comparing the experience of those who seek out and accept SCHIP coverage with that of those who do not. Unobservable characteristics differ between SCHIP enrollees and eligible individuals who decline to enroll. For avoiding confounding by these characteristics, evaluations of public programs such as SCHIP on the basis of the intention to treat compare the outcomes of entire eligible populations before and after the introduction of the new program.
In view of the potential for SCHIP programs to influence immunization rates, we used the recently released National Immunization Survey (NIS) to associate between state implementation of SCHIP and changes in immunization coverage rates. We used poverty status to characterize SCHIP eligibility, and we interpreted the results as estimates of the intention to treat. This study addresses 2 questions. First, how have vaccine coverage rates changed among poor, near-poor, and nonpoor children before and after implementation of SCHIP Second, is there any evidence that SCHIP increased the uptake of new vaccines
METHODS
The NIS
We used data from the NIS for the years 1995 to 2002. The NIS is a national probability sample of children aged 19 to 35 months that annually surveys >34000 households from all 50 states and 28 selected metropolitan areas. Details of the NIS along with the public use data files are available from the NIS web site (www.cdc.gov/nis/datafiles.htm).
Information on each surveyed child in the NIS is gathered from a respondent in each household as well as from the child's health care provider. The proportion of households with complete data from providers ranges from 62% to 70% between 1996 and 2002 and 51% in 1995.8–13 The provider data are considered the most reliable. Those with complete provider data, however, are more likely to be white, be better educated, and have greater incomes than households without provider data. Aware of the potential problems associated with selective reporting, administrators of the NIS use propensity scores within each stratum to adjust sampling weights for households with nonprovider data.14 To reduce the loss of households, we also used the surveys of respondents who use a shot card to recall receipt of vaccines. There is evidence that shot cards provide reliable information regarding vaccines but that parental recall alone does not.15,16 We assumed that there are no false positives on either the shot card or the provider survey. Thus, if a child is reported to be up to date for a vaccine on the basis of either the shot card or the provider survey, then we assume that the child is up to date for that vaccine. The addition of shot card data increases the proportion of households with complete vaccine information to between 72.9% and 81.8%.
SCHIP
The purpose of SCHIP is to extend health insurance to children in near-poor families who do not qualify for Medicaid but who are uninsured. To qualify for SCHIP federal funding, states had the option of extending Medicaid, creating a new insurance program, or combining both approaches. Eighteen states and the District of Columbia chose to extend Medicaid, 15 established separate programs, and 17 had combinations. Evidence suggests that the distinction between Medicaid extension and separate programs is not important with respect to its impact on the lack of insurance among children.4 Consequently, we did not analyze differences by program type.
We specified SCHIP in 2 ways. First, we used dummy variables to capture the timing of program implementation: 3 states began enrollment of children aged 1 to 5 in 1997, 29 in 1998, 13 in 1999, 4 in 2000, and 1 in 2001. We also used income eligibility thresholds to measure variation in extensiveness of the expansion. SCHIP allows states to raise the income eligibility thresholds above the Medicaid thresholds up to any level.1 As of 2002, 14 states had income eligibility thresholds for children 1 to 5 years of age at above 200% of the FPL, 26 states at 200% of FPL, and 11 states at <200% of FPL. For years before SCHIP, we used the Medicaid eligibility threshold for children 0 to 5 years of age. (Timing and eligibility thresholds of SCHIP for infants and children up to 5 years of age were compiled by Lisa Dubay at the Urban Institute. Note that Wisconsin did not change income eligibility threshold for children <6 under SCHIP.) In all states the relevant thresholds for infants <1 year of age are the Medicaid thresholds after which point the SCHIP thresholds apply.
We used differences in the age of the child and income eligibility thresholds between infancy and childhood to develop an algorithm to estimate the average SCHIP income eligibility threshold applicable to each surveyed child. The details of this algorithm have been published elsewhere.17
The NIS uses an algorithm that is based on reported family income and family size to create an indicator of poverty status defined as below 100% of the federal poverty level. In 1999, the NIS added a new variable that measures the ratio of income to poverty. To create a consistent series of income to poverty for years before 1999, we used the income categories and family size variable to create an income to poverty ratio for 1995–1998. We assumed that families with >$50000 per year earned $75000. As a check on our measure, we used our algorithm to compute poverty status (<100% FPL) and compared our poverty indicator with the one computed by NIS. The associated statistic ranged from .93 to .99 between 1995 and 1998. We then defined 3 categories of family income and a separate category for unknown income: poor, or those with <100% of the FPL; near-poor, or those with income 100% and <250% of the FPL; and nonpoor, or those with income 250% of the FPL. We used these as broad measures of eligibility to define exposure to SCHIP.
Because the percentage of families with unknown income varies from 12% to 24% over the study period, we show all results for poor, near-poor, and nonpoor families and families of unknown poverty. Even for those with recorded income, measurement error as a result of self-reporting remained a potential problem. As an alternative measure of socioeconomic status, we used mother's marital status and schooling. The most disadvantaged group includes children with mothers who are 20 years of age or older, who are not married, and who have 12 years of schooling or less; less disadvantaged are children whose mothers are married, who are 20 years of age and older, and who also have no more than a high school education; and the most advantaged includes children of married women with a college degree.
The effect of SCHIP on immunizations should vary by the proportion of uninsured and underinsured children among the poor and near-poor as well as the pre-SCHIP level of immunization coverage rates. States with larger numbers of uninsured children or with lower levels of immunization rates before SCHIP implementation may be expected to experience greater shifts once SCHIP programs are initiated. To exploit these potential sources of variation, we present a series of analyses by states with high rates of uninsured children, by states with large Hispanic populations, and by urban areas specifically designated as immunization action plan areas by the Centers for Disease Control and Prevention. Although immunization action plan areas were to have received additional resources to coordinate local efforts, funds were never forthcoming.18
Outcomes
We used dichotomous indicators of whether a child is up to date for either a particular vaccine or a series of vaccines. We analyzed changes in 2 series: the 4:3:1 (4 doses of diphtheria-tetanus toxoids-pertussis vaccine/3 doses of poliovirus vaccine/1 dose of measles-mumps-rubella vaccine) and the 4:3:1:3:3 (4 doses of diphtheria-tetanus toxoids-pertussis vaccine/3 doses of poliovirus vaccine/1 dose of measles-mumps-rubella vaccine/3 doses of Haemophilus influenzae type B vaccine/3 doses of hepatitis B vaccine) series. A child is up to date for the 4:3:1 series when he or she has had 4 diphtheria, tetanus, and acellular pertussis vaccines, 3 polio vaccines, and 1 measles-containing vaccine by 19 months. Up to date for the 4:3:1:3:3 series requires an additional 3 Haemophilus influenzae and 3 hepatitis B vaccinations (www.cdc.gov/nip/publications/ACIP-list.htm). The 4:3:1:3:3 is the most comprehensive series that is available throughout the study period.
We also analyzed the association between SCHIP and the uptake of new vaccines. We focused on the varicella vaccine because it was licensed by the Food and Drug Administration in March 1995 and became part of the NIS in 1997.19 Thus, the introduction of the varicella vaccine coincides with the implementation of SCHIP. Data on the pneumococcal vaccine were not available until 2001.
Statistical Analysis
We began the analysis by contrasting changes in up-to-date status of the vaccine series for poor, near-poor, and nonpoor children between 1995–1996 and 2001–2002. We present 2 sets of changes. The first shows differences in immunization rates by poverty status before and after implementation of SCHIP. We then estimate what we term relative differences, in which we subtract the change in immunization rates among nonpoor children between 1995–1996 and 2001–2002 from changes among poor and near-poor children. The objective is to eliminate trends in vaccination rates unrelated to SCHIP but related to general improvements in the immunization infrastructure, such as outreach campaigns, improvements in parent reminder systems, registries, and extended hours in clinics and doctors' offices, that occurred to various degrees over the study period.
To refine further the adjustments for confounding, we used regression analysis to estimate linear probability models. The dependent variable is a dichotomous indicator of whether the child is up to date for a vaccine series or the varicella vaccine. On the right-hand side, we include controls for the child's age; the number of children in the household; and whether the child's family is poor, near-poor, or nonpoor. We included characteristics of the mother, such as age, race, marital status, and schooling. To adjust for other initiatives, we included the number of immunization providers in a state that participates in the Vaccine for Children (VFC) program per number of children 0 to 18 years of age. In addition, we included dichotomous indicators for all 50 states to capture time-invariant differences between states in immunization rates that may be related to general immunization policies. We included the annual state unemployment rate to control for business cycle effects on public assistance programs as well as indicators for each year from 1996 to 2002, with 1995 as the reference year. The year indicators capture national trends in immunization rates. Finally, we included indicator variables for the year a state implemented its SCHIP program as well a 1-year lag in the SCHIP indicator to allow for startup delays. We interacted the SCHIP indicators with our indicators of household poverty. The coefficients on the interaction terms show the percentage point change in immunization rates before and after implementation of SCHIP among poor and near-poor children relative to nonpoor children. We refer to these as the adjusted relative changes.
We also estimated an additional set of regressions in which we used the income eligibility threshold for SCHIP instead of the implementation indicators to characterize SCHIP. The eligibility thresholds are an attempt to measure changes in the relative generosity of the SCHIP program. In New Jersey, for example, the income eligibility threshold under SCHIP was raised to 350% of the FPL, up from 133% for children 1 to 5 years of age in 1997. West Virginia, by contrast, increased eligibility thresholds from 133% to 150% of the FPL for children 1 to 5 years of age.
We accounted for the survey design of the NIS in all bivariate comparisons with the survey procedures in STATA 8.0.20 We used ordinary least squares to estimate the regression models.
None of the outcomes is in the extreme tails of the distribution; thus, marginal effects from the linear, logistic, and probit specifications are very similar. The linear probability model, however, allows for easy interpretation and facilitates comparisons between adjusted and unadjusted changes in the probability that a child is immunized, especially in a specification with interaction terms. All regressions were weighted by the sampling probabilities, and we used robust regression procedures to account for the fact that we have only state variation in our measures of program availability.
RESULTS
Characteristics of the Sample
In Table 2, we display the proportion of children who are up to date for the 4:3:1 series, the 4:3:1:3:3 series, and the hepatitis B vaccine in 1995–1996 and 2001–2002. There is an obvious poverty gradient in coverage rates. In 1995–1996, 71.5% of poor children were up to date for the 4:3:1 series as compared with 82.9% of nonpoor children. Children for whom poverty status is unknown have immunization rates similar to poor and near-poor children.
Vaccination coverage rates for the 4:3:1 series were largely unchanged over the study period, which suggests that SCHIP had little impact on this basic series (Table 2). Changes in up-to-date status for the 4:3:1:3:3 series are more substantial (Table 2). Vaccination coverage rates increased 10 to 14 percentage points among all groups between 1995–1996 and 2001–2002. The uniform increase in the 4:3:1:3:3 series largely reflects increases in coverage for the hepatitis B vaccine. As shown in Table 2, the proportion of children who were up to date for hepatitis B increased 11 to 14 percentage points between 1995–1996 and 2001–2002 irrespective of poverty status.
A more formal analysis of the relative changes in vaccine coverage rates is presented in Table 3. In column 1, we show the percentage point change in coverage rates among poor and near-poor children relative to nonpoor children. Except for hepatitis B, there is no evidence that coverage rates increased more among children who were potentially affected by SCHIP relative to children who were unaffected. The estimates in column 3 of Table 3 are from the multiple regression analysis. We refer to these as the adjusted relative changes. The figures are from the coefficients on the interactions terms between SCHIP and poverty status. They show the percentage point change in vaccine coverage rates among poor and near-poor relative to nonpoor children before and after implementation of SCHIP. None of the adjusted changes is of any meaningful magnitude or statistical significance. Indeed, the modest relative increase in hepatitis B vaccine coverage among poor and near-poor children (column 1) is eliminated after adjustment, except for those of unknown poverty.
Changes in Coverage Rates of Varicella
We display the relative changes in coverage rates for varicella in Table 5. In column 1, we show the change in varicella among poor and near-poor children after subtracting the change among the nonpoor. In column 3, we show the relative change as obtained from the regression analysis. The results are robust. In every case but 1, the adjusted changes are less than the unadjusted changes, but even the adjusted changes are relatively large and statistically significant. For instance, coverage for varicella increased by 7.1 percentage points more among near-poor relative to nonpoor children, nationally. Among the subgroup of states and urban areas with children at greater risk for uninsurance, the adjusted relative change in immunization rates for varicella among near-poor children are 10.1 to 14.0 percentage points.
A limitation of this exercise, however, is that we cannot distinguish between the natural diffusion of a new vaccine from the effects of SCHIP. We are concerned that changes among nonpoor children may not be a good counterfactual for children who are potentially affected by SCHIP because coverage rates for varicella among the nonpoor were much higher than among poor and near-poor children in 1997. As an alternative strategy, therefore, we contrasted changes across states among poor and near-poor children only. Specifically, we divided states into 2 groups: those that changed eligibility for children 1 to 5 years of age through SCHIP in 1997 or 1998 and those that changed eligibility in 1999 or later. We then computed annual differences in varicella coverage rates from 1997 to 2002 separately for the 2 groups of states. If SCHIP is associated with changes in varicella coverage rates, then annual increases should be greater between 1997 and 1999 in states that expanded eligibility for young children before 1999 than in states that expanded in 1999 or after. We show the results in Table 6. Annual increases in varicella coverage rates are essentially identical in the 2 groups of states, which is evidence against an association between SCHIP and uptake of the varicella vaccine.
DISCUSSION
The SCHIP represents an unprecedented extension of publicly financed health insurance for near-poor children, a process that began with the Medicaid eligibility expansions in the late 1980s. Numerous studies have linked increases in the availability of health insurance to increases in health care utilization among children.5,21,22 However, measures of utilization are often too general to infer much about the quality of care or its effect on health status. In this study, we associated changes in SCHIP with changes in immunization coverage rates. Vaccines arguably were the greatest public health achievement of the 20th century and an effective measure of both the quality of pediatric care and the improvement in health associated with vaccine-preventable illnesses.
We found little evidence that SCHIP has had a major impact on narrowing the immunization coverage rate gaps among poor and near-poor children relative to their nonpoor counterparts. The 1 exception is the varicella vaccine, for which differences in coverage rates among poor, near-poor, and nonpoor children converged rapidly between 1997 and 2002. Moreover, convergence was faster among poor and near-poor children in states with higher rates of uninsurance among children. We caution, however, against drawing causal inferences from these data, because we observed equivalent convergence in varicella vaccine coverage rates in states in the period before the implementation of SCHIP.
Our analysis, made possible by the recent release of the NIS, represents the first attempt to associate the extension of publicly provided health insurance for children with immunization rates across all 50 states and selected urban areas. Despite its geographic breadth, however, the NIS has less information about household characteristics than is available in smaller, more localized studies. We could not determine a child's precise eligibility for SCHIP, for instance, and we did not know which children already had health insurance that covered vaccinations. Nevertheless, stratifying the data by poor, near-poor, and nonpoor families enabled us to determine whether the timing of SCHIP implementation at the state level was associated with a narrowing of differences in immunization rates among children who likely were exposed relative to those who were unexposed to the change in publicly provided health insurance. Moreover, we were able to stratify the analyses by subsets of states and cities in which the rate of uninsurance among children was greater. Given the large sample size, we were able to detect differences in immunization rates between near-poor and nonpoor children of between 2 and 5 percentage points. That we did not find statistically significant differences in the relative change in immunization rates between near-poor and nonpoor children does not mean that SCHIP has had no effect. It is conceivable, for example, that SCHIP prevented the development of what would otherwise have been a widening of income disparity in immunization rates. Our findings do suggest, however, that any increase that may have been achieved with the advent of SCHIP has not been large enough to lessen differences in vaccine coverage rates by poverty status.
We emphasize that our study is an estimate of the intention to treat. We compared changes in immunization status among potentially eligible children before and after states implemented their SCHIP programs with changes in immunizations among nonpoor children who likely were ineligible for SCHIP. The advantage of an estimate that is based on the intention to treat is that program effects are largely free of selection bias at the individual level.
An alternative estimate of SCHIP's effect on immunization rates could be obtained from a comparison of changes in immunizations rates among children who were newly enrolled in SCHIP with changes among eligible children who were not enrolled or with changes among ineligible children with insurance coverage over the entire period. We were unable to undertake such an analysis because the NIS lacks information on SCHIP participation. However, unless enrollment in SCHIP is randomly assigned, estimates of program effectiveness on the basis of a comparison before and after enrollment in SCHIP may be confounded with unobservable characteristics of the children or families. Mothers who enroll their children in SCHIP, for example, may be more knowledgeable as to the importance of vaccines and may be more likely to have their children vaccinated in the absence of SCHIP than eligible nonparticipants. A finding that children who enroll in SCHIP are more likely to be up to date for immunizations therefore may reflect more the characteristics of the mother than the effect of health insurance. By contrast, an estimate that is based on intention-to-treat analysis, although not informative as to the effect of insurance coverage on immunization rates per se, will provide an unbiased estimate of the effect of making insurance available to a targeted population.
Our study has certain limitations. First, we could proxy eligibility for SCHIP only within relatively broad income categories. In addition, we lacked information on family income for almost 20% of our sample. However, results for children from families with unknown income were similar to those of poor and near-poor children. As an additional check, we used mother's schooling and marital status to characterize households that likely were eligible and ineligible for SCHIP. We found that the level of up-to-date immunization status and its change before and after SCHIP among children whose mothers were unmarried and who had no more than a high school education were similar to those for poor and near-poor children (results available on request), reassuring us that the missing income data for some families did not bias our estimates significantly.
We used the experience of children from nonpoor families as a counterfactual for the experience of poor and near-poor children. However, the nonpoor defined as families with income 250% of the FPL may include too many well-off families to serve as an appropriate comparison group. As an alternative, we redefined our groups so that near-poor families had incomes between 100% and 200% of the FPL and nonpoor families had incomes between 200% and 300% of FPL. The findings did not change.
Our results at the national level are similar to those from evaluations of New York State's Child Health Plus, the state's precursor to SCHIP.6,7 In 1 of the 2 evaluations, researchers found gains in immunization rates among previously uninsured children, but they observed statistically similar gains among the insured.7 Other recent studies have not found health insurance to be an important predictor of up-to-date status.23,24 Although samples in both studies were drawn from a single city, maternal employment and Women, Infants, and Children enrollment were more robust correlates of immunization status than was health insurance.
We adjusted for state variation in the unemployment rate as a control for business cycle effects. SCHIP was rolled out during a period of great economic growth. Employment growth increases the demand for preschool-aged child care. This can affect immunization rates, because many states require that children be up to date for numerous vaccines before enrollment in child care.23 Our study period also coincided with the initiation of welfare reform. There is some evidence that the Temporary Assistance for Needy Families Program, the replacement for Aid to Families With Dependent Children, resulted in a loss of Medicaid coverage for women and children whose cash assistance ended, but the effects are modest.25 However, welfare reform should have affected primarily poor families, because few near-poor families were eligible for cash assistance. Our finding that results for the poor mirrored those of the near poor suggests that welfare reform was not an important confounding factor.
The implementation of SCHIP may have affected the site at which immunizations are obtained without any major impact on vaccination rates. We tested for such changes in a previous analysis and found no association between SCHIP availability and the site of care.17 It is also possible that the VFC program may have limited the potential impact of SCHIP on immunizations by raising the pre-SCHIP level of immunization rates among poor and near-poor children. However, this could not be true for the varicella vaccine, which did not become widely available until 1996. Second, we controlled for variation in the number of VFC providers per child in each state as well as the type of VFC state. Third, there has been little growth in the number of VFC providers since 1997. Fourth, the VFC was a supply-side intervention that did not cover the administration of vaccines or the cost of the office visit for patients. SCHIP, by contrast, was a demand-side policy that covered the costs of the vaccine and the visit. In sum, we do not believe that VFC obscured the impact of SCHIP.
Our initial results suggested that SCHIP was associated with a narrowing of the poverty gradient with respect to the varicella vaccine. Relative changes in varicella coverage, both unadjusted and adjusted, were substantially greater among the poor and near poor compared with the nonpoor between 1997 and 2002. Moreover, relative changes were greater among Hispanics and among children in states with the highest rate of uninsurance. However, we questioned whether these associations were causal. We observed large increases in coverage rates for varicella among poor and near-poor children in the years before SCHIP among states that implemented SCHIP relatively late.
In sum, the good news is that in 2002 immunization rates for individual vaccines exceeded 75% before age 3, even among the poor (authors' tabulations of the 2002 NIS based on information from providers and/or respondents with shot cards). Income gradients still exist, however, and are larger for series of vaccine such as the 4:3:1:3:3. Health insurance may be a necessary condition for continued growth in coverage rates, especially as the number of new and more expensive vaccines increases, but clearly other aspects of the immunization delivery system will need strengthening if we are to achieve the Healthy People objective of 90% coverage rates for all pediatric vaccines by 2010 (www.healthypeople.gov/document/html/objectives/14-22.htm).
ACKNOWLEDGMENTS
This research was made possible by a grant from the Robert Wood Johnson Foundation Changes in Health Care Financing Organization Initiative to the Baruch College Fund.
We thank Lisa Dubay of the Urban Institute for detailed information on SCHIP.
FOOTNOTES
Accepted Nov 18, 2004.
Ted Joyce, PhD, National Bureau of Economic Research, 365 Fifth Ave, 5th Floor, New York, NY 10016. ted_joyce@baruch.cuny.edu
No conflict of interest declared.
REFERENCES
Dubay L, Hill I, Kenney G. Five Things Everyone Should Know About SCHIP. Washington, DC: Urban Institute Press; 2002
Congressional Budget Office. How Many People Lack Health Insurance and For How Long. Washington, DC: US Government Printing Office; 2003
Dubay L, Kenney G, Haley J. Children's Participation in Medicaid and SCHIP: Early in the SCHIP Era. Washington, DC: Urban Institute Press; 2002
LoSasso AT, Buchmueller TC. The effect of the state children's health insurance program on health insurance coverage. J Health Econ. 2004;23 :1059 –1082
Lave JR, Keane CR, Lin CJ, Ricci EM, Amersbach G, LaVallee CP. Impact of a children's health insurance program on newly enrolled children. JAMA. 1998;279 :1820 –1825
Szilagyi PG, Zwanziger J, Rodewald LE, et al. Evaluation of a state health insurance program for low-income children: implications for state child health insurance programs. Pediatrics. 2000;105 :363 –371
Rodewald L, Szilagyi PG, Holl J, Shone LR, Zwanziger J, Raubertas RF. Health insurance for low-income working families: effect on the provision of immunizations to preschool-age children. Arch Pediatr Adolesc Med. 1997;151 :798 –803
Centers for Disease Control and Prevention. National, state and urban area vaccination coverage levels among children aged 19–35 months—United States December 1995. MMWR Morb Mortal Wkly Rep. 1997;46 :176 –182
Centers for Disease Control and Prevention. National, state and urban area vaccination coverage levels among children aged 19–35 months—United States 1997. MMWR Morb Mortal Wkly Rep. 1998;47 :547 –554
Centers for Disease Control and Prevention. National, state and urban area vaccination coverage levels among children aged 19–35 months—United States 1998. MMWR Morb Mortal Wkly Rep. 2000;49 :1 –26
Centers for Disease Control and Prevention. National, state and urban area vaccination coverage levels among children aged 19–35 months—United States 2000. MMWR Morb Mortal Wkly Rep. 2001;50 :637 –641
Centers for Disease Control and Prevention. National, state and urban area vaccination coverage levels among children aged 19–35 months—United States 2001. MMWR Morb Mortal Wkly Rep. 2001;51 :664 –666
Centers for Disease Control and Prevention. National, state and urban area vaccination coverage levels among children aged 19–35 months—United States 2002. MMWR Morb Mortal Wkly Rep. 2001;52 :728 –732
Smith PJ, Battaglia MP, Huggins VJ, et al. Overview of the sampling design and statistical methods used in the National Immunization Survey. Am J Prev Med. 2001;20 (suppl):17 –24
Fierman AH, Rosen CM, Legano LA, Lim SW, Mendelsohn AL, Dreyer BP. Immunization status as determined by patients' hand-held cards vs. medical records. Arch Pediatr Adolesc Med. 1996;150 :863
Joyce TJ, Racine AD. CHIP Shots: Association Between the State Children's Health Insurance Programs and Immunization Coverage and Delivery. New York, NY: National Bureau of Economic Research; 2003
Institute of Medicine. Calling the Shots: Immunization Finance Policies and Practices. Committee on Immunization Finance Policies and Practices, Division of Health Care Services and Division of Health Promotion and Disease Prevention. Washington, DC: National Academy Press; 2000
American Academy of Pediatrics, Committee on Infectious Diseases. Recommendations for the use of live attenuated varicella vaccine. Pediatrics. 1995;95 :791 –796
StataCorp. Stata Statistical Software: Release 8.0. College Station, TX: Stata Corp; 2003
Newhouse JP, Insurance Experiment Group. Free for All Lessons From the RAND Health Insurance Experiment. Cambridge, MA: Harvard University Press; 1993
Newacheck P, Stoddard JJ, Hughes DC, Pearl M. Health insurance and access to primary care for children. N Engl J Med. 1998;338 :513 –519
Brenner RA, Simons-Morton BG, Bhaskar B, et al. Prevalence and predictors of immunization among inner-city infants: a birth cohort study. Pediatrics. 2001;108 :661 –670
Shaheen MA, Frerichs RR, Alexopoulos N, Rainey JJ. Immunization coverage among predominantly Hispanic children, aged 2–3 years, in central Los Angeles. Ann Epidemiol. 2000;10 :160 –168
Kaestner R, Kaushal N. Welfare reform and health insurance coverage of low-income families. J Health Econ. 2003;22 :959 –981(Ted Joyce, PhD, and Andre)
National Bureau of Economic Research, New York, New York
Department of Pediatrics, Albert Einstein College of Medicine, New York, New York
ABSTRACT
Objective. The Balanced Budget Act of 1997 established the State Children's Health Insurance Program (SCHIP), which makes health insurance available to children in near-poor families who are ineligible for Medicaid. SCHIP mandates that all state plans cover the cost and administration of childhood vaccines. Whether SCHIP has narrowed immunization coverage rates between near-poor and nonpoor children is unknown. The objective of this study was to use data from the National Immunization Survey from 1995 to 2002 to analyze changes in immunization coverage rates among poor, near-poor, and nonpoor children before and after implementation of SCHIP.
Methods. A prepost analysis was made of changes in immunization rates among poor, near-poor, and nonpoor children before and after implementation of SCHIP in all 50 states and 28 Immunization Action Plan areas from 1995 to 2002. All children in the National Immunization Survey for whom information on vaccinations was available from the respondents' shot cards and/or from the children's immunization providers (N = 264214) were studied. Up-to-date status for the 4:3:1 (4 doses of diphtheria-tetanus toxoids-pertussis vaccine/3 doses of poliovirus vaccine/1 dose of measles-mumps-rubella vaccine) and the 4:3:1:3:3 (4 doses of diphtheria-tetanus toxoids-pertussis vaccine/3 doses of poliovirus vaccine/1 dose of measles-mumps-rubella vaccine/3 doses of Haemophilus influenzae type B vaccine/3 doses of hepatitis B vaccine) series as well at the hepatitis B and varicella vaccines was measured.
Results. The probability that a poor or near-poor child was up to date for the 4:3:1:3:3 vaccine series increased 11 percentage points after implementation of SCHIP. However, we observed a similar increase for nonpoor children. The proportion of poor and near-poor children who were up to date for the varicella vaccine increased between 7 and 8 percentage points more than among nonpoor children after implementation of SCHIP. Relative increases among poor and near-poor children were greater in the 28 Immunization Action Plan areas, in states with high rates of uninsured children, and among Hispanics.
Conclusion. SCHIP seems not to be associated with changes in the up-to-date status of poor and near-poor children for the 4:3:1 and the 4:3:1:3:3 vaccine series. Vaccine coverage rates increased broadly among all income groups between 1995 and 2002.
Key Words: immunizations up-to-date SCHIP
Abbreviations: SCHIP, State Children's Health Insurance Program FPL, federal poverty level NIS, National Immunization Survey 4:3:1, 4 doses of diphtheria-tetanus toxoids-pertussis vaccine/3 doses of poliovirus vaccine/1 dose of measles-mumps-rubella vaccine 4:3:1:3:3, 4 doses of diphtheria-tetanus toxoids-pertussis vaccine/3 doses of poliovirus vaccine/1 dose of measles-mumps-rubella vaccine/3 doses of Haemophilus influenzae type B vaccine/3 doses of hepatitis B vaccine VFC, Vaccine for Children
The Balanced Budget Act of 1997 established the State Children's Health Insurance Program (SCHIP), which makes health insurance available to children in near-poor families who are ineligible for Medicaid. The motivation for SCHIP came from the recognized high rates of uninsured children among the working poor. Approximately 23% of children from families with incomes between 100% and 200% of the federal poverty level (FPL) and 27% of children in poverty lacked health insurance in 1996.1 A recent analysis indicated that the proportion of children without insurance for some part of the year may be as high as 40%.2 The $40 billion appropriated over 10 years for SCHIP represents the largest extension of publicly provided health insurance since the creation of the Medicaid program.
Efforts to evaluate SCHIP have largely focused on take-up rates.3,4 The few studies of outcomes pertain to the precursors of SCHIP in selected states and have focused primarily on changes in utilization.5,6 An evaluation of New York State's Child Health Plus program in 1991, a model for the current SCHIP, found that the program was associated with increased immunization rates, although it is unclear whether the relationship was causal.7 A principal methodologic consideration to be addressed by any analysis of the effect of SCHIP is the bias inherent in comparing the experience of those who seek out and accept SCHIP coverage with that of those who do not. Unobservable characteristics differ between SCHIP enrollees and eligible individuals who decline to enroll. For avoiding confounding by these characteristics, evaluations of public programs such as SCHIP on the basis of the intention to treat compare the outcomes of entire eligible populations before and after the introduction of the new program.
In view of the potential for SCHIP programs to influence immunization rates, we used the recently released National Immunization Survey (NIS) to associate between state implementation of SCHIP and changes in immunization coverage rates. We used poverty status to characterize SCHIP eligibility, and we interpreted the results as estimates of the intention to treat. This study addresses 2 questions. First, how have vaccine coverage rates changed among poor, near-poor, and nonpoor children before and after implementation of SCHIP Second, is there any evidence that SCHIP increased the uptake of new vaccines
METHODS
The NIS
We used data from the NIS for the years 1995 to 2002. The NIS is a national probability sample of children aged 19 to 35 months that annually surveys >34000 households from all 50 states and 28 selected metropolitan areas. Details of the NIS along with the public use data files are available from the NIS web site (www.cdc.gov/nis/datafiles.htm).
Information on each surveyed child in the NIS is gathered from a respondent in each household as well as from the child's health care provider. The proportion of households with complete data from providers ranges from 62% to 70% between 1996 and 2002 and 51% in 1995.8–13 The provider data are considered the most reliable. Those with complete provider data, however, are more likely to be white, be better educated, and have greater incomes than households without provider data. Aware of the potential problems associated with selective reporting, administrators of the NIS use propensity scores within each stratum to adjust sampling weights for households with nonprovider data.14 To reduce the loss of households, we also used the surveys of respondents who use a shot card to recall receipt of vaccines. There is evidence that shot cards provide reliable information regarding vaccines but that parental recall alone does not.15,16 We assumed that there are no false positives on either the shot card or the provider survey. Thus, if a child is reported to be up to date for a vaccine on the basis of either the shot card or the provider survey, then we assume that the child is up to date for that vaccine. The addition of shot card data increases the proportion of households with complete vaccine information to between 72.9% and 81.8%.
SCHIP
The purpose of SCHIP is to extend health insurance to children in near-poor families who do not qualify for Medicaid but who are uninsured. To qualify for SCHIP federal funding, states had the option of extending Medicaid, creating a new insurance program, or combining both approaches. Eighteen states and the District of Columbia chose to extend Medicaid, 15 established separate programs, and 17 had combinations. Evidence suggests that the distinction between Medicaid extension and separate programs is not important with respect to its impact on the lack of insurance among children.4 Consequently, we did not analyze differences by program type.
We specified SCHIP in 2 ways. First, we used dummy variables to capture the timing of program implementation: 3 states began enrollment of children aged 1 to 5 in 1997, 29 in 1998, 13 in 1999, 4 in 2000, and 1 in 2001. We also used income eligibility thresholds to measure variation in extensiveness of the expansion. SCHIP allows states to raise the income eligibility thresholds above the Medicaid thresholds up to any level.1 As of 2002, 14 states had income eligibility thresholds for children 1 to 5 years of age at above 200% of the FPL, 26 states at 200% of FPL, and 11 states at <200% of FPL. For years before SCHIP, we used the Medicaid eligibility threshold for children 0 to 5 years of age. (Timing and eligibility thresholds of SCHIP for infants and children up to 5 years of age were compiled by Lisa Dubay at the Urban Institute. Note that Wisconsin did not change income eligibility threshold for children <6 under SCHIP.) In all states the relevant thresholds for infants <1 year of age are the Medicaid thresholds after which point the SCHIP thresholds apply.
We used differences in the age of the child and income eligibility thresholds between infancy and childhood to develop an algorithm to estimate the average SCHIP income eligibility threshold applicable to each surveyed child. The details of this algorithm have been published elsewhere.17
The NIS uses an algorithm that is based on reported family income and family size to create an indicator of poverty status defined as below 100% of the federal poverty level. In 1999, the NIS added a new variable that measures the ratio of income to poverty. To create a consistent series of income to poverty for years before 1999, we used the income categories and family size variable to create an income to poverty ratio for 1995–1998. We assumed that families with >$50000 per year earned $75000. As a check on our measure, we used our algorithm to compute poverty status (<100% FPL) and compared our poverty indicator with the one computed by NIS. The associated statistic ranged from .93 to .99 between 1995 and 1998. We then defined 3 categories of family income and a separate category for unknown income: poor, or those with <100% of the FPL; near-poor, or those with income 100% and <250% of the FPL; and nonpoor, or those with income 250% of the FPL. We used these as broad measures of eligibility to define exposure to SCHIP.
Because the percentage of families with unknown income varies from 12% to 24% over the study period, we show all results for poor, near-poor, and nonpoor families and families of unknown poverty. Even for those with recorded income, measurement error as a result of self-reporting remained a potential problem. As an alternative measure of socioeconomic status, we used mother's marital status and schooling. The most disadvantaged group includes children with mothers who are 20 years of age or older, who are not married, and who have 12 years of schooling or less; less disadvantaged are children whose mothers are married, who are 20 years of age and older, and who also have no more than a high school education; and the most advantaged includes children of married women with a college degree.
The effect of SCHIP on immunizations should vary by the proportion of uninsured and underinsured children among the poor and near-poor as well as the pre-SCHIP level of immunization coverage rates. States with larger numbers of uninsured children or with lower levels of immunization rates before SCHIP implementation may be expected to experience greater shifts once SCHIP programs are initiated. To exploit these potential sources of variation, we present a series of analyses by states with high rates of uninsured children, by states with large Hispanic populations, and by urban areas specifically designated as immunization action plan areas by the Centers for Disease Control and Prevention. Although immunization action plan areas were to have received additional resources to coordinate local efforts, funds were never forthcoming.18
Outcomes
We used dichotomous indicators of whether a child is up to date for either a particular vaccine or a series of vaccines. We analyzed changes in 2 series: the 4:3:1 (4 doses of diphtheria-tetanus toxoids-pertussis vaccine/3 doses of poliovirus vaccine/1 dose of measles-mumps-rubella vaccine) and the 4:3:1:3:3 (4 doses of diphtheria-tetanus toxoids-pertussis vaccine/3 doses of poliovirus vaccine/1 dose of measles-mumps-rubella vaccine/3 doses of Haemophilus influenzae type B vaccine/3 doses of hepatitis B vaccine) series. A child is up to date for the 4:3:1 series when he or she has had 4 diphtheria, tetanus, and acellular pertussis vaccines, 3 polio vaccines, and 1 measles-containing vaccine by 19 months. Up to date for the 4:3:1:3:3 series requires an additional 3 Haemophilus influenzae and 3 hepatitis B vaccinations (www.cdc.gov/nip/publications/ACIP-list.htm). The 4:3:1:3:3 is the most comprehensive series that is available throughout the study period.
We also analyzed the association between SCHIP and the uptake of new vaccines. We focused on the varicella vaccine because it was licensed by the Food and Drug Administration in March 1995 and became part of the NIS in 1997.19 Thus, the introduction of the varicella vaccine coincides with the implementation of SCHIP. Data on the pneumococcal vaccine were not available until 2001.
Statistical Analysis
We began the analysis by contrasting changes in up-to-date status of the vaccine series for poor, near-poor, and nonpoor children between 1995–1996 and 2001–2002. We present 2 sets of changes. The first shows differences in immunization rates by poverty status before and after implementation of SCHIP. We then estimate what we term relative differences, in which we subtract the change in immunization rates among nonpoor children between 1995–1996 and 2001–2002 from changes among poor and near-poor children. The objective is to eliminate trends in vaccination rates unrelated to SCHIP but related to general improvements in the immunization infrastructure, such as outreach campaigns, improvements in parent reminder systems, registries, and extended hours in clinics and doctors' offices, that occurred to various degrees over the study period.
To refine further the adjustments for confounding, we used regression analysis to estimate linear probability models. The dependent variable is a dichotomous indicator of whether the child is up to date for a vaccine series or the varicella vaccine. On the right-hand side, we include controls for the child's age; the number of children in the household; and whether the child's family is poor, near-poor, or nonpoor. We included characteristics of the mother, such as age, race, marital status, and schooling. To adjust for other initiatives, we included the number of immunization providers in a state that participates in the Vaccine for Children (VFC) program per number of children 0 to 18 years of age. In addition, we included dichotomous indicators for all 50 states to capture time-invariant differences between states in immunization rates that may be related to general immunization policies. We included the annual state unemployment rate to control for business cycle effects on public assistance programs as well as indicators for each year from 1996 to 2002, with 1995 as the reference year. The year indicators capture national trends in immunization rates. Finally, we included indicator variables for the year a state implemented its SCHIP program as well a 1-year lag in the SCHIP indicator to allow for startup delays. We interacted the SCHIP indicators with our indicators of household poverty. The coefficients on the interaction terms show the percentage point change in immunization rates before and after implementation of SCHIP among poor and near-poor children relative to nonpoor children. We refer to these as the adjusted relative changes.
We also estimated an additional set of regressions in which we used the income eligibility threshold for SCHIP instead of the implementation indicators to characterize SCHIP. The eligibility thresholds are an attempt to measure changes in the relative generosity of the SCHIP program. In New Jersey, for example, the income eligibility threshold under SCHIP was raised to 350% of the FPL, up from 133% for children 1 to 5 years of age in 1997. West Virginia, by contrast, increased eligibility thresholds from 133% to 150% of the FPL for children 1 to 5 years of age.
We accounted for the survey design of the NIS in all bivariate comparisons with the survey procedures in STATA 8.0.20 We used ordinary least squares to estimate the regression models.
None of the outcomes is in the extreme tails of the distribution; thus, marginal effects from the linear, logistic, and probit specifications are very similar. The linear probability model, however, allows for easy interpretation and facilitates comparisons between adjusted and unadjusted changes in the probability that a child is immunized, especially in a specification with interaction terms. All regressions were weighted by the sampling probabilities, and we used robust regression procedures to account for the fact that we have only state variation in our measures of program availability.
RESULTS
Characteristics of the Sample
In Table 2, we display the proportion of children who are up to date for the 4:3:1 series, the 4:3:1:3:3 series, and the hepatitis B vaccine in 1995–1996 and 2001–2002. There is an obvious poverty gradient in coverage rates. In 1995–1996, 71.5% of poor children were up to date for the 4:3:1 series as compared with 82.9% of nonpoor children. Children for whom poverty status is unknown have immunization rates similar to poor and near-poor children.
Vaccination coverage rates for the 4:3:1 series were largely unchanged over the study period, which suggests that SCHIP had little impact on this basic series (Table 2). Changes in up-to-date status for the 4:3:1:3:3 series are more substantial (Table 2). Vaccination coverage rates increased 10 to 14 percentage points among all groups between 1995–1996 and 2001–2002. The uniform increase in the 4:3:1:3:3 series largely reflects increases in coverage for the hepatitis B vaccine. As shown in Table 2, the proportion of children who were up to date for hepatitis B increased 11 to 14 percentage points between 1995–1996 and 2001–2002 irrespective of poverty status.
A more formal analysis of the relative changes in vaccine coverage rates is presented in Table 3. In column 1, we show the percentage point change in coverage rates among poor and near-poor children relative to nonpoor children. Except for hepatitis B, there is no evidence that coverage rates increased more among children who were potentially affected by SCHIP relative to children who were unaffected. The estimates in column 3 of Table 3 are from the multiple regression analysis. We refer to these as the adjusted relative changes. The figures are from the coefficients on the interactions terms between SCHIP and poverty status. They show the percentage point change in vaccine coverage rates among poor and near-poor relative to nonpoor children before and after implementation of SCHIP. None of the adjusted changes is of any meaningful magnitude or statistical significance. Indeed, the modest relative increase in hepatitis B vaccine coverage among poor and near-poor children (column 1) is eliminated after adjustment, except for those of unknown poverty.
Changes in Coverage Rates of Varicella
We display the relative changes in coverage rates for varicella in Table 5. In column 1, we show the change in varicella among poor and near-poor children after subtracting the change among the nonpoor. In column 3, we show the relative change as obtained from the regression analysis. The results are robust. In every case but 1, the adjusted changes are less than the unadjusted changes, but even the adjusted changes are relatively large and statistically significant. For instance, coverage for varicella increased by 7.1 percentage points more among near-poor relative to nonpoor children, nationally. Among the subgroup of states and urban areas with children at greater risk for uninsurance, the adjusted relative change in immunization rates for varicella among near-poor children are 10.1 to 14.0 percentage points.
A limitation of this exercise, however, is that we cannot distinguish between the natural diffusion of a new vaccine from the effects of SCHIP. We are concerned that changes among nonpoor children may not be a good counterfactual for children who are potentially affected by SCHIP because coverage rates for varicella among the nonpoor were much higher than among poor and near-poor children in 1997. As an alternative strategy, therefore, we contrasted changes across states among poor and near-poor children only. Specifically, we divided states into 2 groups: those that changed eligibility for children 1 to 5 years of age through SCHIP in 1997 or 1998 and those that changed eligibility in 1999 or later. We then computed annual differences in varicella coverage rates from 1997 to 2002 separately for the 2 groups of states. If SCHIP is associated with changes in varicella coverage rates, then annual increases should be greater between 1997 and 1999 in states that expanded eligibility for young children before 1999 than in states that expanded in 1999 or after. We show the results in Table 6. Annual increases in varicella coverage rates are essentially identical in the 2 groups of states, which is evidence against an association between SCHIP and uptake of the varicella vaccine.
DISCUSSION
The SCHIP represents an unprecedented extension of publicly financed health insurance for near-poor children, a process that began with the Medicaid eligibility expansions in the late 1980s. Numerous studies have linked increases in the availability of health insurance to increases in health care utilization among children.5,21,22 However, measures of utilization are often too general to infer much about the quality of care or its effect on health status. In this study, we associated changes in SCHIP with changes in immunization coverage rates. Vaccines arguably were the greatest public health achievement of the 20th century and an effective measure of both the quality of pediatric care and the improvement in health associated with vaccine-preventable illnesses.
We found little evidence that SCHIP has had a major impact on narrowing the immunization coverage rate gaps among poor and near-poor children relative to their nonpoor counterparts. The 1 exception is the varicella vaccine, for which differences in coverage rates among poor, near-poor, and nonpoor children converged rapidly between 1997 and 2002. Moreover, convergence was faster among poor and near-poor children in states with higher rates of uninsurance among children. We caution, however, against drawing causal inferences from these data, because we observed equivalent convergence in varicella vaccine coverage rates in states in the period before the implementation of SCHIP.
Our analysis, made possible by the recent release of the NIS, represents the first attempt to associate the extension of publicly provided health insurance for children with immunization rates across all 50 states and selected urban areas. Despite its geographic breadth, however, the NIS has less information about household characteristics than is available in smaller, more localized studies. We could not determine a child's precise eligibility for SCHIP, for instance, and we did not know which children already had health insurance that covered vaccinations. Nevertheless, stratifying the data by poor, near-poor, and nonpoor families enabled us to determine whether the timing of SCHIP implementation at the state level was associated with a narrowing of differences in immunization rates among children who likely were exposed relative to those who were unexposed to the change in publicly provided health insurance. Moreover, we were able to stratify the analyses by subsets of states and cities in which the rate of uninsurance among children was greater. Given the large sample size, we were able to detect differences in immunization rates between near-poor and nonpoor children of between 2 and 5 percentage points. That we did not find statistically significant differences in the relative change in immunization rates between near-poor and nonpoor children does not mean that SCHIP has had no effect. It is conceivable, for example, that SCHIP prevented the development of what would otherwise have been a widening of income disparity in immunization rates. Our findings do suggest, however, that any increase that may have been achieved with the advent of SCHIP has not been large enough to lessen differences in vaccine coverage rates by poverty status.
We emphasize that our study is an estimate of the intention to treat. We compared changes in immunization status among potentially eligible children before and after states implemented their SCHIP programs with changes in immunizations among nonpoor children who likely were ineligible for SCHIP. The advantage of an estimate that is based on the intention to treat is that program effects are largely free of selection bias at the individual level.
An alternative estimate of SCHIP's effect on immunization rates could be obtained from a comparison of changes in immunizations rates among children who were newly enrolled in SCHIP with changes among eligible children who were not enrolled or with changes among ineligible children with insurance coverage over the entire period. We were unable to undertake such an analysis because the NIS lacks information on SCHIP participation. However, unless enrollment in SCHIP is randomly assigned, estimates of program effectiveness on the basis of a comparison before and after enrollment in SCHIP may be confounded with unobservable characteristics of the children or families. Mothers who enroll their children in SCHIP, for example, may be more knowledgeable as to the importance of vaccines and may be more likely to have their children vaccinated in the absence of SCHIP than eligible nonparticipants. A finding that children who enroll in SCHIP are more likely to be up to date for immunizations therefore may reflect more the characteristics of the mother than the effect of health insurance. By contrast, an estimate that is based on intention-to-treat analysis, although not informative as to the effect of insurance coverage on immunization rates per se, will provide an unbiased estimate of the effect of making insurance available to a targeted population.
Our study has certain limitations. First, we could proxy eligibility for SCHIP only within relatively broad income categories. In addition, we lacked information on family income for almost 20% of our sample. However, results for children from families with unknown income were similar to those of poor and near-poor children. As an additional check, we used mother's schooling and marital status to characterize households that likely were eligible and ineligible for SCHIP. We found that the level of up-to-date immunization status and its change before and after SCHIP among children whose mothers were unmarried and who had no more than a high school education were similar to those for poor and near-poor children (results available on request), reassuring us that the missing income data for some families did not bias our estimates significantly.
We used the experience of children from nonpoor families as a counterfactual for the experience of poor and near-poor children. However, the nonpoor defined as families with income 250% of the FPL may include too many well-off families to serve as an appropriate comparison group. As an alternative, we redefined our groups so that near-poor families had incomes between 100% and 200% of the FPL and nonpoor families had incomes between 200% and 300% of FPL. The findings did not change.
Our results at the national level are similar to those from evaluations of New York State's Child Health Plus, the state's precursor to SCHIP.6,7 In 1 of the 2 evaluations, researchers found gains in immunization rates among previously uninsured children, but they observed statistically similar gains among the insured.7 Other recent studies have not found health insurance to be an important predictor of up-to-date status.23,24 Although samples in both studies were drawn from a single city, maternal employment and Women, Infants, and Children enrollment were more robust correlates of immunization status than was health insurance.
We adjusted for state variation in the unemployment rate as a control for business cycle effects. SCHIP was rolled out during a period of great economic growth. Employment growth increases the demand for preschool-aged child care. This can affect immunization rates, because many states require that children be up to date for numerous vaccines before enrollment in child care.23 Our study period also coincided with the initiation of welfare reform. There is some evidence that the Temporary Assistance for Needy Families Program, the replacement for Aid to Families With Dependent Children, resulted in a loss of Medicaid coverage for women and children whose cash assistance ended, but the effects are modest.25 However, welfare reform should have affected primarily poor families, because few near-poor families were eligible for cash assistance. Our finding that results for the poor mirrored those of the near poor suggests that welfare reform was not an important confounding factor.
The implementation of SCHIP may have affected the site at which immunizations are obtained without any major impact on vaccination rates. We tested for such changes in a previous analysis and found no association between SCHIP availability and the site of care.17 It is also possible that the VFC program may have limited the potential impact of SCHIP on immunizations by raising the pre-SCHIP level of immunization rates among poor and near-poor children. However, this could not be true for the varicella vaccine, which did not become widely available until 1996. Second, we controlled for variation in the number of VFC providers per child in each state as well as the type of VFC state. Third, there has been little growth in the number of VFC providers since 1997. Fourth, the VFC was a supply-side intervention that did not cover the administration of vaccines or the cost of the office visit for patients. SCHIP, by contrast, was a demand-side policy that covered the costs of the vaccine and the visit. In sum, we do not believe that VFC obscured the impact of SCHIP.
Our initial results suggested that SCHIP was associated with a narrowing of the poverty gradient with respect to the varicella vaccine. Relative changes in varicella coverage, both unadjusted and adjusted, were substantially greater among the poor and near poor compared with the nonpoor between 1997 and 2002. Moreover, relative changes were greater among Hispanics and among children in states with the highest rate of uninsurance. However, we questioned whether these associations were causal. We observed large increases in coverage rates for varicella among poor and near-poor children in the years before SCHIP among states that implemented SCHIP relatively late.
In sum, the good news is that in 2002 immunization rates for individual vaccines exceeded 75% before age 3, even among the poor (authors' tabulations of the 2002 NIS based on information from providers and/or respondents with shot cards). Income gradients still exist, however, and are larger for series of vaccine such as the 4:3:1:3:3. Health insurance may be a necessary condition for continued growth in coverage rates, especially as the number of new and more expensive vaccines increases, but clearly other aspects of the immunization delivery system will need strengthening if we are to achieve the Healthy People objective of 90% coverage rates for all pediatric vaccines by 2010 (www.healthypeople.gov/document/html/objectives/14-22.htm).
ACKNOWLEDGMENTS
This research was made possible by a grant from the Robert Wood Johnson Foundation Changes in Health Care Financing Organization Initiative to the Baruch College Fund.
We thank Lisa Dubay of the Urban Institute for detailed information on SCHIP.
FOOTNOTES
Accepted Nov 18, 2004.
Ted Joyce, PhD, National Bureau of Economic Research, 365 Fifth Ave, 5th Floor, New York, NY 10016. ted_joyce@baruch.cuny.edu
No conflict of interest declared.
REFERENCES
Dubay L, Hill I, Kenney G. Five Things Everyone Should Know About SCHIP. Washington, DC: Urban Institute Press; 2002
Congressional Budget Office. How Many People Lack Health Insurance and For How Long. Washington, DC: US Government Printing Office; 2003
Dubay L, Kenney G, Haley J. Children's Participation in Medicaid and SCHIP: Early in the SCHIP Era. Washington, DC: Urban Institute Press; 2002
LoSasso AT, Buchmueller TC. The effect of the state children's health insurance program on health insurance coverage. J Health Econ. 2004;23 :1059 –1082
Lave JR, Keane CR, Lin CJ, Ricci EM, Amersbach G, LaVallee CP. Impact of a children's health insurance program on newly enrolled children. JAMA. 1998;279 :1820 –1825
Szilagyi PG, Zwanziger J, Rodewald LE, et al. Evaluation of a state health insurance program for low-income children: implications for state child health insurance programs. Pediatrics. 2000;105 :363 –371
Rodewald L, Szilagyi PG, Holl J, Shone LR, Zwanziger J, Raubertas RF. Health insurance for low-income working families: effect on the provision of immunizations to preschool-age children. Arch Pediatr Adolesc Med. 1997;151 :798 –803
Centers for Disease Control and Prevention. National, state and urban area vaccination coverage levels among children aged 19–35 months—United States December 1995. MMWR Morb Mortal Wkly Rep. 1997;46 :176 –182
Centers for Disease Control and Prevention. National, state and urban area vaccination coverage levels among children aged 19–35 months—United States 1997. MMWR Morb Mortal Wkly Rep. 1998;47 :547 –554
Centers for Disease Control and Prevention. National, state and urban area vaccination coverage levels among children aged 19–35 months—United States 1998. MMWR Morb Mortal Wkly Rep. 2000;49 :1 –26
Centers for Disease Control and Prevention. National, state and urban area vaccination coverage levels among children aged 19–35 months—United States 2000. MMWR Morb Mortal Wkly Rep. 2001;50 :637 –641
Centers for Disease Control and Prevention. National, state and urban area vaccination coverage levels among children aged 19–35 months—United States 2001. MMWR Morb Mortal Wkly Rep. 2001;51 :664 –666
Centers for Disease Control and Prevention. National, state and urban area vaccination coverage levels among children aged 19–35 months—United States 2002. MMWR Morb Mortal Wkly Rep. 2001;52 :728 –732
Smith PJ, Battaglia MP, Huggins VJ, et al. Overview of the sampling design and statistical methods used in the National Immunization Survey. Am J Prev Med. 2001;20 (suppl):17 –24
Fierman AH, Rosen CM, Legano LA, Lim SW, Mendelsohn AL, Dreyer BP. Immunization status as determined by patients' hand-held cards vs. medical records. Arch Pediatr Adolesc Med. 1996;150 :863
Joyce TJ, Racine AD. CHIP Shots: Association Between the State Children's Health Insurance Programs and Immunization Coverage and Delivery. New York, NY: National Bureau of Economic Research; 2003
Institute of Medicine. Calling the Shots: Immunization Finance Policies and Practices. Committee on Immunization Finance Policies and Practices, Division of Health Care Services and Division of Health Promotion and Disease Prevention. Washington, DC: National Academy Press; 2000
American Academy of Pediatrics, Committee on Infectious Diseases. Recommendations for the use of live attenuated varicella vaccine. Pediatrics. 1995;95 :791 –796
StataCorp. Stata Statistical Software: Release 8.0. College Station, TX: Stata Corp; 2003
Newhouse JP, Insurance Experiment Group. Free for All Lessons From the RAND Health Insurance Experiment. Cambridge, MA: Harvard University Press; 1993
Newacheck P, Stoddard JJ, Hughes DC, Pearl M. Health insurance and access to primary care for children. N Engl J Med. 1998;338 :513 –519
Brenner RA, Simons-Morton BG, Bhaskar B, et al. Prevalence and predictors of immunization among inner-city infants: a birth cohort study. Pediatrics. 2001;108 :661 –670
Shaheen MA, Frerichs RR, Alexopoulos N, Rainey JJ. Immunization coverage among predominantly Hispanic children, aged 2–3 years, in central Los Angeles. Ann Epidemiol. 2000;10 :160 –168
Kaestner R, Kaushal N. Welfare reform and health insurance coverage of low-income families. J Health Econ. 2003;22 :959 –981(Ted Joyce, PhD, and Andre)