Methods to assess the costs and health effects of interventions for im
http://www.100md.com
《英国医生杂志》
1 Health Systems Financing, Evidence and Information for Policy, World Health Organization, Geneva, Switzerland, 2 Costs, Effectiveness, Expenditure and Priority Setting, World Health Organization, 3 Health Systems Financing, World Health Organization, 4 School of Population Health, University of Queensland, Australia
Correspondence to: D B Evans evansd@who.int
Assessment of the cost effectiveness of interventions designed to achieve the millennium development goals for health is complex. The methods must be capable of showing the efficiency with which current and possible new resources are used, and incorporating interactions between concurrent interventions and the effect of expanding coverage on unit costs.1 They should also allow valid comparisons across a wide range of interventions. Here we describe how the standardised cost effectiveness methods used in the World Health Organization's Choosing Interventions that are Cost Effective (CHOICE) project have tackled these issues.
Level of analysis
The analysis was performed for 14 regions classified by WHO according to their epidemiological grouping (table A on bmj.com). The regional results (except if not relevant to the disease area, for example, malaria) are available at www.who.int/choice, but the papers in this series give details for just two regions: Afr-E, which includes countries in sub-Saharan Africa with high child mortality and very high adult mortality, and Sear-D, which comprises countries in South East Asia with high child and adult mortality.2-6
Definition and selection of interventions
The term intervention is defined to include any preventive, promotive, curative, or rehabilitative action that improves health. Interventions are analysed individually and then in combinations or packages that could be undertaken together (box 1), taking into account interactions in costs or effectiveness, or both.
Interventions were chosen for analysis either because they are commonly used or because disease control experts have advocated their introduction. In each case, some evidence was needed that the intervention could be effective. The list is not exhaustive, and excluding an intervention does not imply it is cost ineffective.
All interventions and combinations are assessed assuming they are implemented for 10 years starting in 2000, the year the Millennium Declaration was signed. Good policy making would then require a reassessment of strategies. Costs, therefore, are only incurred over 10 years, but all benefits accruing because of actions taken between 2000 and 2010 are included.
Intervention costs
Costs are measured from the perspective of society as a whole, to understand how best to use resources regardless of who pays for them, or indeed, whether they are paid for at all. For example, village volunteers working for maternal health must be included because they could be working for some other health programme if they did not work for maternal health. All resources used for each intervention or combination are included and valued.
Data on the costs incurred by people to access services (such as travel costs) are rarely available, and we, like most other studies, have excluded them.7 8 We also excluded domestic taxes; from the perspective of the society, they simply transfer financial resources from one person to another and do not use up a physical resource such as capital or labour. The effect of interventions on the time and potential earnings of patients and unpaid carers—that is, work time lost—is a vexing question in cost effectiveness analysis but, as here, it is often excluded on ethical grounds. Inclusion would give priority to extending the life of people who earn more (box A on bmj.com).9 10
Costs are evaluated assuming 80% use of capacity (technical efficiency) for all interventions; this is relatively high but achievable. Use of a constant figure ensures that differences in cost effectiveness ratios are due to fundamental differences in the technologies or strategies adopted and not simply to an intervention being implemented poorly in a dysfunctional health system. Accordingly, the results provide practical information about the appropriate mix of interventions for various levels of resource, assuming interventions are implemented relatively efficiently.
Measles immunisation is one of the key interventions for child health
Credit: P VIROT/WHO
Box 1: Analysis of individual and combined interventions
Interventions are analysed individually and then in combinations that could be undertaken together. This requires a decision about whether interventions are independent (they can be done at the same time in a population, with or without interactions) or mutually exclusive (if one is chosen, another cannot be). Independent interventions can be added to existing interventions, while mutually exclusive interventions must replace an existing intervention.
Consider three individual antenatal interventions: tetanus immunisation, screening for pre-eclampsia, and screening and treatment for asymptomatic bacteriuria. They are independent, so all possible combinations of the interventions would be:
Tetanus immunisation + screening for pre-eclampsia
Tetanus immunisation+screening and treatment for asymptomatic bacteriuria
Screening for pre-eclampsia+screening and treatment for asymptomatic bacteriuria
Tetanus immunisation+screening for pre-eclampsia and asymptomatic bacteriuria.
All the scenarios are assessed for different coverage levels (50%, 80%, 95%), introducing the idea of mutual exclusivity. Coverage at 95% must replace coverage at 80%. Each of the combinations of interventions is analysed taking into account all possible interactions in costs or effectiveness. Combinations of interventions must be plausible. For example, emergency obstetric care interventions are not included unless skilled birth attendance is implemented at the same time. The number of possible combinations increases rapidly with the addition of each new intervention.
Classification and measurement of costs
Costs are divided into patient and programme levels. Patient level costs include face to face delivery by a health provider—for example, medicines, outpatient visits, inpatient stays, and individual health education messages. Programme level costs include all resources required to establish and maintain an intervention: administration, publicity, training, and delivery of supplies.
We used a standardised ingredients approach to measure costs, requiring information on the quantities of physical inputs needed and their unit cost. Total costs are quantities of inputs multiplied by their unit costs. To obtain programme level costs, we commissioned costing experts to collect data in 17 countries on the physical inputs (human resources, office space, vehicles, electricity, other services, and consumables) required to introduce and run an intervention using a standard template.11-13 This was supplemented by information from programme managers in other countries known by WHO staff.
For patient level costs, quantities were taken from various sources. When published studies of effectiveness were available we used these to identify the resources necessary to ensure the observed level of effectiveness. In other cases, we estimated the resources required from the activities outlined in WHO guidelines for treatment.14-17 Programme experts checked the face validity of all estimated quantities. We could not always determine the quantities of primary inputs (human resources, consumables), so we estimated quantities and prices at an intermediate level for several inputs: inpatient days at different hospital levels, outpatient visits, and health centre visits.
Unit costs for each input were derived from an extensive search of published and unpublished literature and databases along with consultation with costing experts. The full costing database is available at www.who.int/choice. For goods that are traded internationally, we used the most competitive price available internationally. For example, drug prices were taken from the latest WHO negotiated prices. A mark-up was included for transportation costs. For goods available only locally (human resources, inpatient bed days, etc) unit costs vary substantially within countries. We therefore ran cross country regressions using the collected data to estimate the average cost (with adjustments for use of capacity) for each setting.11-13 18
Variations in scope and scale
Some interventions can share resources like vehicles, buildings, and administrative staff, so the costs of doing both together are less than the sum of the costs of the two individual programmes. We carefully identified possible shared costs when combinations of interventions were evaluated (box B on bmj.com).
Costs were estimated at three standard levels of coverage: 50%, 80%, and 95%. We assumed that interventions are first provided to, or obtained by, easy to reach populations. Coverage then expands to more outlying, sparsely populated areas. Facilities still need to be built despite the lower population density, so use of capacity will be lower, and the costs per patient treated correspondingly higher at higher levels of coverage. On the other hand, some interventions require a fixed number of staff, office space, or equipment to begin work. The same people and resources cope with increases in coverage so the programme level costs can be spread over a larger population, reducing the costs per person covered. Both types of effects are included in the costing exercise, and the net effect varies by intervention.
Costs are reported in international ($I) rather than US dollars with 2000 the base year (box 2). Future costs are discounted to 2000 values using a 3% discount rate.19 These costs do not translate directly into the financial or cash requirements to run or expand interventions. Estimates of the cash requirements are already available.20-24 Here, we estimate the opportunity cost of all resources required to provide interventions, regardless of who pays for them, to explore the combination of interventions that makes the best use of these resources.
Assessing the effect of interventions
Interventions improve health through their effect on incidence, remission, severity, and mortality. We obtained efficacy data from (in order of preference) systematic reviews with meta-analysis; randomised studies, and before and after evaluations of country programmes. Effectiveness was obtained by adjusting efficacy by a factor between 0 and 1 to allow for less than perfect adherence to recommended practices. The adjustment factor was taken from the literature whenever possible, with expert opinion the last resort. Finally, the eligible population was multiplied by coverage to determine the total numbers benefiting from estimated effectiveness. No information was available on the effect of increasing coverage on health effects. We assumed, therefore, that the health benefit of each child treated for pneumonia (unit effectiveness), for example, did not vary with coverage.
Box 2: International dollars
International dollars are used to account for differences in price levels across countries. The exchange rate for domestic currency into international dollars is the amount of domestic currency required to purchase the same quantity of goods and services as $1 could purchase in the US. For low income countries, national income measured in terms of international dollars is generally higher than it is in US dollars because domestic costs for many goods are lower than in the US. Table B on bmj.com gives conversions from international to US dollars.
It was rarely possible to obtain information on the joint effectiveness of multiple interventions implemented simultaneously. We therefore assumed that the health effect was additive for interventions that affect different health outcomes. For interventions acting on the same outcome the joint effect was estimated by multiplying the individual relative risks (box C on bmj.com).
Population health effects
We used a population model, PopMod, to project the effect of interventions on the aggregate healthy years of life lived by a population, combining incidence, prevalence, and mortality and estimates of disease severity with information on intervention coverage and effectiveness.25 The exception was for HIV and AIDS, for which we used a model that allows death rates to increase with time spent ill. Regional epidemiology was taken from the latest internal WHO update of the global burden of disease study.26
PopMod projects regional population over time, allowing people to move in and out of disease states, or die, in accordance with incidence and remission rates. To derive a single indicator of population health, time spent in each state is given a weight (health state valuation or disability weight) reflecting disease severity. Weights were taken from Murray and Lopez.27 The population is projected for the length of time necessary for people affected by the intervention to live their full life course. The difference between the aggregate number of healthy years lived by the projected population in the intervention and in the "do nothing" scenario is the population health gain attributable to the intervention.
The outcome indicator is essentially the number of healthy years of life gained by an intervention, also called the number of disability adjusted life years (DALYs) averted. The mechanics of estimating DALYs are virtually identical to those required to estimate quality adjusted life years (QALYs), although there are some differences in the interpretation (box D on bmj.com). The base case analysis discounts DALYs averted in the future at 3% and gives greater weight to DALYs averted during adulthood. This is relaxed in the sensitivity analysis.
Calculating cost effectiveness
Cost effectiveness analysis generally considers the costs and effects of adding new interventions to current practice or of replacing an existing intervention with another targeting the same condition. Here we evaluate the full set of existing interventions by first considering what would happen to population health if they all ceased to be implemented today. This is the null or do nothing scenario (box E on bmj.com). The current population is followed over time in PopMod assuming that all interventions cease, using the information on epidemiology, effectiveness, and the current coverage of interventions described earlier.
Next, we trace the implications for population health of adding all possible interventions singly and in various combinations, against the baseline of doing nothing. The difference is the gain in health due to the reduction in disease burden from the intervention(s) (DALYs averted). The costs of each scenario are then compared with the gain to identify the most cost effective set of interventions at different levels of resource availability. The comparison of the current mix against the optimal set for the resources currently available shows areas of inefficiency. The optimal set for higher resource levels shows what should be done if existing efforts to raise more resources are successful.
Interpreting results
For each paper in this series, the cost of every intervention and package is plotted against the corresponding population health effect relative to the do nothing scenario for that condition. We then determine the order in which interventions would be chosen and combined at given levels of resource availability if cost effectiveness were the only consideration. The graphical depiction we call the expansion path. To understand this process, two cost effectiveness ratios are defined. The incremental cost effectiveness ratio denotes the additional costs and benefits of a new intervention or package relative to what existed before, the previous point on the expansion path. Comparison of the incremental cost effectiveness ratio to doing nothing is sometimes called the average cost effectiveness ratio. Box F on bmj.com provides an example of how the expansion path is determined and interpreted.
Uncertainty
Cost effectiveness is an inexact science, particularly when data are limited and of poor quality. Many sources of uncertainty cannot be captured by statistical confidence intervals because no sampling is involved. Probabilistic uncertainty analysis has been developed for these circumstances, but technical limitations do not yet allow it to include the complex correlations inherent in combining the large set of interventions in this series (box G on bmj.com).28 29
Summary points
To achieve the millennium development goals, the resources currently available need to be used effectively
In addition, resources need to be found to scale up many interventions simultaneously
The methods used in this series allow for the assessment of the costs and health effects of current interventions and those that could be used when new resources are available
They incorporate interactions in costs and effects when interventions are undertaken concurrently, and at varying levels of coverage
Standardised methods are used to enhance comparability
The focus is on two regions where unmet health needs are enormous, one in sub-Sarahan Africa and one in South-East Asia
Accordingly, the individual papers incorporate relatively simple forms of sensitivity analysis that are easier to understand. In the final paper, the results of all the disease specific analyses are integrated to develop priorities from the perspective of a policy maker responsible for all health millennium development goals, not just one. For this cross disease analysis, we believe it is not possible to recommend that an intervention shown to cost $45 per DALY averted is more efficient than one costing $60, given the nature of the uncertainties. However, we are much more confident that $45 per DALY is better than $450 per DALY. For the broad comparison across goals we, therefore, interpret cost effectiveness figures in broad order of magnitude ranges (box H on bmj.com).
Conclusions
Progress in the five years since the Millennium Declaration was signed has been disappointing.1 More funds to improve health in poor countries are urgently required, but money alone will not be sufficient to achieve the goals. Open discussion of ways of getting more from the available resources is required, both to allow countries to improve the health of their populations more quickly and to give potential donors confidence that new funds will be used effectively. Application of the methods outlined in this paper, as described in the next six articles in this series, will facilitate open debate and encourage changes in strategies where necessary.
This article is part of a series examining the cost effectiveness of strategies to achieve the millennium development goals for health
Editorial p 1093, Papers p 1107 and Education and debate p 1133
Members of the WHO-CHOICE Millennium Development Goals Team and further details of the methods are on bmj.com
We thank Megha Mukim, Jason Lee, and Marilyn Vogel for thoughtful help with referencing.
Contributors: All authors contributed to the development of the methods. DBE wrote the first draft. TA, TTTE, and SSL made substantial comments and modifications, and DBE and SSL prepared the final version. Members of the WHO-CHOICE MDG team commented on the initial outline and provided guidance on the implications of their papers for the final summary. Christopher Murray was influential in developing many of the methods used in the CHOICE project. DBE is the guarantor. The opinions expressed in this paper are those of the authors and not necessarily of the organisations they represent.
Competing interests: None declared.
References
Evans DB, Adam T, Tan-Torres Edejer T, Lim SS, Cassels A, Evans TG, et al. Achieving the millennium development goals for health: Time to reassess strategies for improving health in developing countries. BMJ 2005;331: 1133-6.
Tan-Torres Edejer T, Aikins M, Black R, Wolfson L, Hutubessy R, Evans DB. Achieving the millennium development goals for health: Cost effectiveness analysis of strategies for child health in developing countries. BMJ 2005 Nov 10; epub ahead of print (doi: 10.1136/bmj.38652.550278.7C).
Adam T, Lim SS, Mehta S, Bhutta ZA, Fogstad H, Mathai M, et al. Achieving the millennium development goals for health: Cost effectiveness analysis of strategies for maternal and neonatal health in developing countries. BMJ 2005;331: 1107-10.
Morel CM, Lauer JA, Evans DB. Achieving the millennium development goals for health: Cost effectiveness analysis of strategies to combat malaria in developing countries. BMJ 2005 Nov 10; epub ahead of print (doi: 10.1136/bmj.38639.702384.AE).
Hogan DR, Baltussen R, Hayashi C, Lauer JA, Salomon JA. Achieving the millennium development goals for health: Cost effectiveness analysis of strategies to combat HIV/AIDS in developing countries. BMJ 2005 Nov 10; epub ahead of print (doi: 10.1136/bmj.38643.368692.68).
Baltussen R, Floyd K, Dye C. Achieving the millennium development goals for health: Cost effectiveness analysis of strategies for tuberculosis control in developing countries. BMJ 2005 Nov 10; epub ahead of print (doi: 10.1136/bmj.38645.660093.68).
Jamison DT, Mosley WH. Disease control priorities in developing countries: health policy responses to epidemiological change. Am J Public Health 1991;81: 15-22.
Tengs TO, Adams ME, Pliskin JS, Safran DG, Siegel JE, Weinstein MC, et al. Five-hundred life-saving interventions and their cost-effectiveness. Risk Anal 1995;15: 369-90.
World Health Organization. Making choices in health: WHO guide to cost-effectiveness analysis. Geneva: WHO, 2003.
Olsen JA, Richardson J. Production gains from health care: what should be included in cost-effectiveness analyses? Soc Sci Med 1999;49: 17-26.
Johns B, Adam T, Evans DB. Enhancing the comparability of costing methods: Cross-country variability in the prices of non-traded inputs to health program. Geneva: WHO, 2005.
Johns B, Baltussen R, Hutubessy RCW. Programme costs in the economic evaluation of health interventions. Cost Eff Resour Alloc 2003;1: 1.
Adam T, Evans DB, Murray CJL. Econometric estimation of country-specific hospital costs. Cost Eff Resour Alloc 2003;1: 3.
World Health Organization. Managing complications in pregnancy and childbirth: a guide for midwives and doctors. Geneva: WHO, 2003.
World Health Organization. Pregnancy, childbirth, postpartum and newborn care: a guide for essential practice. Geneva: WHO, 2003.
World Health Organization. Managing newborn problems: a guide for doctors, nurses and midwives. Geneva: WHO, 2003.
World Health Organization. WHO antenatal care randomized trial: manual for the implementation of the new model. Geneva: WHO, 2002.
Johns B, Baltussen R. Accounting for the cost of scaling-up health interventions. Health Econ 2004;13: 1117-24.
Gold MR, Siegel JE, Russel LB, Weinstein MC. Cost-effectiveness in health and medicine. New York: Oxford University Press, 1996.
Commission on Macroeconomics and Health. Macroeconomics and health: investing in health for economic development. Boston: Center for International Development at Harvard University, 2001. www.cid.harvard.edu/cidcmh/CMHReport.pdf (accessed 17 Oct 2005).
Devarajan S, Miller M, Swanson EV. Goals for development: history, prospects, and costs. Washington, DC: World Bank, 2002. (Policy research working paper No 2819.)
Gutierrez JP, Johns B, Adam T, Bertozzi S, Tan-Torres Edejer T, Greener R, et al. Achieving the WHO/UNAIDS antiretroviral treatment 3 by 5 goal: what will it cost? Lancet 2004;364: 63-4.
UN Millennium Project. Investing in development: a practical plan to achieve the millennium development goals. New York: Earthscan, 2005.
World Health Organization. The world health report 2005: make every mother and child count. Geneva: WHO, 2005.
Lauer JA, Rohrich K, Wirth H, Charette C, Gribble S, Murray CJL. PopMod: a longitudinal population model with two interacting disease states. Cost Eff Resour Alloc 2003;1: 6.
Murray CJL, Lopez AD, Jamison DT. The global burden of disease in 1990: summary results, sensitivity analysis and future directions. Bull World Health Organ 1994;72: 495-509.
Murray C, Lopez A, Mathers C, Stein C. The global burden of disease 2000 project: aims, methods and data sources. Geneva: WHO, 2000.
Briggs AH, Goeree R, Blackhouse G, O'Brien BJ. Probabilistic analysis of cost-effectiveness models: choosing between treatment strategies for gastroesophageal reflux disease. Med Decis Making 2002;22: 290-308.
Claxton K, Sculpher M, McCabe C, Briggs A, Akehurst R, Buxton M, et al. Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra. Health Econ 2005;14: 339-47.(David B Evans, director1, Tessa Tan-Torr)
Correspondence to: D B Evans evansd@who.int
Assessment of the cost effectiveness of interventions designed to achieve the millennium development goals for health is complex. The methods must be capable of showing the efficiency with which current and possible new resources are used, and incorporating interactions between concurrent interventions and the effect of expanding coverage on unit costs.1 They should also allow valid comparisons across a wide range of interventions. Here we describe how the standardised cost effectiveness methods used in the World Health Organization's Choosing Interventions that are Cost Effective (CHOICE) project have tackled these issues.
Level of analysis
The analysis was performed for 14 regions classified by WHO according to their epidemiological grouping (table A on bmj.com). The regional results (except if not relevant to the disease area, for example, malaria) are available at www.who.int/choice, but the papers in this series give details for just two regions: Afr-E, which includes countries in sub-Saharan Africa with high child mortality and very high adult mortality, and Sear-D, which comprises countries in South East Asia with high child and adult mortality.2-6
Definition and selection of interventions
The term intervention is defined to include any preventive, promotive, curative, or rehabilitative action that improves health. Interventions are analysed individually and then in combinations or packages that could be undertaken together (box 1), taking into account interactions in costs or effectiveness, or both.
Interventions were chosen for analysis either because they are commonly used or because disease control experts have advocated their introduction. In each case, some evidence was needed that the intervention could be effective. The list is not exhaustive, and excluding an intervention does not imply it is cost ineffective.
All interventions and combinations are assessed assuming they are implemented for 10 years starting in 2000, the year the Millennium Declaration was signed. Good policy making would then require a reassessment of strategies. Costs, therefore, are only incurred over 10 years, but all benefits accruing because of actions taken between 2000 and 2010 are included.
Intervention costs
Costs are measured from the perspective of society as a whole, to understand how best to use resources regardless of who pays for them, or indeed, whether they are paid for at all. For example, village volunteers working for maternal health must be included because they could be working for some other health programme if they did not work for maternal health. All resources used for each intervention or combination are included and valued.
Data on the costs incurred by people to access services (such as travel costs) are rarely available, and we, like most other studies, have excluded them.7 8 We also excluded domestic taxes; from the perspective of the society, they simply transfer financial resources from one person to another and do not use up a physical resource such as capital or labour. The effect of interventions on the time and potential earnings of patients and unpaid carers—that is, work time lost—is a vexing question in cost effectiveness analysis but, as here, it is often excluded on ethical grounds. Inclusion would give priority to extending the life of people who earn more (box A on bmj.com).9 10
Costs are evaluated assuming 80% use of capacity (technical efficiency) for all interventions; this is relatively high but achievable. Use of a constant figure ensures that differences in cost effectiveness ratios are due to fundamental differences in the technologies or strategies adopted and not simply to an intervention being implemented poorly in a dysfunctional health system. Accordingly, the results provide practical information about the appropriate mix of interventions for various levels of resource, assuming interventions are implemented relatively efficiently.
Measles immunisation is one of the key interventions for child health
Credit: P VIROT/WHO
Box 1: Analysis of individual and combined interventions
Interventions are analysed individually and then in combinations that could be undertaken together. This requires a decision about whether interventions are independent (they can be done at the same time in a population, with or without interactions) or mutually exclusive (if one is chosen, another cannot be). Independent interventions can be added to existing interventions, while mutually exclusive interventions must replace an existing intervention.
Consider three individual antenatal interventions: tetanus immunisation, screening for pre-eclampsia, and screening and treatment for asymptomatic bacteriuria. They are independent, so all possible combinations of the interventions would be:
Tetanus immunisation + screening for pre-eclampsia
Tetanus immunisation+screening and treatment for asymptomatic bacteriuria
Screening for pre-eclampsia+screening and treatment for asymptomatic bacteriuria
Tetanus immunisation+screening for pre-eclampsia and asymptomatic bacteriuria.
All the scenarios are assessed for different coverage levels (50%, 80%, 95%), introducing the idea of mutual exclusivity. Coverage at 95% must replace coverage at 80%. Each of the combinations of interventions is analysed taking into account all possible interactions in costs or effectiveness. Combinations of interventions must be plausible. For example, emergency obstetric care interventions are not included unless skilled birth attendance is implemented at the same time. The number of possible combinations increases rapidly with the addition of each new intervention.
Classification and measurement of costs
Costs are divided into patient and programme levels. Patient level costs include face to face delivery by a health provider—for example, medicines, outpatient visits, inpatient stays, and individual health education messages. Programme level costs include all resources required to establish and maintain an intervention: administration, publicity, training, and delivery of supplies.
We used a standardised ingredients approach to measure costs, requiring information on the quantities of physical inputs needed and their unit cost. Total costs are quantities of inputs multiplied by their unit costs. To obtain programme level costs, we commissioned costing experts to collect data in 17 countries on the physical inputs (human resources, office space, vehicles, electricity, other services, and consumables) required to introduce and run an intervention using a standard template.11-13 This was supplemented by information from programme managers in other countries known by WHO staff.
For patient level costs, quantities were taken from various sources. When published studies of effectiveness were available we used these to identify the resources necessary to ensure the observed level of effectiveness. In other cases, we estimated the resources required from the activities outlined in WHO guidelines for treatment.14-17 Programme experts checked the face validity of all estimated quantities. We could not always determine the quantities of primary inputs (human resources, consumables), so we estimated quantities and prices at an intermediate level for several inputs: inpatient days at different hospital levels, outpatient visits, and health centre visits.
Unit costs for each input were derived from an extensive search of published and unpublished literature and databases along with consultation with costing experts. The full costing database is available at www.who.int/choice. For goods that are traded internationally, we used the most competitive price available internationally. For example, drug prices were taken from the latest WHO negotiated prices. A mark-up was included for transportation costs. For goods available only locally (human resources, inpatient bed days, etc) unit costs vary substantially within countries. We therefore ran cross country regressions using the collected data to estimate the average cost (with adjustments for use of capacity) for each setting.11-13 18
Variations in scope and scale
Some interventions can share resources like vehicles, buildings, and administrative staff, so the costs of doing both together are less than the sum of the costs of the two individual programmes. We carefully identified possible shared costs when combinations of interventions were evaluated (box B on bmj.com).
Costs were estimated at three standard levels of coverage: 50%, 80%, and 95%. We assumed that interventions are first provided to, or obtained by, easy to reach populations. Coverage then expands to more outlying, sparsely populated areas. Facilities still need to be built despite the lower population density, so use of capacity will be lower, and the costs per patient treated correspondingly higher at higher levels of coverage. On the other hand, some interventions require a fixed number of staff, office space, or equipment to begin work. The same people and resources cope with increases in coverage so the programme level costs can be spread over a larger population, reducing the costs per person covered. Both types of effects are included in the costing exercise, and the net effect varies by intervention.
Costs are reported in international ($I) rather than US dollars with 2000 the base year (box 2). Future costs are discounted to 2000 values using a 3% discount rate.19 These costs do not translate directly into the financial or cash requirements to run or expand interventions. Estimates of the cash requirements are already available.20-24 Here, we estimate the opportunity cost of all resources required to provide interventions, regardless of who pays for them, to explore the combination of interventions that makes the best use of these resources.
Assessing the effect of interventions
Interventions improve health through their effect on incidence, remission, severity, and mortality. We obtained efficacy data from (in order of preference) systematic reviews with meta-analysis; randomised studies, and before and after evaluations of country programmes. Effectiveness was obtained by adjusting efficacy by a factor between 0 and 1 to allow for less than perfect adherence to recommended practices. The adjustment factor was taken from the literature whenever possible, with expert opinion the last resort. Finally, the eligible population was multiplied by coverage to determine the total numbers benefiting from estimated effectiveness. No information was available on the effect of increasing coverage on health effects. We assumed, therefore, that the health benefit of each child treated for pneumonia (unit effectiveness), for example, did not vary with coverage.
Box 2: International dollars
International dollars are used to account for differences in price levels across countries. The exchange rate for domestic currency into international dollars is the amount of domestic currency required to purchase the same quantity of goods and services as $1 could purchase in the US. For low income countries, national income measured in terms of international dollars is generally higher than it is in US dollars because domestic costs for many goods are lower than in the US. Table B on bmj.com gives conversions from international to US dollars.
It was rarely possible to obtain information on the joint effectiveness of multiple interventions implemented simultaneously. We therefore assumed that the health effect was additive for interventions that affect different health outcomes. For interventions acting on the same outcome the joint effect was estimated by multiplying the individual relative risks (box C on bmj.com).
Population health effects
We used a population model, PopMod, to project the effect of interventions on the aggregate healthy years of life lived by a population, combining incidence, prevalence, and mortality and estimates of disease severity with information on intervention coverage and effectiveness.25 The exception was for HIV and AIDS, for which we used a model that allows death rates to increase with time spent ill. Regional epidemiology was taken from the latest internal WHO update of the global burden of disease study.26
PopMod projects regional population over time, allowing people to move in and out of disease states, or die, in accordance with incidence and remission rates. To derive a single indicator of population health, time spent in each state is given a weight (health state valuation or disability weight) reflecting disease severity. Weights were taken from Murray and Lopez.27 The population is projected for the length of time necessary for people affected by the intervention to live their full life course. The difference between the aggregate number of healthy years lived by the projected population in the intervention and in the "do nothing" scenario is the population health gain attributable to the intervention.
The outcome indicator is essentially the number of healthy years of life gained by an intervention, also called the number of disability adjusted life years (DALYs) averted. The mechanics of estimating DALYs are virtually identical to those required to estimate quality adjusted life years (QALYs), although there are some differences in the interpretation (box D on bmj.com). The base case analysis discounts DALYs averted in the future at 3% and gives greater weight to DALYs averted during adulthood. This is relaxed in the sensitivity analysis.
Calculating cost effectiveness
Cost effectiveness analysis generally considers the costs and effects of adding new interventions to current practice or of replacing an existing intervention with another targeting the same condition. Here we evaluate the full set of existing interventions by first considering what would happen to population health if they all ceased to be implemented today. This is the null or do nothing scenario (box E on bmj.com). The current population is followed over time in PopMod assuming that all interventions cease, using the information on epidemiology, effectiveness, and the current coverage of interventions described earlier.
Next, we trace the implications for population health of adding all possible interventions singly and in various combinations, against the baseline of doing nothing. The difference is the gain in health due to the reduction in disease burden from the intervention(s) (DALYs averted). The costs of each scenario are then compared with the gain to identify the most cost effective set of interventions at different levels of resource availability. The comparison of the current mix against the optimal set for the resources currently available shows areas of inefficiency. The optimal set for higher resource levels shows what should be done if existing efforts to raise more resources are successful.
Interpreting results
For each paper in this series, the cost of every intervention and package is plotted against the corresponding population health effect relative to the do nothing scenario for that condition. We then determine the order in which interventions would be chosen and combined at given levels of resource availability if cost effectiveness were the only consideration. The graphical depiction we call the expansion path. To understand this process, two cost effectiveness ratios are defined. The incremental cost effectiveness ratio denotes the additional costs and benefits of a new intervention or package relative to what existed before, the previous point on the expansion path. Comparison of the incremental cost effectiveness ratio to doing nothing is sometimes called the average cost effectiveness ratio. Box F on bmj.com provides an example of how the expansion path is determined and interpreted.
Uncertainty
Cost effectiveness is an inexact science, particularly when data are limited and of poor quality. Many sources of uncertainty cannot be captured by statistical confidence intervals because no sampling is involved. Probabilistic uncertainty analysis has been developed for these circumstances, but technical limitations do not yet allow it to include the complex correlations inherent in combining the large set of interventions in this series (box G on bmj.com).28 29
Summary points
To achieve the millennium development goals, the resources currently available need to be used effectively
In addition, resources need to be found to scale up many interventions simultaneously
The methods used in this series allow for the assessment of the costs and health effects of current interventions and those that could be used when new resources are available
They incorporate interactions in costs and effects when interventions are undertaken concurrently, and at varying levels of coverage
Standardised methods are used to enhance comparability
The focus is on two regions where unmet health needs are enormous, one in sub-Sarahan Africa and one in South-East Asia
Accordingly, the individual papers incorporate relatively simple forms of sensitivity analysis that are easier to understand. In the final paper, the results of all the disease specific analyses are integrated to develop priorities from the perspective of a policy maker responsible for all health millennium development goals, not just one. For this cross disease analysis, we believe it is not possible to recommend that an intervention shown to cost $45 per DALY averted is more efficient than one costing $60, given the nature of the uncertainties. However, we are much more confident that $45 per DALY is better than $450 per DALY. For the broad comparison across goals we, therefore, interpret cost effectiveness figures in broad order of magnitude ranges (box H on bmj.com).
Conclusions
Progress in the five years since the Millennium Declaration was signed has been disappointing.1 More funds to improve health in poor countries are urgently required, but money alone will not be sufficient to achieve the goals. Open discussion of ways of getting more from the available resources is required, both to allow countries to improve the health of their populations more quickly and to give potential donors confidence that new funds will be used effectively. Application of the methods outlined in this paper, as described in the next six articles in this series, will facilitate open debate and encourage changes in strategies where necessary.
This article is part of a series examining the cost effectiveness of strategies to achieve the millennium development goals for health
Editorial p 1093, Papers p 1107 and Education and debate p 1133
Members of the WHO-CHOICE Millennium Development Goals Team and further details of the methods are on bmj.com
We thank Megha Mukim, Jason Lee, and Marilyn Vogel for thoughtful help with referencing.
Contributors: All authors contributed to the development of the methods. DBE wrote the first draft. TA, TTTE, and SSL made substantial comments and modifications, and DBE and SSL prepared the final version. Members of the WHO-CHOICE MDG team commented on the initial outline and provided guidance on the implications of their papers for the final summary. Christopher Murray was influential in developing many of the methods used in the CHOICE project. DBE is the guarantor. The opinions expressed in this paper are those of the authors and not necessarily of the organisations they represent.
Competing interests: None declared.
References
Evans DB, Adam T, Tan-Torres Edejer T, Lim SS, Cassels A, Evans TG, et al. Achieving the millennium development goals for health: Time to reassess strategies for improving health in developing countries. BMJ 2005;331: 1133-6.
Tan-Torres Edejer T, Aikins M, Black R, Wolfson L, Hutubessy R, Evans DB. Achieving the millennium development goals for health: Cost effectiveness analysis of strategies for child health in developing countries. BMJ 2005 Nov 10; epub ahead of print (doi: 10.1136/bmj.38652.550278.7C).
Adam T, Lim SS, Mehta S, Bhutta ZA, Fogstad H, Mathai M, et al. Achieving the millennium development goals for health: Cost effectiveness analysis of strategies for maternal and neonatal health in developing countries. BMJ 2005;331: 1107-10.
Morel CM, Lauer JA, Evans DB. Achieving the millennium development goals for health: Cost effectiveness analysis of strategies to combat malaria in developing countries. BMJ 2005 Nov 10; epub ahead of print (doi: 10.1136/bmj.38639.702384.AE).
Hogan DR, Baltussen R, Hayashi C, Lauer JA, Salomon JA. Achieving the millennium development goals for health: Cost effectiveness analysis of strategies to combat HIV/AIDS in developing countries. BMJ 2005 Nov 10; epub ahead of print (doi: 10.1136/bmj.38643.368692.68).
Baltussen R, Floyd K, Dye C. Achieving the millennium development goals for health: Cost effectiveness analysis of strategies for tuberculosis control in developing countries. BMJ 2005 Nov 10; epub ahead of print (doi: 10.1136/bmj.38645.660093.68).
Jamison DT, Mosley WH. Disease control priorities in developing countries: health policy responses to epidemiological change. Am J Public Health 1991;81: 15-22.
Tengs TO, Adams ME, Pliskin JS, Safran DG, Siegel JE, Weinstein MC, et al. Five-hundred life-saving interventions and their cost-effectiveness. Risk Anal 1995;15: 369-90.
World Health Organization. Making choices in health: WHO guide to cost-effectiveness analysis. Geneva: WHO, 2003.
Olsen JA, Richardson J. Production gains from health care: what should be included in cost-effectiveness analyses? Soc Sci Med 1999;49: 17-26.
Johns B, Adam T, Evans DB. Enhancing the comparability of costing methods: Cross-country variability in the prices of non-traded inputs to health program. Geneva: WHO, 2005.
Johns B, Baltussen R, Hutubessy RCW. Programme costs in the economic evaluation of health interventions. Cost Eff Resour Alloc 2003;1: 1.
Adam T, Evans DB, Murray CJL. Econometric estimation of country-specific hospital costs. Cost Eff Resour Alloc 2003;1: 3.
World Health Organization. Managing complications in pregnancy and childbirth: a guide for midwives and doctors. Geneva: WHO, 2003.
World Health Organization. Pregnancy, childbirth, postpartum and newborn care: a guide for essential practice. Geneva: WHO, 2003.
World Health Organization. Managing newborn problems: a guide for doctors, nurses and midwives. Geneva: WHO, 2003.
World Health Organization. WHO antenatal care randomized trial: manual for the implementation of the new model. Geneva: WHO, 2002.
Johns B, Baltussen R. Accounting for the cost of scaling-up health interventions. Health Econ 2004;13: 1117-24.
Gold MR, Siegel JE, Russel LB, Weinstein MC. Cost-effectiveness in health and medicine. New York: Oxford University Press, 1996.
Commission on Macroeconomics and Health. Macroeconomics and health: investing in health for economic development. Boston: Center for International Development at Harvard University, 2001. www.cid.harvard.edu/cidcmh/CMHReport.pdf (accessed 17 Oct 2005).
Devarajan S, Miller M, Swanson EV. Goals for development: history, prospects, and costs. Washington, DC: World Bank, 2002. (Policy research working paper No 2819.)
Gutierrez JP, Johns B, Adam T, Bertozzi S, Tan-Torres Edejer T, Greener R, et al. Achieving the WHO/UNAIDS antiretroviral treatment 3 by 5 goal: what will it cost? Lancet 2004;364: 63-4.
UN Millennium Project. Investing in development: a practical plan to achieve the millennium development goals. New York: Earthscan, 2005.
World Health Organization. The world health report 2005: make every mother and child count. Geneva: WHO, 2005.
Lauer JA, Rohrich K, Wirth H, Charette C, Gribble S, Murray CJL. PopMod: a longitudinal population model with two interacting disease states. Cost Eff Resour Alloc 2003;1: 6.
Murray CJL, Lopez AD, Jamison DT. The global burden of disease in 1990: summary results, sensitivity analysis and future directions. Bull World Health Organ 1994;72: 495-509.
Murray C, Lopez A, Mathers C, Stein C. The global burden of disease 2000 project: aims, methods and data sources. Geneva: WHO, 2000.
Briggs AH, Goeree R, Blackhouse G, O'Brien BJ. Probabilistic analysis of cost-effectiveness models: choosing between treatment strategies for gastroesophageal reflux disease. Med Decis Making 2002;22: 290-308.
Claxton K, Sculpher M, McCabe C, Briggs A, Akehurst R, Buxton M, et al. Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra. Health Econ 2005;14: 339-47.(David B Evans, director1, Tessa Tan-Torr)