Common currency in health assessments

This page is a method. The page identifier is Op_en4692
Moderator:Olli (see all)
Give your opinion to the peer rating of the content of this page. {{ #opasnet_rater: }}
Upload data

Scope

We may qualitatively compare things very different in nature, apples and oranges. In cases with very large and clear differences between the choices, qualitative comparisons might be enough. However, the question easily becomes value driven. Common measure is an answer to this problem. This article reviews identifying a common metric to be used in benefit-risk assessments. Several European research projects have already been dealing with this issue, therefore their conclusion are presented here. In the end we select the most suitable common currency.

--# : a short introduction describing the context of the project --Olli 13:17, 6 October 2011 (EEST)

PlantLIBRA (acronym of PLANT food supplements: Levels of Intake, Benefit and Risk Assessment) is a project co-financed in the context of the 7th EU Framework Program. PlantLIBRA aims to foster the safe use of food supplements containing plants or botanical preparations, by increasing science-based decision-making by regulators and food chain operators.
 PlantLIBRA is structured to develop, validate and disseminate data and methodologies for risk and benefit assessment and implement sustainable international cooperation. International cooperation is necessary to ensure the quality of the plants imported in the EU. Common currency methodology answers to the need to compare different health endpoints caused by the intake of food supplements.

Most established options for common currency

There are several common currency measures available. Six of them are briefly introduced in this section.

Incidences

Incidence information as a health measure is the absolute number of cases during a give time. Commonly, it is expressed as a proportion or a rate, where number of new cases is divided by the population initially at risk. It can also be used for presenting person-time incidence rate, where the denominator is the sum of the person-time of the at risk population. Incidence should not be confused with prevalence which is a measure of the total number of cases of disease in a population rather than the rate of occurrence of new cases.

Disability adjusted life years (DALY)

DALY is a measure of overall burden of disease. It contains both mortality and morbidity, which makes it useful for comparing the health effects of different nature, and it has become increasingly common in the field of public health. Methodologically, it summarizes years of life lost (YLL) and years lived with disease (YLD). The less DALY, the better. One DALY is equal to one year of healthy life lost. Each disease is assigned with severity weights which have been derived in combination of expert panels and patient reports. Severity weight of death is assigned the value of 1.0, and perfect health 0. (WHO 2011, Murray and Lopez 1995)

Quality Adjusted Life Years (QALY)

QALY is very much like DALY. The greatest difference is that it measures the positive health state, apart from DALY. Similarly, it includes both the quality and the quantity of life lived. The more QALYs, the better. Using QALY requires utility independent, risk neutral, and constant proportional tradeoff behavior. Each year in perfect health is assigned the value of 1.0, and value of 0.0 for death. Similarly to DALY, weights are assigned to health states. Determining the weight associated with a particular health state can be done using standard descriptive systems which categorizes health states according to the following dimensions: mobility, self-care, usual activities (e.g. work, study, homework or leisure activities), pain/discomfort and anxiety/depression. (Schlander 2010, Health Economics 2010)

Days of work lost

This measure tries to capture the disease burden of working age population to be further analyzed in more sophisticated measures, such as cost. The information is derived from medical statistics, readily available in some countries. Another advantage is that it is a societal measure and can be used to assess association between productivity and health. The biggest shortage of this measure is that it totally ignores other than working population. Therefore using this measure to assess public health is misleading.

Cost in money

One can also translate health outcomes into money, for example estimating the treatment costs for illnesses, and reduced labor. It can also be used for estimating willingness to pay to avoid a disease. The advantage is that it can easily be compared with other societal costs, and therefore it can be a convenient measure for decision-making. In some countries data requirements are quite easy to fulfill, however, in others very challenging. Additionally, willingness to pay vary depending on economic wealth, and the objectivity of the measure can also be questionable as hidden judgements might be involved in the process. (Holland et al. 2005)

Utility

Utility is a mathematical expression that assigns a value to all possible choices. The presumption is that the decision-maker has to be indifferent about options (can not choose over options). The worst result for a decision-maker is defined as value 0 and the best as value 1. If the best option happens with probability p, and option under consideration happens with probability 1, then the utility of option under consideration is given value p. (Lintley 1998, Lintley 1991)

Previous experience on using common currency

Several projects have already assessed the use of common currency. The next sections contain background information and conclusions from these projects.

EFSA 2006

In 2006 EFSA suggested that the assessments with both risk and benefit ideally should be performed under the same criteria for weighing the evidence and identifying the uncertainties. The presentation of the results of the risk-benefit assessment must fit the predefined purpose of the request and make clear where the certainties and uncertainties are in order to compare the relative confidence on the benefits with the risks. The following possible common scale measures were mentioned:

• Incidences
• Disability Adjusted Life Years DALY
• Quality Adjusted Life Years QALY
• Days of work lost
• Costs in money

Further EFSA 2006 concluded that QALY are, nevertheless, based on a number of assumptions, and are more difficult to quantify than DALY. The difficulty in expressing results from toxicological studies in experimental animals as DALY needs to be overcome. Using costs requires equal cost structures across countries/world and is difficult to communicate. It was agreed that more research and experience with different approaches are needed.

Advantages using DALY

• The advantages of using DALYs are that they represent an established procedure to compare risks of different nature
• DALYs may provide guidance to the risk manager on how to prioritize the direction of targeted intervention measures
• DALYs have a time-scale

Challenges using DALY

• DALYs are applied at the societal, rather than the individual, level. It is possible to apply DALYs in risk-benefit assessment, but appropriate data may seldom be available.
• The difficulties in using DALYs are that clear messages are needed so that the numbers generated are not taken out of context.
• It also seems difficult to include preventive aspects (such as effects of preservation) or absence of risk rather than benefit.
• The difficulty in expressing results from toxicological studies in experimental animals as DALYs needs to be overcome.
• Because the scale is likely to differ for different analysis, no generally applicable measurement scale is likely to be developed.

EFSA 2010

Scientific committee, appointed by the EFSA 2010, recommended a stepwise approach for assessing the benefits and risks of food. The last of the three steps utilizes common currency approach to combine two or more of the following elements: increases or decreases in morbidity, mortality, disease burden, and quality of life. The choice of composite metrics should be made on a case by case basis, based on the specific risk-benefit question, identified hazards and positive health effects. Health effects can be assessed in a number of different dimensions, such as incidence of effect, severity of effect, morbidity and mortality rate, and in the case of positive health effects also quality of life. More than one metric will be needed to capture all dimensions of health for a risk-benefit assessment.

Advantages using DALY

• Composite metrics, such as DALY or QALY, can be used for direct comparison of effects
• Effects expressed in a common metric can be compared, but care must be exercised in the interpretation of the comparison.

Challenges

• It is important to recognize that not all relevant dimensions are captured in these metrics, for example, whether the effect is in children or adults. This is because these metrics combine incidence with life years to obtain an estimate of years saved or lost respectively, so that a few young people with many years of potential life can give an equivalent value as a larger number of elderly people with far fewer years of potential life.
• Some of the DALY or QALY weightings are open for discussion.
• When reporting to the risk-benefit manager the risk benefit assessor needs to consider that the result “is more than a number” and should be considered together with the outcome of the Step 2 assessment.
• Currently, generally agreed metrics for positive health effects and well being are lacking, in part because there are no agreed weighting factors for positive health effects.

The Scientific Committee recommends that composite metrics are used to combine two or more of the following elements: increases or decreases in morbidity, mortality, disease burden, and quality of life. The choice of composite metrics should be made on a case by case basis, based on the specific risk-benefit question, identified hazards and positive health effects. The Scientific Committee recommends however, when reporting to the risk-benefit manager on the outcome of the risk-benefit assessment, to provide as well the respective health impact values expressed in the selected composite metric for each relevant health effect and each relevant sub population with their respective uncertainties.

Brafo

Brafo project tried to develop a framework that allows quantitative comparison of human health risks and benefits of foods and food compounds based on a common scale of measurement. It will be based on the evaluation of changes in the quality/duration of life using a system that allows weighting of data quality and severity of effect, with quantification by QALY or DALY-like methodology.

The framework will take into account how risks and benefits interrelate. It is intended that the methodology developed is sufficiently transparent to serve as a reference for the harmonisation of the evaluation methods used within the European Union and more widely in international evaluations.

Options others than to combine risk and benefit in a common scale were suggested, e.g. (1) to give a detailed risk-benefit description and leave any decisions to the risk-benefit manager, or (2) to express the assessment results as changes in risk or benefit (increments) and calculate the risk + benefit difference. It was agreed that more research and experience with different approaches are needed.

QALIBRA

In many cases, conventional risk assessment may show that adverse effects are unlikely. In other cases, a qualitative evaluation may be sufficient to conclude that either the risks or the benefits dominate. When this is not the case, it may be necessary to quantify not only the incidence of adverse and beneficial effects, but also the magnitude of their impact on health, their duration, and their impact on life expectancy. In addition, it may be helpful to combine these different dimensions of health impact into a single integrated measure such as disability-adjusted life years (DALYs).

Qalibra project developed a tool for the higher (quantitative) tiers of tiered approaches to risk-benefit assessment, such as those being considered by the EFSA and the related EU project Brafo.

Bepraribean

Bepraribean project produced seven peer reviewed publications (see Bepraribean papers in the end) on how to manage benefit-risks in different fields, and found that the most used integrated health measures in food-related benefit-risk analysis are DALY and QALY. QALY’s are traditionally mostly used to measure health gains at micro scale, for example to compare two interventions. For many diseases, disability weights, which are currently being revised in the Global Burden of Disease 2010 study, are available at the WHO website. Further, often the choice for DALY or QALY is a pragmatic one, based on data availability or experience of use rather than on a fundamental choice.

Definition of benefits

There are several definitions for benefits, depending on scope and viewpoint. EFSA uses the following general definition "The probability of a positive health effect and/or the probability of a reduction of an adverse health effect in an organism, system, or (sub)population, in reaction to exposure to an agent" (EFSA 2010)

Closer to the food supplement field we can use the following definition: "The attainment of specific physiological objectives, such as reduction of risk factors for chronic diseases and the maintenance of the human homeostasis, which is the body’s capability to physiologically regulate wellbeing and ensure stability and balance in response to changes in the external environment."

Comparison between alternative common currencies

Parameter	Incidences	Disability Adjusted Life Years (DALY)	Quality Adjusted Life Years (QALY)	Days of work lost	Costs in money	Utility
Does it sum up to a single metric?	No	Yes	Yes	Yes	Yes	Yes
Can it be used for non-health endpoints?	No	No	No	No	Yes	Yes
What is the basis for weighting?	No weighting	Severity weight, which is specific to a disease diagnosis.	Quality indices such as pain, cognitive capacity, orientation, anxiety, performance of daily routines. Specific to the condition of a patient.	Reduced capacity of a patient in the labour market.	Costs of treatment, opportunity loss due to a disease, and willingness to pay to avoid a disease.	Decision maker expresses his/her values about all possible outcomes on a scale from 0 to 1.
Where can the weights be used?	Universally; however, background rates vary from country to country and between sub-populations	Universally, although different cultures might have somewhat different severity weights.	Universally, although different cultures might have somewhat different severity weights.	Within a country, because different treatment practices and social security systems may lead to different loss of work for the same disease.	Within a country, because treatment costs and willingness to pay vary depending on economic wealth.	Only for a specific case. The weights depend on the value judgements of a specific decision maker. However, it is also possible to derive more generic weights for standard decision situations.
Who decides about the weights?	No weighting	An international panel of medical doctors (organised by WHO) using person trade-off technique (Salomon 2010).	Large panels of patients and doctors.	Can be derived from medical statistics for a country.	Contingent valuation studies performed in a country.	The decision maker of each decision.
What is the variable that is weighted?	Number of patients with specific diagnoses.	Number of patients with specific diagnoses, duration of each disease.	Number of people with different levels of health loss, duration of health loss.	Number of patients with specific diagnoses.	Number of patients with specific diagnoses, costs of treatment.	Any outcomes that the decision maker considers important.
Are the weights easily available?	Yes	Yes, provided by WHO.	No	Yes for some countries	Rather easy for some countries.	Only if the decision maker is involved in the assessment.
Is the summary indicator easy to understand without further explanations?	Yes	Fairly easy, deeper understanding requires insight	Fairly easy, deeper understanding requires insight	Yes	Yes, although readers may not understand that there are hidden value judgements of WTP.	No
Can the outcome results be compared to other cases?	Yes	Yes	Yes	Yes within a country, between countries only with caution	Yes within a country, between countries only with caution	Only if the decision maker is the same or his/her values are shared with in the other case.
Overall conclusion	Incidences are calculated anyway in a health assessment, and they should be published although it is not actually a summary measure.	DALY is the preferred method and better than QALY because the easy availability of weights and diagnoses.	QALY is also a good method and arguably more precise than DALY but information about health conditions (compared with diagnoses in DALY) and their weights is not easily available.	In many cases a practical metric, but ignores diseases in the young and elderly, and also varies between countries.	If non-health endpoints are considered, money should be considered as DALYs don't work. It is widely used and thus makes comparison easy (at least within a country).	Utility is theoretically the best alternative, and any of the other methods can be seen as a sub-method for utility. However, using utilities requires extensive cooperation with decision maker which is not always possible. On the other hand, this kind of cooperation is recommended anyway because of other reasons.
Recommendation: Which metrics should be used? (Note: there is no need to limit to only one.)	Use always	Use always	Use if data on health conditions and weights is available	Do not use routinely	Use always when non-health endpoints should be considered.	Should be used if there is a possibility for cooperation with the decision maker.

Conclusions

Based on the table, we identified DALYs, QALYs as potential measures for common currency. DALY is the most widely used measure and therefore the first choice for the work. Incidences, as the most transparent indication of the health effect should be presented along with common currencies. If one prefers to present something else than changes in health statuses, cost and utility are the best options for this.

Because calculating common currency often demands special data and some understanding about the methods, developing common currency calculation tools, such which has been developed in the EU-projects INTARESE (INTARESE 2011) and QALIBRA (QALIBRA 2011, Hoekstra et al. 2011), is highly appreciated.

See also

http://en.opasnet.org/w/Common_currency_in_health_assessments

References

Health Economics. 2010. What is QALY? http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/QALY.pdf Accessed September 2011.

Hoekstra, J., Hart, A., Owen, H., Zeilmaker, M., Bokkers, Bas., Thorgilson, B., Gunnlaugsdottir, H. 2011. Fish, contaminants and human health; Quantifying and weighing benefits and risks. Submitted manuscript.

Hunsley, J., and Westmacott, R. 2007. "Interpreting the magnitude of the placebo effect: mountain or Molehill?". J Clin Psychol 63 (4): 391–9.

INTARESE. Integrated Assessment of Health Risks of Envirnonmental Stressors in Europe. Front page: http://www.intarese.org/ Accessed September 2011.

Kaptchuk, JT., Kelley, JM., Deykinc, A., Waynea, PM., Louis C. Lasagnad, LC., Epsteine, IO., Kirschf, I., and Wechsler, ME. Do “placebo responders” exist? Contemporary Clinical Trials Volume 29, Issue 4, July 2008, Pages 587-595

LINDLEY, DV. 1991. SUBJECTIVE-PROBABILITY, DECISION-ANALYSIS AND THEIR LEGAL CONSEQUENCES. JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES A-STATISTICS IN SOCIETY. Volume 154, Pages 83-92

Lintley, DV. 1998. Decision analysis and bioequivalence trials. STATISTICAL SCIENCE. Vol 13, Issue 2, Pages 136-141

Margo, CE. 1999. The Placebo Effect. Survey of Ophthalmology Volume 44, Issue 1, 8 July 1999, Pages 31-44

Murray, CJL., and Lopez, AD. 1995. Global mortality, disability, and the contribution of risk factors: Global Burden of Disease Study. Vol 349, Issue 9063, 1436-1442

Opasnet 2011. Impact calculation tool. http://en.opasnet.org/w/Impact_calculation_tool Accessed September 2011.

QALIBRA. 2011. Quality of life - integrated benefit and risk. Calculation tool: http://www.qalibra.eu/ accessed September 2011.

Salomon 2010. New disability weights for the global burden of disease. Bulletin of the World Health Organization. Volume 88, Number 12, December 2010, 877-953.

Schlander, M. 2010. Measures of efficiency in healthcare: QALMs about QALYs? Z. Evid. Fortbild. Qual. Gesundh. wesen (ZEFQ) 104 (2010) 209–226

Holland, M., Watkiss, P., Pye, S., de Oliveira, A., and Van Regemorter, D. 2005. Cost-Benefit Analysis of Policy Option Scenarios for the Clean Air for Europe programme. AEA Technology Environment.

WHO. 2011. Health statistics and health information systems. http://www.who.int/healthinfo/global_burden_disease/metrics_daly/en/index.html Accessed September 2011.

Bepraribean papers:

Verhagen, H., Tijhuis, MJ., Gunnlaugsdόttir, H., Kalogeras, N., Leino, O., Luteijn, JM., Magnússon, SH., Odekerken, G., Pohjola, MV., Tuomisto, JT., Ueland ,O., White, BC., and Holm, F. 2011. State of the art in benefit-risk analysis: Introduction. Food Chem Toxicol. 2011 Jun 12. Introduction paper

Tijhuis, MJ., de Jong, N., Pohjola, MV., Gunnlaugsdóttir, H., Hendriksen, M., Hoekstra, J., Holm, F., Kalogeras, N., Leino, O., van Leeuwen, FX., Luteijn, JM., Magnússon, SH., Odekerken, G., Rompelberg, C., Tuomisto, JT., Ueland, O., White, BC., and Verhagen, H. 2011a. State of the art in benefit-risk analysis: Food and nutrition. Food Chem Toxicol. 2011 Jun 12. Food and nutrition paper

Ueland, O., Gunnlaugsdottir, H., Holm, F., Kalogeras, N., Leino, O., Luteijn, JM., Magnússon, SH., Odekerken, G., Pohjola, MV., Tijhuis, MJ., Tuomisto, JT., White, BC., and Verhagen, H. 2011. State of the art in benefit-risk analysis: Consumer perception. Food Chem Toxicol. 2011 Jun 12. Consumer perception paper

Magnússon, SH., Gunnlaugsdóttir, H., van Loveren, H., Holm, F., Kalogeras, N., Leino, O., Luteijn, JM., Odekerken, G., Pohjola, MV., Tijhuis, MJ., Tuomisto, JT., Ueland, O., White, BC., and Verhagen, H. 2011. State of the art in benefit-risk analysis: Food microbiology. Food Chem Toxicol. 2011 Jun 12. Food and microbiology paper

Kalogeras, N., Odekerken, G., Pennings, JM., Gunnlaugsdόttir, H., Holm, F., Leino, O., Luteijn, JM., Magnússon, SH., Pohjola, MV., Tijhuis, MJ., Tuomisto, JT., Ueland, O., White, BC., and Verhagen, H. 2011. Food Chem Toxicol. 2011 Jun 12. Economics and marketing-finance paper

Luteijn, JM., White, BC., Gunnlaugsdóttir, H., Holm, F., Kalogeras, N., Leino, O., Magnússon, SH., Odekerken, G., Pohjola, MV., Tijhuis, MJ., Tuomisto, JT., Ueland, O., McCarron, PA., and Verhagen, H. 2011. State of the art in benefit-risk analysis: Medicines. Food Chem Toxicol. 2011 Jun 12. Medicines paper

Pohjola, MV., Leino, O., Kollanus, V., Tuomisto, JT., Gunnlaugsdóttir, H., Holm, F., Kalogeras, N., Luteijn, JM., Magnússon, SH., Odekerken, G., Tijhuis, MJ., Ueland, O., White, BC., and Verhagen, H. 2011. State of the art in benefit-risk analysis: Environmental health. Food Chem Toxicol. 2011 Jun 12. Environmental health paper

Tijhuis, MJ., Pohjola, MV., Gunnlaugsdóttir, H., Kalogeras, N., Leino, O., Luteijn, JM., Magnússon, SH., Odekerken, G., Potof, M., Tuomisto, JT., Ueland, O., White, BC., Holm, F., and Verhagen, H. 2011b. Looking beyond Borders: Integrating Best Practices in Benefit-Risk Analysis into the Field of Food and Nutrition. xxxx xxxx xxxx To be submitted in September 2011.

Projects

<mfanonymousfilelist></mfanonymousfilelist>