Talk:Comparison criteria
From Testiwiki
Contents
Health endpoints to consider
Statements: These health endpoints and other criteria should be considered in the comparison:
Resolution: Criteria or endpoints that should be considered are a) price, b) invasive pneumococcal disease, and c) technical criteria. However, if there were enough reliable data, also pneumonia and other respiratory endpoints should be included. Otitis media should be included at least as a sensitivity analysis. (A stable resolution, when found, should be updated to the main page.) |
| Argumentation:
⇤# : An endpoint or criteria should NOT be considered unless there are clear reasons to do so. This is the nature of a tendering process. --Jouni (talk) 13:43, 4 September 2014 (UTC)
|
Discussion on comparison criteria
Comments from Pfizer
Comment K3 --Pfizer Oy, 2014-08-27 13:47:39 - R upload
PCV tender criteria from other Nordic countries show that quality is valued and given greater weight than price also where the winning vaccine is considered the “economic most advantageous”. Quality points should be given to several dimensions, not just for number of serotypes in the vaccine, for example:
Norway (2011)
Quality 50-70%, price 20-40%, service after delivery 5%. “The offer that achieves the highest point score after adding up the calculated points for each of the weighted criteria for assignment will be considered as the financially most favorable offer.”
Criteria:
- Requirement: It must be possible to store the vaccine at 25°C for at least 48 hours, without altering the vaccine properties. Documentation: stability studies
- The expected number of preventable cases of systemic pneumococcal disease. The efficacy of the vaccine against systemic pneumococcal disease caused by the different serotypes in the vaccine will be of significance. Other relevant conditions, including clinical documentation on the efficacy against otitis caused by S.pneumoniae.
- The best possible safety profile in this particular population. The offered vaccine may be part of a vaccination program where both available pneumococcal vaccines can be used. The prevalence in Norway of the individual disease-developing serotypes will be considered.
- The efficacy and risk/benefit assessment of the vaccines will be assessed on a rough estimate based on the presented documentation. Documentation assessed by competent authorities in the EEA-union will be of particular importance.
- Price pr. dose size is given as one price, which is applicable independently of the delivered dose, i.e. if it is delivered as one dose pack or as 10-dose packs.
- Service after delivery (5%) Service availability and technical assistance will be emphasized
Denmark (2014)
Quality 75%, price 25%. “The framework contract will be awarded on the basis of the award criterion the economic most advantageous tender .
Criteria:
- The effectiveness is assessed on the basis of the vaccine's coverage of invasive pneumococcal diseases in children < 2 years (35%) and the product's direct and indirect coverage of invasive pneumococcal diseases in the rest of the population (25%) in Denmark in 2009 based on the applicable SPC (Summary of Product Characteristics). The tenderer must complete a form concerning the indication of effectiveness of pneumococcal serotypes contained in the vaccine on the basis of the applicable SPC.
- Type and frequency of adverse effects (10%): assessed on the basis of the applicable SPC. As few and least invasive adverse effects as possible are desired.
- The active ingredients/trace elements of the products (5%): it is assessed whether the product is manufactured by or includes materials or trace elements of animal or human origin or other ingredients that may potentially pose a risk to the child/patient.
- The assessment is made on the basis of an applicable SPC, a completed form relating to active ingredients/trace elements enclosed with the tender documents as well as any statement as to no active ingredients and other relevant material, as described in section 10.4. As few active ingredients and trace elements that may potentially pose a risk to the child/the patients as possible are desired.
- The price will be assessed on the basis of the average price for the period 2014-2017. The lowest price possible is desired.
Sweden, Stockholm (2015)
Quality emphasized. If the tender gets 100 points for quality, the price will be multiplied by 1. If, for example, the tender gets 50 quality points, the price will be multiplied by 1,50.
Criteria:
- Clinical efficacy, safety (max 90 points): Range of serotype protection, documentation on efficacy and effectiveness, documentation on risk groups, documentation on antibody response, side effect profile, excipients and preservatives. Very good safety profile, good clinical efficacy, optimal protection in terms of serotype coverage, documentation on risk groups: 90 points:
- Very good safety profile, good clinical efficacy but not optimal protection in terms of serotype coverage, no documentation on risk groups: 20 points Line width, practical management, labelling
- (max 10 points) Size, needles, design (risk of mix up), clear labelling and presence of bar code, storage conditions, handling in general
Sweden, VGR (Region Västra Götaland)
Quality 70%, price 30%
Criteria:
- Clinical efficacy (max 60 p): Number of serotypes, clinical efficacy on individual level, proven herd immunity, approved indications. Based on RCT and/or other studies, official authority statements
- Very Good:48-60 points
- Good: 34-47 points
- Acceptable: 20-33 points
- Less good: 10-19 points
- Poor or not gradable: 0-9 points:
- Clinical safety (max 60 p): Side effect profile in SmPC/WHO, RCT and/or other studies, official authority statements. Aluminum content.
- Very good: 48-60 points
- Good: 34-47 points
- Acceptable: 20-33 points
- Less good: 10-19 points
- Poor or not gradable: 0-9 points
- Product range (max 10 p) Will be evaluated if relevant, otherwise maximum points will be awarded. *Package size: 0-5 points
- Dosing schedule: 0-5 points
- Practical management (max 30 p) Will be evaluated if relevant, otherwise maximum points will be awarded
- Storage conditions (e.g. light sensitive, storage in cold, durability):0-10 points
- Formulation aspects (e.g. dry matter/vial/pre-filled syringe, preparation instructions): 0-5 points
- Product itself (e.g. manageability at the preparation and administration, packaging design): 0-15 points
- Marking and labelling (max 20 p) The bidder shall submit Mock-ups on the outer packaging and images on the inner packaging to the bid, and blister on all offered products. If need of pharmaceutical samples, the company will be contacted.
- Barcode (outer and inner packaging): 0-10 points
- Labelling (outer and inner packaging, readability on the syringe label, removable label): 0-10 points
Comment K4 --Pfizer Oy, 2014-08-27 13:47:39 - R upload
It is appreciable Opasnet is a source of transparency. As JCVI and ACIP do, is it planned to inform the general public by publishing regularly the evaluation and decision processes, including analysis and results, at Opasnet or somewhere else?
- ←Kauq1: : The current use of opasnet is a pilot project. The main aims are to make available reasoning that underlies the tendering criteria for the next round of the national procurement of pneumococcal conjugate vaccine in Finland. Whether or not the same, or similar, approach will be adopted in the future has not yet been decided upon. Kauq, Sep 7, 2014
Comment K5 --Pfizer Oy, 2014-08-27 13:47:39 - R upload
Vaccine efficacy on non-typeable Haemophilus influenzae cannot be taken into account as comparison criterion because neither of the vaccines have prevention of NTHi in their indication. Moreover, for both vaccines the Summary of Product Characteristics (SmPC) approved by European Medicines Agency states in section 4.4., Special warnings and precautions, that the vaccines do not provide protection against other bacteria: Synflorix: There is insufficient evidence that Synflorix provides protection against pneumococcal serotypes not contained in the vaccine or against non-typeable Haemophilus influenzae. Synflorix does not provide protection against other micro-organism Prevenar 13: Prevenar13 will only protect against Streptococcus pneumonia serotypes included in the vaccine, and will not protect against other microorganism that cause invasive disease, pneumonia or otitis media. Based on the same sentences in SmPCs of the vaccines, the vaccines do not provide protection against other serotypes than those contained in the vaccines. Therefore, adjustability of the serotype composition in the economic assessment (http://en.opasnet.org/w/Economical_assessment ) is purposeless and the user defined option should be removed.
- ⇤Kauq2: Possible effects of pneumococcal conjugate vaccination on other than pneumococcal endpoints are not taken into account. This is basically due to lack of sufficient evidence to include such endpoints.
- Regarding the question about including other serotypes in the epidemiological model, we have retained to emphasise that this is a generic approach, not necessarily confined to the currently available vaccines. In the actual comparison of the two licensed vaccines, some assumptions have been based particularly on their reported efficacy and/or effectiveness (see Economic evaluation). Kauq, Sep 7, 2014
Comment K6 --Pfizer Oy, 2014-08-27 13:47:39 - R upload
According to the information on THL’s webpage, the duration of FinIP, Finnish Invasive Pneumococcal Disease Vaccine trial, is until the end of 2018. Would FinIP follow-up impact the tender? Ref. Lääkärilehti 22.8.2014, page 2017 Budjettileikkuri iskee rokotuksiin.
- ⇤Kauq3: FinIP as such would not impact the tender, apart from providing data applicable to both conjugate vaccine formulations (e.g. the assessment of the true incidence of pneumococcal invasive disease among <3 year olds; see Epidemiological modelling) Kauq, Sep 7, 2014
Comments about Question
- "Benefits refer to the decrease in disease burden caused by large scale use of the vaccine. As a result health care costs decrease and the quality of life attributed to improved health increases."
- ⇤GSK1 : Because there are no head-to-head studies which directly compare the effects of the vaccines, this leads to intrinsic uncertainties around the potential differences in the impact of individual formulations, and given that there are much less uncertainties around the cost of the vaccination programme, these two factors should be weighted accordingly.
- The selection criteria should also explicitly value the wealth of clinical evidence generated in randomized clinical trials, as well as in post marketing surveillances, including data generated in Finland itself, in addition to the use of modelling to predict the impact of vaccination. Assumptions used in the model should be reflective of this clinical evidence and applicants should be invited to make proposal for these assumptions with appropriate substantiation. -- GSK 12:58, 3 September 2014 (UTC)
- ⇤Kauq4: It is true that there is considerable uncertainty in the predicted impact of individual vaccine formulations. Because of this, extensive uncertainty analyses have been undertaken to identify the most influential assumptions leading to clearly different impacts of the two currently available conjugate vaccines. These most influential of assumptions concern the expected effectiveness of individual serotypes included in the vaccines (see the sensitivity analyses on pages Epidemiological modelling and Economic evaluation). Of note, although there are uncertainties in the estimated QALYs or medical costs, these affect both vaccines in their comparative evaluation.
- There is no 'uncertainty' as such regarding the cost of vaccination as these will be determined solely by the providers. Importantly, however, the overall cost effectiveness of pneumococcal conjugate vaccination in Finland may need to be understood on a broad scale, to appreciate the significance of any (predicted) differences between the impact of the vaccines to be compared.
- There is no indication in which respect data specific to Finland had not been taken into account. The model is based on evidence on the impact of large-scale and long-term use of pneumococcal conjugate vaccines, including their proven or inferred effect on (vaccine-type) transmission in Finland and elsewhere. The time-scale chosen for the analysis (i.e. 5-10 years in the future) as well as the fact that the overall impact in the Finnish population is assessed means that direct efficacy in the target population is only part of the answer. In particular, current status data on the epidemiology in any vaccinated population may provide only a ‘snap-shot’ of the underlying trends.
- Regarding the request of ‘appropriate substantiation’, these web pages provide the forum for further discussion. Kauq, Sep 6, 2014
Comments about Answer
- "Benefits are quantified as the expected decrease in invasive pneumococcal disease incidence due to vaccination."
- ⇤GSK2 : The Opasnet page entitled Tendering process for pneumococcal conjugate vaccine, section Question/Scope (http://en.opasnet.org/w/Tendering_process_for_pneumococcal_conjugate_vaccine) stipulates: “The preparation of the criteria is based on current knowledge of the impact of pneumococcal conjugate vaccination.”
- According to the current knowledge, notably supplemented recently with data generated in a double blind randomized clinical trial in Finland (e.g. Palmu, Lancet, 2013, Palmu Lancet Resp Med, 2013, Palmu ISRAOM, 2013, Kilpi ESPID, 2013), the benefits of reduction of invasive disease are representing a relatively minor fraction of the total infections prevented by PCVs.
- It has been demonstrated that majority of reductions is seen in respiratory infection episodes, especially mild upper respiratory infections. This translates to reductions of all healthcare costs, which are dominated by prevention of otitis media related outcomes. (Palmu et al. ESPID, 2014).
- Restricting the assessment of benefits to invasive disease fails to reflect the full benefits of PCV vaccination; thus reductions in pneumonia and acute otitis media related outcomes should be taken into account. --GSK 12:58, 3 September 2014 (UTC)
- ⇤Kauq5: In the more recent version of the model, the vaccine-preventable incidence of “IPD-like” disease, as reported by Palmu et al. (Lancet Resp Med, 2013) is taken into account in children <3 years of age. Other endpoints were not included in the analysis because of lack of reliable data on serotype-specific etiologic fractions, i.e. because the vaccine-preventable incidence was too uncertain. Moreover, the tendering process concerns comparison of two (or more) vaccines, in which case any assumptions made about the impact of different vaccines should be made on equal footing. This applies in particular to the vaccine-preventable incidences of disease.
- Of note, should all endpoints would be included in the analysis, PCV vaccination would likely be cost-saving. However, even then the relevant question in the tender process is how comparisons would be affected by the wider range of disease entities included in the analysis. Kauq, Sep 6, 2014
Comments about Rationale
- "To assess the health benefits of vaccination, the following items needs to be known or assessed: pneumococcal serotypes included in the vaccine..."
- ⇤GSK3 : Serotype content alone is an insufficient basis for vaccine comparisons, even if only IPD effects are considered.
- Based on efficacy and effectiveness data gathered to date, differences in the protection against overall IPD conferred by PCV10 or PCV13 are not clearly demonstrated.
- This is because overall IPD protection results from a combination of vaccine effect against vaccine and vaccine-related serotypes, along with the vaccine effect on non-vaccine-type replacement.
- For example, it is not possible to conclude on efficacy against serotype 3 based on the available body of evidence (e.g. Mrkvan, ICAAC 2013, Dagan, CID 2013; Ben-Shimol, CID 2014). In addition, emerging evidence indicates that vaccination with PCV10 is likely to offer significant protection against IPD due to vaccine-related serotypes 6A and 19A (up to 100% and 82% reductions, respectively; Jokinen WSPID 2013, Domingues, Lancet Resp Med 2014) have been observed post- PCV10 introduction to UMVs) and it is unclear whether this protection would substantively differ from that offered by PCV13, because of the absence of efficacy/effectiveness data generated in head-to-head studies.
- Therefore, using the pneumococcal serotypes included in the vaccine does not seem to meet the requirements of the public procurement law, which stipulates that the comparison criteria related to the differences must be clearly demonstrated. In fact using pneumococcal serotypes included in the vaccines as a selection criterion would put tenderers on an unequal footing without sufficient scientific basis and would therefore be contrary to the requirement of non-discriminatory treatment in procurement law.The vaccines should be rather evaluated based on the merits of the vaccine effects demonstrated in clinical trials and/or post licensure implementation. -- GSK 12:58, 3 September 2014 (UTC)
- ⇤Kauq6: As a general principle, serotypes commons in the two current conjugate vaccinates have been treated equally, i.e. we have assumed that they would induce the same very high level of overall effectiveness in the Finnish population. The question then relates to the three additional serotypes in PCV13 as compared to PCV10, as referred to in the above comment. In addition, cross-immunity against serotype 6C for PCV13 is considered. The tentative (and now updated) base-case model is based on (a) assuming only direct efficacy for serotype 3 in PCV13; (b) assuming direct efficacy for serotype 6A for PCV10, due to cross-immunity to 6B; (c) assuming direct efficacy for serotype 19A in PCV10, due to cross-immunity; (d) assuming direct efficacy for serotype 6C for PCV13, due to cross-immunity to 6A. The sensitivity of model predictions to these assumptions has been studied. The most influential assumption concern serotype 3 and serotypes 6A and 6C.
- In Finland, the health and economic impacts of pneumococcal conjugate vaccination have never been evaluated by the sheer number of serotypes included. Decisions have been made on the basis of a careful analysis of the expected effectiveness of conjugate vaccination. In particular, the epidemiological model is used to assess the full impact of PCV vaccination in the Finnish population. This requires assumptions on each serotype’s direct efficacy and/or impact on pneumococcal transmission, as well a model for serotype replacement.
- The model-based evaluation of PCV vaccination is based on the accumulated evidence from both vaccine efficacy and effectiveness in Finland and elsewhere. The eventual tender criteria will be based on this evaluation. Eventually, however, the outcome of the cost effectiveness analysis may need to be translated into corresponding (i.e. equivalent) tender criteria acceptable by to the law of public procurements. Kauq, Sep 6, 2014
- "The tentative assumption is that the use of either the 10-valent (PCV10) or 13-valent (PCV13) pneumococcal conjugate vaccine (PCV) in the childhood immunization programme significantly reduces the disease incidence attributable to the serotypes included in the vaccine and that this protection extends to the population at large. This assumption may be modified so that the vaccine also reduces disease incidence caused by certain serotypes not included in the vaccine (due to cross-protection). "
- ⇤GSK4 : In light of the comments above, as far as possible, the assumption for serotype specific vaccine effectiveness for vaccine and vaccine-related types should be based on available clinical evidence. -- GSK 12:58, 3 September 2014 (UTC)
- ⇤Kauq7: We refer to the assumptions as explained above. Kauq, Sep 6, 2014
- "For other pneumococcal disease endpoints, reliable estimates of the vaccine-preventable disease incidences are not available."
- ⇤GSK5 : Although the impact on other than invasive disease endpoints is more difficult to model, assessment of the impact of PCVs on respiratory diseases (including pneumonia and acute otitis media related endpoints) should be included because these are the major drivers of the cost effectiveness of PCV programmes and the scope of the procurement of PCV is to enable the choice of the economically most advantageous tender.
- It should be considered that vaccine preventable disease incidence estimates have been reported for a number of non-invasive disease endpoints, including hospital-diagnosed and hospital-treated pneumonia, tympanostomy tubes placements, antimicrobial purchases (largely related to otitis media treatment) in a nation-wide randomized clinical trials conducted in Finland recently (Palmu Lancet Resp Med, 2013, Palmu ISRAOM, 2013, Kilpi ESPID, 2013), and demonstrated that the benefits in terms of both episodes and associated costs reduced are comparatively more significant to invasive disease alone (Palmu, ESPID 2014, poster). -- GSK 12:58, 3 September 2014 (UTC)
- ⇤kauq8: We refer to our responses to an earlier comments above. Regarding otitis media, the serotype distribution is known to resemble closely that in nasopharyngeal carriage. Carriage in turn will be immediately replaced by non-vaccine type carriage. Put together, this means that the impact of PCV on otitis media is expected to be relatively small in the long run. Kauq, Sep 6, 2014
Other comments
--GSK6 : We acknowledge that inclusion of the non-invasive disease endpoints as a part of criteria for assessment of the vaccines requires to rely, in part, on assumptions where solid comparative data are missing. However, as the data from head-to-head studies are not available even for the IPD endpoints, such assumptions is in fact also needed if only IPD endpoints are considered, and thus, this uncertainty should not be a reason to restrict the assessment to invasive disease endpoint only.
The inclusion of pneumococcal conjugated vaccine into the national vaccination programme in Finland was supported by a health economic study, as detailed in the “Kansanterveyslaitoksen asettaman lasten pneumokokkirokotustyöryhmän selvitys, KTL, 2008”. The analysis indicated, as recently confirmed in a cluster randomized trial FinIP, that adding the benefits of prevention of non-invasive endpoints, both in terms of episodes and cost associated, prevail over the burden represented by invasive disease only. There is indirect evidence, generated in clinical studies, as well as in post marketing surveillances, suggesting that the available PCVs may differ in their capacity to prevent mucosal infections, e.g. otitis media, and therefore, the inclusion of criteria allowing to assess the impact of the vaccines on non-invasive endpoints should be explored, at the very least, in a sensitivity analysis of the cost effectiveness model. Given the high incidence and associated public health costs related to non-invasive infections, focusing the quality criteria on IPD endpoints only may not allow to meet the ultimate objective of the assessment, which is to choose the economically most advantageous tender. --Anna Hanttu, GSK 10:49, 12 September 2014 (UTC)
