Talk:Comparison criteria

Discussion on comparison criteria

Comment K3

1. PCV tender criteria from other Nordic countries show that quality is valued and given greater weight than price also where the winning vaccine is considered the “economic most advantageous”. Quality points should be given to several dimensions, not just for number of serotypes in the vaccine;

• see examples:

Norway (2011)
Quality 50-70%, price 20-40%, service after delivery 5% “The offer that achieves the highest point score after adding up the calculated points for each of the weighted criteria for assignment will be considered as the financially most favorable offer.”

Criteria: Requirement: It must be possible to store the vaccine at 25°C for at least 48 hours, without altering the vaccine properties. Documentation: stability studies The expected number of preventable cases of systemic pneumococcal disease. The efficacy of the vaccine against systemic pneumococcal disease caused by the different serotypes in the vaccine will be of significance. Other relevant conditions, including clinical documentation on the efficacy against otitis caused by S.pneumoniae. The best possible safety profile in this particular population. The offered vaccine may be part of a vaccination program where both available pneumococcal vaccines can be used. The prevalence in Norway of the individual disease-developing serotypes will be considered. The efficacy and risk/benefit assessment of the vaccines will be assessed on a rough estimate based on the presented documentation. Documentation assessed by competent authorities in the EEA-union will be of particular importance. Price pr. dose size is given as one price, which is applicable independently of the delivered dose, i.e. if it is delivered as one dose pack or as 10-dose packs. Service after delivery (5%) Service availability and technical assistance will be emphasized

Denmark (2014)
Quality 75%, price 25% “The framework contract will be awarded on the basis of the award criterion the economic most advantageous tender . Criteria: The effectiveness is assessed on the basis of the vaccine's coverage of invasive pneumococcal diseases in children < 2 years (35%) and the product's direct and indirect coverage of invasive pneumococcal diseases in the rest of the population (25%) in Denmark in 2009 based on the applicable SPC (Summary of Product Characteristics). The tenderer must complete a form concerning the indication of effectiveness of pneumococcal serotypes contained in the vaccine on the basis of the applicable SPC. Type and frequency of adverse effects (10%): assessed on the basis of the applicable SPC. As few and least invasive adverse effects as possible are desired. The active ingredients/trace elements of the products (5%): it is assessed whether the product is manufactured by or includes materials or trace elements of animal or human origin or other ingredients that may potentially pose a risk to the child/patient. The assessment is made on the basis of an applicable SPC, a completed form relating to active ingredients/trace elements enclosed with the tender documents as well as any statement as to no active ingredients and other relevant material, as described in section 10.4. As few active ingredients and trace elements that may potentially pose a risk to the child/the patients as possible are desired. The price will be assessed on the basis of the average price for the period 2014-2017. The lowest price possible is desired.

Sweden, Stockholm (2015)
Quality emphasized. If the tender gets 100 points for quality, the price will be multiplied by 1. If, for example, the tender gets 50 quality points, the price will be multiplied by 1,50. Criteria: Clinical efficacy, safety (max 90 points): Range of serotype protection, documentation on efficacy and effectiveness, documentation on risk groups, documentation on antibody response, side effect profile, excipients and preservatives. Very good safety profile, good clinical efficacy, optimal protection in terms of serotype coverage, documentation on risk groups: 90 points: Very good safety profile, good clinical efficacy but not optimal protection in terms of serotype coverage, no documentation on risk groups: 20 points Line width, practical management, labelling (max 10 points) Size, needles, design (risk of mix up), clear labelling and presence of bar code, storage conditions, handling in general

Sweden, VGR (Region Västra Götaland)
Quality 70%, price 30% Criteria: Clinical efficacy (max 60 p): Number of serotypes, clinical efficacy on individual level, proven herd immunity, approved indications. Based on RCT and/or other studies, official authority statements Very Good:48-60 points Good: 34-47 points Acceptable: 20-33 points Less good: 10-19 points Poor or not gradable: 0-9 points: Clinical safety (max 60 p): Side effect profile in SmPC/WHO, RCT and/or other studies, official authority statements. Aluminum content. Very good: 48-60 points Good: 34-47 points Acceptable: 20-33 points Less good: 10-19 points Poor or not gradable: 0-9 points Product range (max 10 p) Will be evaluated if relevant, otherwise maximum points will be awarded. Package size: 0-5 points Dosing schedule: 0-5 points Practical management (max 30 p) Will be evaluated if relevant, otherwise maximum points will be awarded Storage conditions (e.g. light sensitive, storage in cold, durability):0-10 points Formulation aspects (e.g. dry matter/vial/pre-filled syringe, preparation instructions): 0-5 points Product itself (e.g. manageability at the preparation and administration, packaging design): 0-15 points Marking and labelling (max 20 p) The bidder shall submit Mock-ups on the outer packaging and images on the inner packaging to the bid, and blister on all offered products. If need of pharmaceutical samples, the company will be contacted. Barcode (outer and inner packaging): 0-10 points Labelling (outer and inner packaging, readability on the syringe label, removable label): 0-10 points

Comment K4

It is appreciable Opasnet is a source of transparency. As JCVI and ACIP do, is it planned to inform the general public by publishing regularly the evaluation and decision processes, including analysis and results, at Opasnet or somewhere else?

Comment K5

Vaccine efficacy on non-typeable Haemophilus influenzae cannot be taken into account as comparison criterion because neither of the vaccines have prevention of NTHi in their indication. Moreover, for both vaccines the Summary of Product Characteristics (SmPC) approved by European Medicines Agency states in section 4.4., Special warnings and precautions, that the vaccines do not provide protection against other bacteria: Synflorix: There is insufficient evidence that Synflorix provides protection against pneumococcal serotypes not contained in the vaccine or against non-typeable Haemophilus influenzae. Synflorix does not provide protection against other micro-organism Prevenar 13: Prevenar13 will only protect against Streptococcus pneumonia serotypes included in the vaccine, and will not protect against other microorganism that cause invasive disease, pneumonia or otitis media. Based on the same sentences in SmPCs of the vaccines, the vaccines do not provide protection against other serotypes than those contained in the vaccines. Therefore, adjustability of the serotype composition in the economic assessment (http://en.opasnet.org/w/Economical_assessment ) is purposeless and the user defined option should be removed.

Comment K6

According to the information on THL’s webpage, the duration of FinIP, Finnish Invasive Pneumococcal Disease Vaccine trial, is until the end of 2018. Would FinIP follow-up impact the tender? Ref. Lääkärilehti 22.8.2014, page 2017 Budjettileikkuri iskee rokotuksiin.

Question:

• "Benefits refer to the decrease in disease burden caused by large scale use of the vaccine. As a result health care costs decrease and the quality of life attributed to improved health increases."

⇤# : Because there are no head-to-head studies which directly compare the effects of the vaccines, this leads to intrinsic uncertainties around the potential differences in the impact of individual formulations, and given that there are much less uncertainties around the cost of the vaccination programme, these two factors should be weighted accordingly.

The selection criteria should also explicitly value the wealth of clinical evidence generated in randomized clinical trials, as well as in post marketing surveillances, including data generated in Finland itself, in addition to the use of modelling to predict the impact of vaccination. Assumptions used in the model should be reflective of this clinical evidence and applicants should be invited to make proposal for these assumptions with appropriate substantiation. -- GSK 12:58, 3 September 2014 (UTC)

Anwer:

• "Benefits are quantified as the expected decrease in invasive pneumococcal disease incidence due to vaccination."

⇤# : The Opasnet page entitled Tendering process for pneumococcal conjugate vaccine, section Question/Scope (http://en.opasnet.org/w/Tendering_process_for_pneumococcal_conjugate_vaccine) stipulates: “The preparation of the criteria is based on current knowledge of the impact of pneumococcal conjugate vaccination.”

According to the current knowledge, notably supplemented recently with data generated in a double blind randomized clinical trial in Finland (e.g. Palmu, Lancet, 2013, Palmu Lancet Resp Med, 2013, Palmu ISRAOM, 2013, Kilpi ESPID, 2013), the benefits of reduction of invasive disease are representing a relatively minor fraction of the total infections prevented by PCVs.

It has been demonstrated that majority of reductions is seen in respiratory infection episodes, especially mild upper respiratory infections. This translates to reductions of all healthcare costs, which are dominated by prevention of otitis media related outcomes. (Palmu et al. ESPID, 2014).

Restricting the assessment of benefits to invasive disease fails to reflect the full benefits of PCV vaccination; thus reductions in pneumonia and acute otitis media related outcomes should be taken into account. --GSK 12:58, 3 September 2014 (UTC)

Rationale:

• "To assess the health benefits of vaccination, the following items needs to be known or assessed: pneumococcal serotypes included in the vaccine..."

⇤# : Serotype content alone is an insufficient basis for vaccine comparisons, even if only IPD effects are considered.

Based on efficacy and effectiveness data gathered to date, differences in the protection against overall IPD conferred by PCV10 or PCV13 are not clearly demonstrated.

This is because overall IPD protection results from a combination of vaccine effect against vaccine and vaccine-related serotypes, along with the vaccine effect on non-vaccine-type replacement.

For example, it is not possible to conclude on efficacy against serotype 3 based on the available body of evidence (e.g. Mrkvan, ICAAC 2013, Dagan, CID 2013; Ben-Shimol, CID 2014). In addition, emerging evidence indicates that vaccination with PCV10 is likely to offer significant protection against IPD due to vaccine-related serotypes 6A and 19A (up to 100% and 82% reductions, respectively; Jokinen WSPID 2013, Domingues, Lancet Resp Med 2014) have been observed post- PCV10 introduction to UMVs) and it is unclear whether this protection would substantively differ from that offered by PCV13, because of the absence of efficacy/effectiveness data generated in head-to-head studies.

Therefore, using the pneumococcal serotypes included in the vaccine does not seem to meet the requirements of the public procurement law, which stipulates that the comparison criteria related to the differences must be clearly demonstrated. In fact using pneumococcal serotypes included in the vaccines as a selection criterion would put tenderers on an unequal footing without sufficient scientific basis and would therefore be contrary to the requirement of non-discriminatory treatment in procurement law.The vaccines should be rather evaluated based on the merits of the vaccine effects demonstrated in clinical trials and/or post licensure implementation. -- GSK 12:58, 3 September 2014 (UTC)

• "The tentative assumption is that the use of either the 10-valent (PCV10) or 13-valent (PCV13) pneumococcal conjugate vaccine (PCV) in the childhood immunization programme significantly reduces the disease incidence attributable to the serotypes included in the vaccine and that this protection extends to the population at large. This assumption may be modified so that the vaccine also reduces disease incidence caused by certain serotypes not included in the vaccine (due to cross-protection). "

⇤# : In light of the comments above, as far as possible, the assumption for serotype specific vaccine effectiveness for vaccine and vaccine-related types should be based on available clinical evidence. --Jouni (talk) 12:58, 3 September 2014 (UTC)

• "For other pneumococcal disease endpoints, reliable estimates of the vaccine-preventable disease incidences are not available."

⇤# : Although the impact on other than invasive disease endpoints is more difficult to model, assessment of the impact of PCVs on respiratory diseases (including pneumonia and acute otitis media related endpoints) should be included because these are the major drivers of the cost effectiveness of PCV programmes and the scope of the procurement of PCV is to enable the choice of the economically most advantageous tender.

It should be considered that vaccine preventable disease incidence estimates have been reported for a number of non-invasive disease endpoints, including hospital-diagnosed and hospital-treated pneumonia, tympanostomy tubes placements, antimicrobial purchases (largely related to otitis media treatment) in a nation-wide randomized clinical trials conducted in Finland recently (Palmu Lancet Resp Med, 2013, Palmu ISRAOM, 2013, Kilpi ESPID, 2013), and demonstrated that the benefits in terms of both episodes and associated costs reduced are comparatively more significant to invasive disease alone (Palmu, ESPID 2014, poster). -- GSK Jouni (talk) 12:58, 3 September 2014 (UTC)