Research and analysis

Report of the Commission on Human Medicines Expert Working Group on COVID-19 vaccine safety surveillance

Published 5 February 2021

Summary

In May 2020, the Commission on Human Medicines established an Expert Working Group (EWG) to advise the Medicines and Healthcare products Regulatory Agency (MHRA) on its safety monitoring strategy for COVID-19 vaccine(s).

The EWG held four meetings from May to October 2020, during which it considered proposals and methodologies for MHRA-led vigilance activities. Based on this advice, the MHRA has developed, and now has in place, a four-stranded approach to vigilance, which is summarised in this report

Background

Since the emergence of the COVID-19 pandemic, research and development of candidate vaccines to protect against the SARS-CoV-2 virus has gathered pace at global level. In the UK, a Government Vaccine Task Force (VTF) has been established to expedite and co-ordinate efforts to research, produce and supply a COVID 19 vaccine.

Several vaccines have now been authorised for use, and many more are at an advanced stage of development, at global level. These are based on a range of technology, some of which is very well-established in other authorised vaccines (such as inactivated virus or purified protein subunits, with or without an adjuvant), some are based on viral vector platforms, including those used in recently-authorised vaccines (such as Ebola vaccine) and others are based on emerging mRNA technology.

In the UK, as of 14 January 2021, the Medicines and Healthcare products Regulatory Agency (MHRA) has authorised the supply of the Pfizer/BioNTech, the Oxford University/AstraZeneca and Moderna vaccines, following a thorough review of the safety, quality and efficacy. These are now being deployed in the UK in accordance with the recommendations of the UK Joint Committee on Vaccination and Immunisation (JCVI).

The need for post-authorisation vigilance

The intense focus, rapid funding, recruitment and prioritised regulatory oversight of trials at global level has allowed clinical trials for COVID-19 vaccines to proceed at pace, without compromising any of the usual, high standards of scientific rigour. In accordance with the usual requirements to support an authorisation of a new vaccine, tens of thousands of subjects have been included in trials and all are subject to very close safety follow-up over several months.

As with the development of any new vaccine or medicine, the size of clinical trials invariably means that very rare side effects can only be identified and/or fully characterised when the products are used in large populations. And certain groups who may benefit from, and be recommended to receive a vaccine, such as those with underlying chronic illnesses, may have been excluded from clinical trials.

It is for these reasons that post-authorisation, ‘real world’ safety vigilance of new vaccines and medicines is a crucial part of the product lifecycle and the public health programme. As well as authorising the use of new vaccines and medicines, the MHRA has statutory responsibility for undertaking post-authorisation safety monitoring in the UK. The MHRA also oversees the manufacturers’ legal responsibilities to undertake such vigilance.

Independent expert oversight of the MHRA’s activities

To inform its decision-making, the MHRA seeks independent expert advice from the Commission on Human Medicines (CHM). In May 2020, the CHM established an Expert Working Group (EWG), consisting of experts in medicine, infectious disease, pharmacoepidemiology and data analytics to provide the MHRA with independent oversight and advice on its COVID-19 vaccine vigilance activities.

The EWG held four meetings from May to October 2020, during which it considered proposals and methodologies for MHRA-led vigilance activities. Based on this advice, the MHRA has developed, and now has in place, a four-stranded approach to vigilance. To ensure the necessary communications, data flows and linkages are in place to fulfil these activities, the MHRA has worked in close collaboration with public health partners across the UK, including Public Heath England (PHE), the respective public health authorities in Scotland, Wales and Northern Ireland, as well as the Department for Health and Social Care (DHSC), NHSEI, NHSD and NHSX. The MHRA has also incorporated scientific collaboration with the NIHR-funded Health Protection Research Unit, within the London School of Hygiene and Tropical Medicine.

This collaborative approach harnesses collective expertise across the UK public health sector, and to make best use of the data sources and methodologies available, to implement a robust vigilance strategy.

This report summarises the activities that the MHRA will have in place for proactive vigilance of COVID-19 vaccines. Although this focuses on the post-marketing safety of the vaccines, the ‘real world’ effectiveness and population impact of the vaccine(s) are key to overall continuing benefit-risk balance and will include longevity of protection, any need for boosters and evaluation of other vaccine characteristics such as prevention of viral transmission.

Public Health England published its COVID-19 vaccine surveillance strategy on 11 January 2020 setting out how it will independently monitor these other important aspects of COVID-19 vaccines following their roll-out. This will be in accordance with the clinical recommendations of the JCVI.

Proactive vigilance for COVID-19 vaccines

Identifying side effects, and distinguishing these from coincidental medical events

Given the likely scale of a COVID-19 mass immunisation programme, with many millions of doses of one or more novel vaccines administered across the UK over a relatively short time period, vigilance needs to be continuous, proactive and as near real-time as is possible. The importance of this is two-fold.

First and foremost to rapidly detect, confirm, characterise and quantify any new risks that were not detected in clinical trials, to weigh these against the expected benefits and take any necessary action to minimise risks to individuals.

Secondly, it needs to be very quickly established if any serious events which are temporally-related to vaccination are merely a coincidental association, and to do this in a robust, evidence-based way so that public confidence in a vaccine is not eroded unnecessarily. Indeed, such associations may be more likely whilst we are still in the midst of a national epidemic, and because most of the millions of people offered the vaccine in the early phase of a vaccination campaign will be elderly and/or have underlying medical conditions, which increases the likelihood of unrelated illnesses occurring soon after vaccination.

Four main strands of our proactive vigilance

There are four strands to the MHRA’s strategy, which combine to address the relative strengths and weaknesses of each form of vigilance.

1. Enhanced passive surveillance – ‘observed vs expected’ analysis

The Yellow Card scheme underpins medicines and vaccines safety monitoring in the UK. Through this scheme, members of the public and healthcare professionals voluntarily submit reports of suspected side effects to the MHRA. Drug companies also submit such reports as part of their legal requirements. Safety scientists at the MHRA continuously evaluate Yellow Card reports to generate “signals” of potential safety issues. It is important to point out that just because a Yellow Card has been submitted, it does not necessarily mean that the vaccine caused the reaction – as outlined above, it may be also coincidental. The MHRA encourages anyone to report any suspicion or concern they have – reporters do not need to be sure of a link between a medicine or vaccine and a suspected side effect, and encouraged to report if in doubt. Every report is taken seriously, and we may get in contact reporters to obtain further information.

The MHRA has developed a dedicated COVID-19 interface to the Yellow Card scheme focused on the capture of suspected side effect reports for COVID-19 products, which will be expanded to include vaccines.

Our standard Yellow Card site, and mobile apps can also be used to report to us. Although paper-based Yellow Card reports are still accepted, the pandemic situation may delay access to such reports and on-line reporting is strongly recommended.

As with any system of safety vigilance, the ability to very rapidly detect a new safety concern in the midst of a mass immunisation campaign is dependent on the early presentation and diagnosis of symptoms. The key strength of the Yellow Card scheme is that it allows any member of the public or health professional across the UK to immediately alert us to any concerns they have without a formal diagnosis. And because anyone across the UK can report to the MHRA at any time, unlike studies which are limited in size, the scheme is able to identify the rarest of side effects.

A team of MHRA scientists will continually review individual reports and will contact reporters to obtain more information, where required. Scientific and clinical assessment will be used to determine if an individual or series of reports indicate a new safety ‘signal’. An established statistical approach known as empirical Bayes geometric mean (EBGM) will be used to facilitate signal detection.

Whilst Yellow Cards in isolation are sufficient to allow signal detection, the MHRA will enhance the system by analysing reports in the context of near real-time information on the number of doses of administered at the relevant time point, stratified by age and gender, and the background rate of the event of interest in the absence of vaccination. This will allow continuous evaluation of the ‘observed’ number of reports of a suspected serious side effect compared to ‘expected’ numbers – i.e. based on the naturally-occurring rate that would normally happen in a given time period in the same sized cohort and in the absence of vaccination.

The background rate used to estimate the expected numbers of cases will be extracted from anonymised GP electronic healthcare records and linked secondary care records within the Clinical Practice Research Datalink (CPRD) supported by additional analyses using full England-wide secondary care data for the rarest events. The MHRA will then continually compare the ‘observed’ vs ‘expected’ numbers to determine whether more events are occurring after the vaccine than we might expect by coincidence, and therefore whether it could signal a possible vaccine-related side effect. By applying a statistical method known as ‘MaxSPRT’7 to this analysis, we reduce the chance of false signals caused by repeated interrogation of the data. This is a vigilance approach now well-established within the MHRA for major new vaccines (HPV and 4CMenB).

Because every passive surveillance system suffers from variable under-reporting, the MHRA will conduct sensitivity analyses based on a range of under-reporting assumptions. Everyone receiving a vaccine should be provided with an information leaflet, which will provide a link to the Yellow Card site, and which should help to reduce any under-reporting.

2. Rapid Cycle Analysis and Ecological analysis

Any form of passive surveillance relies on someone suspecting or ‘making a connection’ between the medicine or vaccine and an unexplained illness, and then reporting it. It is important, therefore, that other forms of vigilance are included to supplement the Yellow Card scheme. Analysing anonymised electronic healthcare records that are routinely collected in clinical practice is one way to do this. The MHRA has access to CPRD data and routinely uses this in vaccine vigilance.

The CPRD Aurum dataset now captures daily data from ~20% of GP practices in England, now including 13 million currently registered patients. The advantage of supplementing vigilance activities with such data is that it does not rely on people directly reporting their concerns. But, unlike passive surveillance, a limitation of using electronic healthcare records for this purpose is that it relies on the timely and accurate recording or linkage in GP IT systems of vaccinations given, as well as any referrals/diagnoses for illness. It is therefore not as real-time as Yellow Card reporting for safety signal detection.

However, as COVID-19 vaccination records (i.e. those given outside of GP surgeries) begin to get updated within GP systems, the MHRA will implement a form of active surveillance known as ‘Rapid Cycle Analysis’. This method involves proactive, weekly analysis of a range of pre-defined events (theoretical side effects) to quickly identify safety signals – it again involves ‘observed vs expected’ analyses (i.e. comparing rates after vaccination to rates in unvaccinated comparator groups) but doesn’t rely on people directly reporting any concerns through the Yellow Card scheme. It is also a more robust way to quickly determine if rates are likely to be consistent with a coincidental association. It also uses the MaxSPRT approach with adjustments made for the expected delays in the recording of events presenting to and diagnosed in secondary care settings. The list of pre-defined events of special interest is not fixed and can be expanded at any time.

The MHRA will also use the CPRD data to conduct ‘ecological analyses’. This involves monitoring trends in the rates of pre-defined events within given population cohorts, based on prioritisation groups for vaccine roll out, to see if they are occurring to a greater extent amongst those targeted for vaccination after it is deployed compared to historical rates from the pre-deployment period. Comparisons can also be made to trends seen in groups not targeted for vaccination at the same time. This approach is most useful when we see high vaccine uptake and is another way to quickly detect a potential safety signal.

Each of these methods will need very careful evaluation to tease out any change in rates over time that may be a direct or indirect consequence of the SARS-CoV-2 epidemic, rather than an effect of the vaccine.

3. Targeted active monitoring – Yellow Card Vaccine Monitor

Another form of vigilance that the MHRA will implement is targeted active monitoring of certain groups of vaccinees, focused particularly on those who may have been excluded or under-represented in clinical trials. Through the call/recall system which the NHS will use to invite people to register to receive the vaccine, a random selection of vaccinees from certain cohorts will be invited to voluntarily register for follow-up via a new platform, called the Yellow Card Vaccine Monitor, which the MHRA has developed.

This vigilance activity will seek enrolment prior to vaccination (and thereby before any suspected side effect is experienced) and vaccinees will then be contacted at set intervals (for example 7 days, 28 days, 3-6 months) to ask whether any adverse reaction occurred. The objective of this is not necessarily to detect very rare risks, as the intention is to recruit the same numbers that are generally included in a clinical trial (i.e. several thousand), but to compare the frequency and severity of side effects to groups that were included in trials to allow further characterisation of the safety profile. This would allow, for example, further evaluation of the safety profile in people with underlying immunosuppression.

4. Formal epidemiological studies

The above three methods are essentially ‘signal detection’ and ‘signal strengthening’ tools – i.e. their main purpose is to quickly flag up whether there might be a new, rare side effect and to build the volume of data on safety. They cannot confirm if it is a side effect. Similarly, whilst they can provide some strong evidence to indicate if something is likely to be coincidental, they can not always confirm this. A formal epidemiological study, designed and powered specifically to test a given hypothesis in an unbiased way, is usually necessary to confirm and quantify a suspected rare side effect. These will be undertaken on an ad hoc basis should the need arise based on other vigilance activities.

Examples of such studies undertaken by the MHRA in the past include the association between human papillomavirus (HPV) vaccine and chronic fatigue syndrome and the safety of pertussis vaccine in pregnancy (pertussis and HPV.

There are a number of data sources and study designs that could be utilised for generating robust evidence regarding specific risks should this be required. It is important that for any specific issue the strongest data set for further evaluating the risk is identified. This will be dependent upon the nature of the potential risk that has been identified. The MHRA can make direct use of the CPRD data. Should the signal originate from our analyses of CPRD Aurum data mentioned above, use of alternative data bases would be preferred in the first instance (for example, through OpenSafely) although use of the CPRD Gold data set15 (which differs from CPRD Aurum in that it contains data contributed by GP practices using the Vision® rather than EMIS Web® electronic patient record system) and inclusion of data from linked secondary care data would help mitigate concerns of hypothesis testing in the same data to which the hypothesis was generated. PHE also have a long record of conducting epidemiological studies using active data collection methods and secondary care data through Hospital Episode Statistics (HES). Studies can be triggered by both the MHRA and PHE using established processes.

The self-controlled case-series method was specially designed for rapid unbiased assessment of vaccine safety issues. In this approach, cases act as their own controls as the incidence of the event of interest in pre-defined risk-periods following vaccination is compared to the incidence outside the risk period. However, as with the choice of data set it is important that the most appropriate study design is used for the issue identified.

Engaging with academia and other experts

The conduct of independent studies is also highly valuable and so the MHRA are working with PHE and the Health Protection Research Unit in Immunisation at LSHTM to establish a framework for the rapid conduct of epidemiological studies in OpenSAFELY17. A template protocol is being written which will allow the investigation of key theoretical adverse events in the first instance and which can be rapidly updated to include additional events if the need arise.

The plans described in this report may be further adapted and extended and the MHRA continues to have dialogue with individual experts on surveillance plans. This may include incorporating additional methods, data sources or further collaboration with other UK and international academic partners into these plans.

What the MHRA does with the data we generate

The main objective of the safety monitoring process is to identify any new risks that may emerge as the vaccines are used. Such risks could include a new side effect, an apparent change in the nature of a known side effect, identification of factors that increase the chances of having a side effect, batch-related problems or issues related to inappropriate use of the vaccines.

If a new risk is confirmed, this will be fed into a continuous evaluation by the MHRA of the balance of benefits of a vaccine versus risks. The MHRA will consult the Commission on Human Medicines (CHM) and its Expert Groups and, if deemed necessary, regulatory action would be taken to minimise risk and support safe use of a given vaccine (e.g. adding warnings to the product information, sending out communications to healthcare professionals and patients, restricting its use). This would also be communicated to DHSC, PHE, devolved Governments and public health partners in the devolved nations to inform any decisions regarding the immunisation programme.

What information the MHRA will provide to the public on vaccine safety

The MHRA will operate a transparent process. On a regular basis, the MHRA will produce an up to date summary of the safety experience, including aggregate Yellow Card reports, on our website.