Michael Colacci, MD, Michael Fralick, MD, PhD, SM
The safety of a newly approved medication is based primarily on the results of preapproval clinical trials.1,2 This can be problematic because it means that many medications are approved according to the results of 1 or 2 clinical trials. These trials typically enrol fewer than 1000 participants, who are often healthier than patients in routine clinical care.3 In addition, although preapproval trials may accurately estimate the rate of common adverse events, rare and serious adverse events may go undetected.4–6 Therefore, postmarketing safety studies are needed to identify potential rare adverse events associated with newly approved medications. This practice is referred to as pharmacovigilance, which the World Health Organization has defined as “the science and activities relating to the detection, assessment, understanding and prevention of adverse effects or any other possible drug-related problem.”7
In Canada, pharmacovigilance occurs predominantly through spontaneous reporting to Health Canada.8 Adverse reactions can be reported—by patients, health care professionals, or drug manufacturers— to the Canada Vigilance Adverse Reaction Online Database, which is similar to the Adverse Event Reporting System of the US Food and Drug Administration (FDA).9 Many adverse drug reactions have been detected through this mechanism; however, there are limitations to this approach.10 First, reporting is voluntary and thus prone to selection bias. Second, the quality and completeness of reports are highly variable.11 Third, because the total number of patients who received the drug (i.e., the denominator) is not reported, the relative and absolute risks cannot be accurately quantified. For example, in 2012, after dabigatran was approved, spontaneous reports of bleeding associated with this drug greatly outnumbered reports of bleeding associated with warfarin use.11 This might suggest a higher rate of bleeding with dabigatran than with warfarin. However, a cohort study of more than 140 000 patients conducted in response to these reports showed that the rate of bleeding was about 2-fold higher with warfarin than with dabigatran.11 Thus, the higher number of spontaneous reports associated with dabigatran resulted from reporting bias, likely because dabigatran was a new medication and warfarin was not.
An alternative to reliance on spontaneous reporting for pharmacovigilance is the use of data mining. This method uses advanced statistical methods to identify patterns and associations in large data sets. In contrast to a traditional research study, in which the researcher starts with a hypothesis and designs a study to test it, data mining involves a data-driven process in which the researcher “lets the data speak for themselves”. The data sources may include health care databases (e.g., those held by ICES), prospective registries, or electronic health records.10 Several data-mining approaches exist, including tree-based statistical scanning (described in more detail below), Gamma Poisson Shrinker, and text mining through natural language processing.12,13
The tree-based scan statistic has been applied in North America. TreeScan, one of the more common data-mining software programs, was developed specifically for pharmacovigilance and was first introduced in 2013.12 The term “tree” refers to the hierarchical grouping of related diagnostic codes used with this approach.12 For example, within the International Statistical Classification of Diseases and Related Health Problems (ICD) coding system, the code I21 (acute myocardial infarction) is considered to be a branch. The codes I21.0 (ST-elevation myocardial infarction of the anterior wall) and I21.4 (non-ST elevation myocardial infarction) are considered to be sub-branches. These codes can be further specified to the second decimal digit (e.g., I21.01 for ST-elevation myocardial infarction of the left-anterior descending artery); this terminal level is referred to as a leaf. One strength of TreeScan is that the investigator does not have to specify a priori the outcome of interest or the level of detail of the outcome. Instead, TreeScan evaluates data across all possible branches, sub-branches, and leaves to identify potential adverse events.12
Recently, TreeScan was used to identify potential adverse events associated with the quadrivalent human papillomavirus (HPV) and live attenuated herpes zoster vaccines.14,15 Among 1.9 million people who received the HPV vaccine, and across 6551 potential ICD codes, TreeScan identified only 2 potential signals: cellulitis and complications of the injection.14 Similarly, in a study of more than 1.2 million herpes zoster vaccinations, TreeScan found that local skin reactions and skin infections were the only statistically significant adverse events.15
Data mining has several advantages over spontaneous reporting systems. First, it leverages large sample sizes, which allows for the detection of rare adverse events.5 Second, it does not require a priori (hypothesis-free) knowledge of a potential association between a medication and an adverse event.12 This advantage is particularly important given that knowledge of potential adverse events is often limited when a drug first enters the market, and it therefore allows for comprehensive evaluation of all possible adverse events. Third, in the case of TreeScan, all results are adjusted for multiple-hypothesis testing, to limit the number of potential false signals.12
Data mining also has important limitations. First, it often uses ICD codes; therefore, associations can be measured only for diagnoses with a relevant ICD code. Second, the validity of ICD codes is variable depending on the diagnosis. Third, data mining produces statistical association signals that may not represent true adverse events (e.g., because of confounding).12 Therefore, signals detected from data mining should be formally evaluated with directed pharmacoepidemiologic studies (e.g., new-user active-comparator cohort study).5
In 2017, the FDA released the Sentinel Initiative: Final Assessment Report, which outlined how the agency planned to modernize the process of postmarketing drug safety surveillance, including through implementation of TreeScan and other data-mining tools.16 In Canada, the Drug Safety and Effectiveness Network (established by the Canadian Institutes of Health Research) created CNODES, the Canadian Network for Observational Drug Effect Studies, in 2011, which is able to access data for millions of patients across the country. CNODES now plays an essential role by conducting pharmacoepidemiologic studies in response to requests from Health Canada. A natural extension of this work would be the incorporation of TreeScan or another data-mining technique to advance the current process of pharmacovigilance in Canada with the ultimate goal of preventing adverse events.
1 Mostaghim SR, Gagne JJ, Kesselheim AS. Safety related label changes for new drugs after approval in the US through expedited regulatory pathways: retrospective cohort study. BMJ. 2017;358:j3837.
2 Downing NS, Shah ND, Aminawung JA, Pease AM, Zeitoun JD, Krumholz HM, et al. Postmarket safety events among novel therapeutics approved by the US Food and Drug Administration between 2001 and 2010. JAMA. 2017;317(18):1854–63.
3 Downing NS, Aminawung JA, Shah ND, Krumholz HM, Ross JS. Clinical trial evidence supporting FDA approval of novel therapeutic agents, 2005–2012. JAMA. 2014;311(4):368–77.
4 Fralick M, Juurlink DN, Marras T. Bleeding associated with coadministration of rivaroxaban and clarithromycin. CMAJ. 2016;188(9):669–72.
5 Fralick M, Schneeweiss S, Patorno E. Risk of diabetic ketoacidosis after initiation of an SGLT2 inhibitor. N Engl J Med. 2017;376(23):2300–2.
6 Fralick M, Macdonald EM, Gomes T, Antoniou T, Hollands S, Mamdani MM, et al.; Canadian Drug Safety and Effectiveness Research Network. Co-trimoxazole and sudden death in patients receiving inhibitors of renin-angiotensin system: population based study. BMJ. 2014;349:g6196.
7 Essential medicines and health products: Pharmacovigilance [website]. World Health Organization; 2004 [cited 2020 Jan 15]. Available from: www.who.int/medicines/areas/quality_safety/safety_efficacy/pharmvigi/en
8 Canada vigilance adverse reaction online database. Government of Canada, 2020 [cited 2020 Jan 15]. Available from https://www.canada.ca/en/health-canada/services/drugs-health-products/medeffect-canada/adverse-reaction-database.html
9 An introduction to drug safety surveillance and the FDA Adverse Event Reporting System. US Food and Drug Administration; 2018 [cited 2020 Jan 15]. Available from: https://www.fda.gov/about-fda/fda-pharmacy-student-experiential-program/introduction-drug-safety-surveillance-and-fda-adverse-event-reporting-system/
10 Harpaz R, DuMouchel W, Shah NH, Madigan D, Ryan P, Friedman C. Novel data mining methodologies for adverse drug event discovery and analysis. Clin Pharmacol Ther. 2012;91(6):1010–21.
11 Southworth MR, Reichman ME, Unger EF. Dabigatran and postmarketing reports of bleeding. N Engl J Med. 2013;368(14):1272–4.
12 Kulldorff M, Dashevsky I, Avery TR, Chan AK, Davis RL, Graham D, et al. Drug safety data mining with a tree-based scan statistic. Pharmacoepidemiol Drug Saf. 2013;22(5):517–23.
13 Brown JS, Petronis KR, Bate A, Zhang F, Dashevsky I, Kulldorff M, et al. Drug adverse event detection in health plan data using the Gamma Poisson Shrinker and comparison to the tree-based scan statistic. Pharmaceutics. 2013;5(1):179–200.
14 Yih WK, Maro JC, Nguyen M, Baker MA, Balsbaugh C, Cole DV, et al. Assessment of quadrivalent human papillomavirus vaccine safety using the self-controlled tree-temporal scan statistic signal-detection method in the sentinel system. Am J Epidemiol. 2018;187(6):1269–76.
15 Yih WK, Kulldorff M, Dashevsky I, Maro JC. Using the self-controlled tree-temporal scan statistic to assess the safety of live attenuated herpes zoster vaccine. Am J Epidemiol. 2019;188(7):1383–8.
16 Sentinel initiative: final assessment report. US Food and Drug Administration; 2017 Sep [cited 2020 Jan 15]. Available from: https://www.fda.gov/media/107850/download
1 Fornasier G, Francescon S, Leone R, Baldo P. An historical overview over pharmacovigilance. Int J Clin Pharm. 2018;40(4):744–7.
2 The Canadian tragedy: the tragedy of thalidomide in Canada. Thalidomide Victims Association of Canada; [cited 2020 Jan 15]. Available from: https://thalidomide.ca/en/the-canadian-tragedy/
3 About thalidomide. Thalidomide Trust; [cited 2020 Jan 15]. Available from: https://www.thalidomidetrust.org/about-us/about-thalidomide/
4 Webb J. Canadian thalidomide experience. Can Med Assoc J. 1963; 89(19):987–92.
5 Caster O, Edwards IR. Reflections on attribution and decisions in pharmacovigilance. Drug Saf. 2010;33(10):805–9.
6 Avery AJ, Anderson C, Bond CM, Fortnum H, Gifford A, Hannaford PC, et al. Evaluation of patient reporting of adverse drug reactions to the UK ‘Yellow Card Scheme’: literature review, descriptive and qualitative analyses, and questionnaire surveys 2011. Health Technol Assess. 2011; 15(20):1–234.
7 McLernon DJ, Bond CM, Hannaford PC, Watson MC, Lee AJ, Hazell L, et al; Yellow Card collaboration. Adverse drug reaction reporting in the UK: a retrospective observational comparison of Yellow Card reports submitted by patients and healthcare professionals. Drug Saf. 2010;33(9): 775–88.
8 Essential medicines and health products: Pharmacovigilance [website]. World Health Organization; [cited 2020 Jan 14]. Available from: www.who.int/medicines/areas/quality_safety/safety_efficacy/pharmvigi/en/
9 Reporting adverse drug reactions: definitions of terms and criteria for their use. Council for International Organizations of Medical Sciences; 1999 [cited 2020 Jan 20]. Available from: https://cioms.ch/wp-content/uploads/2017/01/reporting_adverse_drug.pdf
10 Hazell L, Shakir SA. Under-reporting of adverse drug reactions: a systematic review. Drug Saf. 2006;29(5):385–96.
11 Evaluation Directorate. Appendix C: Supplementary data tables. In: Evaluation of the human drugs program 1999–2000 to 2011–2012. Health Canada and Public Health Agency of Canada; 2014 [cited 2020 Jan 15]. Available from: https://www.canada.ca/en/health-canada/corporate/about-health-canada/accountability-performance-financial-reporting/evaluation-reports/evaluation-human-drugs-program-1999-2000-2011-2012.html#appendixc
12 Domperidone: risks of cardiac side effects. Medicines and Healthcare Products Regulatory Agency [UK]; 2014 Dec 11 [cited 2020 Jan 13]. Available from: https://www.gov.uk/drug-safety-update/domperidone-risks-of-cardiac-side-effects
13 FDA warns about serious breathing problems with seizure and nerve pain medicines gabapentin (Neurontin, Gralise, Horizant) and pregabalin (Lyrica, Lyrica CR) in patients with respiratory risk factors. US Food and Drug Administration; 2019 Dec 19 [cited 2020 Jan 13]. Available from: https://www.drugs.com/fda/fda-warns-serious-breathing-problems-seizure-nerve-pain-medicines-gabapentin-neurontin-gralise-14336.html
14 Contribution of Yellow Cards to identifying safety issues. Medicines and Healthcare Products Regulatory Agency [UK]; [updated 2020 Jan; cited 2020 Mar 23]. Available from: https://yellowcard.mhra.gov.uk/the-yellow-card-scheme
15 Avandia diabetes drug suspended. UK National Health Service; 2010 Sep 24 [cited 2020 Jan 13]. Available from: https://www.nhs.uk/news/diabetes/avandia-diabetes-drug-suspended/
16 Metters J, chairman. Report of an independent review of access to the Yellow Card Scheme. The Stationery Office [UK]; 2004 Apr [cited 2020 Mar 24]. Available from: https://solidarites-sante.gouv.fr/IMG/pdf/Report_of_an_Independent_Review_of_Access_to_the_Yellow_Card_Scheme.pdf
17 Educational support for mandatory reporting of serious ADRs and MDIs by hospitals. Canadian Patient Safety Institute; [cited 2020 Jan 13]. Available from: https://www.patientsafetyinstitute.ca/en/toolsResources/Vanessas-Law/Pages/default.aspx
Competing interests: None declared. ( Return to Text )
Christine M Bond, BPharm, PhD, MEd
It has been suggested that the dawn of pharmacovigilance occurred in 1848, when a young English girl died after undergoing chloroform-induced anesthesia.1 As a result of this and other anesthetic-related deaths, The Lancet established a commission exhorting all doctors to report any deaths associated with anesthesia. Formal systems were established in the United States in 1906, after the Pure Food and Drug Act was passed. Its successor, the Federal Food, Drug, and Cosmetic Act (1938), ruled that the safety of all drugs should be demonstrated before marketing.
The wake-up call of the thalidomide tragedy occurred in the 1950s, the first example of an effective licensed medicine having widespread, serious adverse effects. First marketed in 1956 in West Germany as a sedative and hypnotic, thalidomide was also strongly promoted to treat nausea in early pregnancy. Ultimately, it was prescribed in 46 countries, including Canada. Somewhat ironically, though, the US Food and Drug Administration (FDA) withheld approval because of a lack of evidence of safety in pregnancy, as identified by Dr Frances Kelsey (a Canadian doctor working for the FDA as a pharmacist).2 In 1959, the first cases of congenital deformities—involving not only limbs but also internal organs—were reported. Initially, the manufacturers denied the possibility of any causal association, but the evidence became overwhelming and the drug was withdrawn: in Germany and the United Kingdom in December 1961, and in Canada in March 1962. This was not in time to prevent the estimated 10 000 cases of affected children worldwide,3 including more than 100 in Canada.4 Had there been in place systems of pharmacovigilance to indicate a link between medicine taken by the mother and effects on her unborn child, actions could have been taken earlier to alert doctors to the potential risks.5 The disaster triggered the establishment, worldwide, of national systems of licensing and safety monitoring for all medicines.
In Canada, legislation regarding the control of new drugs was reinforced in late 1962,2 and the Canadian Adverse Drug Reaction Information System was established in 1965. Now, consumers, health care professionals, and product manufacturers can report suspected adverse events to the Canada Vigilance Adverse Reaction Online Database (https://www.canada.ca/en/health-canada/services/drugs-health-products/medeffect-canada/adverse-reaction-database.html).
In the United Kingdom, 1963 saw the establishment of the Committee on Safety of Drugs (renamed the Committee on Safety of Medicines in 1970), and in 1964 letters were circulated to all doctors and dentists asking them to report “any untoward condition in a patient which might be the result of drug treatment”. This was the precursor of the current Yellow Card Scheme, so called because in its original incarnation, reports were prepared on a yellow card. Indeed, these yellow cards are still used, although much of the reporting is now done online. Since the scheme was introduced, reporting rights have been given to other health care professionals, initially nurses and pharmacists and now any health care professional. In 2004, patient reporting was introduced, on the assumption that it would increase the number of reports and lead to earlier detection of signals. There were concerns that patient reports might be less valid, and hence create false signals from background noise, but this has not proved to be the case.6,7
International collaborations were also established, increasing the sample size of exposed individuals. In 1968, the World Health Organization (WHO) instituted its Programme for International Drug Monitoring.8 Participation has grown from an initial 10 countries to about 150 countries, all of whom are eligible to submit reports of adverse reactions associated with medicinal products to the program’s global database, VigiBase. In 2001 the European Agency for the Evaluation of Medicinal Products and the European Commission developed a single European database, EudraVigilance, to which all member states must submit any details of “serious” reports, as defined by the Council for International Organizations of Medical Sciences.9
Currently, although there are differences between national schemes in terms of eligibility to report and what to report, all of the above approaches, however systematically introduced, whether voluntary or mandatory, depend on a system known as spontaneous reporting. This has been much criticized for underreporting, even in countries where reporting is mandatory, such as Sweden, France, and Italy. Indeed a systematic review of 37 studies conducted in 12 countries suggested a median underreporting rate of 94% (range 6%–100%).10 In Canada, although more than 90% of pharmacists and 63% of physicians were aware of how to report an adverse reaction, this proportion was reduced to just 55% for health professionals overall.11
Despite a certain level of under-reporting, this is not the time to abandon a well-established system that has prevented another disaster on the scale of thalidomide. Because of the level of detail requested at the point of reporting, generation of an adverse event signal need not necessarily result in withdrawal of a useful drug, but there will be warnings about use. For example, a warning might refer to contraindications, such as the recent restriction of domperidone to people over 12 years of age,12 because of a lack of evidence of benefit in younger children, or the recommendation that gabapentin not be prescribed to patients with respiratory risk factors.13 Some warnings may relate to drug-drug interactions, such as the interaction between fluconazole and citalopram causing serious cardiovascular events, or food-drug interactions, such as the interaction between grapefruit juice and a range of common medicines.14 Sometimes a medicine will be withdrawn completely; examples have included both prescribed medicines (e.g., rosiglitazone, because of cardiovascular effects15) and non-prescribed over-the-counter or herbal medicines (e.g., Aristolochia in Chinese medicines, because of renal failure).16
As premarketing safety assessments become more rigorous and well informed, we can hope that drug withdrawals will become less common. However, premarketing exposure to a drug is limited to perhaps hundreds of people, and it remains likely that rare and potentially fatal events may only be identified once thousands of people are using the drug. Any system can always be improved, but that is no reason to discard it. Efforts are needed to increase professional and public engagement with current spontaneous reporting systems. Approaches could include better education, individualized feedback, multiple reporting routes, and local initiatives. New approaches linked to big data may also provide complementary information but should not replace current systems.
In Canada, the Protecting Canadians from Unsafe Drugs Act, also known as Vanessa’s Law,17 will strengthen Canada’s ability to collect information and make decisions about potential health risks from treatments. It is now mandatory for hospitals to report serious adverse events related to drugs and devices within 30 days after first documentation of the event (reporting by manufacturers was already mandatory). Multiple reporting routes are available. As experts in medicines, pharmacists must ensure adherence with the new law, so that patients can continue to take medicines as needed, in the knowledge that effective surveillance systems are in place.
1 Fornasier G, Francescon S, Leone R, Baldo P. An historical overview over pharmacovigilance. Int J Clin Pharm. 2018;40(4):744–7.
2 The Canadian tragedy: the tragedy of thalidomide in Canada. Thalidomide Victims Association of Canada; [cited 2020 Jan 15]. Available from: https://thalidomide.ca/en/the-canadian-tragedy/
3 About thalidomide. Thalidomide Trust; [cited 2020 Jan 15]. Available from: https://www.thalidomidetrust.org/about-us/about-thalidomide/
4 Webb J. Canadian thalidomide experience. Can Med Assoc J. 1963; 89(19):987–92.
5 Caster O, Edwards IR. Reflections on attribution and decisions in pharmacovigilance. Drug Saf. 2010;33(10):805–9.
6 Avery AJ, Anderson C, Bond CM, Fortnum H, Gifford A, Hannaford PC, et al. Evaluation of patient reporting of adverse drug reactions to the UK ‘Yellow Card Scheme’: literature review, descriptive and qualitative analyses, and questionnaire surveys 2011. Health Technol Assess. 2011; 15(20):1–234.
7 McLernon DJ, Bond CM, Hannaford PC, Watson MC, Lee AJ, Hazell L, et al; Yellow Card collaboration. Adverse drug reaction reporting in the UK: a retrospective observational comparison of Yellow Card reports submitted by patients and healthcare professionals. Drug Saf. 2010;33(9): 775–88.
8 Essential medicines and health products: Pharmacovigilance [website]. World Health Organization; [cited 2020 Jan 14]. Available from: www.who.int/medicines/areas/quality_safety/safety_efficacy/pharmvigi/en/
9 Reporting adverse drug reactions: definitions of terms and criteria for their use. Council for International Organizations of Medical Sciences; 1999 [cited 2020 Jan 20]. Available from: https://cioms.ch/wp-content/uploads/2017/01/reporting_adverse_drug.pdf
10 Hazell L, Shakir SA. Under-reporting of adverse drug reactions: a systematic review. Drug Saf. 2006;29(5):385–96.
11 Evaluation Directorate. Appendix C: Supplementary data tables. In: Evaluation of the human drugs program 1999–2000 to 2011–2012. Health Canada and Public Health Agency of Canada; 2014 [cited 2020 Jan 15]. Available from: https://www.canada.ca/en/health-canada/corporate/about-health-canada/accountability-performance-financial-reporting/evaluation-reports/evaluation-human-drugs-program-1999-2000-2011-2012.html#appendixc
12 Domperidone: risks of cardiac side effects. Medicines and Healthcare Products Regulatory Agency [UK]; 2014 Dec 11 [cited 2020 Jan 13]. Available from: https://www.gov.uk/drug-safety-update/domperidone-risks-of-cardiac-side-effects
13 FDA warns about serious breathing problems with seizure and nerve pain medicines gabapentin (Neurontin, Gralise, Horizant) and pregabalin (Lyrica, Lyrica CR) in patients with respiratory risk factors. US Food and Drug Administration; 2019 Dec 19 [cited 2020 Jan 13]. Available from: https://www.drugs.com/fda/fda-warns-serious-breathing-problems-seizure-nerve-pain-medicines-gabapentin-neurontin-gralise-14336.html
14 Contribution of Yellow Cards to identifying safety issues. Medicines and Healthcare Products Regulatory Agency [UK]; [updated 2020 Jan; cited 2020 Mar 23]. Available from: https://yellowcard.mhra.gov.uk/the-yellow-card-scheme
15 Avandia diabetes drug suspended. UK National Health Service; 2010 Sep 24 [cited 2020 Jan 13]. Available from: https://www.nhs.uk/news/diabetes/avandia-diabetes-drug-suspended/
16 Metters J, chairman. Report of an independent review of access to the Yellow Card Scheme. The Stationery Office [UK]; 2004 Apr [cited 2020 Mar 24]. Available from: https://solidarites-sante.gouv.fr/IMG/pdf/Report_of_an_Independent_Review_of_Access_to_the_Yellow_Card_Scheme.pdf
17 Educational support for mandatory reporting of serious ADRs and MDIs by hospitals. Canadian Patient Safety Institute; [cited 2020 Jan 13]. Available from: https://www.patientsafetyinstitute.ca/en/toolsResources/Vanessas-Law/Pages/default.aspx
Competing interests: Christine Bond has received grants from the University of Aberdeen to evaluate patients’ reporting to the Yellow Card system. She was also a member of a group that undertook an independent review of access to the Yellow Card system in 2004 (cited as reference 16 in the current article). ( Return to Text )
Dr Bond is also an Associate Editor with the Canadian Journal of Hospital Pharmacy. ( Return to Text )
Canadian Journal of Hospital Pharmacy, VOLUME 73, NUMBER 2, March-April 2020