This text is provided for reference. The official version is published by the Turkish Medicines and Medical Devices Agency (TİTCK).
English translation of the TİTCK guideline (version dated 20.12.2016). The Turkish original is the legally binding text.
Part I – Introduction
1.1. Introduction
The purpose of this Module is to provide general guidance on structure and processes of signal management. To this end, signals obtained from monitoring of the data from spontaneous reporting systems shall be the starting point of signal management process. The same principles apply to data obtained from other sources.
In order to better understand the signal management process, it is necessary to consider the following three basic elements:
1. The starting point of signal detection is a certain amount of ambiguity and always requires a reasonable judgment. The signal probability can be at different levels depending on the sufficient nature and strength of the given data. Evidence can be found in different sources, it can have different strengths and it can accumulate and be collected over time. 2. Signal is a hypothesis. It does not show that there is a definite causal association in all cases. For example; The European Medicines Agency (EMA) evaluated 2372 possible signals in 2015, of which only 61 have been verified. 3. Not all signs are risky, and after the necessary examinations are done, there may not be a need for additional regulatory formal action (e.g., Summary of Product Characteristics (SmPC) updates) for all of the signals. In this Module, the methods used in signal analysis will also be outlined in order to provide guidance for marketing authorization holders.
1.1.1. Terminology
Signal: Information arising from one or multiple sources, including observations and experiments, which suggests a new potentially causal association, or a new aspect of a known association between an intervention and an event or set of related events, either adverse or beneficial, that is judged to be of sufficient likelihood to justify verificatory action [Article 4(1)s of the Regulation on Drug Safety].
New aspects of a known association may include changes in the frequency, duration, severity or outcome of the adverse event.
A signal often relates to all medicinal products containing the same active substance, including combination products. Certain signals may only be relevant for a particular medicinal product while some signals may be relevant for a specific indication, strength, pharmaceutical form or route of administration. Some signals may apply to a whole class of medicinal products.
Signal management process: The set of activities performed to determine whether there are new risks associated with an active substance or a medicinal product or whether known risks have changed based on mass data collected from individual case safety reports (ICSR), active surveillance systems or studies, literature data and other data sources as well as any related recommendations, decisions, communications and tracking regarding such risks.
The signal management process includes the following activities from signal detection to recommendation for action:
- Signal detection
- Signal validation
- Signal confirmation
- Signal analysis and prioritization
- Signal evaluation
- Recommendation for action
Signal detection: The act of looking for and/or identifying signals using data from any source.
Signal validation: The process of evaluating the data supporting a detected signal in order to verify that the available documentation contains sufficient evidence to justify that there is a new causal association or a new aspect of a known association and to further analysis of the signal.
This evaluation should take into account the strength of the evidence, the clinical relevance and the previous awareness of the association (see 2.3.)
Signal confirmation: The process during which the competent authority examines and discusses a validated signal. This action is taken by EMA within the European Economic Area (EEA).
Emerging safety issue: A safety issue considered by a marketing authorization holder in relation to an authorized medicinal product under its responsibility to require urgent attention of the competent authority because of the potential major impact on the risk-benefit balance of the product and/or on patient or public health, that could warrant prompt regulatory action and communication to patients and healthcare professionals.
Chapter II – Structures and Processes
2.1. Sources of data and information
The data sources for identifying new signals are diverse. They potentially include all scientific information concerning the use of medicinal products and the outcome of the use (i.e. quality data, non-clinical data and clinical data (including pharmacovigilance and pharmacoepidemiological data).
Common sources for signal detections are:
- Spontaneous reporting systems,
- Active surveillance systems,
- Systematical examinations and studies including meta analyses,
- Scientific literature.
Signals from spontaneous reports may be detected from:
- Monitoring of ICSRs, suspected adverse reaction databases, articles from the scientific literature,
- Information provided by marketing authorization holders in the context of regulatory procedures (e.g. post-authorization commitments, variations, renewals, periodic safety update reports (PSURs), Risk Management Plan (RMP) updates),
- From other activities related to the continuous monitoring of the risk-benefit balance of medicinal products.
Signal detection is periodically conducted in large databases such as European Union Adverse Reaction Database (EudraVigilance), the US FDA Adverse Event Reporting System (FAERS) or the database of the Programme for International Drug Monitoring (VigiBase).
2.2. Signal Detection
Signal detection should be based on a multidisciplinary approach. It should follow an appropriate methodology, which may vary depending on the nature of data and on the type of medicinal product concerned. Vaccines, orphan medicinal products and drugs used on children require specific methodological strategies.
Signal detection may involve a review of ICSRs, statistical analyses, or a combination of both, depending on the size of the data set. When it is not feasible to assess each individual case (e.g. signals detected from published studies or healthcare record data), aggregated data should be considered.
2.3. Evaluation of the evidence supporting a signal
The following elements should be considered when evaluating the evidence supporting a detected signal:
- Strength of the evidence from ICSRs: The total number of cases (after exclusion of duplicates), and amongst those, the number of supportive cases, e.g. cases showing a compatible temporal association, positive de- or rechallenge, lack of potential alternative causes, assessed as possibly related by the reporting healthcare professional, supportive results of relevant investigations,; Additional cases reported with related terms (e.g. other MedDRA terms indicating clinical complications or different stages of the same reaction),; Consistency of the evidence across cases (e.g. pattern with repeated observations of an association),; Quality of the data and their documentation,; Cases matching internationally agreed case definitions if applicable,; Plausibility of a biological and pharmacological relationship / possible mechanism,; Number of cases in the context of patient exposure,; Measures of disproportionality, if applicable,
- Clinical relevance, for example;
o Seriousness and severity of the reaction, o Reactions occurring in the context of drug-drug interactions, o Reactions occurring in vulnerable populations (e.g. pregnant women, children or the older population or in patients with pre-existing risk factors), o Reactions occurring in different patterns of use (e.g. overdose, misuse, off- label use, medication errors), o Whether the signal may provide additional insight on an expected reaction in terms of e.g. its severity, outcome, incidence or management.
- Previous awareness, for example;
o The extent to which information is already included in the product information (i.e. the summary of product characteristics (SmPC), the patient information leaflet (PIL) and the labelling), o Whether the reaction is already included in the SmPC for other products including the same substance (Some signals may only be relevant to a specific medicinal product), o Whether the association has already been assessed in the initial application for marketing authorization, the RMP, the PSUR or any other regulatory procedure; Example: It is not always easy to check whether the adverse event is included in Chapter 4.8 of SmPC.
- The event could be similar but expressed with a different PT (e.g. it might be included as anaphylactic shock, anaphylaxis, vertigo, dizziness, paraglossa, angioedema). - The event could be included in SmPC, but the signal might provide new information (e.g. anatomical characteristics, severity or duration). - The term included in SmPC might not corresspond to the event (e.g. rash includes morbilliform rash but not Steven-Johnson syndrome; it does not mean that SmPC covers hepatitis fulminant just because it includes liver injury).
- Additional sources of information may provide further evidence on the association. For example;
o Clinical trial data, o Findings regarding similar cases in the scientific literature, including information on substances of the same class of medicinal products, o Experimental or non-clinical findings, o Databases with larger datasets, when the signal was detected from national or company-specific databases, o Healthcare databases that may provide information on characteristics of exposed patients and medicines utilization patterns, o Information from other regulatory authorities. The evaluation of the evidence supporting a signal may involve several rounds of expert discussions and different levels of decision-making. This may result in various decisions.
- Closing the signal: When the available data do not support a causal relationship or when there is sufficient information on the association in the product information. The signal may be re-opened at a later stage if new evidence arises.
- Monitoring the signal: The signal is monitored by reviewing new information from ICSRs or the scientific literature at appropriate time intervals to determine whether the new data is supportive of a causal relationship.
- Proposing actions: Such as changes to the product information by means of a variation, if there is sufficient evidence of a causal relationship.
2.4. Signal Prioritization
A key and continuous consideration of the signal management process is to promptly identify signals that may have an important impact on patient or public health and/or on the risk-benefit balance of the medicinal product.
The following should be considered when evaluating this impact:
- The severity, seriousness, outcome and reversibility of the adverse reaction and the potential for prevention,
- The patient exposure and the estimated frequency of the adverse reaction,
- The patient exposure in vulnerable populations and/or in populations with different patterns of use, where appropriate,
- The consequences of treatment discontinuation and the availability of other therapeutic options,
- The expected extent of the regulatory intervention (e.g. addition of adverse reactions, warnings, contraindications, taking additional risk minimization measures, suspension or revocation of the authorization of the medicinal product),
- Whether the signal is likely to apply to other substances of the same class of medicinal products.
In some circumstances (e.g. adverse events following mass immunization), special consideration may be given to signals that may cause media attention and/or public concerns.
How the signal is further managed including timelines depends on the prioritization. Because prioritization is a continuous process, appropriate measures should be considered at any stage if the information available supports the conclusion that there is a risk that requires prevention or minimization in a timely manner. Such measures may be required before a formal assessment of the signal is concluded. Professional judgment and flexibility should be applied throughout the process.
2.5. Quality Requirements
Signal management is a critical process. The signal management system should be clearly documented to ensure that the system functions properly and effectively, that the roles, responsibilities and required tasks are clear and standardized, that these tasks are conducted by staff with appropriate qualifications and expertise and that such tasks are communicated to all parties involved, and that there are provisions for appropriate control and, when needed, improvement of the system. This includes the rationale for the method and periodicity of signal detection activities. Therefore, there must be a quality assurance system and a quality management system, both of which should be applied to all signal management processes (see Module IV). Detailed procedures for this quality system should be developed, documented and implemented. The performance of the system should be controlled and, when used, performance indicators should be presented in the pharmacovigilance system master file.
The organizational roles and responsibilities for the activities including maintenance of documentation, quality control and review, and for ensuring corrective and preventive action should be assigned and recorded.
As a critical process, signal management activities should be audited at regular intervals, including tasks performed by any service providers and contractors. Data and document confidentiality, security and validity (including data integrity when transferred between organizations) should be guaranteed.
Through a tracking system, all parties should keep an audit trail of signal management activities, and the relevant queries and outcomes, including how signals have been detected, validated, confirmed and evaluated. The audit trail shall ensure the retrospective traceability of how signals have been detected and validated and how the validated signals have been evaluated.
Documentation may be requested by the Agency from marketing authorization holders to demonstrate compliance with these requirements at any time, including justification/evidence for the steps taken and decisions made.
The relevant staff members should be specifically trained in signal management activities in accordance with their roles and responsibilities. Location information for the training system and training records should be documented and the CV and job descriptions of the staff should be archived.
Chapter III – Signal Analysis Methods
3.1. Methods Used in the Signal Detection
3.1.1. Conventional Methods
In spontaneous reporting systems, signal detection is performed in two forms; qualitative and quantitative. From a historical point of view, the classification could be made as conventional and advanced quantitative, statistical or automated signal detection methods. Conventional methods involve both qualitative (e.g. examination of individual cases or case series) and simple quantitative approaches (e.g. frequency/reporting rate, grouping, cross- tabulation, etc.). There has been a need to develop more complex quantitative methods due to the renaissance in statistics approaches, the increase in the number of spontaneous reports, and the growth of the size of the databases.
The analysis for quantitative signal detection should be performed according to MedDRA (Standard Medical Dictionary for Regulatory Activities) "preferred term" (PT) level. 3.1.1.1.Review of cases and case series
One of the most frequently used techniques in conventional pharmacovigilance is the "index case" or "striking case" method. The trained pharmacovigilance officer identifies the signal during the routine examination of the arriving data, usually during the initial examination of ICSRs (spontaneous adverse reaction reports, systematic data collection systems or cases published in the literature). In practice, strong suspicions about possible drug-event associations are often based on case series (clusters) with similar characteristics, although unusually striking and well documented identification of a single ICSR can sometimes be interpreted as a signal. In addition, removal of positive challenge or re- challenge is also an important criterion used to identify the signal. Undoubtedly, such investigations are subjective. For this reason, the related investigator should have adequate knowledge of the pharmacology of the drug, the indications used, and the patient population exposed.
One of the main steps in the examination of case reports is "designated medical events" (DME). DMEs are rare, serious, and drug-related potentially high-risk adverse events. In such cases, one to three reports can be considered possible signals. Typical examples include aplastic anemia, toxic epidermal necrolysis, Stevens-Johnson syndrome,
Torsade de pointes, and hepatic dysfunction. However, there are also events that should be considered as DME if the definition is not certain and does not meet the above criteria. For example: Pancreatitis (Risks depend on alcohol use and gall bladder disease in most adults). For this reason, any event decided to require special attention may also be referred to as DME. There is no universally accepted DME list. There may always be changes to this list. Each organization has its own list. The list of EMA dated August 17, 2016 is included in Appendix 2.
Other events of particular interest related to a particular medicinal product and/or patient population are referred to as targeted medical events (TME). These events are similar to DMEs, but their classification is based on the medicinal product. In this case, scientific information about pharmacovigilance rationale and medicines, treatment indications and/or patient populations make it easier to assess the potential problems that may arise.
A group of selected events considered to be serious is called important medical events (IME). This list has been developed by the organization that forms MedDRA (MedDRA MSSO) and coordinated by EMA.
Other clinical features may require specific attention. For example, hyperacute (needle tip) events need to be carefully assessed. These events are those that occur in a very short period of time during the parenteral administration of the drug and are biologically plausible, while there is a stable state from other aspects. The clinical features of the event indicate that the drug contributes at least to the occurrence of the event. For example; such as kidney or gall stones due to pure medicine.
Apart from the classification of DME and TME in signal detection, there are other criteria that are part of the clinical triage of cases. For example; if a hypothetical drug is known to moderate respiratory depression, but the incoming reports are regarding apnea/respiratory arrest, a biological justification is required for this defined event that happens in a more severe manner.
Preparing and analyzing periodical evaluations for the safety of newly authorized medicines is a very important conventional signal detection method in pharmacovigilance. An example of the reports on the periodical collection of such data is PBRER. Such reports also allow for a comprehensive look at data at a specific time.
The scientific investigation to be conducted by and experience of experts in examining the individual reports and case series are extremely important.
3.1.1.2. Simple analysis of larger data sets
If the observed value for any of the following conditions for an adverse event or a set of adverse events is greater than the expected value, this points to a signal detection:
- The number of a specific adverse event (absolute number)
- The total number of a specific adverse event/drug reports (ratio) or;
- Number of specific adverse events/estimated exposure to the drug (ratio).
ICH E2E Pharmacovigilance Planning Guideline lists post-marketing pharmacovigilance data collection methods. These include passive surveillance (spontaneous reports, case series), active surveillance (Prescription Event Monitoring, Records, etc.), comparative observational studies (case control studies, cross-sectional studies, cohort studies), targeted clinical trials (genetic tests, trials on special population, large simple studies), descriptive studies (natural history of the disease, drug utilization study). When data collection is in the form of passive surveillance, which means when spontaneous reports are routinely collected (e.g. MedWatch in the USA, Yellow Card System in the UK, EudraVigilance in the European Economic Area), or during targeted collection of some report types and extended follow-up (such as chickenpox vaccine, pregnancy record) while signal detection is conducted, striking reports containing positive re-challenges with DME or TMEs, mass reports where spontaneous reports are periodically evaluated, automated scanning of the adverse event database or data mining method can be used.
In cases where the data collection method is active surveillance (such as Prescription Event Monitoring), more advanced methods are used to search for signals.
Signals of disproportionate reporting require careful, detailed case and literature review, including evaluation of pre-clinical and pharmacological data, pharmacoepidemiological and/or clinical trials.
3.1.2. More Complex Quantitative Methods
Quantitative methods have been developed to detect signals in large spontaneous reporting system databases. These methods allow systematic, automated and practical screening of data that cannot be easily scanned by conventional means, while minimizing human error.
Advanced quantitative methods include computer aided statistical methodologies and data mining algorithms. These methods are based on the disproportionality analysis based on the 2x2 probability tables.
3.1.2.1. Disproportionate analyses
Disproportionate analysis (DA) means scanning ICSR databases to detect reporting rates higher than expected, and the main objective is to identify statistically significant reporting relationships between drug and event pairs in the spontaneous reporting system.
Signals of disproportionate reporting (SDR) refers to the numerical output of these analyzes without clinical context and is used in place of the signaling term in statistical calculations.
The main feature of data mining algorithms that support disproportionate analysis is to summarize very complex safety data sets into a 2x2 probability table for each drug-event pair. The statistical 2x2 table is used extensively in drug safety, and forms the basis of calculations for relationship measurement.
The following is used for drug-adverse reaction pairs as the common criterion of disproportionality:
- Proportional reporting ratio,
- Reporting odds ratio,
- Information component,
- Empirical Geometrical Mean.
There are also disproportionality criteria for drug-drug-adverse reaction trios. One of them is Omega (Ω), and it is used by WHO Drug Monitoring Cooperation Program.
3.1.2.1.1. Proportional Reporting Ratio (PRR)
The reporting disproportionate measurement used to obtain SDR in pharmacovigilance databases is PRR. It shows how common an adverse event is for a specific drug compared to all database events.
If an SDR (containing a specific adverse event) has been identified for a drug (P) in this method, it is assumed that this adverse event is reported for this drug (P) relatively more frequently than the other drugs. The relative increase in reported adverse events for the P drug is expressed in the 2x2 probability table as the following table based on the total number of individual cases contained in the pharmacovigilance database:
Table 1: 2x2 probability table used for the calculation of PRR
| Event (E) | All other events | Total | |
|---|---|---|---|
| Drug (P) | A | B | A+B |
| All other drugs | C | D | C+D |
| Total | A+C | B+D | N=A+B+C+D |
The elements in this table are the individual cases in the database. The basic criteria for calculating the PRR are:
Value A is the number of individual cases related to drug P-related adverse event E, Value B is the number of individual cases related to all adverse events other than drug P-related adverse event E,
Value C is the number of individual cases related to adverse event E not related to drug P, Value D is the number of individual cases related to adverse events other than adverse event E and not related to drug P.
PRR is calculated as follows:
PRR = A/(A+B) / C/(C+D)
Example 1 - The proportion of individual nausea cases is 5% among all reports related to drug P. (5 out of 100 reports reported for drug P are nausea. A=5, B=95, A+B =100).
- The proportion of nausea reports among all the reports related to all other medicines in the database (other than drug P) is also equal to 5% (e.g., 5.000 nausea cases out of 100.000 reports reported as associated with all other drugs, similarly C = 5.000, D = 95.000, C + D = 100.000). In this example; PRR is equal to 1 (e.g., 0.05/0.05).
Example 2: - The proportion of individual nausea cases is 15% among all reports related to drug P. (15 out of 100 reports reported for drug P are nausea. A=15, B=85, A+B =100).
- The proportion of nausea reports among all the reports related to all other medicines in the database (other than drug P) is also equal to 5% (e.g., 5.000 nausea cases out of 100.000 reports reported as associated with all other drugs, similarly C = 5.000, D = 95.000, C + D = 100.000). In this example; PRR is equal to 3 (e.g., 0.15/0.05).
3.1.2.1.1.1. 95% Confidence interval of PRR
The data analysis system to be established must also calculate the confidence interval of PRR. The standard error of the natural logarithm of PRR is calculated according to the following formula:
The 95% confidence interval for In (PRR) is calculated as ln (PRR) ±1.96SH. Taking the exponential function, we get the following result:
95% confidence interval for PRR = (PRR/eks (1.96SH), PRR x exp(1.96SH))
3.1.2.1.1.2. Interpretation of SDRs
Disproportionality analysis is not an inferential exercise. Utilizing inferential statistical methods in such exploratory analyzes can lead to great difficulty. If the statistical analyzes in the database are done without discipline, it is possible to lead to subjective decisions in the selection, distribution and interpretation of data mining processes and outputs.
The results of quantitative methods should be carefully interpreted and the limitations of the spontaneous reporting system database should be considered. When interpreting PRR results in analyzes for signal detection, the following points should be taken into consideration:
- a) The PRR measures the reporting relationship between drug P and adverse event on the basis of the relative increase in the proportion of individual cases associated with the adverse event. It does not indicate any causality relationship between the applied drug P and the occurrence of the adverse event. Such statistical disproportionate may reflect one or more bias (such as bias due to the patient's underlying disease), artifacts naturally present in the pharmacovigilance system and also statistical noise. For this reason, SDR determined by quantitative methods must be evaluated in medical terms.
- b) When the first decision is made as to whether further analysis of the drug-event pair is necessary, the threshold value of the applied PRR and other statistical estimates (e.g. the estimated lower limit of the confidence interval) are checked. There is no threshold value accepted as a gold standard for SDRs.
- c) Commonly used thresholds for detecting SDRs are a choice between two conflicting options: If the threshold value is too low, false positive signals can be produced, or if the threshold value is too high, the actual signals may be missed.
- d) PRR also allows you to compare the reporting relationship between the drug P and all other drugs in the database. The PRR value and, consequently, the SDRs determined by this method depend on the data included in the database in which the PRR is calculated. For this reason, the PRR interpretation should be based on the following factors:
- I. Types of drugs in database
- II. Medical terminology used in the reports that establish the database
- III. Coding
- IV. The date the database was created
- V. The source of ICSRs (e.g. all unsolicited reports)
These elements can affect the PRR value and create a masking effect. Alternatively, they may lead to exaggeration of the significance of the statistical relationship between drug and adverse event . e) In addition to the quantitative aspect, it is important to consider other factors in the selection and prioritization of the SDRs defined in the database:
I. Possible effect of SDRs in terms of public health II. Regulatory conditions and especially whether the adverse event is expected/unexpected (especially if there is evidence of a change in the frequency of reporting or the severity of the known adverse drug reaction) III. Whether or not the potential risk associated with the SDR has already been or is already confirmed
These should be evaluated independently of whether the PRR has passed the previously defined threshold. f) The absence of the SDR does not necessarily exclude the possibility of a relationship between drug P and adverse event E. g) Using similar techniques to other SDRs, the PRR can be purified (e.g., bringing many drugs and/or adverse events together, stratifying according to the age and sex of the patient). Based on the PRR, the possibility of masking the drug-adverse event relationship by other drugs should be kept in mind during the analysis of SDRs.
3.1.2.1.1.3. Threshold values defining SDRs in EudraVigilance database
Currently there is no gold standard for determining a universal threshold for the signal of disproportionate reporting. The threshold values used in EudraVigilance are empirical. PRR is a highly sensitive method that can lead to the occurrence of numerous false positive signals, especially when the number of reports is low. For this reason, the number of cases is used as the threshold value (number of reports ≥ 3) in relation to the PRR and the Chi-square statistic is used to reduce the number of false positive results. The following criteria are used in the inquiry to determine the SDR (drug-event combination) in the EudraVigilance Data Analysis System.
- a) When the PRR is shown in the 95% confidence interval:
- The lower limit of the 95% confidence interval is equal to or greater than one
- The number of individual cases is equal to or greater than three These criteria are used for weekly/monthly signal monitoring. Chi-square statistic is also used in relation to PRR. In such cases, the following criteria are used in the inquiry:
- b) When PRR is shown in χ2 statistics:
- PRR ≥ 2
- χð≥ 4
- If the number of individual cases is equal to or greater than three.
3.1.2.1.1.4. Subgroup analyses and stratification
Stratification is a statistical process aimed at reducing mixing effects by making corrections to the relationship between drug and mixing variable and the event and the same mixing variable. PRR calculations can be done on the entire database or on a subgroup of reports. Restrictions on the area can be made by using many variables such as age and gender. Calculation of unadjusted PRR can be adjusted by stratification by age, gender or reporting date. It is also possible to perform subgroup analyzes on the basis of drug or drug class.
3.1.2.1.1.5. Validation
Numerous methodological issues make it difficult to systematically and comprehensively evaluate the performance of quantitative methods used in signal detection. The lack of a gold standard for signal detection makes this assessment difficult. It is possible to reach the following results from the validation studies:
- a) The high sensitivity calculated in these studies is based on a simple definition of the signal (e.g. drug-event pair). This definition of the signal may not reflect the true causal relationship between the drug-event pair in the pharmacovigilance clinical assessment context.
- b) The low specificity of the statistical methods emphasized in the validation studies is an indication that a large number of false positive signals may occur.
3.1.2.1.2. Information component
The Information Component made available through the Bayesian Confidence Propagating Neural Network (BCPNN) method, developed by the WHO Drug Monitoring Cooperation Program to detect signals from adverse drug reactions in VigiBase, is a measure of the disproportionality between the observed and expected reports of a drug- adverse reaction pair.
The positive IC value indicates that a given combination of drug-adverse reactions is reported more than expected from the remaining reports in the database. The IC value of 0 indicates that there is no quantitative dependence, while the negative IC value indicates that the combination appears statistically less frequently than expected in the database.
The IC value is calculated based on the following information:
- Total number of case reports related to drug X (Cx)
- Total number of case reports related to the adverse reaction Y term (Cy)
- Total number of reports for a specific drug-reaction pair (Cxy)
- Total number of reports in the database (C) New reports can cause the IC value to increase or decrease. If the IC value is calculated from large numbers, a new report cannot cause a large fluctuation in the IC value. The standard deviation of each IC value is a measure of the strength of the value. The greater the values of Cx, Cy and Cxy, the smaller the confidence interval will be. The lower limit value of the 95% confidence limit for the IC value is IC025.
The IC value does not give information about the qualitative causality of the combination of drug-adverse reactions. The IC shows the quantitative dependence between the adverse reaction and the drug, based on the reports in VigiBase. If the IC value increases with time and the value of IC25 is positive, there is a high likelihood of a positive quantitative relationship between drug and adverse reaction, but clinical evaluation is still necessary.
As of September 30, 2016, there are 14 million ICSRs in the WHO Drug Monitoring Cooperation Center (UMC) database (VigiBase). The data in the VigiBase is periodically analyzed within the routine signaling process of the UMC and unidentified adverse drug reactions are detected. Clinical evaluation is performed after the combination of drug- adverse reactions is detected automatically. It reduces the number of drug-adverse reaction combinations and increases the likelihood of finding the most important signal by using the disproportionality measure (IC) and other statistics between the triage method and the observed and expected reports of drug-adverse reaction combinations. UMC's ability to predict in the data mining method has been investigated; positive predictive value is 44% while negative predictive value is 85%. To determine the signal in the signal detection process, the following procedure is followed for the combination of drug-adverse reaction:
1. Relative reporting rates are calculated for all drug-adverse reaction combinations reported to VigiBase. The IC value indicates that the drug-adverse reaction combination is reported unexpectedly more frequently than in the past of the database. 2. For the more commonly reported drug-adverse reaction combination, the triage algorithm selects combinations containing ICSRs originating from at least two countries, and these combinations must meet at least one of the following criteria:
a. New drug and serious reaction (the drug has entered the database in the last five years and the adverse reaction is one of the critical terms of the WHO- ART (WHO's adverse reaction terminology)) b. A significant rise in IC since the previous quarter
3. Combinations of drug-adverse reactions filtered by the triage algorithm are then checked by the members of UMC staff in terms of whether the drug is in the summary of product characteristics or in the literature. If the reaction is not found or is not well defined, individual case reports are taken from VigiBase. 4. ICSRs are evaluated by members of UMC staff or UMC signal review panel.
As a result, a signal search is made using a combination of data mining and clinical evaluation of experts.
Clinical evaluation of the potential signal by experts:
- The quantitative power of the relationship: Number of ICSRs; Statistical disproportionality
- Consistency of the form (pattern)
- Exposure-response relationship: Region, timing, dose, reversibility
- The biological probability of the hypothesis (biologically plausible) (likelihood)
- Pharmacological, pathological
- Experimental findings: For example; removal of challenge, re-challenge, blood levels, metabolites, drug-dependent antibodies
- The nature and quality of the data: Impartiality, documentation, causality evaluation
During the evaluation, details of the reports from the National Centers are also requested.
3.1.3. Integration of Statistical Methods with Classical Methods
Quantitative methods used in pharmacovigilance signal detection should be used in conjunction with other conventional methods. Conventional methods may allow detection of safety signals that can not be detected by data mining methods.
If a drug is new to the market and only a certain number of ICSRs is received, it is more plausible to evaluate the ICSRs individually, rather than on a statistical method. One of the reasons is that the reliability of statistical scanning is limited for a small number of ICSRs. Moreover, the information about the safety profile in the early days when the drug is released to the market is basically based on clinical experience. For this reason, scanning of individual cases adds more value to the monitoring process.
Regular and systematic evaluation of all new ICSRs is still one of the basic foundations of pharmacovigilance. Statistical methods allow prioritization of ICSRs and consideration of additional factors. For example; the PRR method does not routinely examine the accompanying drug-event pairs for drug-drug interaction. They need to be assessed by experts who have studied this situation. Knowledge about the nature of the data is vital. Because, indications of the same medicine vary from country to country.
3.1.3.1. Systematic evaluation of SDRs
The critical point in the integration of the classical methods used in signal detection in pharmacovigilance by statistical methods is the systematic evaluation of SDRs. There is a scientific acceptance that signals of disproportionate reporting, defined by statistical methods, should be considered together with medical judgment. The SDRs that need further evaluation should be systematically reviewed through medical examination.
The SDR review process should include the following steps: - Identifying possible duplications: - Checking the quality of the data: Reviewing information such as the completeness of the information provided and the correctness of the codes - Providing additional information if possible: If the information received in the evaluation of the case is insufficient, requesting new information - Control of authorization conditions: For more information, checking the drug's SmPC and PIL (e.g. whether the adverse event is expected or not, possible drug- drug interactions) - Control of additional information: As a support in the evaluation of SDRs, the following should be utilized: Application file, RMP, Suspicious Unexpected Serious Adverse Reactions (SUSAR), PBRER, post-authorization commitments and post- authorization studies.
3.1.3.2. Special conditions for the use of quantitative methods used in vaccines and drugs for children
There are special requirements for analyzes to be made on vaccines and medicines used in children. Detection of new signals for such products or related to this population should be evaluated separately with appropriate scientific experts.
Chapter IV – Duties and Responsibilities
4.1. Duties and Responsibilities of the Marketing Authorization Holder for Signal Management
It is vitally important to monitor the safety of medicines while in the market. This requires that marketing authorization holders ensure that all information sources are scanned to detect any possible signal, that the appropriate action is taken to provide the requisite response to new evidence affecting the benefit/risk balance of the drug, and that the Agency, healthcare professionals and patients are informed of such change in benefit/risk balance.
There are many methods to be used for signal detection. The method to be used should be selected by looking at the product portfolio and the number of suspicious adverse reaction reports. However, it is necessary for all marketing authorization holders to have systems and methods for systematic signal detection.
All marketing authorization holders should have official written procedural documents identifying and describing the signaling method used, such as the standard working method. These documents should provide detailed information about the roles and responsibilities of all staff involved in signal detection, the information sources involved in the analysis, and the method used for signal detection. The master file of the pharmacovigilance system should also have written procedures for signal generation, detection and evaluation (see Module V).
In addition, the marketing authorization holder must also have written procedures for actions to be taken based on the results of the signal detection activity. These procedures should include further analyses after the identification of the potential signal.
The marketing authorization holder must adequately document the signal detection activities, including decisions and conclusions taken during these activities.
Procedures for signal analysis should be arranged as to include at least the following aspects: 1. Increase in frequency, severity, and mortality rate of adverse reaction reports: How to manage adverse drug reaction reports is described in Module I. The marketing authorization holder must inform the TÜFAM immediately in the cases mentioned above and, against the likelihood of it being a condition originating from the nature of the drug, review which processes have any change in the production of the drug, follow up the changed processes closely, make the necessary analyses and notify the Agency on the conclusion. 2. Periodical signal search in the cumulative dataset: PBRERs form an important part of the signal detection system in terms of enabling all data in the reporting period to be reviewed. However, in cases where it is necessary to prepare a report, especially every three years, this time may be too long for signal detection. Regular monitoring of the drug safety is the responsibility of all marketing authorization holders, regardless of the product portfolio. For this reason, if the marketing authorization holder only analyzes safety data in the process of producing PBRER, he should consider that this issue will be investigated during pharmacovigilance audits. 3. Taking the necessary actions in line with the suggestions by following the signals published every month under the heading "Recommendations of the Pharmacovigilance Risk Assessment Committee on Safety Signals" on the official EMA website at http://www.ema.europa.eu/ema/index.jsp?curl=pages/regulation/document_listing/ document_listing_000375.jsp:
a. If the proposal requires any addition to SmPC/PIL, the marketing authorization holder of the original drug should apply to the Agency for variation. b. If additional information is requested from the marketing authorization holder, this issue shall be notified to the Agency and additional information must be submitted to the Agency at the same time as the EMA. c. Other proposals should be taken into account, including monitoring
PBRERs, making routine pharmacovigilance, or reporting that there is no need to do anything at this stage. Documents demonstrating this situation should be archived and promptly presented to the Agency upon request. 4. Taking the necessary actions in line with the suggestions by following the signals published every three months under the heading "Possible signal of new safety information/serious risks detected in the FDA adverse event reporting system (FAERS)" on the FDA's official website at http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Surveillan ce/AdverseDrugEffects/UCM082196 5. Observation of the results of the evaluations made in the PBRs in the EEA and taking the necessary actions: Taking the necessary actions by following the list published under the heading "Evaluation results of periodic safety update reports" on the official EMA website at http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/landing/psusa_sea rch.jsp\&mid=WC0b01ac0580902b8d:
a. If the regulatory result is "continue", no action is taken. b. If the regulatory result is "variation"; the marketing authorization holder of the original drug must make a variation application to the Agency for the issue to be added to SmPC/PIL. When the Agency is informed about the signal detection, the actions planned and performed in the EEA and the actions considered to be done in our country should be explained. The Agency may decide one or more of the following actions after evaluating the signal in question:
- No action is required except routine pharmacovigilance.
- Additional information must be provided by the marketing authorization holder within a specified time period.
- RMP must be submitted or RMP must be updated.
- Variation must be made in the marketing authorization.
- Additional risk minimization measures should be applied.
- The marketing authorization holder must sponsor the post-authorization work in accordance with an agreed protocol and submit the results of the work to the Agency.
- Emergency safety restriction must be applied.
- The marketing authorization must be suspended.
- An audit should be performed to confirm whether or not the marketing authorization holder meets the pharmacovigilance requirements. The Agency may also decide to take any appropriate action not listed above.
Appendices
Appendix 1 – Signal Validation and Evaluation Checklist
Signal validation
Important sources: SmPC, PBRER, RMP, TİTCK website and other regulatory processes
1. Is the event included in the SmPC of the drug for the active substance?
i. Review the SmPC of the drug.
- For active substances with generics, check the SmPC of the innovative drug first.
- Check chapters 4.3, 4.4, 4.5, 4.6, 4.8, 4.9 of the SmPC for the adverse event.
- Check whether the adverse event is included in the SmPC with a similar term or higher level term.
- Check whether the adverse event is included in the SmPC of another drug that carries the same active substance.
- Check whether the adverse event is included in the SmPC of another drug of the same class.
- If there is an interaction, check whether there is information on interaction in the SmPC of other drugs.
2. Does the adverse event show a new dimension of a known relationship?
- Increase in the frequency of occurrence
- Change in its duration or in the time until adverse reaction occurs
- Change in severity
- Change in occurrence pattern (e.g. affecting the private population)
- Change in the previously reported result (e.g. new fatal cases)
3. Has the relationship been described before in regulatory processes?
- Check the PBRER of the drug.
- Check the RMP of the drug.
- Check the websites of EMA and FDA.
- Check whether the variation application has been made with the same safety warning.
Signal evaluation
Important sources: ICSRs, national databases, literature, EudraVigilance database, WHO database
1. Review of cases supporting the signal
- The number of cases supporting the relationship (after the duplicate cases and cases with inappropriate temporal relations are excluded)
- The number of cases appropriately documented with adequate information (e.g. suspected drug, reported event, age and gender, indication, outcome, concomitant medications)
- Consistency of the evidence across cases (e.g. patterns)
- Route of administration and formulation of the drug
- Report clustering, e.g. many reports can be sent by the same reporter.
- Cases that fulfill the diagnostic criteria of the event
2. Strengths
- Biologically and pharmacologically plausible (possible mechanism)
- Dose relationship
- Number of cases of removal of positive challenge
- Number of positive re-challenge cases
- Low background incidence of the event
- Time until it starts
3. Clinical significance
- Seriousness/severity of the event
- Whether the event is reversible
- Events that affect special populations (e.g., pregnant women, children, elderly) or patients with underlying risk factors
- Events that occur in different patterns of use (e.g. off-label use, abuse, overdose, medication errors)
- The relationship that can be applied to other active substances in the same class
- Prevention possibility
4. Other issues to be considered
- Possible class effect
- Possible drug-drug interaction
- Possible medication error
- Possible quality defect
- Possible off-label use
- Possible dependence, abuse, misuse
5. Weaknesses
- Data quality of case reports is poor
- The multiplicity of the number of cases with mixing factors/alternative explanations
- Stimulated reporting indications (e.g. increased media attention)
- Abnormal reporting pattern
- Presence of other event-related risk factors: underlying disease, co- morbidities, co-medications
Additional Sources of Information
Important sources: Literature, other adverse reaction databases, clinical trial data, records, drug use data, studies
1. Additional data
- National databases (e.g. signals detected in literature)
- Databases containing larger datasets such as EudraVigilance or VigiBase
- Information on active substances in the same class
- Literature findings on similar case reports, pharmacoepidemiological studies or studies on possible mechanism of action
- Clinical studies
- Pre-clinical data
2. Information on exposure
- Drug use data obtained from PBRER or national drug use records (national exposure, patterns of use)
- Health databases
3. Additional information
- RMP of the drug (information on ongoing or planned work that may generate evidence for relationship)
- Records
Appendix 2 – List of Designated Medical Events
- Acute hepatic failure
- Acute kidney injury
- Agranulocytosis
- Anaphylactic reaction
- Anaphylactic shock
- Anaphylactoid reaction
- Anaphylactoid shock
- Angioedema
- Aplasia pure red cell
- Aplastic anaemia
- Autoimmune haemolytic anaemia
- Autoimmune hepatitis
- Autoimmune pancreatitis
- Azotaemia
- Blindness
- Bone marrow failure
- Deafness
- Deafness neurosensory
- Deafness permanent
- Deafness transitory
- Dermatitis exfoliative
- Dermatitis exfoliative generalised
- Drug reaction with eosinophilia and systemic symptoms
- Drug-induced liver injury
- Erythema multiforme
- Febrile neutropenia
- Granulocytopenia
- Haemolysis
- Haemolytic anaemia
- Hepatic failure
- Hepatic infarction
- Hepatic necrosis
- Hepatitis fulminant
- Immune thrombocytopenic purpura
- Intestinal perforation
- Ischaemic pancreatitis
- Neutropenic colitis
- Neutropenic infection
- Neutropenic sepsis
- Oedematous pancreatitis
- Optic ischaemic neuropathy
- Pancreatitis
- Pancreatitis acute
- Pancytopenia
- Product contamination microbial
- Progressive multifocal leukoencephalopathy
- Pulmonary arterial hypertension
- Pulmonary fibrosis
- Pulmonary hypertension
- Renal failure
- Reye's syndrome
- Rhabdomyolysis
- Stevens-Johnson syndrome
- Sudden cardiac death
- Sudden hearing loss
- Sudden visual loss
- Thrombotic thrombocytopenic purpura
- Torsade de pointes
- Toxic epidermal necrolysis
- Toxic optic neuropathy
- Transmission of an infectious agent via product
- Ventricular fibrillation
References
- 1. Eudravigilance Expert Working Group (EV-EWG), Guideline on the use of statistical signal detection methods in the Eudravigilance data analysis system. 26 June 2008, EMEA/106464/2006 rev. 1.
- 2. Council for International Organizations of Medical Sciences (CIOMS). Report of CIOMS Working Group VIII on Practical Aspects of Signal Detection in Pharmacovigilance. Geneva:CIOMS; 2010.
- 3. Guideline on good pharmacovigilance practices (GVP) Module IX – Signal Management, 22 June 2012, EMA/827661/2011.
- 4. Guideline on good pharmacovigilance practices (GVP) Module IX – Signal Management (Rev 1), 4 August 2016, EMA/827661/2011 Rev 1 Draft for Public Consultation.
- 5. Guideline on good pharmacovigilance practices (GVP) Module IX Addendum I: Methodological Aspects of Signal Detection from Spontaneous Reports of Suspected Adverse Reactions, 4 August 2016, EMA/209012/2015 Draft for Public Consultation.
- 6. http://who-umc.org/DynPage.aspx?id=115096&mn1=7347&mn2=7252&mn3=7613&mn4=761 (Added on October 31, 2016).
- 7. http://who-umc.org/graphics/28463.pdf (Added on October 31, 2016).
