Professional Documents
Culture Documents
SURVEILLANCE OF
IN VITRO DIAGNOSTICS
POST-MARKET
SURVEILLANCE OF
IN VITRO DIAGNOSTICS
1
P ost-market surveillance of in vitro diagnostics
1.In Vitro Techniques. 2.Product Surveillance, Postmarketing. 3.Equipment Safety. 4.Equipment Failure. I.World Health Organization.
All rights reserved. Publications of the World Health Organization are available on the WHO website (www.who.int) or can be purchased
from WHO Press, World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland (tel.: +41 22 791 3264; fax: +41 22 791 4857;
e-mail: bookorders@who.int).
Requests for permission to reproduce or translate WHO publications –whether for sale or for non-commercial distribution– should be
addressed to WHO Press through the WHO website (www.who.int/about/licensing/copyright_form/en/index.html).
The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever
on the part of the World Health Organization concerning the legal status of any country, territory, city or area or of its authorities, or
concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on maps represent approximate border lines for which
there may not yet be full agreement.
The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by the
World Health Organization in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names
of proprietary products are distinguished by initial capital letters.
All reasonable precautions have been taken by the World Health Organization to verify the information contained in this publication.
However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the
interpretation and use of the material lies with the reader. In no event shall the World Health Organization be liable for damages arising
from its use.
Printed in France
Contact:
Irena Prat, Essential Medicines and Health Products - WHO - 20, Avenue Appia - 1211 Geneva 27- Switzerland
DISCLAIMERS 5
INTRODUCTION 6
1 BACKGROUND 6
2 SCOPE AND INTENDED AUDIENCE OF THIS GUIDANCE 6
3 REFERENCE DOCUMENTS, DEFINITIONS AND ACRONYMS 8
4 BASIC PRINCIPLES 12
5 STAKEHOLDERS’ ROLES IN POST-MARKET SURVEILLANCE FOR IVDS 15
ANNEXES 45
ANNEX 1– LOT TESTING DATA COLLECTION FORM 45
ANNEX 2– TESTING REPORT FORMAT FOR LOT VERIFICATION TESTING 46
ANNEX 3 – IVD COMPLAINT REPORTING FORM 55
ANNEX 4 – ADVERSE EVENT MANUFACTURER INVESTIGATION REPORTING FORM 52
ANNEX 5 – FIELD SAFETY CORRECTIVE ACTION REPORT FORM 55
ANNEX 6 – EXAMPLE FIELD SAFETY NOTICE 57
ANNEX 7 – POST-MARKET INFORMATION EXCHANGE REPORTING FORM FOR NRAS 58
3
P ost-market surveillance of in vitro diagnostics
LIST OF TABLES
TABLE 1 – STAKEHOLDERS’ ROLES IN POST-MARKET SURVEILLANCE FOR IVDS 15
TABLE 2 – STEPS IN VERIFYING COMPLAINTS 20
TABLE 3 – RATIONALE AND PROCESSES FOR LOT VERIFICATION TESTING 22
TABLE 4 – EXAMPLES OF ADVERSE EVENT CLASSIFICATIONS 25
TABLE 5 – LOT VERIFICATION SPECIMEN PANEL FOR RDTS THAT DETECT ANTIBODIES TO HIV-1/2 39
TABLE 6 – LOT VERIFICATION SPECIMEN PANEL FOR RDTS THAT DETECT ANTIBODIES TO HIV-1/2 AND HIV-1 P24 ANTIGEN 40
TABLE 7 – ACCEPTANCE CRITERIA FOR HIV-1/2 RDTS 44
LIST OF FIGURES
FIGURE 1 – STEPS FOR POST-MARKET SURVEILLANCE OF WHO-PREQUALIFIED IVDS 13
FIGURE 2 – RISK MANAGEMENT PROCESS FOR MEDICAL DEVICES (INCLUDING IVDS) 14
FIGURE 3 – FLOW CHART FOR USERS TO REPORTING COMPLAINTS RELATED TO IVDS 18
FIGURE 4 – FLOW CHART OF REPORTING COMPLAINTS FOR WHO-PREQUALIFIED IVDS 33
FIGURE 5 – DILUTION SENSITIVITY OF TWO SPECIMENS ON ONE HIV-1/2 RDT 40
FIGURE 6 – STANDARD CURVE FOR HIV-1 P24 ANTIGEN IN 4TH GENERATION HIV-1/2 RDTS 41
The experts who participated in two rounds of consultative meetings (June 2014 and October 2014) and reviewed the
draft guidance were: Sue Best, Patience Dabula, Joelle Daviaud, Dianna Edgil, Martine Guillerm, Jan Jacobs, Deidre Healy,
Sandra Incardona, Joel Kuritsky, Victor Muchunguzi, Heiner Scheiblauer, Aisseta Tourè, Elsa Tran, Stuart Turner, Vincent
Wong, and Bibiana Zambrano.
The draft guidance was then made available for public comment on the WHO website for the period of one month in
early 2015.
The WHO staff who participated in the development and review of the guidance were: Helena Ardura, Jane Cunningham,
Shona Dalal, Guy-Michel Gershey-Damet , Cheryl Johnson, Robyn Meurant, Irena Prat, Sabine Ohse, Julie Samuelson, Anita
Sands and Willy Urassa.
DISCLAIMER
This guidance was developed based on the in vitro diagnostic (IVD) technology in use at the time, and therefore may require
certain adaptation as technology formats develop. The illustrative examples used in this guidance are not an exhaustive list.
5
P ost-market surveillance of in vitro diagnostics
INTRODUCTION
1. BACKGROUND
Prequalification of The World Health Organization (WHO) Prequalification of In Vitro Diagnostics Programme is
IVDs coordinated through the department of Essential Medicines and Health Products. The aim of the
WHO Prequalification of In Vitro Diagnostics Programme is to promote and facilitate access to
safe, appropriate and affordable in vitro diagnostics (IVDs) of good quality in an equitable manner.
Focus is placed on IVDs for priority diseases and their suitability for use in resource-limited settings.
Beneficiaries The findings of the WHO Prequalification of In Vitro Diagnostics Programme1 are used to provide
independent technical information on safety, quality and performance of IVDs, principally to other
United Nations (UN) agencies but also to WHO Member States and other interested organizations.
The WHO prequalification status, in conjunction with other procurement criteria, is used by UN
agencies, WHO Member States and other interested organizations to guide their procurement
of IVDs.
Post-market The purpose of post-market surveillance is to protect individual health and public health through
surveillance continued surveillance of IVDs once they are placed on the market by reducing any risks. Such
activities should ensure the manufacturer’s obligations are fulfilled through ensuring they are
aware of event which enables them to undertake and assessment of any risks, and as appropriate
any suggested steps to risk mitigation.
In the context of the WHO Prequalification of In Vitro Diagnostics Programme, this guidance aims to
ensure the ongoing compliance of WHO prequalified IVDs with WHO prequalification requirements
once they are placed on the market. Manufacturers of WHO prequalified IVDs are obliged to report
regularly post-market information to the relevant national regulatory authorities, and to WHO.
1 Prequalification does not imply any approval by WHO of the product and manufacturing site(s). Moreover, prequalification does not constitute any endorsement or
warranty by WHO of the fitness of any product for a particular purpose, including its safety, quality, or performance.
2 As of 2015, the scope of the WHO Prequalification of In Vitro Diagnostics Programme includes HIV rapid diagnostic tests, manual HIV enzyme immunoassays, HIV
supplemental assays, CD4 enumeration technologies, HIV nucleic acid tests (qualitative and quantitative), malaria rapid diagnostic tests, hepatitis C rapid diagnostic tests,
manual HCV enzyme immunoassays, hepatitis B surface antigen rapid diagnostic tests, and manual hepatitis B surface antigen enzyme immunoassays.
This guidance was developed with the underlying assumption that adequate pre-market assessment
data is available for the national regulatory authorities to determine if post-market information
will discern differences in post-market vs. pre-market safety, quality and performance of IVDs.
Therefore, this guidance should not be wholly or partially adopted without access to adequate
pre-market assessment data, such as the data generated in WHO Prequalification of In Vitro
Diagnostics Programme.
For regulatory purposes, IVDs are considered to be a subset of medical devices. Therefore, certain
elements of this guidance are adapted from best practices for regulation of medical devices.
Build on existing This document is intended to supplement, and not substitute the internal procedures for
systems post-market activities which are expected to be an integral part of the manufacturer’s quality
management system.
National regulations might require manufacturers and/or end users of IVDs to perform post-market
activities and submit relevant post-market information to national regulatory authorities. WHO
recognises that certain jurisdictions have implemented regulatory requirements for post-market
surveillance of IVDs. This guidance does not intend to replace any requirements that might already
be in place, and aims to be harmonized with their processes and procedures.
However, the principles laid down in this document should be considered by national regulatory
authorities when developing or amending existing national post-market surveillance regulations.
It can also be used by procurement agencies and other entities that procure IVDs and wish to be
assured of their continued quality, safety and performance.
Guidance for This document intends to give an overview of the technical aspects of post-market surveillance
adaptation for IVDs, in particular for those IVDs that fall within the scope of the WHO Prequalification of IVDs
Programme.
Manufacturers of WHO prequalified IVDs are obliged to follow these guidelines as part
of their on-going commitment to WHO prequalification.
Other stakeholders are invited to adopt these guidelines in relation to the resources available, i.e.
a phased implementation may be most appropriate. For instance, stakeholders may choose to
begin with pre-distribution lot testing and later add post-distribution lot testing, or to start with
complaint reporting and later add lot verification testing.
7
P ost-market surveillance of in vitro diagnostics
The following reference documents have been used in preparing this guidance.
POST-MARKET SURVEILLANCE
• ISO 9000:2005 Quality management systems - Fundamentals and vocabulary
• ISO 9001:2008 Quality management systems - Requirements
• ISO 13485:2012 Medical devices - Quality management systems - Requirements for regulatory purposes
• ISO/TR 14969:2004 Medical devices - Quality management systems - Guidance on the application of ISO 13485:2003
• ISO 14971:20072007 Medical devices -- Application of risk management to medical devices
• World Health Organization, PQDx_152, A risk based approached for assessment of in vitro diagnostics (IVDs). Accessed 31
October 2014 at http://www.who.int/diagnostics_laboratory/evaluations/140513_risk_based_assessment_approach_
buffet.pdf?ua=1
LOT TESTING
• EN 13612:2002 Performance evaluation of in vitro diagnostic medical devices
• BS EN 13975:2003 Sampling procedures for acceptance testing of in vitro diagnostic medical devices- Statistical aspects
• ISO 2859-10:2006 Sampling procedures for inspection by attributes - Part 10: Introduction to the ISO 2859 series of
standards for inspection by attributes
VIGILANCE
• GHTF/SG2/N54R8:2006 Medical Devices Post Market Surveillance: Global Guidance for Adverse Event Reporting for
Medical Devices
• GHTF/SG1/N045:2008 Principles of In Vitro Diagnostic (IVD) Medical Devices Classification
• GHTF/SG1/N046:2008 Principles of Conformity Assessment for In Vitro Diagnostic (IVD) Medical Devices
• GHTF/SG2/N008R4:1999 Guidance on How to Handle Information Concerning Vigilance Reporting Related to Medical
Devices
• GHTF/SG2/N57R8:2006 Medical Devices Post Market Surveillance: Content of Field Safety Notices
• GHTF/SG3/N15R8:2005 Implementation of risk management principles and activities within a Quality Management System
• MEDDEV 2 12-1 rev. 8 Vigilance, European Commission guidelines on a medical devices vigilance system
3.2 DEFINITIONS
Abnormal use Act or omission of an act by the operator or user of a medical device as a result of conduct that
is beyond any reasonable means of risk control by the manufacturer.3
Adverse event Defined as a product defect (i.e. malfunction or failure, deterioration in characteristics or performance,
(incident) or inadequacy of labelling or of instructions for use) that, directly or indirectly, might lead to or
might have led to serious medical consequences, namely death or serious deterioration in the
state of health of the patient, user or another person. Also called an incident.4
Analytical Analytical sensitivity measures a test’s ability to detect a low concentration of a given substance.
sensitivity Sometimes used interchangeably as limit of detection5 or detection limit.6
Accuracy The accuracy of an analytical procedure expresses the closeness of agreement between the
value which is accepted either as a conventional true value or an accepted reference value and
the value found.7
3 IEC 60601-1-6:2004 Medical electrical equipment. General requirements for safety - Collateral standard: Usability.
4 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
5 CLSI, EP17-A, 24, 34, Oct 2004. Protocols for Determination of Limits of Detection and Limits of Quantitation.
6 ICH guideline Q2(R1), Nov 2005. Validation of Analytical Procedures: Text and Methodology.
7 CLSI, EP17-A, 24, 34, Oct 2004. Protocols for Determination of Limits of Detection and Limits of Quantitation.
Clinical sensitivity The number of true positive specimens identified by a given assay as positive divided by the
number of specimens identified by the reference assays as positive, expressed as a percentage.9
Clinical specificity The number of true negative specimens identified by a given assay as negative, divided by the
number of specimens identified by the reference assays as negative, expressed as a percentage.10
Conformity The systematic examination of evidence generated and procedures undertaken by the manufacturer,
assessment under requirements established by the national regulatory authority (NRA), to determine that a
medical device is safe and performs as intended by the manufacturer and, therefore, conforms
to the Essential Principles of Safety and Performance of Medical Devices.12
Component Any raw material, substance, piece, part, software, firmware, labeling, or assembly which is intended
to be included as part of the finished, packaged, and labeled device.13
Complaint Any written, electronic, or oral communication that alleges deficiencies related to the identity,
quality, durability, reliability, safety, effectiveness, or performance of a device after it is released
for distribution.14
Complaints may be administrative (i.e. contractual) in nature or technical.
Corrective action Action to eliminate the cause of a detected nonconformity or other undesirable situation and to
prevent recurrence.16
External quality Monitoring of performance through either direct observation and supervision or inter-laboratory
assessment comparisons made possible by participation in an external quality assessment scheme (sometimes
known as proficiency testing).17
Field safety Action taken by the manufacturer to reduce a risk of death or serious deterioration in the state of
corrective action health associated with the use of a medical device that is already placed on the market. 18
(FSCA)
Field safety notice A communication sent out by the manufacturer or its representative to the device users in relation
(FSN) to a field safety corrective action.19
8 US Food and Drugs Administration, CFR - Code of Federal Regulations Title 21, 21CFR803.3
9 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
10 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
11 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
12 Essential Principles of Safety and Performance of Medical Devices are described in the GHTF/SG1/N41R9:2005 document.
13 US Food and Drugs Administration CFR – Code of Federal Regulations Title 21, 21CFR820.3
14 US Food and Drugs Administration CFR – Code of Federal Regulations Title 21, 21CFR820.3
15 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
16 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
17 WHO Laboratory Quality Management Systems Handbook, accessed 25 March 2014 at http://whqlibdoc.who.int/publications/2011/9789241548274_eng.pdf?ua=1
18 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
19 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
9
P ost-market surveillance of in vitro diagnostics
Harm Physical injury or damage to the health of people or damage to property or the environment.20
In vitro diagnostic A device, whether used alone or in combination, intended by the manufacturer for the in-vitro
(IVD) examination of specimens derived from the human body solely or principally to provide information
for diagnostic, monitoring or compatibility purposes. This includes reagents, calibrators, control
materials, specimen receptacles, software and related instruments or apparatus or other articles.22
Lot Defined amount of material, either starting material, intermediate or finished product which is
uniform in its properties and has been produced in one process or series of processes.23
Manufacturer24 The natural or legal person responsible for design, production, assignment of intended purpose,
packaging and labeling of the diagnostic product - whether these tasks are performed by that
person or on their behalf - and who represent themselves as the manufacturer on the diagnostic
product labeling.24
National Reference A testing laboratory which - in agreement with a specified laboratory community or through
Laboratory (NRL) appointment by a competent organization - provides reference values25 in a specific technical
field, i.e. property values of materials or products to which test results can be related or traced
back and whose quality is fit for the purpose.26
National Regulatory A government body or other entity that exercises a legal right to control the use or sale of medical
Authority (NRA) devices within its jurisdiction, and that may take enforcement actions to ensure that medical
products marketed within its jurisdiction comply with legal requirements.27
Preventive action Action to eliminate the cause of a potential nonconformity or other undesirable situation and to
prevent occurrence.29
Risk Combination of the probability of occurrence of a harm and the severity of that harm.31
Sample One or more units of product, either components or finished devices, drawn from a lot without
regard to the quality of the units.32
Surveillance Continuous, systematic collection, analysis and interpretation of health-related data needed for
the planning, implementation, and evaluation of public health practice.33
20 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
21 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
22 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
23 EN 13975:2003 Sampling procedures used for acceptance testing of in vitro diagnostic medical devices – statistical aspects.
24 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2). WHO has adopted this internationally accepted approach of defining a
manufacturer to ensure that there is a clear understanding of the term “manufacturer” for this product across international markets.
25 A reference values is a property value of a specified material or product that has been determined with an accuracy fit for use as a source of traceability of test results
obtained on comparable materials or products.
26 EUROLAB Position paper No. 1/2007, March 2007 on reference laboratories in the field of testing
27 GHTF, Principles of Conformity Assessment for Medical Devices GHTF/SG1/N78:2012
28 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
29 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
30 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
31 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
32 British Standard, Sampling procedures used for acceptance testing of in vitro diagnostic medical devices – statistical aspects, BS EN 13975:2003.
33 Adapted from the WHO website. Accessed on 19 November 2014 http://www.who.int/topics/public_health_surveillance/en/
Testing algorithm A testing algorithm describes the combination and sequence of specific HIV assays used within
a given HIV testing strategy.34
Testing strategy Generic description of a testing approach for a specific need (for example, blood transfusion and
transplantation safety, HIV surveillance, and/or diagnosis of HIV infection in both client-initiated
and provider-initiated testing and counselling), taking into consideration the presumed HIV
prevalence in the population being tested.35
Trend reporting A reporting type used by the manufacturer when a significant increase in the events not normally
considered to be incidents occurred and for which pre-defined trigger levels are used to determine
the threshold for reporting.36
Unanticipated A death or serious injury is considered unanticipated if the condition leading to the event was not
considered in a risk analysis performed during the design and development phase of the device.
There must be documented evidence in the design file that such analysis was used to reduce the
risk to an acceptable level.37
Use error An act, or omission of an act, that has a different result to that intended by the manufacturer or
expected by the operator.38
Verification Confirmation, through the provision of objective evidence, that specified requirements have
been fulfilled.39
Vigilance One of the post-market activities undertaken by the manufacturer to protect the health and
safety of patients, which relates to monitoring of adverse events (according to the definition of
an adverse event given above), investigation of adverse events to determine root causes and the
consequent corrective and preventive action .40
3.3 ACRONYMS
34 WHO Service delivery approaches to HIV testing and counselling (HTC): a strategic framework, accessed 25 March 2014 http://www.who.int/hiv/pub/vct/htc_framework/en/
35 WHO Service delivery approaches to HIV testing and counselling (HTC): a strategic framework, accessed 25 March 2014 http://www.who.int/hiv/pub/vct/htc_framework/en/
36 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
37 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
38 Definition taken from AAMI HE 74:2001² and IEC/CD2 60601-1-6:2002
39 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
40 GHTF, Glossary and Definitions of Terms Used in GHTF Documents, GHTF/SC/N4:2012 (Edition 2)
11
P ost-market surveillance of in vitro diagnostics
4. BASIC PRINCIPLES
Many countries The lack of regulatory oversight of IVDs in many countries, both for pre-market assessment and
lack effective IVD post-market activities, has widely been acknowledged as a shortcoming for assuring the safety,
regulation quality and performance of IVDs. The type of IVDs that are most appropriate and well adapted for
use in resource-limited settings are often used in jurisdictions without existing comprehensive
regulatory control; thus they may escape any stringent pre-market and post-market regulatory
oversight.
Pre-market A degree of pre-market assessment of IVDs is recommended for any product prior to entry into the
assessment as a marketplace in each country of intended use. While pre-market assessment of IVDs can provide
basis information on a product’s safety, quality and performance, there might be questions that cannot
be answered in the pre-market stage or issue that may arise after the product is marketed.
Post-market The safety, quality and performance of IVDs should be further verified upon delivery and before
surveillance to distribution to laboratories and other testing sites. Post-market information on IVDs empowers
protect public NRAs and WHO to detect, investigate, communicate and contain events that threaten public
health health security and to take appropriate action.
Reactive PMS Information on quality, safety or performance of an IVD on the market is collected reactively
through notification by users and evaluation by manufacturers of complaints, including
adverse events. The reactive nature of this statement refers to the fact that the problem has
already occurred, and may have affected a clinical decision.
Proactive PMS Additional information on quality, safety or performance may also be collected proactively through
lot verification testing. This relates to proactively trying to identify a problem before it affects a
clinical decision. Lot verification testing is conducted after shipment to the buyer (countries) and
can be performed both pre-distribution and post-distribution to end users.
Manufacturers should also collect post-market surveillance through actively gathering evidence
from the literature on their product or similar products, through seeking feedback from customers,
and post-market clinical follow up. This is a critical aspect that will not be widely covered in this
guidance as has been published elsewhere.41
Pre-distribution Post-distribution
Complaint
Other data Data on the quality, safety and performance of IVDs in the post-market phase may also come from
on-going external quality assessment schemes (EQAS), also known as proficiency testing. These
programmes collect testing results from testing sites that receive the same blinded specimens
to test. Lot numbers used to test these specimens should be recorded to make these useful data
sets. In resource-limited settings, national reference laboratories usually coordinate this data
collection and should provide feedback to WHO.
Quality control (QC) programmes with comparable data sets on particular test kits may also yield
important post-market information.
Risk management is a guiding principle in most aspects of health product manufacture and regulation. Risk management
principles have been considered in developing this guidance.
Risks of IVD use The risk associated with use of an IVD depends among other factors on the intended setting. In
depend on the high-resource settings, clinical testing typically takes place in accredited laboratories by certified
setting and proficient users using products that have undergone established pre-market assessment for
their appropriateness to be used in that intended setting.
However, in many countries, HIV and malaria testing for diagnosis using RDTs is performed outside
of the traditional laboratory setting and often by unsupported users with minimal training. In
addition, the risk that an incorrect test result will lead to serious consequences (death or serious
deterioration in health) for an individual or the population is greater in settings with higher disease
prevalence and lower access to diagnosis, care and treatment services.
13
P ost-market surveillance of in vitro diagnostics
What is meant by Indeed “risk management is a complex subject because each stakeholder places a different value on the
“risk”? probability of harm occurring and its severity. It is accepted that the concept of risk has two components:
a) the probability of occurrence of harm;
b) the consequences of that harm, that is, how severe it might be.” 42
The acceptability of a risk to an individual varies depending upon their cultural and economic
background, and many other factors.
How stakeholders Risk management applies to all stakeholders, including manufacturers, national authorities
manage risks (regulators and reference laboratories) and end users as well as procurers and implementing partners.
Each manufacturer is obliged to have undertaken risk management and risk assessment with
respect to their IVD before placing it on the market, implementing the most stringent controls
to those aspects of design, and manufacturing and control steps where the risk is greatest. When
a complaint is received, the manufacturer should review and update the risk management file
for the IVD accordingly.
Manufacturers, regulators and procurers will use the information that comes from complaints and
any other information and experience with a given IVD to determine the scope and stringency of
post-market actions in the future. For example, continued acceptable results in pre-distribution lot
verification testing may lead to a change from systematic sampling of each lot towards random
sampling of lots.
Figure 2 describes the process of risk management specifically for medical devices, and therefore, IVDs.
Risk analysis
Intended use and identification of characteristics related to the safety of the IVD
Risk assessment
Identification of hazards
Estimation of the risk(s) for each hazardous situation
Risk evaluation
Risk control
Risk management
*Adapted from ISO 14971 Medical devices -- Application of risk management to medical devices
42 Quoted from ISO 14971 Medical devices -- Application of risk management to medical devices, p.V
Each country should be responsible for establishing and strengthening systems for post-market surveillance, with the
necessary procedures in place to guide processes, and to define roles and responsibilities as these may differ from country
to country depending on the existing regulatory arrangements.
I End users, 1. Identify problems • End users should document any problems, and report complaints
procurers/ 2. Document problems (including adverse events) to the manufacturer, the relevant NRA and to
implementers WHO.
(see Part I of this 3. Report complaints
document) 4. Cooperate in lot verification testing • Trained and qualified personnel from testing sites (or laboratories) are
responsible for sampling of test kits for post-distribution lot verification
testing conducted under the oversight of NRAs.
• Procurers (specialized procurement agencies or implementing agencies)
should contribute to these activities on behalf of end users and in
accordance with quality assurance policies that govern their procurement
and distribution of IVDs.
III National regulatory 1. Collect reports of complaints • NRAs should conduct pre-market assessment and active rather than passive
authorities (NRAs) 2. Oversee lot verification testing post-market surveillance for products on sale within their market.
(see Part III of this
document) 3. Collect other post-market • Regulatory controls should be phased in depending on available regulatory
information capacity and resources, and using a risk-based approach.
4. Take regulatory action
IV National reference 1. Receive and store samples of test kits • Reference testing laboratories should conduct lot verification testing on
laboratories (NRLs) from central medical stores and from behalf of NRAs.
or other designated end users
testing laboratories • An example of a lot verification testing protocol for HIV RDTs is included in
2. Prepare and maintain lot verification Part IV this document.
(see Part IV of this testing panels
document)
3. Conduct testing and record data
4. Analyze data and report results to
NRAs
V WHO Provide support for post-market • WHO assures that WHO-prequalified IVDs continue to uphold their safety,
Prequalification of surveillance of IVDs quality and performance, and ensures traceability of WHO-prequalified
IVDs Programme IVDs. It provides support to manufacturers, NRAs/NRLs, and end users
facing problems with WHO-prequalified IVDs.
• The WHO Prequalification of In Vitro Diagnostics Programme reserves the
right to conduct follow-up inspections to ensure that corrective action have
been implemented where necessary following a complaint, and to inform
stakeholders.
15
P ost-market surveillance of in vitro diagnostics
Centralized PMS A centralized collection of post-market data on WHO-prequalified IVDs, including results from lot
data verification testing, and complaint collection/evaluation ensures traceability of information on
WHO prequalified IVDs and enables coordinated action in WHO Member States. These post-market
data are submitted to WHO in the form of lot testing reports and IVD complaint forms as defined in
Annexes 2 and 3. Certain adverse event reports are notified through vigilance information exchange
to other NRAs and procurers/implementing partners such as non-governmental organizations.
WHO action for Its network of post-market data sourcing and management enables WHO to take action with
prequalified IVDs regard to WHO-prequalified IVDs as appropriate. These might include:
• Post market surveillance information exchange with NRAs;
• Post market surveillance information exchange with manufacturers;
• Publishing certain post market surveillance information on WHO’s website;
• Removal of the product from the list of WHO-prequalified IVDs, if needed;
• Inspection of manufacturing site to ensure that corrective and preventive actions (CAPA) as
a result of any complaint have been implemented.
Crucial role End user feedback on the IVD’s performance in the field is of crucial importance for post-market
surveillance. It is through end users reporting on problems experienced with the use of IVDs that
manufacturers capture an essential part of the product’s post-market data.
This section describes the responsibilities of end users, and the post-market surveillance
activities that they should undertake – in cooperation with international procurers or programme
implementers –for post-market surveillance of IVDs.
End users should handle and use IVDs according to manufacturer’s instructions for use to maintain their quality, safety
and performance.
Quality The principles for quality systems in medical laboratories are laid down in ISO 15189 Medical
management laboratories — Particular requirements for quality and competence and include: organization,
system personnel, equipment, purchasing and inventory, process control (quality control), information
management, documents and records (standard operating procedures, standardized worksheets,
reports), occurrence management, assessment (external quality assessment schemes and
supervision), process improvement, customer service, and facilities and safety.
Storage Users must ensure proper storage of the test kits according to the manufacturers’ instructions for
use (either at climate-controlled room temperature or in a refrigerator), and should monitor the
temperature of the storage facility.
COMPLAINT REPORTING
The end users should notify the manufacturer of all complaints related to the use of their product. Furthermore, the relevant
NRA and WHO should be notified of any serious, moderate or change in the trend of mild adverse event related to an IVD.
These classifications will be described later in this guidance. In any case where the manufacturer is not aware of a complaint,
WHO or the relevant NRA should ensure they are informed. Complaint reporting is a reactive post-market surveillance
measure. It covers activities undertaken after any party becomes aware of adverse events, malfunctions, results of testing
or other relevant information about an IVD placed on the market. It is based on a cooperative and effective exchange of
information between all the parties.
Verify complaints In case of perceived complaints, the end user (in conjunction with appropriate technical expertise)
should document the complaint fully by determining all aspects (lot number, expiry date, storage
temperatures, etc.) and possible causes such as product quality, safety or performance, use error
and abnormal use.
The NRA (sometimes procurers and implementing partners), in conjunction with end users, should make arrangements
for lot verification testing to ascertain proactively that IVDs continue to conform to their specifications, and have not been
adversely affected by inappropriate storage and transport conditions.
43 Taken from ISO14971 Medical devices — Application of risk management to medical devices
17
P ost-market surveillance of in vitro diagnostics
Figure 3 gives an overview of the steps involved in identifying and addressing user-reported problems
Figure 3–Flow chart for end users to report complaints related to IVDs
1. IDENTIFY COMPLAINTS
Types of Complaints may include:
complaints • administrative/contractual complaints related to any aspect of the procurement contact not
fulfilled e.g. agreed delivery time not adhered to, agreed guaranteed shelf life upon delivery
not adhered to, incorrect product and/or quantity delivered, etc.
• technical complaint, affecting the safety, quality or performance of an IVD, for example:
malfunction or deterioration in the characteristics or performance, inadequate design or
manufacture; inaccuracy in the labelling, inappropriate instructions for use and/or promotional
materials, or any other issues might be reported that result in a significant public health
concern. Information about such issues may become available in other ways than through
reporting (for example through literature and other scientific documentation).
Adverse events Some technical complaints may lead to an adverse event. Adverse events (also called incidents)
(incidents) are consequences of problems with IVDs that may lead to death or serious deterioration in health
of a patient, user or other person. As an IVD is not directly used on an individual, the harm is
indirect “a result of an action taken or not taken on the basis of an incorrect result obtained with an
IVD”.44 Notification and evaluation of adverse events is also known as vigilance.
What should be Adverse events should be reported in any of the following circumstances:
reported? 1. When an incident leads to death of a patient, user or other person.
2. When an incident leads to serious deterioration in health of a patient, user or other
person (also known as serious injury).
3. No death or serious deterioration in health occurs but the event might lead to death or
serious deterioration in health.
4. When an incident might happen as a consequence of a medical decision or action taken
or not taken on the basis of results given by the IVD, typically:
• Misdiagnosis:
• Delayed diagnosis;
• Delayed treatment;
44 Quoted from Guidelines on a medical devices vigilance system, MEDDEV 2 12-1 rev. 8 Vigilance Accessed 18 March 2015 http://ec.europa.eu/growth/sectors/medical-
devices/documents/guidelines/files/meddev/2_12_1_ol_en.pdf
Identifying For IVDs used in a one-assay testing strategy (e.g. malaria IVDs), it may be easier to determine
incorrect test false positive and false negative rates.
results
For IVDs used in a multi-assay testing strategies (i.e. HIV IVDs), it may be difficult to attribute
misdiagnosis of the HIV status to one assay over another. False results might be caused by cross-
reactivity between test kits, which is not a product defect. Information on the testing algorithm
must be captured to understand the specificity and/or sensitivity attributes of a given test kit.
This is particularly important for a test kit that may be used interchangeably as a first line assay
in one country but as a second or third line assay in another country.
Impact of incorrect False positive HIV results may be less likely have an impact on people’s health and survival than
test results false negative HIV results. However, the psychological impact of a false positive HIV test result can
be enormous. Commencing an individual on treatment when they are not indeed positive for an
infection may increase the risk of drug toxicity, resistance, and in any case administering medication
and perhaps ordering additional testing is a waste of resources (both financial and otherwise).
False positive malaria results may cause the operator to assume malaria as the cause of clinical
signs and symptoms and mask another cause of febrile illness that may be life-threatening. False
negative malaria results will likely lead to withholding a prescription of anti-malarial drugs and
hence may have a life-threatening consequence.
Inadequate In the case of potential errors by users, labelling and instructions for use should be carefully
manufacturer reviewed for any possible inadequacy. Inadequacies in the information supplied by the manufacturer
instructions that led or could have led to harm to users, patients or third parties should be reported by the
manufacturer to WHO and/or the NRA.
2. DOCUMENT COMPLAINTS
Users should document any problems with IVDs using information taken from the testing/laboratory logbook and inventory
records including affected product code(s), affected lot number(s),and expiry date(s), affected consignments or test kits,
affected users, and any measures taken.
Photographs of affected test devices and/or test kits should be taken to illustrate the problem.
Users should keep and appropriately store at least 1-2 affected test kits (up to 60 tests) as retention kits for later testing,
if required.
45 Act, or omission of an act, that has a different result to that intended by the manufacturer or expected by the operator. See page 21 for details.
19
P ost-market surveillance of in vitro diagnostics
3. VERIFY COMPLAINTS
Documenting and verifying possible complaints will be conducted by the end user, in association with their site
supervisor. Users should conduct a preliminary investigation to identify complaints that are related purely to use
error or abnormal use, not to the IVD itself. These errors may be corrected at testing site, without need for additional
intervention from the manufacturer.
It may be difficult to determine if an adverse event was the consequence of a problem with the IVD itself, or of an error
by the user or third party. There should be a predisposition to report events to the manufacturer and also to WHO and
to the relevant NRA if suspected to be a serious or moderate adverse event.
A preliminary investigation and documentation step will also generate more detailed information on the circumstances
related to the complaint and enable the manufacturer to conduct a more in depth investigation of their own.
User identifies problem Accurate record-keeping, preferably using standardized To understand if the error can be attributed to a particular
testing/laboratory logbooks, to identify errors. operator, a particular lot number, a particular testing site, etc.
Create a site-specific trouble-shooting guide that includes To guide users on what should be raised as a complaint.
commonly observed anomalies (see below) and defects for
the specific types of IVDs used at the site.
Analyze EQAS and QC data. To reveal quality or performance issues related to specific IVDs.
User documents problem Using testing/laboratory logbook and inventory records, To ascertain which lots and/or consignments are affected, and
document the problem, e.g. affected product code(s), affected the scope of the event.
lot number(s), and expiry date(s), affected consignments or To trace the history of the issue and subsequent actions.
kits, affected users, any measures taken, etc.
Keep and appropriately store at least 1-2 affected test kits (up To confirm kit-related errors.
to 60 tests) as retention kits for later testing.
User investigates problem Test other lots, other consignments or kits. To ascertain which lots and/or consignments are affected, and
the scope of the event.
Repeat testing using concise but exact standard operating To identify operator error due to noncompliance with
procedure as per manufacturers’ instructions for use. instructions.
Interview or observe operators in the affected facilities (see To identify any human factors e.g. reading time used
below for details). (minimum and maximum), specimen transfer device (pipette),
specimen type, invalid rate (instrument failures and unable to
return results error reports), etc.
How to identify Each testing site should create a job aid of common anomalies and defects to watch for which will
anomalies depend on the IVD format. For example, the following list might be used for anomalies related
to RDTs:
• dry alcohol swabs;
• very soft or very stiff specimen transfer devices that let out too much or too little liquid;
• graduation marks on specimen transfer devices that are printed in a way that may be erased easily;
• insufficient buffer to complete all tests within a test kit;
• excessively high background or streaking that obscures the reading window;
• misplaced test strip that means test and control lines are not lined up with the reading legend;
• visually obvious problems with specimen migration, etc.
Photographs of commonly occurring anomalies could be created for each of the IVDs in use for
easier visual identification of defective IVDs.
Use error A use error is an act, or omission of an act, that has a different result to that intended by the
manufacturer or expected by the operator. This might include lapses or mistakes and reasonably
foreseeable misuse. Examples include:
• Operator presses the wrong button;
• Operator misinterprets the icon and selects the wrong function;
• Operator disregards incubation or reading time;
• Unintentional use of pipette out of calibration range; and
• Analyzer placed in direct sunlight causing higher reaction temperature than specified.46
Abnormal use error An abnormal use error is an act, or omission of an act, by the operator or user of a medical device
as a result of conduct that is beyond any reasonable means of risk control by the manufacturer.
Examples include:
• Use of an recently installed IVD prior to completing all initial performance checks as specified
by the manufacturer;
• Failure to conduct IVD’s quality control checks prior to each use as defined by the manufacturer;
• Continued use of an IVD beyond the manufacturer defined planned maintenance interval as
a result of operator’s or user’s failure to arrange for maintenance; and
• Product analysis showed that the IVD was working in accordance to specifications; but further
investigation revealed that the operator was inadequately trained due to failure to obtain
proper training.47
Abnormal use related to WHO prequalified IVDs does not need to be reported to WHO under
complaint reporting procedures. However, it may be useful for users to report abnormal use to
manufacturers, and particularly, if the issue relates to unclear instructions for use.
If manufacturers become aware of instances of abnormal use, they may bring this to the attention
of other appropriate organizations and healthcare facility personnel.
Immediate reporting All verified complaints should be reported by the end user to to the manufacturer as soon as
possible using the format in Annex 3.
In addition, any complaints that are classified as serious, moderate or a change in trend of
mild adverse events should be reported to the relevant NRA, and WHO as soon as possible
using the format in Annex 3.
46 This list does not purport to be definitive and each case should be handled individually.
47 This list does not purport to be definitive and each case should be handled individually.
21
P ost-market surveillance of in vitro diagnostics
The objective of lot verification testing is to verify that each lot supplied continues to meet its safety, quality and performance
requirements, and that transport and/or storage conditions have been well controlled so as not to affect the performance
of the IVD.
Lot verification testing should be organized under the oversight of the NRA, or of the procurement agency in accordance
with their quality assurance policies. The testing should be done by a suitably qualified reference laboratory.
Table 3 gives an overview of the processes of pre- and post-distribution lot verification testing.
Pre-distribution Post-distribution
Physical inspection During sampling the test kits should be inspected for damage or deterioration. Test kits with torn,
ripped, broken outer or inner packaging, test pouches not sealed properly, desiccant not included,
etc., should be documented, and excluded from testing.
Other quality issues such as poorly affixed or illegible labelling should be documented as well as
lack of components (accessories, instructions for use).
Photographs of all components and labelling should be taken for each lot tested as a record.
Transport to testing Samples of test kits should be transported to the reference laboratory without delay, through
laboratory the usually utilized means of transport of test kits within the country. A temperature monitoring
device may be used.
Receive results The testing laboratory will report the results within five days to WHO and the relevant NRA or
other national authority delegated to evaluate post-market information on IVDs. (See lot testing
report in Annex 2).
Outcomes If the lot testing report confirms that the acceptance criteria are met, the lot can be distributed
or continue to be used. After a period of consistently acceptable results, the sampling frequency
for further lot testing may be reduced. WHO and the NRA should monitor the results of lot testing
for any trends or shifts.
Handling If the report states that the criteria fail to be met, WHO should be informed and a joint
non-conforming decision with national authorities about next steps should be made. The use of the lot in
results question should be ceased, and the lot put into quarantine until the matter is resolved.
For any nonconforming lot, WHO and NRA will ensure that the manufacturer undertakes a root
cause analysis and conducts field safety corrective action, if required (see Part II for details).
NB: The replacement of the purchased test kits is not recommended until the root cause
analysis is complete and the proposed corrective action has been verified by WHO and
relevant NRA. The sampling frequency for further lot testing may be increased.
23
P ost-market surveillance of in vitro diagnostics
u OVERVIEW OF RESPONSIBILITIES
Knowledge of Manufacturers of IVDs should be familiar with standards including ISO 9001:2008 Quality
relevant standards management systems - Requirements, ISO 13485:2003 Medical devices - Quality management
systems - Requirements for regulatory purposes and ISO 14971:2007 Medical devices - Application of
risk management, which outline their requirements for compliance with post-market surveillance
aspects of these standards.
Manufacturers Manufacturers of IVDs are expected to adhere to available international standards such as ISO
must verify each 2859:2006 Sampling procedures for inspection by attributes series and ISO 3951: 2013 Sampling
lot pre-shipment procedures for inspection by variables series to verify the safety, quality and performance of each
lot manufactured of their products.
Manufacturers are obliged to perform quality control lot release as part of the requirements
of ISO 13485:2003 Medical devices -- Quality management systems -- Requirements for regulatory
purposes, which states that there must be adequate monitoring and measurement of product and
evidence of conformity with an agreed acceptance criteria. Where manufacturers purchase key
components for the product, these components must be verified to ensure they meet specified
purchasing requirements. Furthermore, there must be a process to identify and control product
that does not conform to requirements and to prevent its unintended use or delivery.
Complaint All types of reports related to complaints (including adverse events) should be maintained by
handling the manufacturer including: initial/follow-up/final manufacturer investigation reports, root cause
and vigilance analysis reports, corrective action/prevention action plans, any Field Safety Corrective Action and
Field Safety Notices, and annual post-market surveillance summary reports. Vigilance information
exchange with NRAs should be managed according to national regulations.
Field safety correc- Manufacturers should have in place procedures to facilitate FSCA, including designated personnel,
tive action (FSCA) and to maintain records to facilitate traceability for lots of IVDs distributed to users.
Reporting 1. Any serious adverse event should be reported by the manufacturer to the relevant NRA
timelines for within their respective timelines, and to WHO within 10 days.
complaints 2. Any moderate adverse event or any change in the trend of mild adverse events should
be reported by the manufacturer to the relevant NRA within their respective timelines,
and to WHO within 30 days.
3. All complaints (both administrative and technical including serious, moderate and mild
adverse events) must be reported by the manufacturer annually to the relevant NRA, and
to WHO as a periodic summary report.
Any relevant NRA (NRA in the country where the complaint takes place) may have their own
reporting templates and specific deadlines. These should be adopted, where appropriate. If no
guidance exists from the relevant NRA, annexes to this guidance should be utilized.
48 Manufacturers of prequalified IVDs should notify WHO of their responsible person for PMS during the WHO prequalification process.
Reporting timelines Manufacturers should submit the initial manufacturer investigation report to WHO in a timely
for manufacturer manner; for any WHO-prequalified product within 15 days of first becoming aware of the adverse
investigation event (See template for adverse event manufacturer investigation reporting form in annex 4).
reports Subsequent follow-up and final manufacturer investigation reports should be submitted in a
timely manner as the nature of the adverse event dictates.
1. CLASSIFY COMPLAINTS
A method of classification should be used identify the quality, safety and performance issues that pose a high risk to
individual health and to public health, and therefore require the most immediate action to protect public health and safety.
As previously described, complaints may include:
• administrative/contractual complaints; and
• technical complaints.
As soon as it is received, any complaint must be classified by the manufacturer as part of the risk management file for the
product, see Introduction (section 4.3) on risk management for IVDs. The degree of risk will determine the timeline for
action, and who should be informed. The requirement for root cause analysis will remain, irrespective of the classification.
In general, most technical complaints will be considered as an adverse event, and should be classified, an example is
shown in Table 4.
Note: not every complaint may need to be considered as an adverse event, and not every adverse event or potential adverse
event may lead to a field safety corrective action.
Table 4 – Examples of adverse event classifications (this is not a list of exhaustive examples)
Serious adverse event • Death of patient, user or other person • One or more individuals receive HIV-contaminated blood
(10 days) product that has been produced from one blood donation that
• Serious injury of patient, user or other person was screened as HIV negative by an HIV-1/2 RDT.
• Death or serious injury of patient, user or other person did not • An individual presenting for ART initiation has testing
occur but might have repeated to confirm their HIV diagnosis. The re-testing results
• Any false negative result are negative.
Moderate adverse event • Any false positive result (that resulted in misdiagnosis) • Invalid rate exceeds 5%.
(30 days)
• Higher than expected rate of anomalies that lead to invalid, • High background for rapid diagnostic tests.
unreturnable or inconclusive results
• Greater than expected discrepant rate between assay 1 and
assay 2 within a testing algorithm.
Mild adverse event • Deficiency found by the user prior to use • Control line does not appear.
(30 days)
• Adverse event caused by patient conditions • Higher than usual background, may or may not obscure
reading window and prevent reading.
• Adverse event caused by device exceeding its service life or
shelf life • Desiccant has changed colour.
• Malfunction protection operated correctly • A component labelled lyophilized is found to be fluid, this is
discovered by the user prior to use.
• Negligible likelihood of occurrence of death or serious injury
• The packaging of a device is labelled with the caution ‘do
• Unexpected and foreseeable side effects not use if the packaging is opened or damaged’. Prior to use,
• Adverse events that might be described by the manufacturer obvious damage to the packaging was observed, and the
in FSN device was not used.
25
P ost-market surveillance of in vitro diagnostics
For each complaint received, the manufacturer should undertake a root cause analysis to determine if the complaint (including
adverse events) can be verified and root cause can be established. Depending on the findings the risk management file
should be updated.
Root cause analysis is a systematic approach to investigating why and how a problem took place, in order to prevent its
reoccurrence. There are a number of tools that may be used such as a fishbone diagram.
Following the investigation of the complaint, the manufacturer should consider the following possibilities:
• No action;
• Immediate correction;
• Additional surveillance of the IVD in use;
• Design modification, manufacturing process modification, etc.;
• Field safety corrective action, including recall or quarantine of existing stock, modification instructions for use or
product labeling;
• Field safety advisory notice issuance, including urgent information to inform those responsible for the device, or affected
by the problem;
• Retraining;
• Other possible action, such as retesting of individuals and/or special monitoring of individuals previously tested using
the affected IVD.
Corrective and Based on the results of root cause analysis, corrective and/or preventive action should be taken,
preventive action where necessary. Corrective action/preventive action (CAPA) are improvements made to the
(CAPA) manufacturing process as part of the overall quality management system to eliminate causes
of nonconformities. Any process for CAPA should focus on the systematic investigation of
discrepancies (failures and/or deviations) in an attempt to prevent their reoccurrence. To ensure
that corrective and preventive actions are effective, the systematic investigation of the failure
incidence is pivotal in identifying the corrective and preventive action undertaken. The degree
of action taken should be dependent upon and related to the risk, size and nature of the problem
and its effect(s) on product quality.
Corrective action should be handled according to ISO 13485:2003, Section 8.3. (control of
nonconforming product) and 8.5.2. (corrective action), depending on whether a nonconforming
IVD is involved or if action is taken to prevent recurrence of a nonconforming IVD.
Definition A field safety corrective action (FSCA) is an action taken by the manufacturer to reduce a risk of
death or serious deterioration in the state of health associated with the use of a medical device
that is already placed on the market.
What can trigger A FSCA is triggered by information about any problem with an already distributed IVD that poses
FSCA an unacceptable increased risk when that IVD is used. Such problems include malfunction or
deterioration affecting the performance or operational characteristics of an IVD, as well as any
inadequacy in the instructions for use which might lead or might have led to the death of a patient,
user or other individual or to a serious deterioration in his/her state of health.
Such information may arise from any aspect of post-market surveillance: pre-distribution or
post-distribution lot testing, report from the field, review of IVD design, changes in production
or component specifications, etc.
Assessing the need In assessing the need for the FSCA, the manufacturer is advised to use the methodology described
for FSCA in the standards: ISO 14971:2007 Medical devices - Application of risk management to medical devices
and “Implementation of risk management principles and activities within a Quality Management
System” (GHTF/SG3/N15R8).
Risk assessment is thus a key element of the manufacturer determining the need for a FSCA.
Appropriate expertise must be used to determine the potential harm and the risk properly.
Undertaking FSCA When the need for a FSCA of an IVD has been established, the manufacturer of the affected product
assumes the responsibility for recovery of the goods and implementation of the corrective action.
WHO and the relevant NRAs may assist by monitoring the overall action.
27
P ost-market surveillance of in vitro diagnostics
Communicating A FSCA is communicated through a Field Safety Notice (FSN); see section 3.3 for details.
FSCA
The manufacturer is required to report any FSCA related to a WHO-prequalified IVD to WHO
Reporting FSCA and to the relevant NRA where the IVD is supplied.
The manufacturer should issue a notification to WHO and to NRAs of all the countries affected
through a FSCA report. A format is proposed in Annex 5.
Follow-up FSCA A follow-up report should be submitted by the manufacturer to WHO within a maximum of 30
report days from the initial notification of the FSCA.
49 The NRA should ensure a record of disposal of affected product, and inform the manufacturer so that they may reconcile product distributed and product destroyed.
If the FSCA includes return of affected stock to the manufacturer or an update of the instructions
for use or a modification/update of existing IVDs on- or off-site, records of completed actions
should be fully reconciled against distribution records in order to maintain control of the progress
of the FSCA.
Purpose Field Safety Notices are an important means of communicating FSCA and related safety information
to users. They may also be used to provide updated information about how an IVD should be used.
Distribution Manufacturers should inform affected users of any FSCA via FSN, with copy to the relevant NRAs
of FSN and to WHO. The manufacturer should ensure that the FSN is distributed to all affected users,
and must keep track of confirmation of receipt of the FSN. A full, detailed distribution list with
contact name and email address for each intended recipient must be kept and must be made
available on request.
Affected users are will usually receive the FSN via their procurement agents or through in-country
distributors who are obligated to inform all users within their region of supply. Distributors may
need to translate the FSN from English or other common language to local language but this
needs to be managed to ensure that the translation is of good quality and interprets the message
of the FSN correctly.
Further purchasing information requirements are specified in the ISO/TR 149696:2004 Medical
devices - Quality management systems - Guidance on the application of ISO 13485:2003.
Content The manufacturer should use a standardized format for a FSN (see example in Annex 6).The FSN
and format should be written on company letter head and in English. It may be translated into the official
language(s) of the country by in-country distributors.
29
P ost-market surveillance of in vitro diagnostics
Prior consultation It is recommended that manufacturers should provide a draft of the FSN to WHO and the
with WHO and NRA relevant NRA, allowing a minimum of 48 hours for comment unless the nature of the FSCA
dictates shorter timescale e.g. for serious public health threat.
In a very few cases, where an urgent FSCA is needed because of serious safety risks, WHO accepts
that prior consultation may not be possible.
Where WHO believes that the FSN does not fully meet the requirements as described in this
document, and in particular explain the risk and how it will be removed/reduced, WHO may issue
its own information notice and send it to the relevant NRAs for further dissemination to users.
WHO information WHO reserves the right to issue information notices for users in certain circumstances: if
notices the manufacturer has not undertaken an appropriate FSCA within an appropriate time frame,
if the manufacturer has not disseminated an appropriate FSN, and to give information to users
about how to interpret the contents of FSN.
WHO information notices may include any recommendations to programmes and implementing
partners for alternative testing arrangements and to procurers for past, on-going or future purchase
orders of affected or potentially affected products.
Obligations for WHO Manufacturers of WHO-prequalified products agree, as a condition of WHO prequalification, to
prequalified IVDs undertake the following post-market surveillance obligations:
• Notify WHO of any post-market events relating to the WHO prequalified product that have
affected (or could have affected) the performance of the assay, safety of the individual being
tested, safety of users of the assay or safety of any person associated with the assay.
-- All complaints (both administrative and technical including serious, moderate and mild
adverse events) must be reported to WHO annually, as a periodic summary report.
-- Any serious adverse event should be reported to WHO within 10 days.
-- Any moderate adverse event or any change in the trend of mild adverse events should be
reported to WHO, within 30 days.
• WHO may request that the manufacturer provide further information relating to the incident,
including details regarding the preventive and corrective action taken.
• Manufacturers of WHO-prequalified IVDs are responsible for activating their complaint reporting
and vigilance system and must inform WHO of reportable adverse events. The manufacturer
should encourage end users to report on problems experienced with the use of an IVD.
• Notify WHO of all events which require FSCAs such as withdrawal of IVDs from sale or distribution,
physical return of the IVD to the manufacturer, IVD exchange, destruction of the IVD, IVD
modification/s or additional advice provision to customers to ensure that the IVD continues
to function as intended; and
• Submit information on all complaints, including any FSCA, carried out in the previous calendar
year as part of the mandatory annual summary reporting.
A national IVD focal point should be assigned. The IVD focal point may already exist, if a functional
unit for IVDs exists within the NRA, but if this is not the case the focal point may come from the
NRL, or the Ministry of Health. Preferably one national IVD focal point should be assigned for all
IVDs (with particular emphasis on WHO-prequalified products), irrespective of the analyte or test
kit format.
Reference testing The NRA should designate a NRL or other recognized laboratory that is assigned the overall
laboratory responsibility for coordinating and conducting pre-distribution and post-distribution lot
verification testing.
Information- The NRA should establish an information circuit between end users (and procurers/implementers)
sharing and IVD manufacturers, and the NRA’s responsible person for post-market surveillance (or national
IVD focal point), allowing post-market information exchange.
Lot verification To ensure that IVDs continue to meet their specifications, national regulatory authorities have
testing the mandate to arrange proactive lot verification testing by a reference laboratory including:
• Pre-distribution to testing sites (when a consignment of test kits arrives in country);and
• Post-distribution to testing sites (after the lot has already been in use).
Complaints The NRA should ensure that any FSCA recommended by the manufacturer are implemented.
Regulatory action The NRA should take regulatory actions as appropriate to address any issues identified through
post-market surveillance activities.
Interaction Post-market data on WHO-prequalified IVDs gathered through pre-distribution and post-distribution
with WHO lot testing and vigilance procedures or collected through other means are encouraged to be shared
with WHO as a part of post-market information exchange process for prequalified IVDs, ensuring
product traceability and contributing to public health protection. See Annex 7 for a post-market
information exchange reporting form template for NRAs to use.
Risk-based Full and complete post-market surveillance for all products is not feasible with available regulatory
approach capacity and resources. Therefore, NRAs are encouraged to adopt a risk-based approach to both
pre-market assessment and post-market surveillance of IVDs placed on their market according to
a set of risk classification rules. The rules set down by the Global Harmonization Task Force (GHTF)
are useful in this regard, given their adoption by the successor of GTHF; the International Medical
Device Regulators Forum (IMDRF). Risk classes A through D were developed taking into account
risk to the individual and risk to the public.50 The level of regulatory scrutiny will depend on the
risk the IVD presents and the setting of its intended use.
Other prioritization criteria may be the risk class of an IVD, its scope of use and other factors. The
information given in Table 4 (Part II) may be useful in this regard.
50 Taken from “A risk based approach for the assessment of in vitro diagnostics (IVDs) accessed 12 December 2014
http://www.who.int/diagnostics_laboratory/evaluations/140513_risk_based_assessment_approach_buffet.pdf?ua=1
31
P ost-market surveillance of in vitro diagnostics
Phased The implementation of post-market surveillance measures will depend on the maturation and
implementation capacity of the NRA to handle complaints. In the absence of fully fledged post-market regulatory
controls, WHO will serve to handle complaints from end users, including procurers and implementers,
and ensure that manufacturers are informed and follow-up appropriately.
End users The NRA should receive reports for certain categories of complaints submitted by end users as
described in Part I of this document and take appropriate regulatory action.
Sentinel sites An alternative arrangement may be to establish sentinel surveillance sites with primary responsibility
to look for issues related to safety, quality and performance of IVDs; these should be under the
supervision of the NRL (or designated competent laboratory) and NRA. A sentinel site would be
responsible for active collection of information on safety, quality and performance of IVDs, through
external quality assessment (on-site supervision, etc.). Sentinel sites may also provide testing sites
within their geographical area with specimen panels for lot verification testing.
The procedure for receiving, reviewing and acting on complaints (including adverse events)
comprises of the following mechanisms:
Manufacturer • Reports from end users (including implementing partners) are forwarded all complaints to
(see also Part II) manufacturer and all serious and moderate adverse events to the relevant NRA and WHO;
• WHO and the relevant NRA ensures manufacturer is aware of complaints, equally manufacturers
must inform WHO and the relevant NRA of all serious and moderate adverse events;
• Manufacturer evaluates the reported complaint, including serious, moderate, and mild adverse
events, and conducts root cause analysis;
• Manufacturer submits initial adverse event manufacturer investigation report to the relevant
NRA and WHO;
• Where appropriate, manufacturer recommends Field Safety Corrective Action and submits a
FSCA report to WHO and/or NRA;
-- Where appropriate, manufacturer disseminates information related to the FSCA to affected
users through a Field Safety Notice;
-- Where appropriate, manufacturer in collaboration with the relevant NRA and WHO
recommend modifications of the IVD or its removal from the market and therefore the list
of WHO-prequalified products;
-- WHO ensure implementing partners are informed;
• Manufacturer submits adverse event manufacturer investigation report to WHO and/or the NRA.
NRA follow-up The NRA follows up on the results of the manufacturer’s report by appropriate regulatory action.
The WHO Prequalification of In Vitro Diagnostics Programme has its own complaints handling procedure (Figure 4). WHO will
handle any complaints received about WHO prequalified IVDs. WHO will notify the manufacturer of the complaint, and will
notify the relevant NRA in the country/region where the product is manufactured and where the product is being supplied.
WHO will also inform affected procurement agencies and other UN organizations of any relevant FSCA, if it is felt that
this information would be of interest. Subject to the protection of commercially sensitive information WHO is entitled to
make public any relevant Field Safety Notices and information notices for users. In addition, WHO reserves the right to
share the post-market surveillance reports with the relevant authorities of interested Member States and UN agencies. The
responsibilities of manufacturers of WHO-prequalified IVDs in this regard are described in Part II.
In sufficient information,
additional information requested
33
P ost-market surveillance of in vitro diagnostics
2.1 RATIONALE
NRAs have the mandate to order lot testing to verify the quality, safety and performance of IVDs. Where the relevant NRA
does not exist or insufficient capacity to conduct post-market surveillance activities, the end user (including procurers and
implementing partners) may request lot verification testing.
In the context of WHO-prequalified IVDs, lot verification testing ensures that manufactured lots continue to meet the WHO
prequalification requirements for safety, quality, and performance51. Secondly, lot testing can assess variation between
lots and captures deviation in performance in comparison with the preceding lot(s).
Lot verification testing is not intended to duplicate the manufacturer’s quality control (QC) lot testing undertaken
throughout the manufacturing process, and at final release of the product. Rather it aims to ascertain that IVDs continue
to meet their standards at all stages of their distribution and use.
Manufactured lots Despite the manufacturers’ obligation to test each lot during production and at release, variations
can differ greatly in the performance characteristics of each lot may occur due to differences in the lots of key
components (starting materials) used, different personnel involved in production processes,
variations in manufacturing processes, and a range of other variables.
What is meant Lot sizes may vary from 5,000 tests per lot to 1,000,000 tests per lot, depending on how the
by “lot”? manufacturing site configures their operations. The EU standard EN 13975: 2003 Sampling
procedures used for acceptance testing of in vitro diagnostic medical devices – Statistical aspect defines
the term lot/batch as follows: “A defined amount of material that is uniform in its properties and
has been produced in one process or series of processes. The material can be either starting material,
intermediate material or finished product”. In the context of these guidelines, lot testing is focused
on a commercially available test kit which is provided with a unique lot number and where the
single components are matched to this kit, e.g. micro-plate or nitrocellulose membrane, antigen,
conjugate, specimen diluent, etc.
Transport and Inappropriate transport and storage conditions such as high or low temperature, high humidity
storage affect IVDs and the presence of direct sunlight can have a considerable effect on the performance and
quality of an IVD.
Pre-distribution The approach described here supposes that adequate pre-market assessment has been conducted
testing and that the clinical and analytical sensitivity and specificity of the IVDs are well known. The objective
of pre-distribution lot verification testing is to ensure lot-to-lot consistency when performance
of a baseline lot is known (through WHO prequalification or otherwise).
The purpose of pre-distribution testing is to ensure that the diagnostics delivered to country meet
requirements for safety, quality and performance and that the manufacturer’s claims of product’s
performance can be verified in the state as it is procured. Lot testing ensures that only lots with
acceptable test results are distributed to laboratories and testing sites all over the country.
Post-distribution Post-distribution lot testing assumes that pre-distribution lot testing has been undertaken on
testing the same lot and brings additional information on the safety, quality and performance of the
IVD after it has been distributed to end users. Testing of samples from the field, in combination
with pre-distribution lot verification testing, guarantees monitoring of IVD quality throughout
the distribution chain thus ensuring that only quality lots of IVDs that fully meet manufacturers’
claims for stability are used by end users.
51 For non-WHO prequalifed IVDs, lot verification testing will ensure that pre-market assessment performance criteria remain adhered to.
This approach to lot verification testing will not necessarily be able to detect quality issues if the lot has not been homogenously
produced. Pre-distribution lot testing cannot detect stability issues that might lead to degradation of the shelf-life of the
product, including all components and accessories. However, post-distribution lot testing may detect stability issues.
2.3 METHOD
Testing known Lot verification testing verifies whether or not the IVD correctly classifies a set of clinically-derived
specimens reference specimens in a panel. Where possible the panel will be locally or regionally derived, so
that the same specimen panel can be used by a number of countries with similar epidemiology
and genetic diversity, and data can be compared. Detail on preparation of specimen panels and
lot verification testing by the reference laboratory is given in Part IV of this document.
Sampling Samples should be taken by appropriately trained and qualified personnel from the NRA (or
central medical stores) for pre-distribution testing, and from the testing sites for post-distribution
testing (see also Part I). Samples should be transported to the reference testing laboratory in such
a way that the integrity of the test kits is not adversely affected and that the appropriate storage
conditions, as specified by the manufacturer, are maintained. Temperature log monitors should
be included within the transportation packing for the samples.
Sampling frame For pre-distribution lot verification testing, each lot should be sampled initially. After a certain
period of acceptable results (12 months or 10 lots, whichever comes first), the sampling frame may
change from systematic sampling and testing of each lot, to random sampling of lots delivered to
countries. The random sampling frame should be selected (every 5th lot). If any issue is observed
with random pre-distribution lot verification testing, the national authorities may elect to re-
commence systematic testing of each lot. The decision about the sampling frame is therefore
made using a risk-based approach.
Post-distribution lot testing carries a different risk as the product has already been in use, and
depending on the through-put of the testing site, it may or may not be close to its expiry date.
Therefore, the sampling frame does not need to include every lot delivered to country, but should
rather be conducted twice per year.
See Part I (Table 3) for details on sampling at central medical stores and in laboratories/testing sites.
Reference A reference laboratory should be identified to perform lot verification testing as described in Part
laboratory IV. The same laboratory should perform both pre-distribution and post-distribution lot verification
testing for the same lot of product. A suggested protocol for HIV RDT lot verification testing, with
provisions for reporting of results and confidentiality, is included in Part IV.
For malaria rapid diagnostic test (RDTs), a system for lot testing through the WHO/FIND Lot Testing Programme already exists.
35
P ost-market surveillance of in vitro diagnostics
EQAS Where it exists, the data generated by an EQAS can be assessed. Although the primary purpose
of EQAS is inter-laboratory comparison, these data can provide very useful information about
the performance of IVDs.
EQAS data analysis may indicate not only operator-related errors (for example transcription errors),
but also errors related to the test itself, especially if large numbers of laboratories/testing sites
are using the same test kit.
QC A QC specimen is a specimen that has reactivity that is just above the cut-off for positivity of a test
kit. It is usually a manufactured specimen that is optimized for the test kit and may be provided by
a national authorities to all laboratories/testing sites using the test kit as part of a QC programme.
All attempts should be made to procure and distribute QC material to any site undertaking testing
QC specimens should be tested in each test run (for an immunochromatographic RDT, no more
than 10 tests per run). The results of QC specimens can be graphically represented and results
outside a pre-determined acceptance range identified and investigated. This approach is typically
not all that useful for RDTs that generally are used in settings outside of the laboratories without
test kit controls and external QC specimens.
Sentinel sites If sentinel surveillance sites are established, these sites could collect information on safety, quality
and performance of IVDs.
The classification of complaints given in Table 4 (Part II) may be useful in prioritizing regulatory action.
At country level, the NRA may not have a function that relates specifically to IVDs. WHO can bridge this lack of capacity
and support/conduct post-market surveillance activities. However, WHO encourages each WHO Member State to
building pre-market assessment and post-market surveillance capabilities, specifically for IVDs.
52 This list does not purport to be definitive and each case should be handled individually.
Pre-distribution The reference laboratory should receive samples for pre-distribution lot verification testing which
lot verification have been sampled from newly arriving consignments at the central medical stores. For laboratories
testing with direct procurement (i.e. no centralized storage as central medical stores), samples should
be taken by their staff and sent to the reference laboratory for testing. The samples should be
transported to the reference laboratory in such a way that the integrity of the IVDs is not adversely
affected and that the storage conditions specified by the manufacturer are maintained. The testing
should be conducted on a standardized lot verification panel. Further details on sampling
procedures at central medical stores and laboratories/testing sites are found in Part I (Table 3). The
reference laboratory should present testing results in the form of a lot testing report as defined
in Annex 2 which would be sent to the relevant NRA and to WHO.
Post-distribution The reference laboratory should receive samples taken from test kits in the field by staff at the
lot verification testing sites themselves (or NRA or central medical stores) and transported to the reference
testing laboratory through the specimen referral network or other network for distributing IVDs within
the country. The testing should be carried out using the same standardized lot verification panel
as the pre-distribution lot testing, and testing results presented in the form of a lot testing report
as defined in Annex 2.
Competencies The staff performing the lot testing should be qualified and competent to undertake the task and
of testing staff to demonstrate that they can perform the test procedure correctly.
37
P ost-market surveillance of in vitro diagnostics
The supervisor and technicians should not proceed with testing until they are confident regarding
every aspect of the testing procedure.
The reference laboratory (in conjunction with the end user) should be able to undertake an
investigation of perceived problems to eliminate all other possible causes that are not test kit-
related (i.e. use errors, abnormal use, poor implementation of the quality management system
at the testing site).
EXAMPLE: PROTOCOL FOR HIV RDT LOT VERIFICATION TESTING (PRE- OR POST-DISTRIBUTION)53
Objective of lot To verify the performance of an IVD (i.e. sensitivity and specificity) in the state that the test kit
verification testing is delivered to the buyer by testing of the lot verification panel of well-characterized biological
specimens.
Each lot is evaluated by testing the same set of specimens so that variation in assay performance
over time can be monitored and any catastrophic product failure identified.
Storage Test kits must be stored at the temperatures indicated by the manufacturer up to the expiry dates
printed on the labels. Since many RDTs allow for storage at a wider temperature range (e.g. 2-30°C),
RDTs may be stored permanently in a consistent temperature range, e.g. in a refrigerator at 2-8°C
or at climate-controlled room temperature. If a test kit package is opened and kit components
are reconstituted, the reagents should be labelled with the appropriate date of opening of the
component and the remaining residual amount of tests. Any separate shorter storage periods in
compliance with the instructions for use should be taken into account.
Safety The testing of all biological specimens should be performed in such a manner as to minimize
occupational risk. For further details see the Laboratory Biosafety Manual, third edition, World
Health Organization, Geneva, 2004 (ISBN 92 4 154650 6).
Best practice Biological specimens should be collected under usual best practice for phlebotomy for use in
the lot verification panel. In order for specimens to be of good quality (free of clotting, lipids,
hemolysis, microbial contamination), all specimens should be processed to serum or plasma within
6 hours of collection and any clots and particulate matter removed. The specimens should then
be aliquotted into appropriate volumes (e.g. 50 – 200 µl) and stored until required.
53 Through the WHO-FIND Malaria RDT Lot Testing Programme, separate protocols are in place for lot verification of malaria RDTs. See the FIND website for further details
http://www.finddiagnostics.org/programs/malaria-afs/malaria/rdt_quality_control/lot_testing/
The specimens should be stored in freezers and refrigerators that are monitored through an alarm
system. Panels may be split among a number of freezers so that the risk of damage to stored
panels due to power outage to a freezer is mitigated.
The HIV RDT lot verification panel will consist of well-characterized HIV seropositive and
seronegative clinical specimens, including some dilution series. This panel should be specifically
constructed for the RDTs to be lot-tested. The claims for performance made by the manufacturer
in the instructions for use should be reflected in the lot testing panel, e.g. detection of HIV type
(HIV-1 and HIV-2), HIV-1 group O, and for HIV-1 p24 antigen should be reflected.
RDTs vs EIAs Different panels will need to be constructed for RDTs and for EIAs, as the analytical sensitivity of
EIAs is expected to be greater than that of RDTs and thus EIAs may not be sufficiently challenged
by RDT lot verification testing panels.
Specimen types To meet the general objective of the lot verification testing, it will be sufficient to test characterized
specimens only, especially if adequate pre-market assessment has been undertaken for all
specimen matrices. For IVDs that utilize specimen types other than serum/plasma and that require
fresh collection and use, i.e. oral fluid or capillary whole blood, consideration should be given to
whether these alternate specimen types should form part of the lot verification testing panel.
National/regional Lot testing panels should be created from locally derived specimens, meaning specimens
panels collected nationally or regionally. A regional lot verification testing panel would be ideal for
initial implementation of lot verification testing as a range of lots of the same IVD from within
the same geographical area could be tested on the same panel, for increased ability to compare
lot verification testing data between countries.
Composition The lot verification specimen panels shown in Tables 5 and 6 are intended for IVDs that have
undergone stringent pre-market assessment such as WHO prequalification assessment. Pre-
market assessment would generate sufficient evidence of clinical sensitivity and specificity, thus
the lot verification can be kept to a minimum and will only be used to verify that performance
is comparable to the performance at the time of approval. Table 5 shows the composition of
a panel for RDTs that detect antibodies to HIV-1/2 . For RDTs that detect HIV-1 p24 antigen, in
addition to HIV-1/2 antibodies, analytical sensitivity to p24 antigen will need to be added to the
lot verification panel (Table 6).
Table 5 –Lot verification specimen panel for RDTs that detect antibodies to HIV-1/2
39
P ost-market surveillance of in vitro diagnostics
Table 6–Lot verification specimen panel for RDTs that detect antibodies to HIV-1/2 and HIV-1 p24 antigen
Notes:
(a) By testing the dilution series in triplicate, the precision of the assay can be evaluated, and expressed as the coefficient of variation (% CV).
(b) The negative dilution matrix used as diluent for the various specimens and dilution series should be included as a negative reference specimen, (i) to monitor
the background reactivity of the IVD, and (ii) to control matrix effects which may impact the analytical sensitivity.
Differences RDTs for detection of HIV-1/2 antibodies may vary considerably in their ability to detect diluted
in analytical specimens. Indeed there is no correlation between detection of the absolute number of HIV-1/2
sensitivity antibodies present in a specimen and lowest antibody concentration that can be detected by
between RDTs a given HIV-1/2 RDT. Nevertheless proportional quantification of antibody reactivity between
different specimens in the same HIV-1/2 RDT. Moreover, the analytical sensitivity also differs
between anti-HIV-1 and anti-HIV-2 by several magnitudes in most HIV-1/2 RDTs.
To select a 3-member dilution series, five HIV positive specimens (four dilution series for HIV-1 and
one dilution series for HIV-2). The dilution series must contain three members that span the assay’s
cut-off . To establish which three dilutions should form the series, each of the five specimens are
diluted 2-fold in normal human serum or plasma (1:2, 1:4, 1:8, 1:16, 1:32, 1:64, 1:128, 1:256, 1:512,
etc.) down to the endpoint (test kit cut-off ). Each set of these dilutions should be tested on the
given RDT and three dilutions that cross the cutoff are selected for inclusion in the lot verification
panel i.e. (1) the reactive dilution two dilutions above the assay´s cut-off, (2) the reactive dilution
just above cut-off, and (3) the first non-reactive dilution (See Figure 5). Each of the three reference
dilutions should be tested in triplicate on the same lot to verify their reactivity.
3.0 anti-HIV-1
anti-HIV-2
2.5
2.0
Test line intensity
1.5
1.0
0.5
0.0
0 80 160 320 640 1280 2560 5120 10240 20480 40960
HIV-1 p24 antigen The WHO international biological reference standard for HIV-1 p24 Ag (NIBSC 90/636) may be
used in the lot verification panel. A dilution series with two-fold dilutions from 20 IU/ml to 0.125
IU/ml has been found to cover the analytical sensitivity of most HIV 4th generation RDTs (see
Figure 6– Standard curve for HIV-1 p24 antigen. Each member of the dilution series was tested
in duplicate to verify their reactivity.
Figure 6– Standard curve for HIV-1 p24 antigen in 4th generation serology assays
35.0 EIA
2 RDT
30.0
Sample/cutoff ratio
20.0
1
15.0
10.0
0.5
5.0
0.0 0
0 5 10 15 20 25 30
1. Each assay should be performed under identical conditions to minimize the chance that differences in lot testing
results were not caused by differences in the environmental and/or testing conditions (including equipment such as
instruments, pipettes, etc.);
2. Testing should be performed by one technician where possible, to avoid technician-dependent differences;
3. All testing on the one lot should be performed on the same day;
4. All specimens should be tested in a randomized and blinded fashion, i.e., the testing personnel should not know the
status of the specimen before testing;
5. Test kits (devices and components) should be bought to room temperature before testing, and should be used
immediately after opening the pouch;
6. Test kits should always be stored at recommended temperatures and must be in good condition. If a desiccant is
included in the package and it has changed color, the kit should not be used;
41
P ost-market surveillance of in vitro diagnostics
8. Reagents from one lot should not be used with those of another lot; and
It is essential to inspect test kits for signs of damage caused by heat or humidity before initiating the assessment. Temperature
or time-temperature indicators are a good means of monitoring of the products’ thermal history.
The assay is to be performed exactly according to manufacturer’s instructions for use. If these contain several different
methods for the test procedure, a consensus on the exact test procedure must be made and then onwards always
followed when testing the lot verification panel on subsequent lots.
Each test should be read at both the minimum and maximum reading times stated in the instructions for use. A maximum
of 10 immunochromatographic (lateral flow) and 5 immunofiltration (flow through) RDTs can be tested in one batch at
one time to maximize logistics of testing.
For subjectively read assays, all test results should be entered as band intensities, e.g. 1+ to 3+ as shown below.
Faint lines should be interpreted according to the manufacturer’s instructions for use as their interpretation may vary from
product to product. Weaker test lines, weaker control lines, and high membrane background that disturbs reading of test
and/or control lines must be recorded.
Invalid results should be recorded when they occur as defined within the instructions for use e.g. control line does not
appear, high background that obscures strip and prevents accurate reading, displaced strip prevents accurate reading, etc.
If a digital camera or smartphone is available, it would be useful to take photographs of test devices side-by-side for electronic
storage of results. This is particularly important for invalid test results and results that do not meet the acceptance criteria.
Visual interpretation of results of subjectively read assays is made independently by two readers (without the knowledge
of the other set of results and blinded to the reference result for the specimen). These results are compared by the operator
carrying out the assay so that any mistakes may be identified and rectified immediately. Should recording errors be
identified, both the original and corrected result are recorded and initialed by the reader. When the two readers interpret
the results differently from each other, the technical supervisor is called to make a third reading. When the three readers
interpret the results differently from each other, the consensus is recorded as that interpretation which occurs two out of
three times. In cases where all three interpretations are different, the result is recorded as inconclusive.
The inter-reader variability is expressed as the percentage of specimens for which the test results were differently interpreted
(i.e. between +/- and 1+, between 1+ and 3+ between +/- and 2+) by the independent readers.
A colour intensity chart should be created for each RDT with photos of the test devices with varying intensity test bands.
Data entry Each assay should be performed and read by the technician according to the site-specific SOP
based on the manufacturer’s instructions for use, and results recorded on the data collection
form, see Annex 1.
A second person should read the test results independently and record the test results on a
same data collection form but with a piece of paper/cardboard that hides the results from the
first reader.
At the end, any discrepancies in reading should be reviewed by the technical supervisor and
resolved. The test results should be entered into the lot testing report by the technical supervisor
and double-checked against the data collection sheets, see Annex 2.
The data collection sheets should be kept in a folder and signed off by the technical supervisor
at the end of each day.
If the results are invalid or inconclusive, they should be recorded as such. The test may be repeated
if there are sufficient test kits available.
If the test is repeated for any reason, all results should be entered into the data collection sheets
as well as the reasons for repeating testing.
All data collection sheets and lot testing reports should be kept for a period of 5 years following
the conclusion of the testing.
For national HIV programmes that use one (or more) validated national testing algorithm(s), the exact assay used will be well
known and so it will be easier to ensure that the lot verification testing panel is suitable, particularly for analytical sensitivity.
The lot testing verification panel should have been tested on each product used within the country to establish the reference
result for each specimen of the lot verification specimen panel. This information will guide the criteria for acceptance.
The recommended acceptance criteria for HIV-1/2 RDTs are shown in Table 7.
43
P ost-market surveillance of in vitro diagnostics
4x HIV-1 dilutions All triplicates to be concordant for each specimen. Non-reactive specimen to be non-reactive; most concentrated
reactive specimen to be reactive.
1x HIV-2 dilution All triplicates to be concordant for each specimen. Non-reactive specimen to be non-reactive; most concentrated
reactive specimen to be reactive.
2x HIV low seropositive specimens Both specimens to be reactive.
Discrepant testing If a result falls outside of the acceptance criteria, the following algorithm should be followed.
results • Rule out aliquot failure (repeat testing on same aliquot);
• Rule out specimen failure (repeat testing on same specimen from a different aliquot);
• Rule out operator failure (repeat with another operator).
• Rule out quality management system failing of the testing laboratory, e.g. verify refrigerators
and freezers that store lot verification testing specimen panels, verify run worksheets for
transcription errors.
• Rule out other robustness issues may be related to precise volumes of specimen and running
buffer and volume variations with drop vials and disposable pipettes leading to e.g. increased
membrane background.
The technical supervisor shall generate a lot testing report following the format in Annex 2. The lot testing report will
be sent to the requestor, usually the relevant NRA and WHO, and will be retained by the testing site. In some cases, the
requestor may be a procurer or implementing partner, in this case any lot testing report should be copied to the relevant
NRA and to WHO when sent to the requestor. For WHO prequalified IVDS, all lot testing reports will be filed with their
prequalification files.
Any observations or unexpected outcomes, e. g. instability of specimen or reagent, defects, etc. as well as any deviation
from the defined procedures shall be recorded in the lot testing report.
4.4 CONFIDENTIALITY
The protection of all confidential data must be ensured during the lot testing process. The testing reports will remain the
property of the NRA (and WHO) which has the mandate to order lot testing. If the NRA wishes, they may release lot verification
testing reports to the manufacturer. This will be a requirement when the lot fails to meet the lot testing acceptance criteria.
If subjectively read assay – ensure that specimens are run in a randomized manner. This template is for
illustrative purposes.
Test results
Panel Specimen ID Lot testing result Reference result
Reading 1 Reading 2
Expand as required
ANNEX 2 – TESTING REPORT FORMAT FOR LOT VERIFICATION TESTING
Pre-distribution lot testing [tick one] Post-distribution lot testing [tick one]
2. Introduction
The objective of lot testing is to verify the performance of the IVD and to ensure that it continues to meet WHO
requirements for prequalification54 by identifying any form of product failure.
The lot testing verification panel of well-characterized specimens constituted of the following specimens. The
specimens were characterized with [state test kit name and manufacturer name].
Anti-HIV-1 analytical 4 HIV-1 specimens 2-fold dilutions (first non-reactive and last two 12x3
sensitivity reactive specimens)
Anti-HIV-2 analytical 1 HIV-2 specimen 2-fold dilutions (first non-reactive and last two 3x 3
sensitivity reactive specimens)
Grand total 50
The IVD was performed exactly according to manufacturer’s instructions for use. For rapid diagnostic tests and
other subjectively read assays, the band intensity was scored.
54 Or continues to meet the initial performance criteria for pre-market assessment for non-WHO-prequalified IVDs.
The evaluation results were interpreted by two technicians, recorded in a standardized data collection worksheet.
These data were then transcribed into this lot testing report.
Test results
Panel Specimen ID Reading 1 Reading 2 Lot testing result Reference result
Series 1,
Dilution 1
Series 1,
Dilution 1
Series 1,
Dilution 1
Series 1,
Dilution 2
Series 1,
Dilution 2
Series 1,
Dilution 2
Series 1,
Dilution 3
Series 1,
Dilution 3
Series 1,
Dilution 3
Series 2,
Dilution 1
Series 2,
Dilution 1
Series 2,
Dilution 1
Series 2,
Dilution 2
Series 2,
Dilution 2
Series 2,
Dilution 2
Series 2,
Dilution 3
Series 2,
Dilution 3
Series 2,
Dilution 3
Series 3,
Dilution 1
Series 3,
Dilution 1
Series 3,
Dilution 1
Series 3,
Dilution 2
Series 3,
Dilution 2
HIV +
HIV +
HIV -
HIV -
HIV -
4x HIV-1 dilution series All triplicates to be concordant for each specimen. First non-reactive
specimen to be non-reactive, last reactive specimen to be reactive.
1x HIV-2 dilution series All triplicates to be concordant for each specimen. First non-reactive
specimen to be non-reactive, last reactive specimen to be reactive.
6. Conclusion
Lot number [add] of [add assay name] with expiry date [add] was tested on [dd/mm/yyyy]. The results showed that
this lot produced acceptable results for the lot verification testing panel. Any unexpected outcomes, e. g. instability
of reagent, defects, software errors etc. as well as any deviation from the defined procedures shall be properly
recorded and be part of the evaluation report.
Telephone: Fax:
2. Product details
Event/problem description narrative (explain what went wrong with the product and the observed or
likely/probable consequences):
Operator/user at the time of the event/problem Has more than one user experienced the problem
(please choose): with the product?
q Laboratory technician/technologist q Yes q No
q (Non-laboratory) health worker
q Other (specify):
Comments:
Annex 3 – Disclaimer: The act of reporting an event is not an admission of manufacturer, user or patient
liability for the event or its consequences. Submission of an adverse event report does not, in itself, represent
a conclusion by the manufacturer that the content of this report is complete or confirmed, that the device(s)
listed failed in any manner. It is also not a conclusion that the device caused or contributed to the adverse
event.
ANNEX 4 – ADVERSE EVENT MANUFACTURER INVESTIGATION
REPORTING FORM
Send to:
The relevant National Regulatory Authority and WHO Prequalification of In Vitro Diagnostics Programme
World Health Organization, 20 Avenue Appia CH-1211, Geneva 27 Switzerland
Email: diagnostics@who.int
1. Recipient details
Country: Telephone:
Event reference number assigned by the Report reference number assigned by NRA:
manufacturer:
State any other NRAs who were also sent this report:
2. Submitter details
Country: Telephone:
Product name:
Expiry date:
4. Event/problem details
Where did the event happen:
Event/problem description narrative (explain what went wrong with the product and the observed or
likely/probable consequences):
Operator/user at the time of the event/problem Has more than one user experienced the problem
(please choose): with the product?
q Laboratory technician/technologist q Yes q No
q (Non-laboratory) health worker
q Other (specify):
Further investigations:
Is the manufacturer aware of similar events with this IVD with a similar root cause?
q Yes q No
7. Signature
Name:
Signature:
Date:
ANNEX 5 – FIELD SAFETY CORRECTIVE ACTION REPORT FORM
Send to:
The relevant National Regulatory Authority and
WHO Prequalification of In Vitro Diagnostics Programme
World Health Organization, 20 Avenue Appia CH-1211, Geneva 27, Switzerland
Email: diagnostics@who.int
1. Recipient details
Type of report:
q Initial report
q Follow-up report
q Final report
Country: Telephone:
2. Manufacturer details
Name of manufacturer:
Country: Telephone:
Product name:
Expiry date:
4. FSCA description
5. Comments
6. Signature
Name:
Signature:
Date:
ANNEX 6 – EXAMPLE FIELD SAFETY NOTICE
FSCA-identifier: [insert]
Type of action: [e.g. return of IVD to supplier, IVD modification (including instructions for use), IVD exchange,
IVD destruction, retrofit of IVD by purchaser of manufacturers modification or design change, advice given by
manufacturer regarding use of the IVD and/or follow-up of patients, users, or others].
Date: dd/mm/yyyy
[Specific details to easily identify the affected IVD e.g. product name, product code, lot number]
[A factual statement explaining the reasons for the FSCA, including description of the problem and a clear
description of the potential hazard associated with the continued use of the IVD and the associated risk to the
patient, user or other person.]
The above recommended action(s) are to be taken by all recipients of this FSN, including action(s) recommended
for people that have previously used or been treated by affected IVDs.
This notice needs to be passed on all those who need to be aware within your organization or to any organization
where the potentially affected product has been transferred. Please be aware of this notice and resulting action
for an appropriate period to ensure effectiveness of the corrective action.
The undersigned confirms that this notice has been notified to the appropriate National Regulatory Authorities.
Signature:
ANNEX 7 – POST-MARKET INFORMATION EXCHANGE REPORTING FORM
FOR NRAS
The following information exchange reporting form should be used by NRAs for sharing post-market surveillance
data on IVDs gathered through pre-distribution lot testing or post-distribution lot testing, vigilance procedures or
analysis of EQA and QC programme data. This reporting form should be completed and the relevant lot testing
report and initial/final investigation report or FSCA report attached to it.
This form is adapted from the document “Medical Devices: Post Market Surveillance: National Competent
Authority Report Exchange Criteria and Report Form” (IMDRF/NCAR WG (PD1)/N14R4 proposed by the
International Medical Device Regulatory Forum (IMDRF). The document describes a system for NRAs to
exchange confidential information about safety, quality and performance of IVDs where there is an anticipated or
unanticipated serious public health threat, defined as:
No death or serious injury occurred but the event might lead to death or serious injury of a patient, user or other
person if the event recurs.
This programme is used to exchange early information on significant concerns or potential trends that individual
NRAs have observed, but that have not yet resulted in FSCA or recall. Participation currently includes the
regulators from Australia, Brazil, Canada, China, Europe, Japan, Russia and the United States of America.
1. Report details
Purpose of exchange:
q Share information
q Events leading or highly likely to lead to unanticipated public health threat
q Observations from national trend analysis
q Request information
q Summary of query findings
Confidentiality:
q Yes q No
Name of NRA:
Country: Telephone:
3. Product details
Product name:
Expiry date:
Manufacturer name:
Country: Telephone:
4. Background information
5. Additional remarks
6. Details of responding NRA
Name of NRA:
Country: Telephone:
7. Signature
Name:
Signature:
Date:
C ONTAC T
Department of Essential Medicines
and Health Products
World Health Organization
20 Avenue Appia
CH-1211 Geneva 27
Switzerland
E-mail: diagnostics@who.int
www.who.int/diagnostics_laboratory
978 92 4 1509213
Prequalification Team-Diagnostics