NGS Assays: Here are the Variables of Clinical Performance

“FDA’s vision is that NGS-based tests can be developed, validated, and offered for clinical use through a process that leverages appropriate standards, quality systems controls and community assessment of clinical validity to streamline the premarket review process.” [1]

Since the early 2000s next-generation sequencing (NGS) has become an invaluable tool in both drug discovery research and clinical/diagnostic settings for disease-specific precision medicine.  Whole genome sequencing (WGS) and whole exome sequencing (WES) are two NGS methods with clinical implications. Clinically, WGS sequences the human genome in clinical samples to determine the roles of genes in disease development. According to Hendriksen et al., WGS is currently used to monitor antimicrobial resistance globally. [2] Targeted sequencing of the protein coding exons of genes makes WES useful for the genetic diagnosing of diseases such as with specific cancer types. [3]

Establishing the clinical validity of next-generation sequencing (NGS) assays is a unique burden in comparison to the requirements for non-genomic in vitro diagnostic devices (IVDs). Per the notifying body, TUV SUD, a clinical performance report must provide evidence for usefulness of the analyte(s) or marker(s) of a device. [4] Because NGS devices are capable of measuring millions of genomic analytes related to numerous conditions, clinical performance reports (CPRs), which must painstakingly delineate diagnostic specificity and sensitivity for every prescribed intended use of an assay, can easily become tedious to develop. In comparison, a non-genomic IVD CPR is written to provide evidence for a limited number of analytes, usually one or two.

A second caveat and distinct feature of NGS assay testing is that a single genomic variant may either hinder or promote disease progression depending upon the region of pathology. For instance, programmed cell death-ligand 1 (PD-L1) genes are understood to prevent cancer cell death by inhibiting T lymphocyte activation in malignant cells. [5] To prolong the survival of cancer patients, a class of immunotherapy known as PD-L1 inhibitors are often prescribed concomitant with chemotherapy.

Validating the clinical performance of a PD-L1 assay will likely generate PD-L1 inhibitor trials which will demonstrate equal success in prolonging progression-free survival in extensive-stage small-cell lung cancer patients [6] and failure to do the same in recurrent and/or metastatic head and neck squamous cell carcinoma [7]. This duality in clinical performance is inherent to NGS assay testing and still proves the usefulness of the assay for monitoring progression of both lung and head and neck cancers.

Clinical Study Literature Review for NGS Assays

When evaluating test performance, incorporating the following parameters into the literature search protocol should provide the most comprehensive results.

• Include in vitro studies on genomic regions (regions of the chromosome at which a particular gene is located), variant types (i.e., mutations, deletions, insertions), and sequence contexts representative of the test’s indications for use, including clinically relevant targets. Because in vitro studies use human samples, these study results may be used to identify patient groups who may benefit from precision medicine. [8]
• Establish test performance on variants in highly homologous, highly polymorphic, or other regions specific to the indications for use of the assay. This is important because regions of high sequence homology and polymorphisms are a challenge for short read technologies often resulting in false positive and false negative results. [9]
• Ensure that included clinical studies use specimens that reflect the indicated specimen types per the assay’s Instructions for Use (IFU). Devices must meet EU labelling requirements in order to be placed on the EU market. [10]
• Ensure that included clinical studies enroll the target population as indicated by the assay’s IFU.
• Include DNA preparation, reagent and specimen acquisition, handling and storage when available to ensure that manufacturer’s specifications are being met.
• If available, include a clinical study that documents precision by comparing the NGS assay to an industry-established or “gold standard” reference sequence of well-characterized samples. The Association of Clinical Research Professionals advises that for novel genomic diagnostic devices, a predicate gold standard may be a biopsy or an alternate biomarker test. [11]
• Read the IFU to ascertain whether biosynthetic samples containing clinically relevant variants pertinent to the assay’s indications may be used as an acceptable alternative to cell lines or fresh clinical samples.
• In lieu of fresh clinical samples, cell lines or biosynthetic materials, in silico constructed sequences containing known sequence variants of various claimed types (e.g., single nucleotide variants, copy number variants, repeat expansions, duplications, and indels) may be used. The construction of in silico sequences should be clearly described, justified, and documented. The Expressed Sequence Tag (EST) database is an in-silico source for cancer-related sequence polymorphism discovery at the whole-genome level.[12]
• Document positive percent agreement and negative percent agreement, separately, for each type of studied variant claimed and sequence context (e.g., highly homologous, highly polymorphic, or other difficult regions) to be assessed by the test.

Complying with the new IVDR requirements may seem like a daunting task, particularly for manufacturers who could previously self-certify, but partnering with Nerac can help you navigate the process and successfully achieve compliance.  Nerac has a knowledgeable team of analysts/medical writers with expertise in clinical literature searches and targeted literature reviews to support regulatory submissions.  The Nerac team assists IVD manufacturers with identifying and analyzing clinical data from the literature as it pertains to state of the art, scientific validity, and device performance, using an approach that has been refined over many years supporting manufacturers with EU medical device submissions. Call us at 860.872.7000 or contact us here to learn more!

Sarah McLeod, M.S.

Sarah McLeod, M.S. is a clinical research specialist with 15 years supporting data acquisition, quality control and regulatory submissions for Phase 1-4 oncology and HIV trials. The culmination of this experience has allowed her to apply a detailed understanding of clinical trials to in vitro diagnostic regulatory environments. Several of her reports have been approved for submission by notified bodies.

Academic Credentials

• M.S. Health Science (Clinical Research Administration)
• Touro University International, Cypress, CA
• Subsidiary of Touro College, NY, NY
• B.A. Human Services, George Washington University, Washington, D.C.

1. Center for Devices and Radiological Health. Considerations for Design, Development, and Analytical Validation of Next Generation Sequencing (NGS) – Based In Vitro Diagnostics (IVDs) Intended to Aid in the Diagnosis of Suspected Germline Diseases. FDA; [cited 2023 Aug 14]. Available from: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/considerations-design-development-and-analytical-validation-next-generation-sequencing-ngs-based
2. Hendriksen RS, Bortolaia V, Tate H, Tyson GH, Aarestrup FM, McDermott PF. Using genomics to track global antimicrobial resistance. Frontiers in Public Health. 2019;7. doi:10.3389/fpubh.2019.00242
3. Hartman P, Beckman K, Silverstein K, Yohe S, Schomaker M, Henzler C, et al. Next generation sequencing for clinical diagnostics: Five Year experience of an academic laboratory. Molecular Genetics and Metabolism Reports. 2019;19:100464. doi:10.1016/j.ymgmr.2019.100464
4. Annex XIII: Performance evaluation, performance studies and post-market performance follow-up [Internet]. TÜV SÜD Akademie GmbH; [cited 2023 Aug 16]. Available from: https://de-mdr-ivdr.tuvsud.com/Annex-XIII-Performance-evaluation-performance-studies-and-post-market-performance-follow-up.html
5. Cell Signaling Technology, Inc.; 2018 [cited 2023 Aug 13]. Available from: https://www.cellsignal.com/pathways/immune-checkpoint-signaling-pathway
6. Yu H, Chen P, Cai X, Chen C, Zhang X, He L, et al. Efficacy and safety of PD-L1 inhibitors versus PD-1 inhibitors in first-line treatment with chemotherapy for extensive-stage small-cell lung cancer. Cancer Immunology, Immunotherapy. 2021;71(3):637–44. doi:10.1007/s00262-021-03017-z.
7. Saâda-Bouzid E, Defaucheux C, Karabajakian A, Coloma VP, Servois V, Paoletti X, et al. Hyperprogression during anti-PD-1/PD-L1 therapy in patients with recurrent and/or metastatic head and neck squamous cell carcinoma. Annals of Oncology. 2017;28(7):1605–11. doi:10.1093/annonc/mdx178
8. Center for Devices and Radiological Health. In vitro diagnostics [Internet]. FDA; [cited 2023 Aug 16]. Available from: https://www.fda.gov/medical-devices/products-and-medical-procedures/in-vitro-diagnostics.
9. Mandelker D, Schmidt RJ, Ankala A, McDonald Gibson K, Bowser M, Sharma H, et al. Navigating highly homologous genes in a molecular diagnostic setting: A resource for clinical next-generation sequencing. Genetics in Medicine. 2016;18(12):1282–9. doi:10.1038/gim.2016.58
10. Medicines and Healthcare products Regulatory Agency. Regulating medical devices in the UK [Internet]. 2023 [cited 2023 Aug 16]. Available from: https://www.gov.uk/guidance/regulating-medical-devices-in-the-uk.
11. Cramer G. Regulatory and development approaches to research for in vitro diagnostics vs. other medical devices-the same or different? [Internet]. 2023 [cited 2023 Aug 16]. Available from: https://www.acrpnet.org/2023/02/22/regulatory-and-development-approaches-to-research-for-in-vitro-diagnostics-vs-other-medical-devices-the-same-or-different/
12. Aouacheria A, Navratil V, López-Pérez R, Gutiérrez NC, Churkin A, Barash D, et al. In silico whole-genome screening for cancer-related single-nucleotide polymorphisms located in human mrna untranslated regions. BMC Genomics. 2007;8(1). doi:10.1186/1471-2164-8-2.