4.8 Pre-clinical studies
Determination of bioinformatic pipeline thresholds Analytical sensitivity •Validation of sample input requirements •Limit of detection (LoD) Analytical specificity •In silico specificity study (primer specificity study) •Interfering substance study •Cross-reactivity study Precision/reproducibility •Assay reproducibility study Linearity/measuring Range (If applicable) Traceability & expected values (controls, calibrators, methods) Cut-off value (same approach as other IVDs) •Microsatellite instability (MSI) score determination and cut off values. •Tumour mutational burden (TMD) score determination and cut off values. Trueness (not applicable) Stability of reagent Specimen stability •Extracted DNA/RNA stability •Genomic DNA/RNA freeze-thaw study Carryover effect/cross-contamination •Sample carryover study Specimen matrix study •Tissue comparability study •Tissue fixation study (for FFPE samples) Determination of Bioinformatics pipeline thresholds •Requirements on target region (e.g. exon, intron, promoter) coverage should be established •Requirements on sample coverage •Requirements on mutation coverage, uniformity of coverage, allele depth and frequency for positive calls •Requirements on fusion gene detection (if applicable) Analytical sensitivity Validation of sample input requirements: Sample input requirements for the assay to work as intended must be specified and validated. They may be validated as part of the workflow in other pre-clinical/clinical studies or as a study on its own. Below are examples of sample input requirements that may be applicable: 1)Primary sample input amount The recommended primary sample input amount should be validated to determine if they would yield the required DNA/RNA amounts required for the assay when extracted using the specified DNA/RNA isolation protocol. Analytical sensitivity 2)DNA/RNA Input: The DNA/RNA input amount/purity requirements should be validated to meet the minimum amount specified in the IFU that is required for the IVD assay to perform as intended. For example: The assay performance can be tested across several DNA/RNA input concentrations. If the expected gDNAinput for an assay is 250 ng, DNA samples can be tested at input levels of 1250, 250, 100, and 25 ng. The upper and lower bounds for DNA input should be determined. The study should demonstrate high percentage of positive and negative variant call rate (>95%) within the DNA input range tested, supporting the specified DNA input amount claimed by the NGS IVD Test kit. Samples with known somatic mutations/fusion genes from cell lines and clinical samples can be used for this validation purpose. Analytical sensitivity 3)Tumour content of primary samples (applicable – for tumour profiling IVDs) The minimum tumour content in primary samples that is required for the assay to perform accurately. For example: Pre-characterized clinical samples (with established number of SNVs, deletions, and gene fusions confirmed by validated reference methods) can be used for tumour content validation process. The tumour cell content of each specimen and region of interest should be estimated before the study. Broad range of samples with varying tumour content percentage (<10% to 60-70%) should be included in this study. For liquid biopsy samples, appropriate sample input amount requirements should be validated to meet the minimum specified tumour fraction requirement for the IVD assay to perform as intended. Analytical sensitivity Limit of Detection (LoD) study For tumour profiling IVDs, the LoDis the lowest Allele frequency (AF) of single nucleotide variants (SNV), Multi-nucleotide polymorphisms (MNP), or deletion variants, or the lowest number of reads of RNA fusion variants that can be detected at least 95% of the time. Different test methods can be used to determine the LoDof the IVD assay. 1.Variant-containing samples can be blended with Wild type (WT) samples at multiple levels (serial dilution) and used as the input material for the test. 2.Confirmation of the LoDcan also be done (Specific LoDof Variant Allele frequency (VAF) claimed by the test kit). For example, IVD test kit with LoDof 5% VAF claim, can test for 5 replicates of a sample for certain number of deletions, insertions and SNVs at 5% minor allele frequency. All variants ideally should have 100% positive call rates. 3.Commercially available reference standards can also be used for LoDvalidation purposes. Analytical specificity In silico specificity study •An in silico cross-reactivity analysis needs to be performed that evaluates all the primers in the NGS IVD Test Kit to determine the specificity of the primers to their targeted sequences. Primers should be checked for specificity to the human genome, human transcriptome, and genomes from bacteria, fungi, and viruses frequently found in human specimens. •Corresponding pseudogenes for any target gene on the NGS panel should be highlighted if they can potentially cause mismapping. False positive rate in well characterised samples/cell lines •To ensure that a variant-free ("blank") sample does not generate an analytical signal that might be classified as a mutation. Wild-type (WT) samples should be evaluated at each variant location that can be detected by the IVD NGS Test kit. Samples that are WT at all locations should produce a "variant not detected" call at each location. This tests provide information regarding the false positive rates. Analytical specificity Interfering substance study •The effect of potential interfering substances on the performance of the assay should be examined. •Examples of potentially interfering substances include bilirubin, hemoglobin, cholesterol, lipid, melanin, short draw interference, buffer component interference, paraffin, xylene, ethanol, protease and therapeutic drugs. •Samples should be spiked with various potential interfering substances prior to nucleic acid extraction. For the assessment of each inhibiting substance, data for each spiked sample should be compared to the control samples. Impact on call rate, reproducibility, and concordance between variant calls in samples with and without interfering substances should be determined. Cross-reactivity study •Potential cross-reactivity to other organisms that may be present in sample type should be assessed. •Potential cross-reactivity to other cancers or diseases should be assessed. Precision/reproducibility The study should be designed to evaluate within-run precision performance (repeatability) and variability across sites, operators, and instrument platforms (reproducibility). The reproducibility and repeatability of variant/fusion gene detection using the NGS IVD Test kit can be assessed with WT samples and variant-positive samples. Each sample should be tested several times at different sites and total number of replicates tested per sample should be stated. Ideally, the study should include samples close to the LoD. The call rate, no call rate, positive call rate, negative call rate, and within-run repeatability should be computed at each variant location of interest. Additional tests that can be performed: •Reagent Lot-to-Lot Reproducibility •Instrument-to-Instrument Reproducibility •Reagent Lot Interchangeability Linearity/measuring range (If applicable) Appropriate linearity/measuring range studies must be provided for assays that are quantitative, for example in residual disease determination and in disease staging. Cut-off value Microsatellite instability (MSI) validation (If /when applicable – for tumour profiling IVDs) Precision of the MSI calling by any specific methods (e.g., MSIsensor) should be demonstrated with a total of (n) specimens which should contain both MSI-H (high) specimens and Microsatellite stable (MSS) specimens. Each DNA extracted sample can be tested with multiple inter- and intra-run replicates. Concordance score between the calls for all replicates tested should be summarised. Precision and Reproducibility testing should include MSI performance as well. Cut-off value Tumour mutational burden (TMB) validation (If /when applicable –for tumour profiling IVDs) •TMB score determination method should be provided in sequencing and analysis part. •TMB concordance should be evaluated by comparing the TMB output in terms of mutations per Mb with appropriate control samples. •Performance of TMB at the recommended tumour purity (e.g., ≥ 20%), should be validated on samples near the claimed TMB cut-off. •Precision and Reproducibility testing should include TMB performance as well. Stability of reagent •Long term stability (3 lots of reagents required) •In-use stability (if reagents are not single use) •On-board stability (if applicable) •Shipping studies Where stability testing is performed on a representative panel, scientific justification must be provided. Specimen stability (if there are specific claims) Specimen stability claims may also be supported with specimen stability studies or with literature/guidelines. Extracted DNA/RNA stability Extracted DNA/RNA stability should include testing of extracted DNA/RNA stored at various temperatures/storage periods as claimed in the IFU of the NGS IVD kit. For example: •DNA extracted from anti-coagulated blood stored at room temperature (20-25°C) for 7 days •DNA extracted from anti-coagulated blood stored at at 2°C to 4°C for 30 days •DNA extracted from anti-coagulated blood storedat -15°C to -25°C for 30 days Genomic DNA/RNA freeze-thaw study: •The impact of repeated freeze-thaws on gDNAsamples isolated should be evaluated. Carryover effect/cross-contamination Sample Carryover study •The goal of this study is to ascertain that sample carryover between samples within an instrument run and between successive sequencing runs meets design requirements. Specimen matrix study Tissue Comparability study (If /when applicable –for tumour profiling IVDs) The goal of the study is to demonstrate that genomic profiling can be performed on DNA/RNA derived from different tissue type (as claimed by the assay). It is also acceptable for the claims for different tissue type to be validated as part of the workflow in other pre-clinical/clinical studies. Tissue fixation study for FFPE samples – (If /when applicable –for tumour profiling IVDs) If formalin fixation time is a potential issue, potential effects of formalin fixation times on the assay performance should be addressed.