Comparison of Molecular Testing Methods for Detecting BRAF V600 Mutations in Melanoma Specimens with Challenging Attributes

Introduction: To ensure the appropriate assignment of vemurafenib to patients with unresectable or metastatic melanoma, accurate detection of activating BRAF mutations is now a clinical imperative. However, the performance of commercially available test kits on challenging samples is unknown. Methods: 126 formalin-fixed, paraffin-embedded melanoma samples were selected for challenging attributes, such as small sample size, that might affect test kit performance. The Qiagen BRAF RGQ PCR version 2 (RGQv2) test kit, intended for research use only, and the FDA-approved companion diagnostic cobas 4800 BRAF V600 Mutation Test were compared for their ability to detect the V600E mutation in challenging samples, using a single 5 μm unstained slide. Results: of the 126 specimens, three samples were invalid by the RGQv2 test, three other samples were invalid by the cobas test, and an additional two samples were found invalid by both tests. For the 118 samples that yielded valid results with both tests, concordant results were observed for 105 (89.0% [95% CI, 82.1%-93.5%]) samples. Of the 12 discordant samples with sufficient material for further testing, next generation sequencing confirmed the cobas test result for 6 (50.0%) and confirmed the RGQv2 test result for the other 6 (50.0%) samples. Five (4.0%) of the RGQv2 test results yielded multiple positive mutation calls and two results had a sample control assay PCR cycle threshold (CT) >33, indicating insufficient amounts of DNA template, but gave accurate mutation calls. Workflow analysis showed that the total time to result was 5.65 hours for the cobas test to process 24 samples and 7.84 hours for RGQv2 assay to process 7 samples. Conclusions: The two commercially available, PCR-based methods demonstrated similar abilities to detect BRAF V600 mutations in challenging melanoma samples. However, the total time-to-result, assay hands-on time, and diagnostic interpretation were more efficient and rapid with the cobas test.


Introduction
Treatment options for malignant melanoma have undergone a true revolution with the introduction of targeted therapies. With the concomitant changes in diagnostic testing, patients with advanced or metastatic melanoma are now routinely screened for mutations in BRAF to identify patients who are most likely to respond to targeted therapies, such as vemurafenib and dabrafenib. Mutations in the BRAF gene are a common event in the development of malignant melanoma, accounting for 50% to 60% of all cases [1,2]. The majority of BRAF mutations occur at codon 600 in exon 15 of the BRAF gene, leading to constitutive activation of the cognate protein and downstream signaling through the MAPK pathway [3]. The most frequent BRAF mutation leads to the substitution of glutamic acid for valine (V600E) and accounts for as many as 90% of BRAF mutations, although other activating mutations are known (e.g., V600K and V600R) [1].
Recognition of the important role played by BRAF kinase mutations in melanomas led to the development and subsequent approval of vemurafenib, a potent and selective inhibitor of mutated BRAF kinase, for the treatment of patients with unresectable or metastatic melanoma with the BRAF V600E mutation. Clinical trials with vemurafenib for metastatic melanoma showed improved overall and progression-free survival in patients with previously untreated melanoma whose tumors tested positive for this genetic change [4,5]. The development of vemurafenib represented a groundbreaking advance in the management of malignant melanoma patients with V600E-mutated tumors. Until recently, advanced melanoma was associated with a dismal prognosis; with few effective treatment options available, overall survival (OS) was measured in months [5,6]. Vemurafenib demonstrated potent activity against melanoma cell lines that carried the V600E mutation, but not against cell lines with wildtype BRAF [7]. In a landmark phase 3 trial of patients with treatmentnaive metastatic melanoma harboring the V600E mutation, vemurafenib significantly improved PFS and OS relative to the control arm [5]. Based on these outcomes, vemurafenib received approval from the FDA for the treatment of patients with unresectable or metastatic melanoma with BRAF V600E mutation as detected by an FDA-approved test. In the clinical trials of vemurafenib, a positive result for the V600 mutation with the cobas® 4800 BRAF V600 Mutation Test was a key eligibility requirement [4,5]. Other commercially available tests are available for the evaluation of BRAF mutations for research rather than diagnostic purposes.
Given the improved melanoma patient outcomes with targeted therapies, it is imperative that test methods are highly sensitive to ensure that eligible patients receive the appropriate treatment. However, assay performance may be hampered if specimens submitted for analysis have challenging attributes, such as a low proportion of mutant cells, small specimen size, poor specimen quality, or high melanin content, which can interfere with PCR. BRAF tests with low sensitivity may yield false negatives or invalid results in those situations, which would result in patients being denied potentially lifeprolonging therapy.
This study compared the analytical performance of the FDAapproved cobas® 4800 BRAF V600E Mutation Test versus the Qiagen BRAF RGQ PCR kit version 2, for research use only, for the detection of BRAF mutations in melanoma specimens with challenging attributes. These two tests were compared because they are the most commonly used PCR-based kits and, at the time of the study, only the cobas kit had approval for BRAF diagnostic testing. Since this study was completed, a new test, the THxID™ BRAF Kit (Hazelwood, MO), has been FDA approved in the US for advanced melanoma patients with BRAF V600E and V600K mutations to be treated with trametenib and dabrafenib.
The Qiagen BRAF RGQ PCR kit version 2 ("RGQv2 test"; Qiagen Inc., Valencia, CA, USA) is an RT-PCR assay for research use only. It uses Scorpions® (bi-functional molecules in which a PCR primer is covalently linked to a probe) and Amplification Refractory Mutation System technologies to detect the V600E (c.1799 T>A), V600E2 (c. 1799_1800TG>AA), V600K (c.1798_1799GT>AA), V600R (c. 1798_1799GT>AG), and V600D (c.1799_1800TG>AT) mutations against a background of wild-type genomic DNA in FFPE tissue specimens [10]. For control and experimental samples, the BRAF RGQ PCR Handbook for the version 2 assay specifies a CT of 20.95-33.00 as an acceptable range indicating the presence of sufficient DNA for optimal kit performance; a CT >33.00 indicates that mutations present at low levels may not be detected and results should be interpreted with caution; and a CT >35.00-45.00 indicates that only a few amplifiable copies of DNA are present and mutations are only likely to be reported if most copies are mutated [10]. The mutation status of a sample is established by calculating the difference between the CT values (ΔCT) from the sample and a control sample as follows: A ΔCT of ≤7.0 indicates that the sample is positive for the V600E mutation.
Next generation sequencing (NGS) was performed according to the manufacturer's instructions on the Ion Torrent PGM (Life Technologies, Thermo Fisher Scientific, Carlsbad, CA) using a validated protocol for BRAF mutation detection with a limit of detection of 1% for V600E mutations [11]. This method is a 5-to 7day process that involves the generation of amplicons which are subject to pooling, ligation, emulsion PCR amplification, and NGS [11].

Study design
The study was conducted using a panel of 146 FFPE melanoma tissue specimens, were collected from a single site, from which 126 specimens with challenging attributes were selected ( Figure 1). The challenging attributes were: (a) tissue sample <60 mm 2 ; (b) CNB samples; (c) >50% necrosis; (d) >10% pigmentation ( Figure 2); or (e) metastatic sites ( Figure 3). Some of these attributes are known to compromise results acquired with PCR-based methods and can result in higher sample failure rates [12][13][14]. For example, decalcification steps for bone metastatic samples can lead to gross degradation of the target DNA. Also, no samples were excluded due to a low percentage of tumor cells. For the 126 samples, tumor content was at least 50% in 87 samples, was between 10% and 50% in 32 samples, and was 5% or less in 7 samples. This study was double blinded. Five 5-μm tissue curls were sectioned from each of the 126 panel specimens and blinded. All sections from each specimen were mounted on a slide; one was stained with hematoxylin and eosin, and all were coded. This section was assessed by a certified pathologist to confirm the diagnosis of melanoma and to assess tumor content, degree of pigmentation, and extent of necrosis according to laboratory protocol. Included samples could have multiple attributes; thus there were 42 samples with 1 attribute, 45 samples with 2 attributes, 25 samples with 3 attributes, and 3 samples with 4 attributes to yield 219 observable attributes.
DNA from one slide-mounted 5-µm section of each of the 126 clinical specimens was isolated using the cobas DNA Sample Preparation Kit and tested with the cobas test ( Figure 4) [15]. DNA from another 5-µm section was isolated as recommended by the manufacturer using the QIAamp® DSP DNA FFPE Tissue Kit and tested on the RGQv2 test ( Figure 5) [10]. Prior to PCR setup for the RGQv2 test, the amount of DNA to be used in the final PCR run was measured by a recommended sample assessment procedure based on CT values from the package insert. Although the cobas test requires macrodissection for samples with less than 50% tumor content, the RGQv2 test does not require macrodissection. In order to allow a fair comparison of outcomes between the 2 different methods, no macrodissection was performed on any samples prior to DNA extraction. This choice could potentially impart a bias favoring the RGQv2 kit. Therefore, to minimize assay performance bias, no samples were macrodissected for either test. The remaining two glass mounted slides were kept for additional discordance analysis, if required, and future studies.   : cobas® 4800 BRAF V600 Mutation Test workflow. The assay is based on two major processes: (1) manual isolation and preparation of DNA from the specimen; (2) PCR amplification and detection. Macrodissection is normally required when specimens have <50% tumor but was not performed for this study for any samples. H&E = hematoxylin and eosin; PCR = polymerase chain reaction

Agreement analysis
Agreement analysis was assessed as positive percent agreement (PPA), negative percent agreement (NPA), and overall percent agreement (OPA) between the cobas test and the RGQv2 test for detecting BRAF V600 mutations. Specimens that gave discordant results between the two methods or invalid results for one of the methods were further subjected to NGS as a third mutation detection method, using existing DNA eluates. False positive (FP) and false negative (FN) rates were calculated for both methods using the following formulae and the cobas test as the reference method:

IRB submission
Specimens used in this study were included in a proposal and presented to the Carolinas HealthCare System Institutional Review Board (IRB File # 03-12-16EX). After review, this study met the criteria for exempt status. The IRB waived the need for consent since the testing was performed on de-identified samples.

Mutation test failure rate
Based on the quality assessment step using CT values with the RGQv2 test, DNA from 15 (11.9%) of the melanoma samples were classified as "interpret with caution" because the CT values were greater than 33 but generated a result due to manual interpretation. The remaining samples were within the acceptable range (CT = 20.95-33.00) for sample assessment control reactions. Of the prior 15 specimens, 13 had sufficient DNA yields for use with the cobas test (i.e., at least 5 ng/µL). The cobas test reported 5/126 (4%) invalid specimens, which were subjected to NGS. Of the 5 specimens called invalid by the cobas test, two specimens were invalid by RGQv2 and one specimen was also invalid by NGS and exceeded the CT >33 cutoff for a good quality sample. The RGQv2 test also reported invalid results from 5 specimens; two specimens were also invalid by the cobas test and three were unique to the RGQv2 test.

Methods correlation
Out of 126 specimens, 8  Thirteen specimens were discordant between the two test methods, and one of these samples did not have adequate DNA for NGS resolution analysis. For the remaining 12 specimens, NGS showed concordance with 6 (50%) of the cobas test results (4 wild-type and 2 mutated by the cobas test) and 6 (50%) of the RGQv2 test results (5 wild-type and mutated by the cobas test; Table 2).

Time to result
An estimation of the total time to result was derived from a single run of each instrument loaded with the maximum number of samples. This analysis resulted in a total time-to-result of 7.84 hours for RGQv2 to process 7 samples (Table 4) and 5.65 hours for cobas to process 24 samples. For the PCR reaction itself, the total reaction times are 2.67 hours for RGQv2 versus 1.75 for cobas, which is relevant for laboratories that use their own DNA extraction and quality assessment methods.

MD
MND Total

Discussion
When this study was performed, the cobas test was the only CE-IVD and FDA-approved test for the identification of patients with tumors harboring the V600E mutation; however, other assays using differing technologies are commercially available for research use only (RUO) [16,17]. Recently, the bioMérieux THxID BRAF test was approved by the FDA for use as a companion diagnostic for the detection of V600E and V600K mutations when making treatment decisions regarding dabrafenib and trametinib for patients with metastatic melanoma [18]. The Qiagen BRAF RGQ PCR version 2 kit is an RUO, CE-IVD approved test and it is commercially available for the detection of the clinically relevant V600E, V600E2, and V600K BRAF mutations, as well as other V600 mutations whose clinical relevance is unknown. In addition to comparing the cobas and RGQv2 test kits, we assessed sample processing turnaround times to allow consideration of whether the tests can achieve acceptable timescales for clinical decision-making. The analytical performance of the cobas test has also demonstrated cross-reactivity to BRAF V600D plasmid at ≥10% mutation, BRAF V600K plasmid at ≥35% mutation, and BRAF V600E2 plasmid at ≥65% mutation [9].
Although the V600E mutation is the most common at this codon, accounting for approximately 80% of valine substitutions, other mutations have been observed. V600K is clearly the next most common mutation and occurs in approximately 10% to 20% of melanoma samples [19,20]. Cross-reactivity against the V600K mutation has been documented for the cobas mutation test kit. In a recent study, the kit detected 75.0% (8 of 12) of BRAF V600K mutations confirmed by NGS [13]. For detection of very rare or unknown mutations, methods such as Sanger sequencing may be useful in samples not requiring the high analytical sensitivity of a PCRbased assay.
In the authors' experience, the majority of samples that receive BRAF testing has at least one challenging attribute. Both the cobas and the RGQv2 methods produced a high rate of valid results from 126 samples with challenging attributes. Eight samples were invalid by the cobas and/or the RGQv2 tests. In addition, 13 samples were discordant between the two tests, and one of these samples was invalid for NGS. After factoring in results from NGS, which was used to resolve discordant samples, the tests yielded an OPA of [111 of 117] 94.9%.
The cobas test yielded 5 false negatives based on the mutation calls that showed agreement with both RGQv2 and NGS; another sample reported the V600E mutation which was wild-type by RGQv2 and NGS [9]. Although the RGQv2 instructions do not require macrodissection, if these specimens had been macrodissected prior to DNA extraction for the cobas test, the lack of sensitivity may have been resolved for these samples. The RGQv2 test yielded three false positives, including two V600E/E2 and one V600K, and it yielded three false negatives which were either V600E (two samples) or V600K (one sample) based on NGS.
Although the sample sizes were too small to allow statistical testing, the samples that provided the discordant calls between the cobas test and the RGQv2 test tended to be smaller samples (<60 mm 2 ) with >10% pigmentation. Although approval of the cobas test by the FDA was specific to V600E mutations, the assay demonstrated crossreactivity to V600K; these mutations are present in >5% of reported melanoma cases and therefore V600K results are reported this study [9]. There were a total of 12 V600K mutations as determined by sequencing and the cobas test and RGQv2 test were able to identify 10 and 11 specimens, respectively.
Minimizing the presence of challenging tumor sample attributes can diminish the risk for invalid results. Nonetheless, in this study, challenging attributes had a nominal impact on assay performance for both the cobas and RGQv2 tests. Moreover, the cobas test demonstrated equivalent assay performance to the RGQv2 test despite the fact that 39 samples with less than 50% tumor content were not macrodissected. Two samples that were invalid by both assays were derived from metastatic bone samples.
Providing timely results for BRAF V600 mutation status is critical to patient management. Depending on the laboratory workflow for the case load of BRAF testing and other molecular tests, time to result becomes a differentiator for laboratory costs. In addition, invalid test results have important implications for patients, as the need to repeat tests, and potentially to re-biopsy patients, can lead to significant delays in effective patient treatment. This study yielded 10 separate samples for which an equal proportion of the results of RGQv2 or cobas testing did not agree with the NGS results. Although samples with insufficient DNA can often be manually evaluated, the process can contribute to a longer time to obtain results and introduces subjective interpretation of results. This issue is underscored by the longer time-to-result derived for a single instrument run of the cobas test versus the RGQv2 test (5.65 hours for 24 samples versus 7.84 hours for 7 samples, respectively). The faster time-to-result for a greater number of samples using the cobas test kit is particularly relevant for research and large clinical studies.

Conclusions
In summary, we have presented a comparison of two commercially available methods for the detection of V600E mutations in the BRAF gene from specimens of malignant melanoma with challenging specimen attributes. The two assays were found to perform similarly in their ability to detect V600E mutations in melanoma samples and neither assay was adversely impacted by the challenging attributes. Overall, the cobas test displayed a higher degree of mutation detection accuracy than the RGQv2 test when including the detection of V600K mutations. In addition, the total time to result, assay hands-on time, and diagnostic interpretation were more efficient with the cobas BRAF test. Current studies are underway to evaluate the performance of the cobas test relative to additional available tests.