M. Bryan Warf*, Julia E. Reid, Krystal L. Brown, Hillary Kimbrell, Kathryn A. Kolquist, Steven Stone and Benjamin Roa
Myriad Genetic Laboratories, Salt lake city, USA
Received Date: July 13, 2015 Accepted Date: July 27, 2015 Published Date: July 30, 2015
Citation: Warf MB, Reid JE, Brown KL, Kimbrell H, Kolquist KA, et al. (2015) Analytical Validation of a Cell Cycle Progression Signature Used as a Prognostic Marker in Prostate Cancer. J Mol Biomark Diagn 5:239. doi:10.4172/2155- 9929.1000239
Copyright: ©2015 Warf MB, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Visit for more related articles at Journal of Molecular Biomarkers & Diagnosis
Background: Prostate cancer is the most common cancer in men in the developed world. Appropriate clinical care requires accurate prognostic information to determine whether definitive treatment or conservative management is most appropriate for a given patient. We previously demonstrated that a gene expression signature, which measures the RNA expression of 31 cell cycle progression (CCP) genes and generates a CCP score, is a robust predictor of patient outcome in cohorts of conservatively managed patients diagnosed by needle biopsy or transurethral resection of the prostate.
Methods: These current studies represent the analytical validation of this gene signature, for the testing of either formalin-fixed paraffin-embedded (FFPE) prostate resection tissue (radical prostatectomy, RP) or FFPE prostate needle biopsy samples.
Results: The measured standard deviation (SD) of the signature was determined to be 0.1 score units, representing 1.6% of the range of scores observed within previous clinical validation studies. Individual amplicons for all genes within the signature had a SD <1 CT, with a median SD of 0.52 CT’s. We observed the median amplification efficiency for all genes was 92.6%. The linear range of the signature was over a ~260-fold range of RNA concentrations. We observed that 100% of RP samples and 99.8% of needle biopsy samples produced sufficient RNA for testing, when RNA was extracted from 7,525 recent prostate samples. Finally, RNA samples were able to reproduce similar CCP scores when stored for up to 8 weeks.
Conclusion: These studies indicate that this prognostic gene signature is robust and reproducible, and is analytically validated for use on FFPE prostate biopsy radical prostatectomy samples.
Prostate cancer; Clinical; Prostatectomy; Gene
Prostate cancer is the most prevalent cancer among men in the developed world, with ~30,000 annual deaths in the US . While PSA screening has significantly decreased disease specific mortality , prostate cancer progression is highly variable. As PSA screening is not capable of discriminating between low and high-risk cancers [3,4] this type of screening results in the over-detection of indolent cancers that do not pose a significant risk of mortality . While clinical and pathologic features, such as Gleason score, clinical stage, and baseline PSA levels, are currently utilized to distinguish aggressive and indolent prostate cancers, this has been shown to have limited accuracy [6-8].
As a result of these clinical limitations, a majority of men with prostate cancer will receive treatment that may include surgery and/or additional therapies, despite the fact that only 15% to 30% of prostate cancers will exhibit oncologic progression [9-11]. The risk of treatmentrelated complications and morbidity for many of these patients may outweigh their prostate cancer risk [5-11]. The lack of adequate screening technologies also results in the under-treatment of men with more aggressive cancers. Ultimately, appropriate clinical care requires accurate prognostic information to determine if removal of the prostate or additional therapies after prostate removal might improve patient outcomes.
We recently developed a gene expression signature that can assess the risk of death from prostate cancer by measuring the RNA expression of 31 cell cycle progression (CCP) genes, which have increased expression in aggressive tumors. The expression of the CCP genes is normalized by the expression of 15 housekeeper genes to generate a CCP score, as previously described . Previous clinical validations have demonstrated that the CCP score is a robust predictor of prostate cancer outcomes, including disease-specific survival, in conservatively managed cohorts.
RNA expression assays require stringent analytical validation of the components of the assay, as well as the tissue being tested. The work presented here represents the analytical validation of this gene signature, for the testing of either formalin-fixed paraffin-embedded (FFPE) prostate resection tissue (radical prostatectomy, RP) or FFPE prostate needle biopsy samples. The aim of these studies is to demonstrate that the CCP score is a robust and reproducible molecular diagnostic tool that is appropriate for clinical use.
RNA extraction and CCP score calculation:
This CCP signature has previously been clinically validated on FFPE prostate resection tissue (radical prostatectomy, RP) or FFPE prostate needle biopsy samples [13-16]. The analytical validation studies presented here were performed using commercial samples of FFPE prostate biopsy or RP tissue (Avaden Biosciences, Inc., Scarsdale, NY), or residual RNA from samples submitted for clinical testing. Upon completion of clinical testing, all samples used in these studies were anonymized. All of these studies were performed within a CLIA certified laboratory under established protocols.
RNA expression was analyzed as previously reported . In brief, every sample required an H&E stained slide and 1-5 sections of unstained tissue, containing a total of 3-25 μm of tissue. An anatomic pathologist identified and circled tumor-enriched areas on the H&E slide, which were then macro-dissected from the unstained tissue. RNA was extracted from the unstained tissue using the RNeasy FFPE kit (QIAGEN, Valencia, CA) on a QIACube instrument (QIAGEN). Isolated RNA was treated with DNase I (Sigma-Aldrich, St. Louis, MO), and quantified using a Nanodrop spectrophotometer (Thermo Scientific, Waltham, MA).
To measure gene expression, 25–500 ng of RNA was used to synthesize cDNA with the High Capacity cDNA Reverse Transcription Kit (Life Technologies, Carlsbad, CA). Unless otherwise noted, samples with <25 ng of total RNA were not tested. All CCP and housekeeper genes were pre-amplified for 14 cycles in a single multiplex reaction, using the TaqMan PreAmp Master Mix and the associated TaqMan expression assays for each gene (Life Technologies). Gene expression was then measured on a custom TaqMan Low Density Array, on an Applied Biosystems 7900HT machine. All samples were run in triplicate, with samples being split into the triplicate measurements directly after cDNA synthesis. The expression of each gene was recorded as the CT (crossing threshold) at a pre-specified threshold. The CCP score was calculated as previously reported .
The precision of the CCP score was assessed in a set of 6 FFPE biopsy and 12 FFPE RP samples, collected by a commercial supplier according to IRB-approved guidelines (Avaden Biosciences Inc). The RP samples had sufficient tissue for 3 replicates, while the biopsy samples had sufficient tissue for 4 or 6 replicates. Samples were required to have mean expression of housekeeper genes ≤24 CT, in order to match the average expression of clinical samples. Potential biological variation between the different unstained tissue sections was minimized by combining a set of interleaving unstained tissue slides for each replicate. The precision for the overall CCP score was defined as the standard deviation captured in the residual variation term from the linear mixed model described as: Yij = μ+αi +εij where Yij was the CCP score, μ was the overall mean effect, αi was the random effect from the ith sample (i = 1, 2… 23), and εij was the residual error ~N(0,σε 2) from the jth run of the ith sample (j = 1, 2, …, 6), which was assumed to be independent of αi. Individual gene SDs were determined within the triplicate measurements.
Assessment of linear range for RNA concentration and cDNA amplification efficiency
The linear range for each gene was tested on three “samples”, each of which was the aggregation of anonymized RNA from both biopsy and RP clinical samples with known CCP scores. RNA for samples with similar scores was first combined, then concentrated using a Savant SpeedVac (Thermo Scientific). Serial 2-fold dilutions were prepared for each sample with RNA concentrations ranging from 125–0.06 ng/ μL. Samples were processed using a fixed volume of input RNA after dilutions were made.
The linear range was defined as the minimum and maximum RNA concentrations for which the CCP scores did not vary more than 1 SD from the mean and the R2 value between the CT and the log concentration was greater or equal to 0.93 for all the genes. Amplification efficiencies were calculated for sample B (Figure 1); the resulting cDNA were estimated by using the formula: Efficiency=100(2-1/slope-1) where the slope is estimated from the regression of CT measurements versus log base 2 RNA concentrations, over the previously determined linear range.
Stability of extracted RNA
RNA was extracted from five biopsy and six RP samples and aliquoted after quantification. One aliquot for each sample was initially tested at time zero. All remaining RNA aliquots were stored at -20°C, and an aliquot of each sample was tested every two weeks, over an eight week period.
Clinical laboratory control measures
A variety of control processes have been implemented to ensure the accuracy and precision of clinical samples run within our CLIA certified laboratory, which are codified within standard operating procedures (SOPs). In brief, a batch of six clinical samples will contain both a positive control (RNA with a known CCP score) and a notemplate negative control. In order for clinical samples in a batch to be reported, both the positive and negative control within the batch must perform within predetermined specifications. New lots of reagents are quality control tested against previous reagent lots, prior to use. Reagents are aliquoted and stored until use, according to manufacturer’s guidelines. All instruments within the laboratory have a preventative maintenance schedule; new or serviced instruments are qualified prior to use, according to current operational qualification SOPs. Laboratory technicians undergo biannual proficiency testing.
Reproducibility and precision of the CCP gene expression signature
The overall precision of the signature was determined by testing 18 samples, with 3–6 biological replicates for each sample (depending on the amount of tissue available). Interleaving unstained tissue sections were used when testing biological replicates of each sample, to minimize potential biological variation between the different unstained tissue sections. This precision measurement includes variation within all steps of the process, from tissue macro-dissection through the quantitative measurement of gene expression.
The overall standard deviation (SD) of the signature was determined to be 0.1 CCP score units (95% CI, 0.08- 0.13) between replicate measurements. We observed CCP scores from -2.0 through 4.1 during recent clinical validation studies [13-16]; this precision represents 1.6% of this range of observed scores, indicating the gene expression signature is reproducible and precise.
During the calculation of the CCP score, the expression of 31 CCP genes is normalized by the expression of 15 housekeeper genes. We also determined the precision of the individual TaqMan amplicons for all 46 genes within the signature, by calculating the SD for each amplicon (Table 1 and Figure 1). We observed that all amplicons had a SD ≤1 CT, with a median SD of 0.325CT units. More specifically, we observed that the housekeeper genes were more precise than CCP genes, with a median SD of 0.43 ΔCT and 0.26 CT units for CCP and housekeeper genes, respectively. This result was anticipated, as the housekeeper genes generally have higher expression than the CCP genes
Stability of stored RNA
To assess the reproducibility and stability of RNA that is stored at -20°C, we tested the reproducibility of CCP scores of 11 samples (5 biopsy and 6 RP) over an 8 week timeframe, testing each sample every 2 weeks. We observed that the CCP scores were reproducible across all time points, with no trend in the scores of any of the individual samples (Figure 2). This indicates that there is not a bias for the CCP score to change as the RNA is stored over this time period. Furthermore, we observed that all samples had a SD equal to or less than 0.1 CCP score units, which is similar to overall precision of the signature.
Yields of RNA extraction from FFPE tissue
This RNA expression signature requires a minimum RNA concentration of 2 ng/μL (25 ng of input RNA). Samples with concentrations greater than 40 ng/μL (500 ng of input RNA) are diluted and tested at an RNA concentration of 40 ng/μL. In order to determine the frequency at which the RNA extraction process provides sufficient RNA for testing, the RNA yields from 952 RP and 6,573 biopsy clinical samples were assessed (the majority of which are less than 2 years in age). We observed that 100% of the RP and 99.8% of the biopsy samples produced sufficient RNA for testing (Figure 3). Furthermore, 82.6% of the RP samples produced RNA concentrations in excess of 40 ng/μL and required subsequent dilution, while only 5.5% of the biopsy samples required dilution (Figure 3). It was expected that RP samples would produce such large quantities of RNA compared to biopsy samples, due to the large relative difference in the sizes of the two types of samples.
Linearity of the RNA concentration
The linearity of the signature was determined in regards to RNA concentration, as the amount of RNA obtained from FFPE samples can vary drastically. To this end, 3 samples were tested across a range of RNA concentrations from 125 to 0.06 ng/μL (1,560 to 1.5 ng of input RNA), which exceeds the clinical range over which we test samples (40 to 2 ng/μL, 500 to 25 ng of input RNA). The 3 samples tested had scores that ranged from -1.5 to 2.1, to assess whether samples with different CCP scores produced consistent scores over the same range of RNA concentrations. We observed that all three samples had consistent CCP scores across the entire range of RNA concentrations that was assessed (Figure 4); however, none of the three samples produced a CCP score at 0.06 ng/μL (1.5 ng of input RNA) because the CCP scores at those concentrations did not pass our quality control measures. Although the low scoring sample failed to give a result at 7.8 ng/μL due to an apparent technical failure, at least two concentrations greater than and less than this concentration gave successful results.
Figure 4: Dilution series for the RNA concentration, for 3 samples with different CCP scores. Note, the Low score sample failed to produce a CCP score at concentrations 7.8 and 0.12 ng/ÂµL, and all three samples failed to produce a CCP score at 0.06 ng/ÂµL because the CCP scores at those concentrations did not pass our quality control measures.
Using these data, we next calculated that the linear range of the RNA concentration was from 62.5 to 0.24 ng/μL. This ~260-fold range exceeds the 20-fold range of RNA concentrations over which the signature was clinically validated and clinical samples are tested (40 to 2 ng/μL). We also observed that every CCP score within this linear range had a CCP score within <1 SD (0.1 score units) of the average CCP score for that sample, for all three samples (Figure 4).
Amplification efficiency of genes within the CCP gene expression signature
The amplification efficiency of each amplicon within the signature was also determined, over the linear RNA concentration range from 62.5 to 0.24 ng/μL (Table 1). We observed no statistical difference in the amplification efficiencies when comparing housekeeper and target genes (p-value, 0.39).
|Gene||Function||Amplification Efficiency (%)||R2||SD||95% CI|
|ASF1B||CCP||111||0.96||0.80||0.71 , 0.91|
|ASPM||CCP||90||0.97||0.36||0.32 , 0.42|
|BIRC5||CCP||90||0.99||0.99||0.88 , 1.14|
|BUB1B||CCP||82||0.98||0.30||0.26 , 0.34|
|C18orf24||CCP||81||0.97||0.71||0.63 , 0.81|
|CDC2||CCP||108||0.95||0.39||0.35 , 0.45|
|CDC20||CCP||101||0.95||0.39||0.35 , 0.45|
|CDCA3||CCP||96||0.96||0.98||0.87 , 1.13|
|CDCA8||CCP||93||0.99||0.21||0.19 , 0.24|
|CDKN3||CCP||90||0.98||0.28||0.25 , 0.32|
|CENPF||CCP||81||0.99||0.45||0.40 , 0.51|
|CENPM||CCP||107||0.98||0.50||0.45 , 0.57|
|CEP55||CCP||93||0.93||0.83||0.73 , 0.95|
|DLGAP5||CCP||79||0.97||0.49||0.44 , 0.56|
|DTL||CCP||82||0.96||0.50||0.44 , 0.57|
|FOXM1||CCP||102||0.98||0.32||0.28 , 0.36|
|KIAA0101||CCP||105||0.96||0.31||0.28 , 0.35|
|KIF11||CCP||78||0.96||0.58||0.51 , 0.66|
|KIF20A||CCP||94||0.95||0.61||0.54 , 0.69|
|MCM10||CCP||111||0.96||0.64||0.57 , 0.73|
|NUSAP1||CCP||105||0.96||0.43||0.38 , 0.49|
|ORC6L||CCP||85||0.97||0.41||0.37 , 0.47|
|PBK||CCP||89||0.97||0.59||0.53 , 0.68|
|PLK1||CCP||83||0.98||0.33||0.29 , 0.38|
|PRC1||CCP||91||0.97||0.26||0.23 , 0.30|
|PTTG1||CCP||98||0.99||0.43||0.38 , 0.49|
|RAD51||CCP||105||0.94||0.57||0.50 , 0.65|
|RAD54L||CCP||105||0.96||0.41||0.37 , 0.47|
|RRM2||CCP||90||0.98||0.32||0.29 , 0.37|
|TK1||CCP||121||0.96||0.28||0.25 , 0.32|
|TOP2A||CCP||84||0.97||0.65||0.58 , 0.74|
|CLTC||HK||107||0.95||0.26||0.23 , 0.29|
|MMADHC||HK||107||0.95||0.32||0.29 , 0.37|
|MRFAP1||HK||117||0.95||0.24||0.21 , 0.27|
|PPP2CA||HK||107||0.97||0.29||0.26 , 0.33|
|PSMA1||HK||91||0.98||0.23||0.20 , 0.26|
|PSMC1||HK||85||0.98||0.32||0.28 , 0.36|
|RPL13A||HK||102||0.97||0.27||0.24 , 0.31|
|RPL37||HK||82||0.99||0.27||0.24 , 0.31|
|RPL38||HK||92||0.97||0.25||0.22 , 0.28|
|RPL4||HK||86||0.99||0.24||0.21 , 0.27|
|RPL8||HK||100||0.96||0.27||0.24 , 0.31|
|RPS29||HK||117||0.96||0.22||0.20 , 0.25|
|SLC25A3||HK||102||0.97||0.19||0.17 , 0.22|
|TXNL1||HK||83||0.97||0.27||0.24 , 0.31|
|UBA52||HK||88||0.98||0.21||0.19 , 0.24|
SD, standard deviation. CI, confidence interval. CCP, cell cycle progression. HK, housekeeper. The listed R2 value is for the fit when determining the amplification efficiency for each amplicon.
Table 1: List of genes, their function within the gene signature, as well as the amplification efficiencies and SD of the CT and ?CT for individual housekeeper and target genes.
Dynamic range of the CCP gene expression signature
We previously determined the dynamic range over which this gene expression signature could produce valid CCP scores . This was accomplished by testing samples with an expected CCP score (as created by pre-specified ratios of the CCP genes to housekeeper genes) and comparing the expected score with the observed CCP score for each sample. We found that the signature had a very wide dynamic range over a 108-fold range of gene expression, from CCP scores of -13 to 14. The range of CCP scores we observed within recent clinical validations in prostate cancer samples (CCP scores from -2.0 to 4.1) [12-17] is well within the dynamic range of the gene expression signature.
The CCP score is calculated from the average expression of 31 CCP genes and 15 housekeeper genes and has been clinically validated as a robust predictor of prostate cancer outcomes in cohorts of conservatively managed men [12-17]. Previous studies have also demonstrated the clinical utility of the CCP score, which has been shown to influence medical management . However, this clinical utility requires high analytical accuracy and precision. Here, we have presented the analytical validation of this signature, assessing the reproducibility, dynamic range, linearity, amplicon efficiency, and precision of the signature
Previous studies have demonstrated a clinical range of CCP scores in biopsy and RP prostate samples from -2.0 to 4.1 [13-16]. The standard deviation of the CCP score determined here is 0.1 CCP score units between sample replicates, which is only 1.6% of the range of clinically observed scores. This is similar to previous analytical validations of RNA expression signatures, including the precision of this CCP expression signature within lung resection tissue [18,20-22]. The individual amplicons for all 46 genes in the signature had a SD ≤1 CT, with the housekeeper genes showing a greater degree of precision than CCP genes (Figure 1 and Table 1). These data are consistent with the higher overall expression of the housekeeper genes compared to the CCP genes in prostate tissue, resulting in higher precision when measuring housekeeper gene expression.
The stability of the RNA extracted from both needle biopsy and RP samples was also examined, as clinical samples occasionally require re-testing. Currently, extracted RNA from clinical samples is stored for one month within our CLIA laboratory, after which RNA must be extracted from new tissue if a sample requires re-testing. We observed that representative biopsy and RP samples were stable over at least an eight week period (Figure 2), which is double the amount of time clinical samples are retained for re-testing.
The quantity of RNA extracted from FFPE tissue can be highly variable, which can impact clinical testing of RNA expression. Here we have shown that 100% of RP samples (n=952) produced sufficient RNA for testing (2 ng/μL, 25 ng of input RNA), with 82.6% having concentrations high enough to require subsequent dilution prior to testing (≥40 ng/μL). Similarly, 99.8% of biopsy samples (n=6,573) had RNA concentrations sufficient for testing, while only 5.5% had concentrations high enough to require dilution. With the large difference of the relative sizes of RP and needle biopsy lesions, it was unsurprising that a large percentage of only RP samples yielded such high concentrations of RNA.
Previous studies have shown that dynamic range of the CCP score is from -13 to 14 . This extends well beyond the clinical range of scores observed in biopsy and RP samples (-2.0 to 4.1). Linearity of the CCP score was observed for RNA concentrations ranging from 62.5 to 0.24 ng/μL (781 to 3 ng of input RNA) (Figure 1), which includes the full range of RNA concentrations specified in the clinical testing protocol (40 to 2 ng/μL). Although this indicates that testing samples with concentrations as low as 0.24 ng/μL could provide an accurate CCP score, the threshold at 2 ng/μL is based on the detection limit of the Nanodrop spectrophotometer used for RNA quantification.
Many analytical validations of RNA expression signatures also require assessment of potential PCR inhibitors found within different types of tissue. For example, melanin found in skin biopsies can inhibit PCR and must be explicitly investigated . However, there are no known PCR inhibitors found in prostate tissue.
The work presented here demonstrates this CCP gene expression signature is reproducible and robust when testing prostate FFPE needle biopsy and FFPE RP samples. The linear and dynamic range of the signature exceeds the parameters utilized in clinical testing, indicating that the test is suitable for use. In conjunction with the previous clinical validation [13,16] and utility studies , these studies indicate that this signature can be useful in providing accurate prognostic information to better inform medical management decisions for men with prostate cancer.