ISSN: 0974-276X
Journal of Proteomics & Bioinformatics
Make the best use of Scientific Research and information from our 700+ peer reviewed, Open Access Journals that operates with the help of 50,000+ Editorial Board Members and esteemed reviewers and 1000+ Scientific associations in Medical, Clinical, Pharmaceutical, Engineering, Technology and Management Fields.
 
Meet Inspiring Speakers and Experts at our 3000+ Global Conferenceseries Events with over 600+ Conferences, 1200+ Symposiums and 1200+ Workshops on
Medical, Pharma, Engineering, Science, Technology and Business

Range Charts for Agreement in Measurement Comparison Studies, With Application to Replicate Mass Spectrometry Experiments

James A. Koziol1*, Anne C. Feng1, Jingyi Yu2, Noelle M. Griffin2,Jan E. chnitzer2
1Department of Molecular and Experimental Medicine, The Scripps Research Institute,  10550 North Torrey Pines Road, La Jolla, CA 92037
2Sidney Kimmel Cancer Center, 10905 Road to the Cure, San Diego, CA 92122
Corresponding Author : Dr. James A. Koziol
Ph.D, Department of Molecular and Experimental Medicine
MEM216, The Scripps Research Institute, 10550 North Torrey
Pines Road, La Jolla, CA 92037,
Tel        : 858-784-2703,
Fax      : 858-784-2664,                                       
E-mail  : koziol@scripps.edu
Received August 02, 2008; Accepted September 02, 2008; Published September 13, 2008
Citation: James AK, Anne CF, Jingyi Y, Noelle MG, Jan ES (2008) Range Charts for Agreement in Measurement Comparison Studies, With Application to Replicate Mass Spectrometry Experiments. J Proteomics Bioinform 1:287-292. doi:10.4172/jpb.1000036
Copyright: © 2008 James AK, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Related article at
DownloadPubmed DownloadScholar Google

Visit for more related articles at Journal of Proteomics & Bioinformatics

Abstract

It is important to investigate the reproducibility of raw mass spectrometry (MS) features of abundance, such as spectral count, peptide number and ion intensity values, when conducting replicate mass spectrometry measurements. Reproducibility can be inferred from these replicate data either formally with analyses of variance techniques or informally with graphical procedures, particularly, Bland-Altman plots on paired runs. In this note, we suggest range plots to provide a suitable generalization of Bland-Altman plots to experiments with more than two replicate runs. We describe range charts and their interpretation, and illustrate their use with data from a recent proteomic study relating to label-free analysis.

Introduction
Bland and Altman, (1986) introduced a useful graphical procedure for comparing two methods of measurement of a continuous biological variable. The Bland-Altman plot of differences (ordinate) versus means (abscissa) of paired measurements is simple and elegant, and has become a standard for agreement studies, especially when combined with limits of agreement. Typically, 95% limits of agreement lines are superimposed on the Bland-Altman plots; these provide an interval within which 95% of differences between future measurements by the two methods would be expected to lie.
If one wishes to compare more than two matched measurements, pairwise Bland-Altman plots can be shown. This might prove cumbersome, however, with increasing numbers of measurements being compared. Bland and Altman, (1999) and Bland and Altman, (2007) have proposed rigorous statistical methods for such studies, devolving from formal analyses of variance of the method comparison studies. Similarly, an informal graphical procedure is available from the quality control literature, namely, Shewhart control charts for the range (Shewhart, 1939; Montgomery, 2001). Conceptually, one plots the ranges (ordinate) versus the means (abscissa) of the matched samples for the range chart.
In this note, we describe range charts for multiplymatched comparison studies, and illustrate some techniques for imposing limits of agreement on the charts. We illustrate this method with data arising from a recent study we had undertaken, to investigate the reproducibility of raw mass spectrometry features of abundance, including spectral count, peptide number and ion intensity values, across replicate 2DLC mass spectrometry measurements.
Reference Intervals
In comparison studies of two methods, the limits of agreement advocated by Bland and Altman can be interpreted as reference intervals, and delineate the range within which most differences between measurements might be expected to lie. The reference interval is generally defined by the range between two centile values of a population, centered about the median value. For example, the standard limits of a 95% reference interval would be the 2.5th and 97.5th centiles.[See Altman and Bland,(1994) for some discussion of terminology regarding centiles, quantiles, and related quantities.]
There are many ways of constructing reference intervals, as elegantly summarized by Wright and Royston, (1999) and Bland and Altman, (2007). We here describe two simple nonparametric techniques, which make no assumptions about the underlying distributions of the data comprising the range charts, followed by a semi-parametric technique, related to underlying normality.
Harrell-Davis
Harrell and Davis (1982) introduced an elegant distribution- free quantile estimator, which may be invoked to estimate the centiles of an underlying distribution.
This approach is formulaically straightforward, but does require computational capabilities involving mathematical functions. For reference, we state Harrell and Davis's formula in the Appendix.
Bootstrap
A second nonparametric approach utilizes the bootstrap (Efron and Tibshirani, 1993), as an extension of the simple approach of estimating empirical centiles directly from the ordered observations. Draw, say, 1000 bootstrap samples of the same cardinality from the underlying observations. From each bootstrap sample, compute the desired empirical centiles, using interpolation if necessary between the nearest two order statistics. Then, average the corresponding empirical centiles over the bootstrap samples to establish the reference interval limits. This approach is computationally involved, but formulaically simple.
Box-Cox Transformation Procedure
Box and Cox, (1964) introduced a power function that has been widely adopted to transform data to approximate normality. In the present context, one would invoke the Box- Cox procedure on the sample ranges, determine limits of agreement on the Box-Cox transformed scale, and then back-transform to the original range scale. There is some computational complexity involved in finding the Box-Cox transformation parameter [see Appendix],but determination of limits of agreement on a putatively normal scale is straightforward.
An Application
We illustrate range charts and reference intervals with the following example. Yu and colleagues, (2008) have undertaken an extensive study of computational and statistical considerations relating to proteomic analyses of label-free mass spectroscopy (MS) data. We utilize one dataset from their study, consisting of normalized spectral indices of 174 peptides found in normal liver tissue from three replicate MS runs. Yu et al., 2008 investigate fundamental issues of reproducibility and effects of normalization in this and other datasets, and we make no attempt to recapitulate their analyses here. Rather, our use of range charts is meant to be complementary to their more comprehensive study.
In Figure 1, we give the range chart for the 174 replicates. Clearly, variability increases with sample mean, an important diagnostic from these plots. We may take a simple log transformation (Bland and Altman, 1986, Bland and Altman,1996) of the spectral indices, with the intent to render the variance independent of the mean. As shown in Figure 2,the log transform goes far toward ameliorating the increasing variability with mean in the original spectral data.
We turn next to the issue of tolerance intervals. In Table 1we give the limits of the 90% and 95% tolerance intervals for the original data as well as the log-transformed data, for the three methods described earlier. That the Harrell-Davis and bootstrap estimates are nearly coincident is unsurprising: Sheather and Marron (1990) have noted the interconnection of the two methods. The Box-Cox method is somewhat deviant from the other methods, particularly in the upper limits for the original data. Here, there are 14 values (8.0%) exceeding 211, 7 (4.5%) exceeding 386 or 396, 5 (2.9%) exceeding 537, and 4 (2.3%) exceeding 887 or 889; the nonparametric approaches appear to have coverages nearer the desired levels of 5% or 2.5% than Box-Cox. We comment that with the original data, Box-Cox was not altogether successful in achieving normality: sample skewness was reduced from 5.27 to 0.004 (target 0), but sample kurtosis was reduced from 33.32 to 3.59 (target 3). Further correction for kurtosis might be desirable. In Figure 1and Figure 2 we have also superimposed the nonparametric twosided 90% tolerance intervals; again, the log transformation seems quite suitable.
Discussion
Our motivation for investigating range charts was a comprehensive study we had undertaken, to investigate the reproducibility of raw mass spectrometry (MS) features of abundance, including spectral count, peptide number and ion intensity values, across replicate 2DLC mass spectrometry measurements. We found that typically raw features were not very reproducible across replicates. We could verify this formally via analyses of variance; our search for an informal graphical procedure led us first to Bland-Altman plots, and then to range plots as an appropriate extension to greater than two replicate runs. The range plots gave us immediate indication that log transformation would ameliorate the increasing variability problem. We went on to develop a more involved normalization technique which also tended to work well with our experimental data.
The reproducibility of a proteomics experiment can be viewed in two ways, 1) reproducibility in terms of the proteins identified, i.e., are we identifying the same proteins in replicate 2 as we did in replicate 1, and 2) reproducibility in the relative quantity of these proteins detected across the replicates, i.e., is the relative abundance of protein A the same in replicate 1 as in replicate 2. In terms of protein identification, Durr et al., (2004) previously showed that any second replicate "shotgun" MS measurement will identify 30-40% of proteins not found in a single MS measurement of an identical sample, inferring an approximate 60% overlap in protein identifications between replicate measurements. This also demonstrates the requirement for multiple replicate measurements in order to achieve significant coverage of the sample of interest; we refer the interested reader to Koziol et al., (2006), where we give some practical guidelines for experimental design. Nevertheless, the Yu et al.,2008 study clearly demonstrated [both formally and with range charts] that the abundance features for these common proteins are not very reproducible across replicate MS measurements. As a result, normalization of these features (by such method as developed by Yu et al., 2008) is required to control for the variation, which in turn enhances the reproducibility of the replicates thus allowing their direct comparison.
Range charts are conceptually straightforward, and are an economical way of depicting levels of agreement among multiple measurements. Compared to pairwise Bland-Altman plots, one range chart summarizes the entire agreement study; however, signed differences revelatory of systematic large or small values in one method compared to a second method may be obscured.
There are various ways of establishing reference intervals on range charts. We have focused on nonparametric and semiparametric methods that are relatively easy to implement, and dispense with sometimes restrictive distributional assumptions attending ranges. We mention in passing that we also explored quantile estimation based on Edgeworth and Cornish-Fisher expansions (Cornish and Fisher, 1937; Stuart and Ord, 1987), but results were less satisfactory than with Box-Cox. The distribution of the range with normally distributed random variables has been extensively studied, leading to parametric approaches to establishing reference intervals; see Montgomery, (2001) for details. And, one could plausibly impose one-sided limits of agreement with range charts, in settings where upper limits of agreement are of primary interest
We remark that, with two matched measurements, the range chart displays the absolute values of the signed differences from the Bland-Altman plot. This relationship generalizes to our setting of three MS determinations, a consequence of the mathematical identity: range{x1, x2, x3} = ½ ( |x1 - x2| + |x2 - x3| + |x3 - x1| ), for any real numbers x1, x2, x3. That is, the range of 3 observations [as in a range chart] can be recovered from the pairwise differences [as in the Bland-Altman plots], though not conversely.
In summary, reproducibility is an important criterion for assessing proteomic experiments, and is an essential property of validation. In this regard, range charts provide immediate graphical representation of comparison studies, from which unacceptable or untoward levels of variability can be discerned. Range charts constitute straightforward assessments of method comparison and agreement, and are complementary to more formal assessments of agreement, as described by Bland and Altman, (1999) and Bland and Altman, 2007.
Acknowledgments
We thank the reviewers for their insightful comments. This research was support in part by grant PO1CA164898 from the National Institutes of Health.
References
  1. Altman DG and Bland JM (1994) Quartiles, quintiles, centiles, and other centiles. BMJ 309: 996. » CrossRef   » PubMed   »  Google Scholar

  2. Bland JM and Altman DG (1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet i: 307-310. » CrossRef   » PubMed   »  Google Scholar

  3. Bland JM and Altman DG (1996) Transforming data. BMJ 312: 770. » CrossRef   » PubMed   »  Google Scholar

  4. Bland JM and Altman DG (1999) Measuring agreement in method comparison studies. Statist Meth Med Res 8: 135- 160. » CrossRef   » PubMed   »  Google Scholar

  5. Bland JM and Altman DG (2007) Agreement between methods of measurement with multiple observations per individual. J Biopharm Statist 17: 571-582. » CrossRef   » PubMed   »  Google Scholar

  6. Box GEP and Cox DR (1964) An analysis of transformations (with discussion). J Roy Statist Soc B 26: 211-252.

  7. Cornish EA and Fisher RA (1937) Moments and cumulants in the specification of distributions, Review of the International Statistical Institute 5: 307-320. » CrossRef  »  Google Scholar

  8. Durr E, Yu J, Krasinska KM (2004) Direct proteomic mapping of the lung microvascular endothelial cell surface in vivo and in cell culture. Nature Biotechnol 22: 985- 992. » CrossRef   » PubMed   »  Google Scholar

  9. Efron B and Tibshirani RJ (1993) An Introduction to the Bootstrap Chapman and Hall London. » CrossRef  »  Google Scholar

  10. Horn PS, Pesce AJ, Copeland BE (1998) A robust approach to reference interval estimation and evaluation. Clinical Chemistry 44: 622-631. » CrossRef   » PubMed   »  Google Scholar

  11. Koziol JA, Feng AC, Schnitzer JE (2006) Application of capture- recapture models to estimation of protein count in MudPIT experiments. Anal Chem 78: 3203-3207. » CrossRef   » PubMed   »  Google Scholar

  12. Montgomery DC (2001) Introduction to Statistical Quality Control 4th Ed John Wiley New York.

  13. Shewhart WA (1939) (with the editorial assistance of Deming W). Statistical Method from the Viewpoint of Quality Control Graduate School US Department of Agriculture Washington DC.

  14. Stuart A, Ord JK (1987) Kendall’s Advanced Theory of Statistics Volume I 5th Ed Oxford University Press New York.

  15. Wright EM and Royston P (1999) Calculating reference intervals for laboratory measurements. Statist Meth Med Res 8: 93-112.

  16. Yu J, Griffin NM, Oh P, Shore S (2008) Quantification and normalization of complex label-free mass spectrometry data for proteomics analysis. submitted.

Select your language of interest to view the total content in your interested language
 
Share This Article
   
 
   
 
Relevant Topics
Disc Applications of Bioinformatics
Disc Bacterial transcriptome
Disc Bioinformatics Algorithms
Disc Bioinformatics Databases
Disc Bioinformatics Tools
Disc Cancer Pharmacogenomics
Disc Cancer Proteomics
Disc Clinical Pharmacogenomics
Disc Clinical Proteomics
Disc Cluster analysis
Disc Comparative genomics
Disc Comparative proteomics
Disc Comparative transcriptomics
Disc Computational drug design
Disc Current Proteomics
Disc Data algorithms
Disc Data mining applications in genomics
Disc Data mining applications in proteomics
Disc Data mining in drug discovery
Disc Data mining tools
Disc Data modelling and intellegence
Disc Data warehousing
Disc Drug Dosage Formulations
Disc Drug Toxicity and Efficacy
Disc Epigenetics
Disc Epigenomic studies
Disc Gene Expression profiling
Disc Gene polymorphism
Disc Genome annotation
Disc Genomic Targets
Disc Genomic data mining
Disc Genomic data warehousing
Disc Glycome
Disc Human Proteome Project Applications
Disc Immune Disorders
Disc Individualized Medicine
Disc Mapping of genomes
Disc Mass Spectrometry in Proteomics
Disc Meta genomics
Disc Metabolome
Disc Microarray
Disc Microarray Proteomics
Disc Molecular and Cellular Proteomics
Disc Mouse transcriptome
Disc Non coding MRNA
Disc Personalized Medicine Studies
Disc Pharmacoeconomics in Drug Development
Disc Pharmacogenetics
Disc Pharmacogenomic Biomarker
Disc Pharmacogenomics Applications
Disc Pharmacogenomics Future Medicine
Disc Pharmacogenomics and Personalized Medicine
Disc Pharmacogenomics for Patient Care
Disc Pharmacoproteomics in Drug development
Disc Profiling
Disc Protein Sequence Analysis
Disc Protein engineering
Disc Proteogenomics
Disc Proteome
Disc Proteome Profiling
Disc Proteomic Analysis
Disc Proteomic Biomarkers
Disc Proteomics Clinical Applications
Disc Proteomics Research
Disc Proteomics Science
Disc Proteomics and Pharmacodynamics
Disc Proteomics data warehousing
Disc Python for Bioinformatics
Disc Quantitative Proteomics
Disc RNA sequencing
Disc RNA sequencing and analysis
Disc Sequencing
Disc Small RNA Sequencing
Disc Statistical data mining
Disc Transcripotme
Disc Transcriptional Attenuation
Disc Transcriptional Regulation
Disc Transcriptome analysis
Disc Translational Medicine
 
Recommended Journals
Disc Transcriptomics Journal
Disc Pharmacogenomics Journal
Disc Data Mining Journal
  View More»
 
Recommended Conferences
Disc 6th Bioinformatics Conference
August 22-23, 2016 Philadelphia, Pennsylvania, USA
Disc 7th International Conference and Expo on Proteomics
October 24-26, 2016 Rome, Italy
View More»
 
Article Tools
Disc Export citation
Disc Share/Blog this article
 
Article usage
  Total views: 11141
  [From(publication date):
September-2008 - May 24, 2016]
  Breakdown by view type
  HTML page views : 7412
  PDF downloads :3729
 
 

Post your comment

captcha   Reload  Can't read the image? click here to refresh

 
OMICS International Journals
 
Make the best use of Scientific Research and information from our 700 + peer reviewed, Open Access Journals
 
 
OMICS International Conferences 2016-17
 
Meet Inspiring Speakers and Experts at our 3000+ Global Annual Meetings
 
 

Contact Us

Agri, Food, Aqua and Veterinary Science Journals

Dr. Krish

agrifoodaquavet@omicsinc.com

1-702-714-7001 Extn: 9040

Clinical and Biochemistry Journals

Datta A

clinical_biochem@omicsinc.com

1-702-714-7001Extn: 9037

Business & Management Journals

Ronald

business@omicsinc.com

1-702-714-7001Extn: 9042

Chemical Engineering and Chemistry Journals

Gabriel Shaw

chemicaleng_chemistry@omicsinc.com

1-702-714-7001 Extn: 9040

Earth & Environmental Sciences

Katie Wilson

environmentalsci@omicsinc.com

1-702-714-7001Extn: 9042

Engineering Journals

James Franklin

engineering@omicsinc.com

1-702-714-7001Extn: 9042

General Science and Health care Journals

Andrea Jason

generalsci_healthcare@omicsinc.com

1-702-714-7001Extn: 9043

Genetics and Molecular Biology Journals

Anna Melissa

genetics_molbio@omicsinc.com

1-702-714-7001 Extn: 9006

Immunology & Microbiology Journals

David Gorantl

immuno_microbio@omicsinc.com

1-702-714-7001Extn: 9014

Informatics Journals

Stephanie Skinner

omics@omicsinc.com

1-702-714-7001Extn: 9039

Material Sciences Journals

Rachle Green

materialsci@omicsinc.com

1-702-714-7001Extn: 9039

Mathematics and Physics Journals

Jim Willison

mathematics_physics@omicsinc.com

1-702-714-7001 Extn: 9042

Medical Journals

Nimmi Anna

medical@omicsinc.com

1-702-714-7001 Extn: 9038

Neuroscience & Psychology Journals

Nathan T

neuro_psychology@omicsinc.com

1-702-714-7001Extn: 9041

Pharmaceutical Sciences Journals

John Behannon

pharma@omicsinc.com

1-702-714-7001Extn: 9007

Social & Political Science Journals

Steve Harry

social_politicalsci@omicsinc.com

1-702-714-7001 Extn: 9042

 
© 2008-2016 OMICS International - Open Access Publisher. Best viewed in Mozilla Firefox | Google Chrome | Above IE 7.0 version