Journal of Obesity & Weight Loss Therapy
Open Access

Body weight is an important clinical variable that reflects health and wellbeing. It is commonly obtained in large scale epidemiologic studies, and it is an important outcome in clinical trials that evaluate interventions targeting weight loss. Obtaining accurate assessments of weight is the cornerstone for ensuring that these studies are internally valid. Patient self-report is a convenient and inexpensive way to obtain weight information and numerous studies has assessed the reliability of patient self-report of weight compared to in person obtained values [1-4]. In general, self-reported weight values are about 4 pounds lower than coincident values obtained in person. Women slightly underreport weight more than men [5], and the longer the time interval between the value obtained by self-report and the in-person value, the greater the discrepancy [6]. However, it is generally felt that selfreported weight is an adequate proxy for in-person values, particularly for epidemiologic and populational studies. What is not known is the degree to which weight values obtained from electronic medical record (EMR) data agrees with values obtained by other methods. EMR obtained data, if it is reliable, has the potential to be more convenient to obtain for large studies than weights obtained by other methods (e.g., patient self-report via telephone). The goal of this analysis is to determine the reliability and agreement of EMR recorded weights with those obtained by self-report.

The Telephone Lifestyle Coaching demonstration program was a 3- year prevention program (2011-2014) led by the VA’s National Center for Health Promotion and Disease Prevention (NCP) and a private contractor, Alere™ Wellbeing (Alere). The intervention was offered to veterans in 24 VA facilities from geographically diverse areas throughout the United States.

Eligible veterans were age 18 or older; fluent in written and spoken English; and of normal cognitive function. Additionally, veterans had to express interest in making one or more behavioral changes (smoking cessation, improved physical activity, improved diet quality, weight loss, decreased alcohol consumption, and/or stress management) and be willing to receive health coaching via telephone. The health coaching occurred through a series of telephone calls (up to 9), over a period of 6 months. The calls focused on the specific behavior change that the veteran was interested in working on (e.g., weight reduction). Referral to the program generally occurred following a primary care clinic visit. The referral was made by the primary care team via the EMR through an electronic consult. This quality improvement program evaluation was deemed exempt from review by the Durham VA Institutional Review Board.

Self-report values

All veterans referred to the program were first asked to complete a telephone survey that included questions about their health behaviors and a question about their current weight in pounds. All 5,000 veterans completed this survey and provided a baseline self-reported weight. At the end of the coaching intervention a repeat telephone survey was attempted, and again, veterans were asked their current weight. We analyzed repeat six-month self-reported weight values for the 2,458 veterans who completed the survey. Based on the self-reported weight values at the six-month survey 605 veterans reported losing >5% of their baseline weight. The telephone surveys were conducted by personnel who were different from the patients’ telephone coach.

Medical record

We used the weight data obtained in the VA’s EMR from 2010-2015. These values are typically measured with a digital scale, the types of which may vary across clinical sites, at an outpatient clinic visit by nurse personnel who conduct an initial patient assessment before the visit. The values from the measurement are then entered into the local EMR, which is then transmitted to a national corporate data warehouse. We extracted weight data from the national corporate data warehouse looking for recorded weights within a window of 3 months prior to enrollment in Telephone Lifestyle Coaching, deleting observations that were outside of a 50-700 pound weight range. Each subject’s longitudinal weight trajectory was cleaned using a method that calculated the standard deviation of consecutive clusters of measurements and excluded extreme outliers; 88 outlier weight observations were removed. Finally, we removed 6 subjects whose survey reported weight was >100 pounds discrepant with their observed weight in the corporate data warehouse. We have used these methods to determine weight in national cohorts of veterans for both cross sectional and longitudinal studies [7,8]. A total of 31,568 weight values for the 5,000 subjects were available in that time period. For these analyses, we used the weight value in the EMR that was closest in date to self-reported value date prior to enrollment, both at the baseline survey and again at the 6-month survey.

Analyses

Reliability was assessed with intraclass correlation coefficients. We also constructed Bland Altman graphs to assess agreement between the two measures across the full range of weights [9].

The 5,000 veterans available for analysis had an average age at enrollment of 56 years. 83% were male, and 67% were white, 27% black. There was extremely high reliability between EMR values and self-reported body weight for both baseline values ICC=0.99. Figure 1 shows the Bland-Altman plot for the baseline self-reported weight and corresponding EMR values. The baseline weights showed an average of 1.0 pounds lower self-reported weight compared to EMR values which remained constant across the range of self-reported weights. The average number of days separating the self-reported value and the EMR value closest, but prior, to that date was 7.4 days. At six months, 2,458 veterans completed the survey and provided self-reported weights. There remained a high degree of reliability between EMR and self-reported weights ICC=0.98. Now, however, there was a 4.6 pound average lower self-reported weight compared to EMR values (Figure 2).

Figure 1: Baseline bland-altman plot: closest cdw weight within three months prior to enrollment

Figure 2: Six month bland-altman closest cdw weight no later than 9 month post enrollment.

The average number of days separating the self-reported value and the EMR value was now 39 days. Based on the six-month self-reported weight 605 veterans reported losing >5% of their baseline weight. These six-month self-reported values were 9.4 pounds lower than the closest EMR value (Figure 3).

Figure 3: Six month bland-altman plat closet cdw weight within one month prior to (or at) 9 month post enrollment for patients who reported >=5% body weight loss.

We performed a sensitivity analysis to determine if restricting the EMR weight value to a tighter time window would improve agreement. Our base case scenario allowed EMR weight values up to 3 months prior to the baseline self-reported survey. When we restricted that time window to only allow weights up to 1 month prior to the baseline selfreported survey, our sample was reduced from 5,000 to 4,643. There was no change in the ICC-0.99, and there was a similar 1.0 pound average lower self-reported weight. We repeated this process for the six-month survey limiting the EMR time window to one-month before the survey. Our sample was reduced from 2,458 to 978 and the bias in under-reporting weight decreased from 4.6 to 3.7 pounds.

There has been an extensive amount of interest in the most accurate and reliable method to obtain patient weight values in the most economical and efficient manner. One of the earliest studies on reliability and validity of patient self-report of weight derived from patients participating in the RAND health insurance experiment showed excellent agreement between self-report and weight values obtained at a research clinic visit [1]. Findings from this study have generally held true in subsequent studies from across the world [2-6]. Trends from these studies show that women tend to under-report weight more than men (3.1 lbs for women vs. 1.6 lbs for men); patient age is not associated with under-reporting; and, there is a positive association between under-reporting of weight as patients’ weight increases [1]. The majority of weight reliability literature was conducted in studies where a separate patient visit was required to obtain the in-person value. The protocols used to obtain these values are often highly specified and carried out by trained research personnel. Therefore, it is unclear to what degree weight values obtained from EMR values, which may be subject to high variability in how they are obtained, would agree with patient self-report.

This study showed that EMR obtained weight values show excellent agreement and reliability with those obtained by self-report. In fact, the one-pound difference between these values that were obtained at the baseline survey is the smallest difference reported to date. Most studies report about a 4-pound difference between self-report and inperson obtained values [1]. Additionally, narrowing the window of time in which we searched for EMR weight values from three months to one month did not improve what was already excellent reliability and agreement. This means that for evaluations that require a weight value at a specific point in time it is reasonable to allow values at least three months around the desired point. An alternative approach is to take all available EMR data and describing a weight trajectory over time for specific patients [7,8]. This approach may be useful in population-based studies.

We did observe a greater mean difference in the EMR obtained weight and self-report at the six-month survey 4.6 pounds, which became even more pronounced (9.4 pounds) for veterans who reported losing weight in the program. One explanation for that may be there was a much greater time interval between the EMR obtained value and the self-report value at six-months (39 days) compared to the baseline (7.4 days) which may worsen recall bias. However, if it was purely due to recall bias we would anticipate both under and over reporting of weight which would not lead to systematic underreporting that we observed in this study. Approximately half of the patients in this study were working on weight loss so the underreporting of weight at six months, particularly for those who reported losing weight, may in fact reflect patients’ tendency to over-estimate their weight loss.

We believe that this study shows that EMR obtained weight values are at least as reliable, and accurate, as those obtained by patient selfreport. Future population based studies and evaluations may choose to obtain EMR data for clinical variables such as weight to increase efficiency and save data collection costs. However, self-reported weight values may not be reliable for interpreting the outcomes of weight management programs because patients tend to over-report weight loss. EMR weight values may be used in lieu of self-reported weights to inform population-level evaluation and they may be a better proxy for interpretation of weight management interventions.

All authors were federal employees at the time of this work, and none of them have any conflict of interest.

National Center for Health Promotion and Disease Prevention, Veterans Health Affairs.

This retrospective evaluation was deemed exempt from IRB Review by the Durham VA IRB because it was classified as a quality improvement project.

Body Weight-C23.888.144; Electronic Health Records-E05.318.308.