Reach Us +447480022765
Impact of A Standardized Recommendation and Electronic Prompts on Follow-Up of Indeterminate Pulmonary Nodules Found on Computed Tomography | OMICS International
ISSN: 2161-105X
Journal of Pulmonary & Respiratory Medicine

Like us on:

Make the best use of Scientific Research and information from our 700+ peer reviewed, Open Access Journals that operates with the help of 50,000+ Editorial Board Members and esteemed reviewers and 1000+ Scientific associations in Medical, Clinical, Pharmaceutical, Engineering, Technology and Management Fields.

Impact of A Standardized Recommendation and Electronic Prompts on Follow-Up of Indeterminate Pulmonary Nodules Found on Computed Tomography

Richard M. Elias1, Anne-Marie G. Sykes2, John M. Knudsen2 and Timothy I. Morgenthaler1,3*

1Department of Internal Medicine, Mayo Clinic, 200 First Street SW, Rochester MN 55905, USA

2Department of Radiology, Mayo Clinic, 200 First Street SW, Rochester MN 55905, USA

3Division of Pulmonary and Critical Care Medicine, Mayo Clinic, 200 First Street SW, Rochester MN 55905, USA

*Corresponding Author:
Dr. Timothy I. Morgenthaler
MD, Division of Pulmonary and Critical Care Medicine
200 First Street SW
Rochester MN 55905, USA
Tel: 507- 264-4348
Fax: 507-266-4372
E-mail: [email protected]

Received date: December 17, 2011; Accepted date: January 09, 2012; Published date: January 11, 2012

Citation: Elias RM, Sykes AG, Knudsen JM, Morgenthaler TI (2012) Impact of A Standardized Recommendation and Electronic Prompts on Follow-Up of Indeterminate Pulmonary Nodules Found on Computed Tomography. J Pulmonar Respirat Med 2:113. doi:10.4172/2161-105X.1000113

Copyright: © 2012 Elias RM, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Visit for more related articles at Journal of Pulmonary & Respiratory Medicine


Purpose:Incidentally discovered pulmonary nodules on computed tomography are common. Executing appropriate follow-up is challenging. The purpose of this study was to assess the impact of a standardized template of follow-up recommendations in radiology interpretation reports and an electronic messaging system on the rate of radiographic follow-up of indeterminate pulmonary nodules identified on computed tomography.

Materials and methods:This retrospective study examined rates of appropriate follow-up, as defined by the Fleischner Society guidelines, of incidental pulmonary nodules over a seven-month period both before (17 patients; mean age 62.7 years) and after (72 patients, mean age 61.6 years) the commencement of the quality improvement initiatives. Further analysis by risk group, patient and nodule characteristics, notification type and location of imaging request were performed.

Results:There was a trend towards improved time-appropriate follow-up from 35.3% (6/17) to 56.9% (49/72) [p=0.18]. The largest change was noted in high-risk patients with an improvement from 11.1% (1/9) appropriate follow-up to 51.4% (18/35) [p=0.06] following the interventions. The largest improvement in on-time follow-up imaging was a reduction in premature imaging, which decreased from 17.6% (3/17) to 6.9% (5/72) [p=0.18]. Rates of on-time follow-up after the interventions were similar irrespective of patient age, nodule size or origin of initial imaging request.

Conclusions: Ensuring use of a rigorous approach to indeterminate pulmonary nodule reporting reporting that includes standardized follow-up recommendations may improve rates of appropriate follow-up.


Pulmonary nodules; Computed tomography; Follow-up; Lung cancer; Quality improvement


Lung cancer is the leading cause of cancer death in both men and women worldwide. In the United States mortality from lung cancer exceeds that of colorectal, breast and prostate cancer combined with greater than 150,000 deaths per year [1]. Significantly, greater than 60% of patients with a cancer diagnosed in Stage IA (T1N0M0) will be alive at five years whereas advanced stage disease carries a less than 5% five year survival rate [2]. It is thus very desirable to diagnose lung cancer at the earliest stages. Often, early diagnosis arises from the incidental discovery of a pulmonary nodule on imaging studies.

Incidental pulmonary nodules are common, particularly in patients with a smoking history, but very small nodules are unlikely to be malignant. Benjamin et al. [3] reviewed all computed tomography (CT) chest studies of patients greater than 18 years of age at their institution over a given period and found that in 9.7% of cases there was a nodule less than 1 cm in diameter for which follow-up was recommended. Henschke et al. [4] enrolled one thousand asymptomatic smokers and ex-smokers for screening CT. Non-calcified nodules were noted in approximately one in four participants. Nodules were ultimately determined to be malignant in 2.7% but only one cancer was <5mm at detection. The Mayo Clinic Lung Cancer Screening Trial reported by Swensen et al. [5] which enrolled a high-risk population of 1520 20+ pack/year smokers aged greater than 50 and imaged them with annual low-dose chest CT scans over a period of four years, found uncalcified pulmonary nodules in a significant majority of participants (74%). Sixty-eight cancers were diagnosed of which approximately 80% were greater than 8 mm in diameter at detection and only three cancers were less than 5 mm at detection [5]. Several other studies have noted the relationship between nodule size and risk of malignancy [6-9]. Overall, the data suggest that the likelihood of a nodule 4 mm or less in diameter representing a lethal cancer is less than 1%, even in smokers, which is markedly different to the 10-20% likelihood for lesions of approximately 8 mm.

The accumulation of these and other data led to the creation of evidence-based guidelines in 2005 for the follow-up of small pulmonary nodules in patients greater than 35 years of age by the Fleischner Society [10]. These guidelines stratified patients into low- and high-risk groups principally on the basis of smoking history and made specific recommendations for follow-up intervals based on nodule size and risk category. These guidelines have been widely recognized and provide a footing for radiologists and clinicians in regards to nodule follow-up.

In clinical practice, follow-up of pulmonary nodules may be suboptimal. The Benjamin et al. [3] study found that in 44% of cases of pulmonary nodules less than 1 cm for which follow-up was recommended, no follow-up CT could be shown to have been undertaken. Even after publication of the Fleischner Guidelines, Eisenberg et al. conducted a study demonstrating that 79% of radiologists were aware of the guidelines and only 59% of radiology practices employed them. No written policy regarding nodule follow-up was present at 37% of the surveyed institutions. Further, management recommendations made by radiologists in three mock clinical scenarios were consistent with the guidelines only 35% to 61% of the time [11].

At Mayo Clinic in Rochester, Minnesota a quality improvement (QI) initiative were undertaken in 2008 in order to address the issue of appropriate radiologic follow-up of pulmonary nodules found incidentally on CT scans of the chest. First, the Radiology Department, in consultation with the Division of Pulmonary Medicine, developed a standardized template of follow-up recommendations based on the Fleischner guidelines (Figure 1) to be inserted into the interpretation report. Secondly, an electronic message flagged as a ‘Semi-Urgent’ finding could be sent to the ordering clinician’s secure in-box (a clinical action list maintained within the electronic medical record) at radiologist discretion. The expectation communicated to the radiologists by e-mail and via the departmental electronic newsletter was that these interventions should be employed for all nodules that could not be considered definitively non-malignant in appearance. We undertook a retrospective review to determine the extent to which these initiatives impacted the rates of appropriate follow-up of incidental pulmonary nodules and to identify factors associated with suboptimal follow-up.

Materials and Methods

This study was reviewed by the Mayo Clinic Institutional Review Board (10-004113) and was deemed not to require full Board review as it is a quality improvement project not constituting research involving human subjects as defined under 45 CFR 46.102. The authors endorse that this research to be in compliance with the Health Insurance Portability and Accountability Act (HIPAA). This study was supported by the National Center for Research Resources, a component of the National Institutes of Health (NIH) [Grant 1 UL1 RR024150-01] and the NIH Roadmap for Medical Research. The Center for Translational Science Activities at Mayo Clinic has NIH funding.

We identified two groups of patients for whom an incidental pulmonary nodule was noted on their first chest CT at our institution, and for which follow-up CT imaging was recommended by the reporting radiologist. The first group underwent their first chest CT scans between April 1 and October 31, 2006, before the initiation of the above described quality-improvement interventions but after publication of the Fleischner Society Guidelines. The second group underwent their first chest CT scans between April 1 and October 31, 2008, after the initiation of the interventions. In order to capture incidental nodules for which follow-up imaging was recommended by the radiologist, reports of all chest CTs undertaken during the above mentioned time periods were searched to identify those which contained the word ‘nodule’ or ‘nodules’ and ‘follow-up’ or ‘repeat’. The above-described strategy resulted in groups of 21 and 85 patients from the 2006 and 2008 periods respectively. We limited our analysis to patients residing within Olmsted County, MN (our primary care area), so that follow-up data might be considered complete. Application of the remainder of the inclusion and exclusion criteria listed in (Table 1) yielded 17 and 72 patients from the 2006 and 2008 periods respectively for analysis.

Inclusion Criteria:
  • Chest computed tomography report contained [‘nodule’ OR ‘nodules’] AND [‘follow-up’ OR ‘repeat’]
  • Age ≥ 35 years
  • Domicile in Olmsted County, MN
Exclusion Criteria:
  • Age < 35 years
  • Domicile outside of Olmsted County, MN
  • Prior history of malignancy other than non-melanoma skin cancer
  • Prior history of lung transplantation
  • Prior cross-sectional imaging revealed pulmonary nodule
  • Detected nodule is outside of the lung
  • Patient died of unrelated causes within the recommended follow-up interval
  • Subsequent CT chest earlier than Fleischner recommendation for a clinical reason other than the pulmonary nodule(s)

Table 1: Inclusion and Exclusion Criteria.

All the CT reports were then reviewed to record nodule size in millimeters and to confirm that follow-up of the nodule was recommended. In patients with multiple nodules the reported diameter of the largest nodule was used. If the report did not explicitly state the nodule size, an experienced thoracic radiologist (A.S.) re-read the images to determine the size. Data was extracted from the patients’ electronic medical records regarding smoking history with anything greater than trivial smoking (defined as less than 1 lifetime pack/year) considered a positive smoking history and resulting in the patient’s designation as high-risk in relation to the Fleischner Society guidelines.

We have defined ‘appropriate follow-up’ as occurring in the month recommended by the guidelines or else one month early or late, thus resulting in a 3-month window. Of note, while the Fleischner guidelines do not recommend repeat imaging for low-risk patients in whom a nodule of 4 mm or less is detected, when our QI initiatives were initially undertaken the decision was made to put in the report template a recommendation for follow-up in 12 months for this group (Figure 1). The rationale for this decision was that our clinicians, having previously been guided by much more follow-up intensive recommendations such as those from Ost et al. [2] and by generally accepted clinical practice, would need a period of transition to the Fleischner Society guidelines prior to being comfortable with the notion of no radiologic follow-up whatsoever for these small nodules. Therefore, for the purposes of this study, patients in this category were deemed to have had appropriate follow-up if they either had no repeat imaging or if they underwent a CT scan at 12 months.


Figure 1: Screenshot of the Standardized Template Inserted Into the Computed Tomography Report..


Groups did not significantly differ at baseline with respect to age, nodule size, or proportion of patients considered at high risk for developing lung cancer (Table 2). Of 72 patients with nodules identified on CT in the post-intervention group, in 17 cases the radiologist neither imprinted the interpretation report with the template nor used the semi-urgent notification. The template was used in 51, and the template plus semi-urgent notification was used in 4 patients. Most scans occurred in the outpatient setting (60/72; 83%). Rates of appropriate follow-up appeared similar whether the scan was initiated in the outpatient, inpatient, or emergency department settings (34/60; 57%, 1/2; 50%, 6/10; 60% respectively).

Age [years]; median (25%-75% quartiles) 67.3
Nodule size [mm]; median (25%-75% quartiles) 6.82
High risk (%)* 52.9 48.6 0.79

Table 2: Baseline Patient Characteristics.

Table 3 summarizes the rates of appropriate follow-up before and after the initiation of the quality improvement measures using the modified Fleischner Society guidelines (Figure 1) as the reference standard. There was a non-significant trend toward overall improvement in time-appropriate follow-up CT following the intervention (35.3% vs. 56.9%; p=0.18). Subgroup analysis demonstrated a trend toward improvement in rates of appropriate follow-up in high-risk (11.1% vs. 51.4%; p=0.06) rather than in low risk patients (62.2% vs. 62.5%; p=1.00). The largest reduction of inappropriately timed scans was in those performed prematurely (17.6% vs. 6.6%; p=0.18). In the postintervention group, rates of appropriate follow-up were very similar irrespective of the notification type, with no notification, imprinting the guidelines alone or with the addition of a semi-urgent notification resulting in rates of 53%, 59% and 50% respectively. (Table 4) provides further subgroup analysis based on patient age and nodule size. The largest improvements in appropriate scanning interval was noted in patients aged 35-49 (0% to 50.0%; p=0.11) and in those with nodules size >4-6mm (0% to 50.0%; p=0.12).

Total appropriate follow-up 6
Patients at high risk with appropriate follow-up 1/9
18/35 (51.4%) 0.06
Patients at low risk† with appropriate follow-up 5/8
23/37 (62.2%) 1.00
Patients scanned ≥ 1 months earlier than recommended 3
Patients scanned ≥ 1 months later than recommended 4
Patients with no follow-up scan 4

Table 3: Follow-Up Rates of Pulmonary Nodules Detected By Computerized Tomography of the Chest by Year.

Patient age (years) 2006 2008 p
35-49 years 0/5 (0%) 8/16 (50.0%) 0.11
50-64 years 2/3 (66.6%) 15/25 (60.0%) 1.00
≥ 65 years                         4/9 (44.4%) 18/31 (58.1%) 0.70
Nodule diameter (mm)      
≤ 4mm 3/6 (50.0%) 20/36 (55.6%) 1.00
> 4-6mm 0/4 (0%) 8/16 (50.0%) 0.12
> 6-8mm None. 5/7 (71.4%) N/A
> 8mm 3/7 (42.9%) 8/13 (61.5%) 0.64

Table 4: Number of Patients with Time-Appropriate Follow-Up by Patient Age and Nodule Diameter.


Electronic reminder systems have been used in a variety of contexts to improve healthcare delivery. As early as 1994, Burack and colleagues demonstrated that a computer generated mammogram reminder form for physicians, in conjunction with other interventions, could significantly increase the rate of mammography [12]. More recently, a study by Feldstein et al. [13] showed that automated phone reminders to patients improved repeat mammography screening among previously screened women. Interventions targeting the reduction in catheter-associated urinary tract infections have found that computer-based urinary catheter ordering in conjunction with computer-generated reminders and prompts significantly decreased duration of catheterization and rates of urinary tract infection [14,15]. Other authors have demonstrated the efficacy of standardized e-mail reminders and electronic alerts on clinical outcomes in a range of situations including heart failure management, intravascular catheterrelated infections and surgical antibiotic prophylaxis [16-18]. To our knowledge, this is the first study to systematically evaluate the role of an electronic reminder system and guideline template provision on improving rates of pulmonary nodule follow-up.

The principal finding of our study is that appropriate followup of pulmonary nodules identified by CT improved following implementation of interventions to standardize and augment communication to the ordering clinician regarding appropriate followup (53% vs. 32%), although this did not reach statistical significance (p=0.17). When risk status is taken into account, the post-intervention trend is more striking, with the rate of appropriate follow-up increasing from 11% to 51% (p=0.06) in high-risk patients. Another notable trend is the reduction in the number of scans performed too early (18% to 7%; p=0.18). Of particular interest from the data are the very low rates of appropriate follow-up in high-risk patients and those aged 35-45 (11% and 0% respectively) in the pre-intervention group suggesting that appropriate re-imaging in these groups is particularly problematic for clinicians.

Although our results suggest a substantial trend towards improved appropriate follow-up, with 21% of patients still receiving no radiological follow-up and 22% receiving it either early or late, clearly there is scope for further improvement. Additionally, few (6%) nodules prompted the radiologist to initiate an electronic semi-urgent notification to the requesting clinician in addition to imprinting the Fleischner-based template into the report. Upon further review, practicing radiologists explained that at our center, they regularly use informal channels such as telephone calls to notify clinicians of an important finding, in lieu of electronic semi-urgent notification although it was not possible to reliably ascertain retrospectively whether or not this had occurred on a case-by-case basis. A substantial proportion of scans which did prompt semi-urgent notification during the study period were either excluded from analysis due to previous history of malignancy (25%) or had close follow-up imaging for a reason other than the nodule (37%), suggesting that radiologists’ determination of whether to initiate a semi-urgent notification was guided by clinical context. Nevertheless, the low rate of semi-urgent notification raises the possibility that incomplete delivery of the intervention contributed to the failure to improve to a significant degree, and represents an opportunity for further improvements in clinical practice. A more robust system enhancement such as automated reminders to the radiologist to send the semi-urgent notification or alerts to clinicians as the time for follow-up imaging approached, driven by computer-assisted decision support algorithms might result in additional improvements. Also of interest was the similar rates of follow-up in the post-intervention group irrespective of the notification type (including no notification) suggesting several possibilities: a) that the quality improvement initiatives had increased awareness amongst clinicians of the Fleischner guidelines generally, b) that with time the guidelines had simply become more widely known, or c) that informal channels of communication between radiologist and clinician were employed.

In the past, we generally followed indeterminate pulmonary nodules with a number of repeat CT scans in the year or two following the discovery of the nodule [2]. This practice was in large measure predicated on two factors. First, the observation, from the era prior to routine use of CT scanning, that a substantial proportion of incidental nodules noted on chest x-ray (CXR) were ultimately determined to be malignant. This does not sufficiently take account of the fact that nodules appreciable on CXR are generally significantly larger than incidental nodules visible by CT scan and therefore carry a higher risk of being malignant once noted. Second, many prior recommendations were largely based on data from lung cancer screening trials and probably do not apply well to incidentally discovered nodules.

The Fleischner society guidelines are generally regarded as the best available evidence-based approach to nodule management. There is very little in the literature addressing the extent to which these guidelines have changed management, but evidence suggests that even amongst radiologists, one in five are unaware of the guidelines and approximately half the recommendations to clinicians are not congruent with the Fleischner society guidelines [11]. Anecdotal evidence from our institution suggests that prior to the institution of this QI initiative there was significant variability of radiologists’ recommendations and uncertainty amongst clinicians. Interestingly, rates of appropriate follow-up appeared improved in the 2008 cohort even for cases where neither Fleischner nor Semi-urgent was used (53%), suggesting that these interventions were successful in disseminating knowledge regarding appropriate follow-up or that the guidelines had become more generally known.

Our study had several limitations. We believe there is a high chance of a beta error. Post-hoc analysis of the data indicates that a sample size of 285 would be required to have 80% power to detect a significant difference between groups. Thus, we were underpowered for even fairly large differences in appropriate follow-up rates to reach statistical significance. The pool of patients for the pre-intervention group was constrained by the relatively narrow time window after the publication of the Fleischner Society guidelines but before the initiation of our quality improvement initiatives and by limiting recruitment to those living in Olmsted County. Additionally, if one assumes that the number of nodules incidentally detected during a six-month period in 2006 and 2008 would be expected to be similar, the smaller group from 2006 suggests that our search strategy did not capture all patients from that period. While this may have introduced bias it also underscores the inconsistent nature of terminology used for reporting these nodules prior to the QI initiative, which may have contributed to clinician uncertainty. Printing the guidelines into the report may have led to more standardized terminology, which has significant implications for the development of future improvements, such as computerized decision support tools that provide automatic reminders, since standardized input is needed for reliable performance. Finally, this study was not designed to detect differences in patient outcomes resulting from closer adherence to the guidelines.

Incidental pulmonary nodules are common and problematic and appropriate follow-up is a significant issue. The Fleischner Society guidelines represent an important step forward in rationalizing the approach to these findings but are inconsistently applied. Ensuring use of a rigorous approach reporting that includes standardized follow-up recommendations may improve rates of appropriate follow-up.


Dr. Elias contributed to the conception and design, data acquisition, analysis and interpretation of data and drafted the submitted article.

Dr. Knudsen contributed to the conception and design, data acquisition, analysis and interpretation of data and revised it critically for important intellectual content and approved the final version.

Dr. Sykes contributed to the conception and design, reviewed CT scans for which no explicit nodule size was reported and revised the manuscript critically for important intellectual content and approved the final version.

Dr. Morgenthaler contributed to the conception and design, data acquisition, analysis and interpretation of data and revised it critically for important intellectual content and approved the final version.

The authors would like to thank Professor David E. Midthun for his thoughtful review of the manuscript, Vicki Schmidt for her assistance in data acquisition from the radiology database and Anna Elias for her assistance in editing the manuscript.


Select your language of interest to view the total content in your interested language
Post your comment

Share This Article

Article Usage

  • Total views: 12023
  • [From(publication date):
    April-2012 - Dec 15, 2019]
  • Breakdown by view type
  • HTML page views : 8234
  • PDF downloads : 3789