A Critical Analysis of Consumer Oriented Abdominal Aortic Aneurysm Information on the Internet

Healthcare consumers are increasingly utilizing the Internet as a medium for gathering information to make decisions on their healthcare. The lack of regulation of the Internet leads to inconsistencies in the quality of available information. The aim of this study was to assess the accuracy of Internet based information regarding Abdominal Aortic Aneurysm surgery compared with the accepted guidelines and to provide search strategies for accessing superior quality websites. There was high variability in information quality, with over half of the studied websites giving recommendations not in accordance with current guidelines. This study demonstrated that reliable consumer information regarding AAA treatment does exist on the Internet; however variation in quality between websites make it difficult for the consumer to ensure they are obtaining accurate information. Certain website attributes are correlated with superior information quality and this can be used to guide patients in their Internet based research.


Introduction
Healthcare consumers are increasingly utilizing the Internet as a medium for gathering information to assist in making decisions about their health [1]. The Internet allows expedient access to large volumes of data, making it a convenient resource for those with reduced access to other sources of information [2]. This ease of accessibility lead to initial conceptions of the Internet becoming a valuable tool for improving the knowledge asymmetries between health provider and consumer by providing access to otherwise difficult-to-find information [3]. More recently, the effectiveness of the Internet as a tool for making such decisions has come into question, with multiple studies showing that the lack of regulation on the Internet has curbed the ability of consumers to reliably access trustworthy and current medical information [4][5][6][7]. The increasing prevalence of poorly researched and un-reviewed information has resulted in difficulties for consumers to separate the accurate information from that which is false, misleading or incomplete.
Abdominal Aortic Aneurysm (AAA) is a common and potentially dangerous manifestation of vascular disease. While the disease can in certain circumstances be managed conservatively, surgery is the only definitive method of treatment for AAA. Intervention has traditionally taken the form of an open surgical procedure where aortic reconstruction is performed through a midline laparotomy. This is an invasive procedure requiring a large incision, intensive post-surgical care and a protracted period of convalescence. The alternative method is endovascular repair (EVAR), a well-established technique performed by the endoluminal placement of a stent graft in the aorta. Endovascular repair is less invasive than the open method, with smaller incisions and shorter recovery time and is therefore more popular with consumers [8,9]. Despite its less invasive nature the endoluminal approach does have drawbacks: it has shown to confer no long term survival advantage, requires significant long term surveillance, increased levels of follow up and an increased rate of re-intervention. The endovascular method is also precluded in some patients due to anatomical considerations. These considerations need to be taken into account and balanced with patient preference when determining the ideal treatment course for the patient.
The complexities of AAA management and the contrast between the two current treatment modalities require the patient to be provided with good quality information to be able to understand the indications and rationale for intervention. As consumers are increasingly turning to the Internet to obtain this information, it is important that the quality and consistency of the information available is well known. There is currently minimal data available to help guide consumers as to how to access higher quality AAA surgical websites. The aim of this paper is to subjectively assess the quality of Internet information regarding AAA management compared with the current evidence base and to provide search strategies that can be used to confidently predict superior quality website information.

Materials and Methods
The quality of consumer oriented AAA related Internet information was evaluated by employing an assessment tool devised for determining the quality of General surgical websites [10]. This method, the Composite Score, has been partially validated and has previously been used to evaluate the quality of Internet information regarding Minimally Invasive Parathyroid Surgery [11]. It was transferable for the purposes of this study and used with the authors' permission. The Composite Score (CS) uses 16 different clinical and non-clinical indices ( Table 1) to predict the quality and utility of information on a website to a surgical patient. These indices are scored either 1 or 0, depending on whether that particular criterion was addressed on the website, with higher scores indicative of information quality. Results 270 websites were returned via the search strategy outlined. Of these there were 100 individual websites identified as fitting the inclusion criteria after repeated sites were excluded. These websites were analysed and scored with the CS tool. The average score of the websites via the CS tool was 9.2 (S.D. 3.2) with a range from 1 to 15. 10% of sites had scores above 13, which indicate very good information quality, while 14% of websites had scores below 3, indicating exceptionally poor websites quality. The majority of websites originated from the United States (94), with the remainder divided between Australia (1), UK (4) and Singapore (1). It was found that there were five main types of information publishers: Governmental, Scientific Societies, Private practice/Biotech, Consumer Information and Universities/Hospitals ( Table 2).

Journal of Vascular Medicine & Surgery
Of the 100 individual websites analysed, three sites had their main focus on topics other than that of AAA repair, 17 of the websites studied only referred to one method of AAA repair and a further 10 of the websites did not mention either form of repair. Of websites examined, 25% contained some form of conflict of interest, while 26% contained misleading or inaccurate statements on the efficacy or indications of surgery. Management recommendations or advice not in accordance with current AAA treatment guidelines were found on 56% of websites studied.
The websites were subdivided into groups and website quality was compared depending on whether or not certain characteristics were present on the site. These were characteristics of authorship, date of last update, publishing institution: either commercial (Private practice/ biotech, Commercial consumer information) or non-commercial (governmental, public hospital/university, scientific society), country of origin of the publishing institution (either USA based or otherwise) and the presence of advertising. The mean CS of these subdivided groups were then compared via unpaired Students t-test to determine In order to collate appropriate consumer health websites related to AAA management, website searches were performed on the top three most widely accessed search engines worldwide; Google, Bing and Yahoo [12]. The web browser Internet Explorer version 8.0 was employed, with website research undertaken throughout February 2013. The search engines were searched with three different, generic search terms related to the surgical management of AAAs: 'Abdominal Aortic Aneurysm surgery', 'Abdominal Aortic Aneurysm treatment' and 'Abdominal Aortic Aneurysm repair'. From each search term in each engine, the first 30 un-sponsored results were recorded. Repeated websites, websites not predominantly in English, sites intended for medical professionals, sites requiring payment and sites that were solely clinical trials were excluded from the analysis. Websites that fulfilled the provisions were then systematically assessed via the criteria for the Composite Score (CS). All internal links were included in the scoring process; links to external sites were not. Any ambiguities in how to score specific indices were assessed by an independent reviewer and settled via discussion.
Further website characteristics including the date of last modification of the website, the nature of the publishing institution and country of origin of the website were also noted. The position the website ranked in the search strategies was also recorded. In addition to the objective recording of scores for assessment via the CS, any claims or statements made by the website that were false or misleading regarding the efficacy, side effects or superiority of either form of treatment were noted and recorded.
Mean and standard deviation of the CS values were calculated. Subgroup analysis based on date of publication, authorship, institution, advertising and country of origin was undertaken using unpaired Students t-test Statistical analysis. Spearmans Correlation coefficient was used to assess correlation between CS values and search engine rankings. Statistical analysis was completed with GraphPad Prism v5.03 (GraphPad Software Inc., La Jolla) correlation between the presence of these characteristics and higher website quality.
This analysis returned a significantly higher mean CS in those sites where the authorship of the information was easily identified (11.3) compared to websites where no authorship was listed (8.2) (p < 0.001). Similarly, websites that published the date of last revision had a significantly higher mean CS (11.0) compared with those that did not (7.2) (p < 0.0001) ( Table 3).
The levels of advertising on a particular website were defined as either high (greater than 25% of the website content) or low (less than 25% of the website content). There was found to be a higher mean CS in the reduced advertising group, however this difference did not achieve significance (P = 0.06) ( Table 3).
When analysed by type of publishing institution, 45 websites were grouped as primarily commercial sites, while 55 of the websites were grouped in the non-commercial category. A significantly higher mean CS was seen in the non-commercially published (10.0) websites when compared to commercial sites (8.2) (p < 0.05) (Table 4). Similarly, websites were grouped by country of origin into either United States (94%) or other (6%) and the mean scores compared. No significant difference was observed in mean CS scores between these two groups ( Table 4).
The position of the each of the websites in the ranking of the results (1-30) of each search strategy was noted and Spearman's linear regression analysis was used to compare the position of the website in the search engine ranking with the determined CS value of the website. The 9 different search strategy combinations were analysed to determine whether sites of higher rank in search results offered superior information, as borne out by a statistically significant negative correlation coefficient. This analysis showed no trend in the quality of information compared with search engine ranking; of the search strategies employed 4 returned negative correlation coefficients and 5 positive correlation coefficients. None of these coefficients showed any significant deviation from zero (Table 5).

Discussion
Surgical intervention is the only definitive management of AAA, whether open or endovascular [13]. The choice of management is dependent upon many complex variables, including, among other things, patient and anatomical factors, and a thorough understanding of these complexities is only possible with the provision of unbiased, accurate and up to date information. The needs and expectations of the consumer need to be carefully balanced and current guidelines suggest consultation with a vascular specialist before deciding on a method of surgery [13,14].
The Internet is the fastest growing and most widely used research tool for consumer's to access healthcare related information [7]. Estimates of between 40-60% of the population regularly accesses health information on the web [7,[15][16][17]. These figures represent a substantial increase from previously estimated usage statistics of 16% in 2001 and 32% in 2007 [15]. In addition, the Internet is progressively becoming the first and only resource used for consumers seeking medical information [17]. Approximately one third of those who access health information on the Internet believe that the information obtained actually affects decisions they make about their health [15], highlighting the importance of ensuring that consumers are basing their health care decisions on accurate and reliable internet information. The methods employed by health care consumers to search and appraise websites have been previously investigated and reported [18]. Consumer searches are generally slow; websites are accessed based upon position in the search results and the source of the information presented is rarely verified. Consumers infrequently access a link ranked lower than 10 th on a search page and rarely access websites on the second page of results. The main drawback to information accessed on the Internet is that the quality of the data available, especially healthcare related, remains inconsistent and unreliable [2,7,19]. The unregulated nature of the Internet allows for the dissemination of un-reviewed information that can often be deceptive, misleading or inaccurate, especially if commercial or ideological interests are involved [20]. Consumers who base their healthcare decisions on inaccurate information have been shown to receive inferior care and experience poorer health outcomes [21].
The effectiveness of the Internet as a resource for healthcare

Institution Number of Sites [out of 100]
Governmental 9 Scientific Societies 9 Private practice/Biotech 13

Commercial Consumer Information 32
Universities/Hospitals 37 Table 2: Distribution of websites by institution of publication.

Update Published
Last Update listed -CS 11.0 ± 0.3 p<0.0001 Last Update Not Listed -CS 7.2 ± 0.4    information has generated much debate in the literature as the benefits of accessibility and convenience are contrasted with the lack of consistency and quality. As consumers turn increasingly towards the Internet for information on their healthcare needs, research has started focusing on progressively more specialised areas of health care provision to determine the quality of the information available for patients to make informed decisions. Studies examining Internet based information related to aspects of orthopaedic surgery [22], plastic surgery [5], bariatric surgery [5], parathyroid surgery [11], laparoscopic surgery [23], varicose vein surgery [24], arthritis [25], renal transplant [6] as well as many other subjects have recently been published. These studies have reported markedly homogeneous findings; that consumer targeted information in these medical fields, while freely available and often accurate, is markedly inconsistent and often contains incorrect, incomplete or misleading information.

CS -Composite Score; NS -Not Significant
A study conducted in 1999 confirmed that Internet information relating to vascular conditions, including AAA, was of a similarly poor standard [26]. It was found that found that information quality and consistency was inadequate and clouded by large amounts of irrelevant material. There have been few studies looking at how the quality of this information has developed since this study in spite of significant progression in both the use of the Internet for medical research and in the volumes of data available online since this study was completed. This intervening period has also seen endovascular repair surpassing open repair as the mainstay of AAA treatment; with both remaining realistic options. To avoid consumer confusion in light of these developments, it has become increasingly important to determine the quality of information available on the Internet and provide strategies for consumers to access superior quality websites to avoid.
This study used an objective website assessment technique to confirm that there has been little improvement in consumer oriented AAA information accessible on the Internet since 1999. The lack of consistency in information across the Internet is detrimental to the overall usefulness of the Internet as a research medium for consumers. Over half of the websites examined under the searching protocol provided recommendations not consistent with current guidelines on AAA management. Most frequently these recommendations suggested intervention on all aneurysms regardless of size or suggesting only one method of intervention was appropriate in all cases. Further, one quarter of websites did not provide information on both forms of surgical intervention and 10% neglected to discuss any form of intervention at all. For consumers utilizing the Internet for decision making this is of great concern. The subtleties of treatment planning in AAA management rely on a thorough understanding of the guidelines. Consumers who have accessed information that is inaccurate or incomplete will find increased confusion and skepticism when meeting with providers if there is a discrepancy between recommendations and what they have researched. The majority of these inconsistent recommendations were found on the websites of private sector institutions and those with conflicts of interest, indicating that those institutions with a commercial involvement in AAA management were far more likely to recommend inappropriate interventions.
Unlike the results of some previous studies into consumer oriented healthcare data, not all of the websites assessed were of poor quality. It was possible to access websites that contained good quality, peerreviewed AAA surgery related content with accurate, guideline-based information. Many sites displayed clear, consumer targeted information that explained the rationale behind the different treatment indications and options. These exposed consumers to unbiased information that would have been useful in weighing up the risks and benefits of surgery. The subjective quality of these websites was confirmed with the CS tool as three websites scored 15 out of 16 possible points and 10 websites scored 13 points or above. Unfortunately these good quality websites accounted for only 10% of the websites examined and a much higher percentage scored 1 or 2, reflecting minimal information quality. This means that good quality information becomes lost in the noise of poor quality sites with little useful information or information that is incorrect. It is this lack of consistency in website quality and the difficulties in determining the good from the bad that makes the Internet a sub-optimal research tool for consumers.
A contributing factor to the lack of consistency in quality of information on the Internet is the sheer amount of information available online. A search for 'Abdominal Aortic Aneurysm' on the Google search engine returns over 1.4 million individual results, while our search strategy returned 100 distinct sites targeted at consumers. The unregulated nature of the Internet results in the vast majority of this information being published without any form of review. With prodigious quantities of information to digest, it remains difficult to ensure consumers obtain clear and consistent information from website searches. This study has demonstrated that there are website attributes, easily identifiable to the consumer, with a significant correlation to superior information quality. The publication of the author's details and the presence of a date of publication on the website correlated positively with significantly higher website quality. In contrast, the amount of advertising on the website or the country of origin of the institution producing the website was shown to have no significant effect on the quality of website information. Along with the increased likelihood of providing incorrect recommendations, websites of commercial institutions were noted to have significantly poorer levels of information quality. These results indicate possible strategies that consumers can use to guide them towards information that is likely to be of a higher standard.
The position of the website on a search engine's list of results was shown to have no correlation to information quality. While consumers have been shown to usually pick a site ranked in the first three hits, there was nothing to suggest that these sites were of superior quality. Websites that ranked last in the sample search engine results analysed were likely to be just as useful as those ranked first. This suggests that the algorithms employed by the search engines are not necessarily leading consumers to the best possible information available, even though they may be reassured by a high place in the rankings. Healthcare consumers should be counseled to look for the characteristics mentioned above when deciding on websites in which to obtain good quality information.
There are limitations presented by this study. The CS tool relies heavily on subjective interpretation of website content and this may lead to some variation between observers in regards to scoring. Furthermore while this method has been validated with regards to higher scores relating to improved website quality, there is no absolute validation as to what score equates to a 'good' website and what score equates to a 'bad' website. As such it may be difficult to compare websites across different disciplines or timeframes. Its use as a tool may be better kept to compare website quality in single topic areas. The criteria used to calculate the CS value are discrete, binary interpretations made by a clinician of what is in reality a spectrum of data. This may skew interpretation as information that in considered as available on the site may in reality be either inadequate or overly complex to the consumer.
A further limitation of any study of Internet based information is the difficulties with generalising data taken from a single snapshot. New websites appear daily, while existing websites may be altered, change in rankings or disappear altogether. This dynamic nature of the Internet means that such studies are poorly reproducible.
It is clear consumers are continuing to use the Internet to access information about their health issues. This increasing utilization calls for improved regulation to ensure information is of a sufficient standard to facilitate well-informed decisions. Failing this, consumers need to be informed about how best to use the Internet for their healthcare needs. While this study has shown that some website attributes do correspond with improved information quality, the Internet remains a resource that must be used circumspectly. Providing directions to websites previously vetted by the clinician may be a more effective way of delivering improved information to consumers. In managing abdominal aortic aneurysms, where dialogue between patient and doctor is paramount, consultation with health professionals remains the most important component in healthcare decision making.