Associate Professor of Clinical Medicine, Columbia University, Director of Clinical Cardiovascular Research, Center for Interventional Vascular Therapy, New York, USA
Received date: June 27, 2012; Accepted date: August 14, 2012; Published date: August 18, 2012
Citation: Foster CS (2012) An Analysis of Volunteer Age and Level of Education on Performance in ‘Readability’ Testing for Patient Information Leaflets (PILs). J Health Med Inform 4:115. doi: 10.4172/2157-7420.1000115
Copyright: © 2012 Foster CS. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Visit for more related articles at Journal of Health & Medical Informatics
The success and safety of a medicine depends on the clarity and completeness of information given with it. Since July 2005 it has been a requirement in the EU that PILs ‘shall reflect the results of consultations with target patient groups to ensure that it is legible, clear and easy to use’. The ‘User Test’ or Readability Test has been the method of choice for most pharmaceutical companies to fulfil this requirement.
Readability Testing; PILs; Age; Level of education; Statistics; Pooled data
The most common intervention in healthcare is the recommendation or otherwise prescribing of a medicine. The success and safety of a medicine depends on the clarity and completeness of information given with it . This information would be expected to be given verbally in the first instance by the healthcare professional recommending or prescribing it. The level of retention of information presented verbally to patients has been shown to be poor [2,3]. When written information is provided in addition to verbal information retention has been shown to improve [2-4]. This is the basis of the legislation requiring that Patient Information Leaflets (PILs) be supplied by law with all medicinal products marketed in the EU. This requirement has been in force since 1999 . The PIL may also act as a reference source to answer further questions from the patient that may emerge later in the treatment cycle. Furthermore, the empowerment of patients and a shift towards deregulation of medicines legal status from Prescription only Medicines (PoM) to Pharmacy only medicines (P) and from P medicines to General Sales List Medicines (GSL) means that self selection of medicines with no consultation is increasing. To ensure that medicines are taken safely and effectively high quality patient information including the PIL is becoming increasingly important .
Studies have shown that patients often do not read the PIL provided with a medicine or do not read it fully before beginning treatment for the first time or subsequently . The reason for this may be that some groups have difficulty in understanding information because of technical language and medical ‘jargon’ sometimes used. Since July 2005 it has been a requirement in the EU that PILs ‘shall reflect the results of consultations with target patient groups to ensure that it is legible, clear and easy to use’ . The most common way that pharmaceutical companies have chosen to comply with the legislation to date has been using the Readability Test or ‘User Test’. Guidance on this type of test has been given by the MHRA and EMEA . This method of testing relies on volunteers from the general public answering questions in a structured interview. Volunteerease of finding information and understanding are scored. The questions asked are based upon the critical points for safety and efficacy for the medicine under study. The method was originally pioneered by Sless in Australia and has been used there for a number of years .
Volunteer characteristics such as age, level of education and sex may be considered to be important in determining performance in a Readability Test. A study in the US found that volunteers with low literacy levels had a corresponding increase in propensity to misinterpret medicine label information . With this in mind the selection of volunteers is paramount in this type of Readability Test. The aim of the test is to ensure that all sections of society can read and understand the information presented in the PIL. In order for us to produce such PILs we need to identify systematically the groups of people most likely to have difficulty in understanding such information. From here we should be better placed to select volunteers for Readability Tests that ensure all or most groups can understand. The inclusion criteria of volunteers for many Readability Tests may have been poorly controlled without consideration to what affect this may have on Readability Test results. This article aims to statistically examine which characteristics which are most important in determining how well a volunteer performs in a Readability Test. The information gained should make it possible to identify the volunteer groups more likely to have difficulty. Based upon this a structured approach to volunteer selection should be possible.
Data from 25 Readability Tests was pooled and the data analysed statistically. The test method used was based on that described in the EU and MHRA Guidance on readability testing . Analysis of Variance (ANOVA) was performed using GraphPad InStat v.3.05 (GraphPad Software Inc, San Diego, California, USA).
Revision of the PILs used in readability testing
The PILs were revised to comply with the QRD template guidance issued by the EMEA where appropriate. The leaflet was checked against guidance issued by the EU on Readability [12,13] and the MHRA 2005 publication “Always read the leaflet” .
The content of the PIL was carefully examined by a pharmacist to determine the most critical points of information for the safe and effective use of each medicine. These issues included the indication of the medicine, dose and any significant side effects or warnings. From these 12 to 15 key points a questionnaire was developed. A mixture of factual and action question types were used. Factual questions were judged to be whether certain key facts can be found and understood. Actions questions are based around the appropriate action to take in an outlined set of circumstances. The questions for each questionnaire were set in an order that did not follow the sequence of the leaflet content. The questionnaire and PIL were then pilot tested in two volunteers to identify ambiguous questions. If necessary, minor amendments were made to the questionnaire at this stage. No further amendments were allowed afterwards.
Recruitment of volunteers
At the start of each interview, each volunteer was given an explanation of the purpose of the study, before giving their consent to participate. Each volunteer was paid £20 for travelling and other expenses. Before the start of the Readability Test volunteers were recruited who were members of the general public who had never used the medicine before, but could potentially have the indicated condition personally or be carers of such people. Twenty volunteers were recruited for Readability Tests fulfilling the criteria as follows. At least 5 volunteers of each gender, at least one volunteers, and no more than 5 people, in each of the following age groups; up to 30 years, 30s, 40s, 50s, 60s and 70s-plus, no more than 5 higher education graduates, able to read and having reading glasses with them (as required) and speaking English to native standard. The following volunteers were excluded from the study. Volunteers currently taking the medicine for the leaflet being tested or those who had taken it in the previous 6 months, those who had taken part in another patient information Readability Test in the past 6 months, current healthcare professionals (doctors, pharmacists, nurses and dentists) and others who routinely work with medicines information and people who are unable to engage with the interviewer whether due to cognitive impairment or due to intoxication.
Interview procedure and scoring of participant responses
Volunteers were interviewed individually. Using the PIL supplied, volunteers were asked to find the information relevant to each question and then to describe it in their own words. For each question, the interviewer noted whether the volunteer had any difficulty finding each piece of information. In this study the number of volunteers who had difficulty in finding information was recorded. EU requirements for readability testing require 90% of volunteers to find each point of information in a leaflet and this corresponds with the term ‘information not found’ in this article. Of these, 90% should be able to show that they understand the information and this corresponds with the term ‘information not understood’ in this article. This gives an overall pass rate of a minimum of 80% of volunteers being able to find and understand each point of information. The term ‘information found with difficulty’ in this article is over and above the requirements of the EMEA and MHRA guidance, it is used as a more sensitive indicator of participants ease or difficulty in finding information for any given PIL. ‘Information found with difficulty’ is defined as requiring more than 2 mins or more than 2 permitted prompts from the interviewer in order to find the relevant information.
In Figure 1 the graph shows that volunteers who studied to GCSE level (or equivalent) or less (1) experienced more difficulty in finding and understanding information as age increased. This increase in difficulty was progressive with age and most marked for the 70s and 80s+ age groups where a statistically significant difference (P < 0.05) occurred compared with volunteers aged < 30 with the same educational background (see Figure 2). Further, there was a tendency for volunteers to have more severe difficulty in finding and understanding information as age increased. Although, it is interesting to note that the same clear pattern of difficulty did not exist with increasing age for either of the groups with higher educational attainment. Those who either studied “A” levels or equivalent (2) or studied at degree level or equivalent (3) experienced a lower more constant level of difficulty even as age increased.
Figure 2: Statistical significance of the increasing difficulty experienced by volunteers as age increased. Scores were compared to the <30 age group within each level of education using Dunns test (P<0.05). Key; * 1 = Studied GCSEs (or equivalent) or left school at 16 years or less; 2 = Studied “A” levels or equivalent; 3 = Studied at degree level (or equivalent) or above. ** Difficulty is defined as required >2min or more than 2 prompts from interviewer to answer find information. S = Statistically significantly different to <30 age group, N = Not statistically significantly different to <30 age group.
The graph also shows that for most age groups a difference in scores occurred between volunteers with different levels of education. In most cases this was statistically significant (P < 0.05) as shown in Figure 3 Volunteers who studied to GCSE level (or equivalent) or left school at 16 years or less (1) scored worse than those who studied at degree level or equivalent (3). This indicated that volunteers on average perform better in Readability Tests when they are more highly educated. The graph also shows that to a large degree age was negated in the two more highly educated groups (2 and 3). However, in the two more highly educated groups a shift was observed towards more volunteers not finding or not understanding information, rather than simply experiencing a minor level of difficulty.
Figure 3: Statistical significance of the increasing difficulty experienced by volunteers as level of education decreased. Scores were compared at the corresponding age groups at each level of education using Dunns test (P<0.05). 1 = Studied GCSEs (or equivalent) or left school at 16 years or less; 2 = Studied “A” levels or equivalent; 3 = Studied at degree level (or equivalent) or above. * Difficulty is defined as required >2min or more than 2 prompts from interviewer to answer find information. S = Statistically significantly different to <30 age group, N = Not statistically significantly different to <30 age group.
Considering that the volunteers experiencing most problems are those with the lowest levels of education and the highest age groups it may be logical to select 20 volunteers over the age of 70 years who studied to GCSE level (or equivalent) or left school at 16 years or below for use in Readability Testing. The studies are small and therefore of low power, if they consisted of more challenging participants it is statistically implied that other groups of volunteers will understand the information presented to them in any given PIL. Furthermore, older volunteers are much more likely to be taking medicines with as many as four in ten of those aged 55 to 64 years taking more than one medicine [14-16]. In contrast, barely one in ten people under 35 year has been found to take more than one medicine [14-16]. The use of prescribed and over-the-counter medicines has been shown to be strongly related to age with increasing use as age increased [14-16].
Readability studies are currently small usually consisting of only 20 volunteers and are therefore of low statistical power. They may be merely measuring random variation between PILs or participants, rather than statistically sound differences in the quality of PILs. To increase the power and reliability of Readability Testing in future they could focus on groups of volunteers who have been shown to have the most difficulty. This would be those over 70 years of age who have been educated to GSCE level (or equivalent) or left school at 16 years or below. This would increase the value and power of these types of studies even with low numbers of volunteers. If these groups find and understand the information then the results in this study support extrapolation to groups who have not been shown to have such difficulty with written information. There may be important exceptions to this rule when volunteers suffer from a rare or complex condition and the medicine in question is unlikely to be used as a first line treatment. Such medicines tend to be complex and often require a level of understanding above that of the naive user.