Identifying Critical Issues in the Perception of Caregivers about Quality of Child Health Care Services Using a Modified Delphi Technique
Received Date: Feb 27, 2018 / Accepted Date: Mar 12, 2018 / Published Date: Mar 15, 2018
Background: Identification of the elements of the perception users of health services is a critical step in the development of measurement scales to assess such perception. This study is the second of a three phase approach to create a validated tool for the assessment of caregiver’s perceived quality of child health care in Primary health care (PHC) facilities. The first phase was exploratory using qualitative methods to assess the perceptions of caregivers about quality of child health care services; it yielded 39 quality-of-care items.
Method: In this current study, we subjected the 39 quality of care items to expert review using the modified Delphi method. This method is useful for providing evidence for content validity in quantitative scale design. A total of 33 experts, including public health specialists, pediatricians and medical sociologists, participated in three iterative rounds. In each round the experts awarded score to the items and consensus amongst experts was determined by calculating inter quartile range of the assigned scores.
Results: At the end of the three rounds of expert review, the total number of question items that had shown consensus considerably amongst the expert reviewers was 23.
Conclusion: For these 23 items, a three point Likert scale response format was framed appropriately as a quantitative scale for assessing perceived quality of child health services in PHC facilities.
Keywords: Delphi technique; Perceived quality of care; Child healthcare; Infant care
Quality of care is an important determinant of health services utilization ultimately influencing health outcomes of public health importance . It is known that the perception of the users about the quality of service offered in a health facility is a determinant of choice of provider and willingness to use or pay for the services . There are gaps in the literature addressing assessment of perceived quality of child health care in Africa. For instance, studies on quality of care in Nigeria have mostly not reported the processes of validation of the tools used as such there are varying degrees to which their reports are reproducible [3-10]. Also, review of African health literature yielded only one locally developed quantitative tool for assessing users’ perspectives of quality of care in health facilities that had been subjected to systematic, quantitative content validation . The tool was however not specifically for child health care services.
We used a three phase approach to create a validated tool for the assessment of caregiver’s perceived quality of child health care. In the first phase we conducted an explorative study using focus group discussions (FGD) sessions to collect qualitative data on perception about the quality of child health care services in PHC facilities. Participants were men and women selected from some communities in South West Nigeria. Analytical framework and codes that focused on identifying recurrent, dominant and divergent opinions were developed. The exchanges in the FGD sessions relating to perception of quality of child health care were organized into 39 quality related items. The 39 items fell into three major areas corresponding to: Perception about health workers; perception about the health services; and perception about the health facility. The report of this first phase is documented elsewhere.
In the second phase we used a modified Delphi technique to identify the critical issues amongst the 39 quality of care related items earlier identified based on expert opinion. The Delphi method is a structured communication technique, originally developed as a systematic, interactive forecasting method which relies on a panel of experts . It is based on the principle that forecasts (or decisions) from a structured group of individuals are more accurate than those from unstructured groups [13,14]. Typically the panelists are requested to contribute their opinions on the subject of interest in a listing fashion or otherwise provide answers to a questionnaire. They are also required to score the opinion based on their priority to the issue of interest. Descriptive statistics are used to analyze scores by calculating means, medians, and standard deviations for each item data for each iteration. Consensus on opinions are determined by computing the inter-quartile range (middle 50% of response) for each of the identified items or priority statement .
Though originally formulated by military strategists, it has been widely used in such fields as public policy, education and health usually in an adapted form for instance: the modified Delphi method . While the original version starts with the expert panel contributing items, modified method usually starts with the researcher developing a list of items from literature review or analysis of qualitative data on the subject of interest. The panelists are then requested to rank the items during several iterations. The method is useful for providing evidence for content validity in quantitative scale design , Expert opinions provide strong basis for including or excluding items in the scale. The third phase of the study tested the scale amongst caregivers of under-five children in selected PHC facilities. This paper reports findings of the second phase.
This study was carried out between October and November 2014. The study population consisted of public health physicians, anthropologist, sociologists and pediatricians practicing in Nigeria. Some of the experts were engaged by peer recommendation and others through their association mailing list, in particular, Association of Public Health Physicians of Nigeria (APHPN). Thirty-three experts practicing in 11 states of the Federal Republic of Nigeria responded to the survey. Of the thirty-three, 27 had post graduate qualification in Public Health, two in Pediatrics, two in Sociology and two others were medically qualified public health practitioners.
An online survey platform was used via a commercial online survey website: www.createsurvey.com. The online method was adopted because it allowed for reaching a significantly large number of potential respondents without requiring their physical presence. The 39 ‘quality’ items that were identified from the FGD were modified to be read as questions and uploaded on the platform. Each expert respondent received an email which contained a link to the online questionnaire. The experts were requested to rate each item on the questionnaire on the appropriateness of the question for soliciting users’ perception of quality of child health care services from caregivers. It was five-point rating scale where 1 represented ‘not appropriate’ and 5 represented ‘very appropriate’.
The expert review was done in three iterative rounds. In each round the experts rated the questionnaire items, gave suggestions for rephrasing any of the questions where they deemed fit and gave suggestion of any other item/question that they believed should be included in the list. The inter-quartile range (IQR) of the scores for each question item from each round were calculated. For each question item, the lower the IQR the higher consensus amongst the reviewers. This means that the items with lower IQR were the more usefulness items for assessing perception of quality of care based on expert consensus. After each of rounds, the questionnaire was modified to include comments and suggestions of reviewers in previous rounds.
The 1st round of the modified Delphi process started with 39 quality related question items derived from FGD sessions. The responses of the reviewers showed that about only 41% of the question items had IQR greater than 1.5 (Tables 1-3).
|In your opinion, the health workers in this health facility are friendly and accommodating||4.45||5||5||0.79||1|
|The health workers in this health facility explain the prescription to caregiver||4.36||5||5||0.93||1|
|The health workers in this health facility give appropriate health education||4.33||4||5||0.82||1|
|The health workers in this health facility explain the side effect of drugs/immunization||4.3||5||5||0.95||1|
|The health worker in this health facility spends adequate time with the patients||4.27||4||4||0.8||1|
|In your opinion, the health workers in this health facility are diligent with their work||4.06||4||5||1.09||1|
|The health workers in this health facility refer your child to the next appropriate hospital only as necessary||3.88||4||4||1.05||1|
|The health workers in this health facility show favoritism||3.45||4||4||1.12||1.5|
|The health workers allow for confidentiality and privacy with their patients||3.97||4||5||1.13||2.00*|
|The number of health workers available in this health facility are sufficient||3.82||4||4||1.16||2.00*|
|In your opinion, the health workers in this health facility absent themselves from work||3.03||3||4||1.29||2.00*|
|The health workers are involved dubious and corrupt practices with the drugs and other resources of the health facility||2.85||3||2||1.28||2.00*|
|The health workers request in this health facility appropriate lab investigation before treatment||3.76||4||4||1.23||2.50*|
|In your opinion, the health workers in this health facility are qualified and competent||3.73||4||5||1.31||2.50*|
|The health workers in this health facility arrive at work late||3.55||4||4||1.3||3.00*|
|SD: Standard Deviation; IQR: Interquartile Range; *Items with IQR>1.5|
Table 1: Round one items and summary statistics (health worker related items).
|The patients wait for long time before they are attended to in this health facility||4.42||5||5||0.94||1|
|The drugs given in this health facility are effective||4.27||5||5||1.01||1|
|The cost of the services in this health facility are high||4.24||5||5||1.03||1|
|Drugs and other medical supplies are sufficiently available in this health Facility||4.12||4||5||1.03||1|
|The services of this health facility are properly organized||4.12||5||5||1.14||1|
|The drugs given at this health facility are of good quality||4.06||5||5||1.25||1.5|
|The health service has sufficient immunization service||4||5||5||1.32||1.5|
|The services in some health facilities are preferable to others in this community||3.82||4||4||1.16||2.00*|
|In your opinion, this health facility has adequate equipment for care of the patient||3.76||4||5||1.25||2.50*|
|SD: Standard Deviation; IQR: Interquartile Range; *Items with IQR>1.5|
Table 2: Round one items and summary statistics (health services related items).
|In your opinion, health facility is clean and hygienic||4.24||5||5||1.03||1|
|The health facility has adequate water supply||4.12||4||5||1.17||1|
|This health facility is near enough to where you live||4.06||4||5||1.09||1|
|This health facility has good access road||4.03||4||4||1.02||1|
|In your opinion, the surrounding of this health facility is clean neat and free from bushes||4.09||4||5||1.13||1.5|
|This health facility has sufficient chairs for patients||4||4||5||1.15||1.5|
|This health facility has regular supply of electricity||3.88||4||5||1.27||1.5|
|In your opinion, health facility has adequate toilet facilities||3.88||4||5||1.36||1.5|
|The windows and doors of health facility is well protected by mosquito net||3.88||4||5||1.27||2.00*|
|The health facility is properly fenced||3.76||4||4||1.25||2.00*|
|This health facility has adequate physical structure in terms of location, size and beauty.||3.61||4||4||1.22||2.00*|
|The number of health facilities in this community are adequate||3.45||4||4||1.28||2.00*|
|This health facility has sufficient beds for patients||3.7||4||4||1.26||2.50*|
|Your children recover from their illness when they are treated in this health facility||3.85||4||5||1.25||2.00*|
|There is evidence of government monitoring and over sight of this health facility||3.52||4||4||1.3||3.00*|
|SD: Standard Deviation; IQR: Interquartile Range; *Items with IQR>1.5|
Table 3: Round one items and summary statistics (Health facility related and unclassified items).
Based on some of the comments of the reviewers, three question items were merged into one because of their similarity. These are: “The drugs given in this health facility are effective”; “The drugs given in this health facility are of good quality”; and “Your children recover from their illness when they are treated in this health facility”. These were merged into the following single question item: “Children recover from their illnesses when they use the drugs provided/prescribed in this health facility”.
Of the remaining items, three that had IQR greater than 1.5 were completely excluded from the subsequent round. These are: “The number of health facilities in this community is adequate” [IQR=2.0]; “This health facility has sufficient beds for patients” [IQR=2.5]; and “In your opinion, this health facility has adequate equipment for care of the patients” [IQR=2.5]. Furthermore, the reviewers suggested the addition of three new question items as follows; “The health workers do follow up after treating children in this health facility”; “You are likely to use the child health services of this health facility again or recommend it to others (friends, relatives etc)”; and “Health workers allow for participation of users in monitoring and improving the quality of child health services in the health facility”. In all, round one started with 39 items, three items were merged into one, three items were excluded and three new items were added. Thus, thirty-seven question items went into the second round. Some of the reviewers’ comments from round one are:
“There are some uncomfortable questions for the clients especially if the study is facility based. The clients may not be entirely truthful in answering some questions bothering on the health workers’ attitude especially on issues like favoritism, lateness, and corrupt practices”.
“The question to ask is: is the patient in a good position to be knowledgeable about the issue he/she has been asked to comment on? In many of the questions, the client is not likely to have good knowledge of the issue. e.g. drugs quality, equipment availability etc”.
“Some of the questions assume that the users will have the competency to ascertain the claims e.g. clients may not be able to determine whether health workers are competent or not. They may also not be able to determine whether the lab request are appropriate or not”.
“There is a need to be more specific with some of these questions and avoid having two or more ideas in one question”.
What is the meaning of “sufficiently available”, I think asking the patient whether drugs prescribed are usually available in the health facility is more appropriate; “sufficient immunization service” is also an ambiguous question, “are the immunization needed for the child always available” is more appropriate.
In the 2nd round, higher consensus was achieved as 87% of the question items had IQR lower than 1.5 (Table 4). Following this high level of consensus on several of the items, the cut off point for good consensus was reduced to IQR to 1. There were nine items with IQR greater than 1 and only these were presented for the third round. Furthermore, amongst the items with IQR less than or equal 1, specific comments from the reviewers indicated that a number of them were redundant or not suitable for the targeted respondents to be able to assess and give appropriate responses (Table 5). Based on the comments of the reviewers, these items were excluded from the final tool. Thus after analysis of responses to the second round, 21 items were considered to already have sufficient consensus and needed no further deliberations upon, 7 items were completely excluded and 9 items were introduced into the third round. Specific comments from the reviewer in the second round include the following:
|The health workers in this health facility explain the prescription to caregivers||4.64||5||5||0.74||0.5|
|The health workers are present anytime children are brought to the health facility to receive treatment||4.61||5||5||0.7||1|
|The health worker in this health facility spends adequate time during consultation or when attending to the patients||4.55||5||5||0.71||1|
|The health workers in this health facility refer children to the next appropriate hospital only when unable to handle the case||4.3||5||5||0.88||1|
|The health workers allow for confidentiality and privacy with their patients||4.27||5||5||1.04||1|
|In your opinion, the health workers in this health facility are diligent with their work||4.36||4||4||0.65||1|
|The health workers in this health facility explain the side effect of drugs/immunization||4.58||5||5||0.83||1|
|in your opinion, the health workers in this health facility are friendly and accommodating||4.58||5||5||0.71||1|
|The health workers in this health facility give appropriate health education||4.52||5||5||0.71||1|
|In your opinion, the health workers in this health facility are competent in their work||4.06||4||4||0.93||1|
|The number of health workers available in this health facility are sufficient||4||4||5||1.12||1.50*|
|The health workers in this facility give preferential treatment to some patients over others||3.88||4||4||1.24||1.50*|
|The health workers in this facility suggest laboratory investigation before treatment.||3.73||4||4||1.04||1.50*|
|The health workers misuse/misappropriate drugs and other resources of the health facility||3||3||2||1.35||2.00*|
|The health workers in this health facility start work late in the mornings||3.88||4||5||1.22||2.00*|
|SD: Standard Deviation; IQR: Interquartile Range; *Items with IQR>1.0|
Table 4: Round two items and summary statistics (health worker related items).
|The cost of the services in this health facility is high||4.45||5||5||0.71||1|
|Drugs and other medical supplies prescribed in the health facility are readily available for purchase in the health facility||4.55||5||5||0.71||1|
|The health service has satisfactory immunization services||4.33||4||5||0.82||1|
|In your opinion patients receive attention promptly in this health facility||4.39||5||5||0.83||1|
|The services of this health facility are properly organized||4.3||5||5||0.98||1|
|Services provided in this health facility for the care of children are preferable to services in other health facility in this community||4.15||4||5||0.91||1|
|SD: Standard Deviation; IQR: Interquartile Range|
Table 5: Round two items and summary statistics (health services related item).
“Health workers are competent in their work and diligent in their work may mean different things but to the patients they may not be able to differentiate the questions”
“What makes a health education appropriate? Is it when it addresses my issue or not? Adequate time during consultation looks subjective to me, why not takes time to attend to patient's problems? What about questions that addresses empathy? What about answers to questions asked by patients? Do they allow them to ask questions?”
“The health service has satisfactory immunization services'- I would have preferred that what is meant by satisfactory is explained if possible- like if the vaccines are available or the staff nice, etc?”
“What do you mean by satisfactory immunization service please explain and possibly rephrase. The issue of service being ‘properly organized’ and ‘preferable to services in other facilities’ need to be rephrased in such a way that the caregiver can give their opinion without prejudice”.
“The follow up after treatment is good but it is important to consider possibility of the home visit as part of health workers' activities. Also consider: Direct community service in form of clearing the surrounding bushes and protecting facility properties against robbers”.
In the third round two approaches were used to seek consensus amongst the reviewers. Of the nine items in round three, six were displayed in the five-point scale format as in the previous rounds. The remaining three were put up for a vote of whether to exclude or include them. The reason stems from comments of some reviewers suggesting that these three items should be removed while others suggesting they should be retained. All the six items displayed in the five-point scale format had IQR more than 1.5 (Table 6). They were all excluded from the final set of items. Of the three items that were voted upon two were voted to be retained but after deliberations amongst the authors, only one of them was retained. Also one of the reviewers suggested a new item which had not featured all along during the Modified Delphi process. Considering the value of the question it was added to the final tool. The question item is: “In your opinion, the health workers in this health facility take time to explain to the caregivers about the illness of their child/ward”.
|Health workers allow for involvement of the community in monitoring and improving the quality of child health services delivered in this health facility||3.39||3||3||1.3||2.5*|
|The health facility is properly fenced||3.24||3||4||1.09||2.0*|
|The physical structure of this health facility is beautiful to see||3.3||4||4||1.1||2.0*|
|The health workers in this facility advise/request laboratory investigation before treatment.||3.45||4||4||1.2||2.0*|
|The health workers in this facility give preferential treatment to some patients over others||3.64||4||4||1.27||2.0*|
|The number of health workers available to attend to users when they visit this health facility are sufficient||3.7||4||4||1.26||2.5*|
|SD: Standard Deviation; IQR: Interquartile Range; *Items with IQR>1.5|
Table 6: Round three items and summary statistics.
At the end of the three rounds of expert review therefore, the total number of question items that had shown consensus considerably amongst the expert reviewers were 23 question items.
For these 23 items, a three point Likert scale response format was framed appropriately as a quantitative scale for assessing perceived quality of child health services in PHC facilities.
The tool development which started with 39 quality related items ended with 23 in the third phase. The 23 items fitted into the three broad categories of perception on health workers, health services and the health facilities. It is important that the number of question in surveys is limited to the barest minimum without compromising on the internal structure and reliability of the tool during the development process. Scales composed of large numbers of items that are irrelevant to the experiences of a specific patient are more susceptible to receiving a biased rating . Furthermore, the shorter the tool the more cooperative the respondents are likely to be and the more likely that correct responses are provided. This is even more important if such tool will be administered as an exit interview in a busy clinic setting. Although the description of the specific expressions given to the items of the new tool were different, they did overlap with the items of the pre-existing tool on perceived quality of care in primary health care facilities created by . The item of the new scale however specifically focused on child health care. Table 4 contains the 23 final items.
Modified Delphi technique is a very useful method of seeking expert opinion through consensus on technical issues. The method helped to create a 23-item tool for assessment of user’s perception of child health care services in PHC facilities in Nigeria from an initial list of 39. Methods such as this help to provide evidence for validity of content when creating quantitative scales from the scratch.
- WHO (2006) Quality of care: A process for making strategic choices in health systems: World Health Organization, Geneva.
- Akin JS, Hutchinson P (1999) Health-care facility choice and the phenomenon of bypassing. Health Policy Plan 14: 135-151.
- Afolabi MO, Erhun WO (2005) Patients' response to waiting time in an out-patient pharmacy in Nigeria. Tropical J Pharmace Res 2: 207-214.
- Asekun-olarinmoye EO, Bamidele JO, Egbewale BE, Asekun-Olarinmoye IO, Ojofeitimi EO (2009) Consumer assessment of perceived quality of antenatal care services in a tertiary health care institution in osun state, nigeria. J Turkish-German Gynecolo assoc.
- Chukwuani CM, Olugboji A, Akuto EE, Odebunmi A, Ezeilo E, et al. (2006) A baseline survey of the primary healthcare system in south eastern Nigeria. Health Policy 77: 182-201.
- Esimai OA, Omoniyi-Esan GO (2009) Wait time and service satisfaction at Antenatal Clinic, Obafemi Awolowo University Ile-Ife. East Afr J Public Health 6: 309-311.
- Eze C, Okaro A (2006) Survey of patient satisfaction with obstetric ultrasound service at university of Nigeria teaching hospital, Enugu, Nigeria. Nigerian J Health Bio Sci 5: 93-97.
- Ofili A, Ofovwe C (2005) Patients' assessment of efficiency of services at a teaching hospital in a developing country. Ann African Med 4: 150-153.
- Rasak OB (2013) Patrons’ perception of quality of healthcare services in primary healthcare centres (PHCs) in Oyo state, Nigeria. Developing Country Studies 3: 75-83.
- Uzochukwu B, Onwujekwe O, Akpala C (2004) Community satisfaction with the quality of maternal and child health services in southeast Nigeria. East Afr Med J 81: 293-299.
- Haddad S, Fournier P, Potvin L (1998) Measuring lay people's perceptions of the quality of primary health care services in developing countries. Validation of a 20-item scale. Int J Qual Health Care 10: 93-104.
- Linstone HA, Turoff M (1976) The Delphi method: Techniques and applications. Addison-Wesley.
- Armstrong JS (2001) Principles of forecasting: A handbook for researchers and practitioners.
- Wicklein RC (1993) Identifying critical issues and problems in technology education using a modified-Delphi technique. J Technolo Educ 5: 54-71.
- Cook DA, Beckman TJ (2006) Current concepts in validity and reliability for psychometric instruments: Theory and application. Am J Med 119: 166-167.
- Atkinson MJ, Sinha A, Hass SL, Colman SS, Kumar RN, et al. (2004) Validation of a general measure of treatment satisfaction, the treatment satisfaction questionnaire for medication (TSQM), using a national panel study of chronic disease. Health Qual Life Outcomes 2: 12.
Citation: Arije OO, Omotosho OS, Alabi OT (2018) Identifying Critical Issues in the Perception of Caregivers about Quality of Child Health Care Services Using a Modified Delphi Technique . J Community Med Health Educ 8: 597. Doi: 10.4172/2161-0711.1000597
Copyright: © 2018 Arije OO, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Select your language of interest to view the total content in your interested language
Share This Article
Internatioanl Conference on Community Nursing
September 17-18, 2018 Cape Town, South Africa
International Public Health and Community Nursing Conference
September 19-20, 2018 Singapore City, Singapore
International Conference on Nursing & Healthcare
Oct 31-Nov 01, 2018 New York, USA
27th World Nursing Education Conference
November 12-14, 2018 Frankfurt, Germany
- Total views: 337
- [From(publication date): 0-2018 - Jun 23, 2018]
- Breakdown by view type
- HTML page views: 312
- PDF downloads: 25