Reach Us +441474556909

Quality Assessment Of Family Planning Services Using Direct Observation And Exit Interview In Salt City, Jordan

Abdulhakeem Okour

Department of Public Health and Family Medicine, Faculty of Medicine, Jordan University of Science and Technology, Jordan

Rami Saadeh

Department of Public Health and Family Medicine, Faculty of Medicine, Jordan University of Science and Technology, Jordan

Abeer Abu Rassaa

Ministry of Health, Salt, Jordan

Muhammad Faizal Bin A Ghani

Department of Educational Management, Planning and Policy, Faculty of Education, University of Malay, Jordan

*Corresponding Author:
Rami Saadeh
Department of Public Health and Family Medicine
Faculty of Medicine
Jordan University of Science and Technology, Jordan
Email
: [email protected]

Received Date: Nov 16, 2018; Accepted Date: Mar 22, 2019; Published Date: Mar 28, 2019

 
Visit for more related articles at Quality in Primary Care

Abstract

Background: The quality of care provided in family planning services is usually measured through women’s perceptions of services. Fewer studies compared their perception with direct observation of health personnel, which might lead to inadequate evaluation of the quality of services provided.

Aim: The objective of this study was to compare the level of agreement between direct observations of health personnel while providing family planning services with women’s perceptions of services.

Methods: A cross-sectional of a convenient sample from three health care centers operated by the Jordanian Ministry of Health recruited women at a reproductive age and collected information on family planning sessions and women perception on the exit. There was a family planning session for each participant (n=503). Chi-Square testing and Kappa coefficients for agreement were used in the analysis.

Results: Women mean age was 31.11 years (+ 7.09), 41% of them were new users. In a binary relationship, the provided information on how to use contraceptive method and side effects of their use were significantly related to womens number of children (p=0.024 and 0.004, respectively). Seven out of eight observed counseling items were significantly related to provider’s level of experience. Using Kappa coefficient, the disagreements between direct observation and exit interview were evident in almost all items.

Conclusion: Client interviews should not be used without direct observation as a quality measurement in settings where health care providers and clients are well-known to each other.

Keywords

Delivery of health care; Jordan; Counseling; Contraception; Family planning services; Reproduction.

Background

Quality of Family Planning (FP) services support the organized family planning efforts to meet demands of fertility regulation [1,2]. Many programs have applied activities to improve quality of care in their facilities and invested resources to develop appropriate measurements of quality [1-4]. In FP, six elements of quality were suggested by the Bruce/Jain framework: choice of method, information to the client, technical competence, interpersonal relations, mechanisms to encourage continuity, and constellation of services [1,3,4]. However, there is no single comprehensive method of evaluation that implemented all elements of quality. Yet, the most commonly used approaches are direct observation and exit interviews. Direct observation measures the accuracy and thoroughness of information conveyed during counseling and assesses the technical competence of the provider. By contrast, the exit interview is the only instrument that taps the clients view of the services received. Nonetheless, direct observation may instigate a bias among providers performing better than their regular conditions [1-7]. Exit interviews as well may introduce courtesy bias; women may perceive that they are required to give positive feedback on the services, or they can be reluctant to undermine the services for fear of being neglected from care if providers know about their opinion. Eventually, women may simply give any answer without thinking, because they want to get home [1].

Quality of Family Planning (FP) services has long been recognized as the principal determinant of uptake and continuation of contraceptive methods [8-10]. In Jordan, the unmet need for contraception among women who wish not to have any more children have reached 16%, with current contraceptive users among married women of 61%, and total fertility rate at 3.38 [11]. Although FP services are widely known and provided for free in the premises of Jordan’s Ministry of Health (MOH), the information about the quality of these services is limited. In addition, most studies addressed FP in terms of perceptions of providers and clients [12,13], counseling couples on FP [14], practices and attitudes of providers [15,16], and few if none have involved comparing methods of evaluation. Thus, the objective of this study was to investigate the FP quality by comparing two methods of evaluation that are commonly used: the direct observation of health personnel while providing the service, and the exit-interview of women who received FP.

Methods

Design

This was a cross-sectional study. The outcome was the level of agreement between two methods of evaluating the quality of FP services.

Sample and settings

The study took place in three health care centers owned and operated by the Jordanian MOH in Salt city, located in the middle of Jordan. The target population was all married women at reproductive age; between 15 and 49 years old, who attended the designated health care centers for FP services. The initial sample of women approached was 575, but only 503 (87%) agreed to participate in this study. Family planning services at the MOH health centers are provided as part of the daily regular services. Women present at a room where multiple personnel including nurses, midwifes, and nurse assistants provide different tasks such as registering, measuring vital signs, asking and or interacting with women about FP choices, and providing contraceptives. As well, counseling is performed in the same room. The service provided was based on “first come first served”.

Ethical approvals

The present study was approved by the Institutional Review Board (IRB) of Jordan University of Science and Technology (JUST), and the Jordanian MOH. A verbal consent was obtained from the counseling providers before monitoring their work in the counseling room. Participating women were asked to sign a written consent, which included information about the study purpose and benefits, and an affirmation of the voluntary nature of participation in the study. The participants were assured confidentiality, and their names were not recorded.

Data collection

Data was collected from June to July 2013. The data collection for evaluation of FP services was achieved by two methods. First, the researcher witnessed, observed, and recorded the performance of each session where the health personnel provided the FP service. This was done by using a predetermined protocol that included eight items related to performance of the provider. Using the same items, the women who finished and exited the health care center were interviewed and asked to evaluate the service that they just had, and the researcher recorded their responses.

Analysis

Socio-demographic information was obtained using descriptive statistics. The relationship of women opinion about the FP service with: number of children, level of education, and provider years of experience was examined using Chi-Square test. Levels of agreement between observation data and exit interviews data for each client were obtained by Kappa coefficients. Analysis of the data was performed using the Statistical Package for the Social Sciences (SPSS), Version 20 (IBM Corp., Armonk, New York, USA). In all analyses, a p-value of 0.05 was considered statistically significant.

Results

There were 503 sessions of FP services, in which 503 women participated. The mean age of the women was 31.11 years (+ 7.09), about 87% of them were between 20 and 39 years old, and most of them had higher than high school of education (Table 1). Approximately, 41% of women were getting FP services for the first time, 68.6% have had less than three pregnancies, and about 70% wanted to have more children. Among those who were previously using contraceptives, 33.9% used pills, 31.8% used IUD, and 25% used condoms. Regarding the FP providers, 71% had 2 to 4 years college education, and 97% had 4 or more years of experience. The FP service sessions took an average time of 10 minutes, while the counseling lasted for an average of 3 minutes (the women who received counseling comprised 39.4%).

  N %
Age / years
15-19
20-29
30-39
40-50
0
7
210
227
59
0
1.4
41.8
45.1
11.7
Education
< 12 years
= 12 years
-
60
443
-
11.9
88.1
Family income
< 350 JDs
350-999 JDs
= 1000 JDs
-
204
289
10
-
40.5
57.5
2.0
Number of children
1-3
4-6
7-12
-
388
108
7
-
77.1
21.5
1.4
Working status
Not working
Working
-
293
210
-
58.2
41.8

Table 1: Women’s characteristics.

Table 2 shows womens opinions in relationship to education and number of children. Provider’s explanation of how to use the contraceptive method and side effects were significantly related to women’s number of children (p-value=0.024 and 0.004, respectively).

Women said yes Years of education Number of children
<12 =12 p-value 1-3 4-6 7-12 p-value
Welcoming women in counseling room 56 (11.5%) 432 (88.5%) 0.735 375 (76.8%) 107 (21.9%) 6 (1.2%) 0.071
Maintaining privacy during counseling 56 (11.3%) 440 (88.7%) 0.390 381 (76.8%) 108 (21.8%) 7 (1.4%) 0.548
Provider explained method effectiveness 55 (11.2%) 441 (88.6%) 0.611 384 (77.1%) 107 (21.5%) 7 (1.4%) 0.616
Provider explained method side- effects 55 (11.2%) 438 (88.8%) 0.150 382 (77.5%) 105 (21.3%) 6 (1.2%) 0.004
Provider explained how to use method 56 (11.4%) 436 (88.6%) 0.921 379 (77.0%) 107 (21.7%) 6 (1.2%) 0.024
Provider explained IUD insertion steps 21 (12.8%) 143 (87.2%) 0.490 128 (78.0%) 34 (20.7%) 2 (1.2%) 0.915
Provider explained method time- duration 57 (11.6%) 434 (88.4%) 0.278 378 (77.0%) 106 (21.6%) 7 (1.4%) 0.937
Women received counseling in enough time duration 57 (11.7%) 429 (88.3%) 0.173 375 (77.2%) 104 (21.4%) 7 (1.4%) 0.747

Table 2: Womens opinions of counseling by level of education and number of children.

All other women’s perceptions were not related to education or number of children.

Table 3 demonstrates the observed counseling in relation to providers level of experience, which revealed a significant relationship of almost all items.

Component % Yes /Total 1-4 years of experience 5-10 years of experience >10 years of experience p-value
Welcomed women in counseling room 44.0 5 (2.3%) 100 (45.5%) 115 (52.3%) 0.014
Maintained privacy during counseling 52.4 3 (1.1%) 114 (43.5%) 145 (55.3%) 0.004
Explained method effectiveness 37.8 5 (2.6%) 87 (46.0%) 97 (51.3%) 0.024
Explained method side- effects 20.2 1 (1.0%) 45 (44.6%) 55 (54.5%) 0.190
Explained how to use method 31.4 7 (4.5%) 79 (50.3%) 71 (45.2%) 0.000
Explained IUD insertion steps 50.0 8 (3.2%) 109(43.6%) 133 (53.2%) 0.048
Explained method’ time- duration 47.6 8 (3.4%) 105 (44.1%) 125 (52.5%) 0.031
Counseling session in enough time duration 85.8 9 (2.1%) 168 (39.2%) 252 (58.7%) 0.013

Table 3: Observed counseling by providers level of experience.

Table 4 illustrates levels of disagreements the two methods of evaluation of FP services. There was a substantial level of disagreement between the two methods in all items except the item that tackled provider'sexplanation of IUD insertion steps (p-value = 0.008).

Component Women exit interview Observation of counseling Agreement Kappa value p-value
Welcoming women in counseling room 97.6% 44% 43.6% -0.012 0.311
Maintaining privacy during counseling 99.2% 52.4% 52.8% 0.009 0.271
Provider explained method effectiveness 99.6% 37.8% 38.2% 0.005 0.269
Provider explained method side effects 98.6% 2.02% 21.6% 0.007 0.180
Provider explained how to use method 98.4% 31.4% 31.8% 0.003 0.708
Provider explained IUD insertion steps 32.8% 50% 44.4% -0.112 0.008
Provider explained method’ time- duration 98.2% 4.6% 48.2% 0.010 0.387
Women received counseling in enough time duration 97.2% 85.8% 83.4% -0.024 0.443

Table 4: Percent agreement and kappa values of direct observation and women’s exit interview.

All other women’s perceptions were not related to education or number of children.

Discussion

Quality of care assessments have traditionally centered on medical records. Later, literature has focused on comparing record reviews and exit interviews with direct observation; these studies proved medical chart audit limitations and indicated that methods other than observation can report consistently on some, but not all aspects of health care process [2,17]. Exit interviews data can measure aspects of quality that are more subjective, such as respectful treatment, client satisfaction, methods discussed by the provider, waiting times, and information given during the counseling session on topics including side effects, method use, and when to return to the facility [1,4,7,9]. Learning more about these processes with the aim of improving them can have important programmatic payoff [8,10,18]. In this study, we restricted the scope of questions to the relevant objectives, so that our questioning would not represent a time constrain for women.

In general, the results of this study showed high disagreement between the data obtained from direct observation and exitinterviews. The women recipients of FP care highly appraised performance of providers during the service sessions (results ranged from 97% to 99.6%, except for one item; 32.8%). On the contrary, researchers evaluated the service in much lower rates, ranging from 2% to 52%, except for one item; 85%. These findings may indicate that the women had high appreciation of the providers performance regardless of the status of care they had, which was shown in almost total satisfaction with providers care. Such perception may be justified while considering that womens priority was not to criticize health care providers, especially that they needed to return for follow-up visits. Similar conclusions were reported by other studies about women’s perceptions of health care facilities and services [7,19,20]. Further, the reliability of exit interview data relies on womens memory, level of attention toward the provider’s actions, expectations about the level of care by the provider, and comfort when talking to an interviewer [7,19].

While women in this study were generally satisfied, other studies reported women’ satisfaction with specific items during counseling [8,12,19,21]. However, only 39.4% of women in this study have received FP counseling, a service that somewhat could be described as inadequate (average time was 3 minutes), and the rest of women did not receive any form of counseling; they were provided with contraceptives only. This may indicate that women in this study were not aware of qualities of an adequate FP services, otherwise, they would have relatively reflected that in their views of the service [7,19]. Although the analysis in this study did not include all women’ characteristics, other factors such as ever use of FP services, desire for more children, and the type of previously used contraceptive methods may have played a role in forming perceptions about providers’ performances. These factors were not analyzed due to the negligent portion of women who described the performance as inadequate. Overall, women’ perceptions of the FP services cannot be solely used as a method of evaluation because disagreement with direct observation was high. In addition, other studies indicated that women tend not to criticize health settings that are familiar to them, fearing that criticism would be interpreted as offensive [7,13].

In our study, the researchers performed an objective-based evaluation which entitled checking for existing standards of care and avoiding personal perceptions. Direct observation intends to measure gaps between desired and actual quality of care delivered [8]. It’s not conservative or strict measurement of quality, but rather an objective evaluation. Providing adequate FP services were affected by providers experience, as indicated by the statistically significant relationship of almost all items observed. Moreover, the adequacy of the provided services has increased proportionally in relation to years of experience. In this aspect, high percentages of studies of contraceptive discontinuation rates have indicated that the principal reason for discontinuation is dissatisfaction with the quality of service [17,21]. Although our results showed unexplained high womens satisfaction with the offered FP services, Jordan’s discontinuation rate and unmet need for FP are still at high rates; 11% and 16%, respectively [11]. It is imperative for the Jordan’s health care system to put forward strategies and policies for continuous monitoring of health care services, in addition to regular evaluation measures.

Our study introduced better understanding of the comparison between direct observation and exit-interview. Many studies attributed the disagreement between the results obtained from applying the two methods to multiple factors, including: courtesy bias, reliability of reporting, the Hawthorne effect, and recall bias were discussed [4,7,13,22-24]. When clients do not report negative aspects of care, or feel uncomfortable doing so, this might be due a courtesy bias [23]. Provider interviews may be unreliable due to providers intentions to show an ideal performance of service delivery rather than what they do in practice [23]. As well, the Hawthorne effect which results when providers demonstrate their best behavior during observations [23]. Most of these information biases may skew the resulting measures of quality in a positive direction of higher perceived quality [4,7,13,23]. In these cases, biased information may result in false study implications and conclusions [23].

With the continuous debates on drawbacks in the provision of health care services, adequate evaluation is a golden standard strategy for quality assurance [17,18,23-26]. Many studies recommend approaches that apply multiple methods of evaluation [4,7,23]. In addition, the method itself should be carefully designed to explore all possible dimensions of evaluation of the quality of health care services [7].

Conclusion

The evaluation of quality of FP services reported in this study has spotted important findings that concerns health care providers and researchers. Client satisfaction interviews should not be used without direct observation, as a quality measurement, in settings where health care providers and clients are well-known to each other. Recommendations for further research includes conducting a longitudinal study on a national scale that follow up with women who attend certain health care centers for family planning services to evaluate the quality of services at different time points and examine any variations noted in findings of these evaluations. This will help to improve the quality of the evaluation process and work on enhancing services based on these evaluations.

Limitations

One major limitation is the cross - sectional nature of the study that only captured womens perception at a certain point and not necessary reflect their opinion over a longer period; for instance, their opinion from all visits since they started visiting a certain family planning health service center. Furthermore, sample population were limited to a small geographical region that certainly affected the generalizability of the findings. Another possible limitation is that womens responses in the exit interview might have been impacted by their expectations of that visit, date scheduled for the next visit, or womens impressions about the service from previous visits, which could have affected their overall satisfaction, and thus, altered their responses.

Funding

There is no funding received for this study.

Acknowledgment

We would like to thank Jordan University of Science and Technology and Jordan Ministry of Health for their support and agreements, and women who agreed to participate in the study.

References

Select your language of interest to view the total content in your interested language

Viewing options

Post your comment

Share This Article

Flyer image
journal indexing image
 

Post your comment

captcha   Reload  Can't read the image? click here to refresh