Validating the EAP Philosophy:

Listening to Satisfaction Surveys

Steven M. Harris, Ph.D., Texas Tech University

Abstract

Despite the intuitive sense put forth by the EAP philosophy that healthy individual make for a healthy work force EAP professionals are faced with the challenge of proving this assertion. This research investigates whether or not client satisfaction surveys can be useful in validating the EAP philosophy. Most EAP professionals collect this information but may not be using it to its fullest potential. A multiple regression procedure, using data collected from satisfaction surveys, validates that positive personal experiences with an EAP may predict improvement in the employee's work life. Despite this finding, EAPs are encouraged to do more than rely upon satisfaction survey data to validate the EAP philosophy. This paper highlights suggestions and recommendations for future research.

Introduction

Few would deny that the EAP philosophy makes intuitive sense. It seems logical to assume that the mentally and emotionally well-adjusted individual is a productive and valuable member of any work force. EAP professionals consider this philosophy as the backbone of their reason for existence. Despite the intuitive sense this philosophy makes, EAP professionals are increasingly faced with the challenge of empirically substantiating the claim that having an EAP is a good business decision and that employees who use the EAP are better off in both their personal and work lives¹.

Statement of the Problem

Although many understand the need to confirm empirically the EAP philosophy, relatively few EAPs employ a strict research regime to show improvement in clients' lives². Furthermore, few are showing that improvement in one's personal life actually translates into an observed or measurable improvement in the workplace (that is, better relationships with coworkers and supervisors, more enjoyment, fulfillment, productivity, etc.). Despite the lack of research, many EAP professionals do collect satisfaction survey data. These surveys perform at least three functions. First, they provide a forum for client feedback. Second, they may increase the client's respect for the EAP, which is actively searching for ways to improve the service. Third, they provide valuable information for the service provider³. These questionnaires are typically used to show client companies' utilization rates and often provide brief narrative testimonials regarding the benefit of the EAP in the clients' lives. Considering that many EAPs are already collecting satisfaction data, is it possible for these surveys to provide an accurate or reliable measure of how a client's personal and work life may have been affected by visiting an EAP?

Self -report satisfaction questionnaires administered by the very institution being evaluated may present threats to the reliability and validity of the results⁴. For example, some researchers believe that those who fill out satisfaction surveys are typically persons at polar extremes. Usually, only the very satisfied⁵ and sometimes the very dissatisfied fill out and return these instruments. Because of this, it is often not accurate to generalize this information as representative of the total population served. Additionally, threats to construct validity may arise due to bias in the wording of the questions on the survey. Customer satisfaction questionnaires, in an attempt to be brief so as to maximize return potential, typically request specific and subjective information instead of trying to assess more objective criteria regarding experiences with the service. Furthermore, in an EAP setting, social desirability bias may exist, depending on whether or not the participant believes that the instrument is confidential, and that answering truthfully will not lead to any negative consequences, such as jeopardizing one's employment status⁶.

Satisfaction surveys do have some inherent flaws, but for many EAPs they represent the extent to which research and data collection is conducted. If this is the case, what can be said about the information generated from a client satisfaction survey? Can it, in any way, validate the EAP philosophy?

Methodology

HelpPeople, a hospital-based internal EAP with multiple external contracts located in Syracuse, New York, collected client satisfaction surveys from October 1995 to July 1996. Surveys were sent to every client of record during this ten-month period. The surveys were to remain anonymous and contained questions designed to assess the client's experience with the EAP office staff and counselor, the client's beliefs regarding how using the EAP affected his/her personal and work life, as well as how referral resources were experienced.

Sample

A total of 1,277 people used the EAP service between October 1995 and July 1996. Of this group, only 738 were either employees or relatives of an employee seeking assistance and presenting in person to the EAP office. The remaining 539 persons of record consisted of employees or relatives who contacted the EAP by phone for simple phone consultation or were supervisors consulting about particular employees. Neither supervisory nor phone consultation clients received the standard client satisfaction survey because these calls were typically anonymous (phone consult) or did not involve the individual in need of direct clinical services (supervisory consult). In all, 83 of the 738 surveys were returned. Therefore, the response rate for returned surveys was 11 percent.

The questions on the survey did not request any identifying demographic information; therefore it was impossible to say who responded. However, of the population surveyed, the two majority age groups were 36 to 45 years old (n=274) and 26 to 35 years old (n=244). Women comprised 59 percent of the overall population (n=435 of N=738). Neither ethnic nor racial identity information was collected. The three most prevalent presenting problems were: 1) relationship problems (n=247), 2) emotional problems (n=172), and 3) family problems (n=122). Only 77 people reported coming to the EAP to work on issues related to their jobs or careers.

Measures

Three scales made up the measures used in the data analysis. Each scored high on tests of reliability; the Counselor Effectiveness Scale (n=81, alpha =.83), Impact on Life Scale (n=74, alpha =.86), and Impact on Work/Job Scale (n=71, alpha =.94). The Counselor Effectiveness Scale consists of three questions that assess how the client perceived the counselor's understanding of the problem, ability to offer alternative ways of looking at the problem, and the counselor's ability to offer techniques to handle the problem.

The Impact on Life Scale consists of four questions designed to assess how the client perceived the EAP affecting his/her life. Clients indicated the degree to which they felt better, felt more confident in making decisions, saw their problems in a new light, and the degree to which family members noticed a positive change in them.

The Impact on Work/Job Scale consists of three questions. Clients indicated the degree to which their participation at the EAP helped them concentrate on work more than on their problems while at work. They also indicated the degree to which they felt they performed their job better and how relationships with coworkers had improved as a result of being at the EAP.

Data Analysis

Two conditions dictate the analysis of this data: I) the limited information provided by the customer satisfaction survey, and 2) the problem statement outlined at the beginning of this document. Specifically, can satisfaction surveys validate an underlying assumption of EAP professionals: that improvement in one's personal life translates to improvement in one's work life? Considering the limitations, data analysis was confined to measures of central tendency as well as a multiple regression. The independent variables for the regression were the scores from the Counselor Effectiveness and Impact on Life Scales. The dependent variable came from the scores on the Impact on Work/Job Scale.

Results

Most of the people who returned the survey had a positive response to their experience with the EAP. For example, regarding the client's initial contact with the EAP, not one respondent was less than somewhat satisfied. The range of possible scores was 4 to 15, with higher scores indicating more satisfaction. The mean score was 13.85 (SD, 1.34) and the observed scores ranged from 11 to 15, with the majority of respondents (98 percent) being either satisfied or very satisfied (50 percent responded with the highest possible score). Another example of participants' positive experience with the EAP is indicated by the fact that 86 percent of respondents agreed or strongly agreed that they would recommend the EAP to coworkers. Overall, respondents reported a favorable experience with the EAP. This finding is consistent with the concern expressed by those who critique satisfaction survey research by suggesting that perhaps only the very satisfied fill out satisfaction questionnaires, thereby skewing the information and compromising the validity of the instrument^4,5. Very few respondents reported negative experiences with the EAP.

The results of the multiple regression are presented in Table I. The explained variance can be attributed directly to the respondents⁷ beliefs about how the EAP had positively impacted their lives while their reports of counselor effectiveness yielded a non-significant effect for the model. The independent variables were the Counselor Effectiveness and the Impact on Life Scales. Higher scores on each of these scales indicate either greater perceived counselor effectiveness, or greater perceived impact on one's life, respectively. The significant results suggest that those who had experienced positive changes in their personal lives as a result of participating in the EAP also experienced positive changes in their work lives. Specifically, the survey results indicate that EAP clients believed they were able to concentrate more on work than on their problems while at work.

Additionally, they believed that they performed their jobs better and that they experienced improvement in their relationships with coworkers since coming to the EAP. These findings suggest that customer satisfaction surveys can be helpful to EAP professionals who are seeking to validate the EAP philosophy.

Discussion and Recommendations

One central limitation of this study was the low response rate. With only an 11 percent response rate, it is hard to say with confidence that the findings are accurate or reflect the experience of every (or even most) EAP c1ient(s). Additionally, an over- whelming majority of the respondents were either satisfied or very satisfied with their EAP experience. As one of the EAP counselors, the author knows that not every client was satisfied with the service. It is hard to imagine that employees mandated to attend the EAP for work performance reasons, or family members forced to go to the EAP for personal reasons had overwhelmingly positive experiences. Satisfaction surveys from these clients may not have been returned. Despite the inherent limitations of satisfaction surveys and the poor response rate, the data do suggest a relationship between improved work performance, improvement in the employee's personal life, and satisfaction with the EAP.

These two factors - low response rate and minimal representation of dissatisfied EAP clients - may be extremely helpful in showing EAP professionals some of the problems associated with conducting outcome research. Consequently, they may point the way EAP professionals need to head to begin conducting meaningful outcome studies.

Although the data support the EAP philosophy, or show that improvement in one's personal life seems to be a good predictor of improvement in one's work life, the limitations of this study merit additional comment. One conclusion researchers and practitioners can draw from these results is that there is a need for more accurate outcome studies in the EAP world. In a sense, EAP professionals need to back up their claims that having an EAP is good for businesses as well as individuals. In an effort to provide some direction, the following paragraphs outline suggestions, considerations, and recommendations for future EAP outcome research.

Specific Designs

Reliable and valid research results are most often obtained when the data collection and analysis are guided by a specific design or a solid conceptualization of what needs to be studied⁷. The design should follow a particular conceptualization. This would call for a strong theoretical orientation to guide the formation of research questions and the structure of the data collection. The satisfaction survey used in this project was limited in how helpful it could be in answering the research question due to the fact that it was created for a different purpose. It was designed as a confidential instrument and requested no personal demographic information. Furthermore, the questions may have been leading as opposed to more objective measures. In the future, it would be helpful to know more about who is using the EAP and how the experiences or effectiveness of the EAP change based on gender, race, employment history, or occupation.

Perhaps the introduction of more experimental or quasi- experimental designs that utilize random assignment of cases and control groups could help control for the effects of extraneous variables⁸. Employing a pre and post-testing procedure could help identify specific effects of the EAP on a client's life. Another important dimension to assess is the effectiveness of the EAP over time. Longitudinal studies may contribute to our understanding of what interventions last the longest and which fade most quickly

Multimethod and Multifaceted Designs

There is nothing wrong with survey research per se; however, one should consider the unidimensionality of self-report survey research. It only taps into one information source. A multi-method design that employs self-report data as well as interviews with other employees, family members, and employers may provide a more accurate picture of the impact of the EAP on the life of the individual and the individual's consequent impact on the workplace. Additionally, having access to records of employment productivity, absenteeism, relationships with coworkers, and other job-specific information may increase the usefulness of the findings⁸.                                                                                         

 Industry-specific concerns should also be considered. For example, do firefighters experience different stressors than nurses?  How are they both different from legal secretaries? Does the instrument in a particular study consider the very different working environments of the participants?

The suggestions in the paragraphs above should be considered a starting place. They should not be considered the end-all of questions necessary to ask to create a solid study. Also, study designers should remember that every research study has flaws. The key to conducting more solid research is clearly to conceptualize what needs to be studied or observed, and select instruments and measures capable of describing the phenomenon in question.

Conclusion

This project began with the author's hope to identify the benefits of using customer satisfaction surveys to validate the importance of EAPs and to show the impact that attendance at an EAP may have on both the personal and work life of the employee. The findings do support, in part, the EAP philosophy. It appears as though people who believe their personal lives have benefited by EAP participation also indicate positive changes in their work lives. However, as EA professionals, we should not be content with the results of a single study based solely on customer satisfaction surveys. This research should be considered as a springboard for future EAP research.

One thing the findings do wonderfully is point to the importance of and need for more structured outcome studies in the EAP world. Despite the helpful information satisfaction surveys may provide client companies about usage and client satisfaction, and even though they can help us see an empirical connection between our personal and work lives, they may not necessarily be the most reliable indicators of program benefit. More involved and well- thought-out research is needed to further validate and confirm the accuracy of the EAP philosophy.

References

^I P.L. Bills. "EAP Marketing: Successful EA programs need to go back to the basics." Employee Assistance: Solutions to the Problems, 2, (1990), pp. 43,45,47.

² R.E. Bergmark, M. Parker, P.H. Dell, & C.L. Polich. "EA programs: The challenge, the opportunity, part III." Employee Assistance: Solutions to the Problems, 4, (1991), pp. 24-28.

³ W.E. Petersen & S.L. Davidow. "Satisfaction surveys improve clinic care." Update: The Newsletter of the Health Outcomes Institute, 2, (1995), pp. 1-3.

⁴ Hill, N. Handbook of customer satisfaction measurement. Hampshire, England: Grower Publishing Limited, 1996.

⁵ Naditch, M. P. "The new frontier: Moving beyond the black hole of outcome data collection." Employee Assistance: Solutions to the Problems, (1995), pp. 14-17.

⁶ E. Naumann & K. Gaiel. Customer satisfaction measurement and management: Using the voice of the customer. Cincinnati, OH: Thomson Executive Press, 1995.

⁷ Wampler, K. S. "Systems therapy and out-patient mental health treatment: Twelve priorities for outcome research." Invited paper presented at the Marriage and Family Therapy Research Conference, Santa Fe, New Mexico, July 1997.

⁸ K. Bruhnsen. "Michigan study shows EAP clients use less sick leave, stay longer." EAPA Exchange, 24, (1994), pp. 11,27.

Steven M. Harris, Ph.D., is an assistant professor in the Marriage and Family Therapy Program, Department of Human Development and Family Studies at Texas Tech University, Box 41162, Lubbock, TX, 79409-1162. He can be reached via e-mail at sharris@ttu.edu.