Objective This study assessed the long-term effects of a workplace intervention aimed at reducing adverse psychosocial work factors (psychological demands, decision latitude, social support and effort–reward imbalance) and mental health problems among health care professionals in an acute care hospital.
Methods A quasi-experimental design with a control group was used. Pre-intervention (71% response rate) and 3-year post-intervention measures (60% response rate) were collected by telephone interviews with validated instruments.
Results Three years after the intervention, all adverse psychosocial factors except one were reduced in the experimental group, and the improvement was statistically significant for 5/9 factors: psychological demands, effort–reward imbalance, quality of work, physical load and emotional demands. In addition, all health indicators improved and 2/5 significantly: work-related and personal burnout. In the control hospital, three work factors improved significantly but two deteriorated significantly: decision latitude and social support. All health problem deteriorated, although not significantly, in the control hospital. Moreover, 3 years after the intervention, the mean of all adverse factors except one (psychological demands) and all health indicators was significantly more favourable in the experimental than the control hospital, after adjusting for pre-intervention measures.
Conclusion These results support the long-term effectiveness of the intervention. The reduction in many psychosocial factors in the experimental hospital may have clinical significance since most health indicators also improved in this hospital. These results support the whole process of the intervention given that significant improvements in psychosocial factors and health problems were observed in the experimental hospital but not in the control hospital.
- Psychosocial work environment
- primary prevention intervention
- evaluative research
- quasi-experimental design
- mental health
- intervention studies
- longitudinal studies
Statistics from Altmetric.com
- Psychosocial work environment
- primary prevention intervention
- evaluative research
- quasi-experimental design
- mental health
- intervention studies
- longitudinal studies
What this paper adds
Many studies have documented the effect of adverse psychosocial work factors on the incidence of mental health problems.
The few studies that have evaluated the impact of interventions aimed at reducing adverse psychosocial work factors and their mental health effects have important limitations.
The results of this study support the long-term effectiveness of an intervention to reduce adverse psychosocial work factors and mental health problems.
The reduction in many psychosocial factors in the experimental hospital may have clinical significance since most heath indicators also improved in this hospital.
The participatory process and the active role of the members of the intervention team in the identification of problems and solutions were important and closely linked elements of the intervention.
Many studies have documented the effect of adverse psychosocial work factors on the incidence and prevalence of mental health problems.1–3 Karasek's job demand–control–support model4 and Siegrist's effort–reward imbalance model5 identify four psychosocial factors in the work environment whose effects on physical and mental health have been the most frequently documented2: high psychological demands, low decision latitude, low social support from colleagues and supervisors and effort–reward imbalance. However, few studies have evaluated the impact of interventions aimed at reducing these adverse psychosocial work factors and their mental health effects. Available studies also have important limitations.6 7 Nevertheless, recent systematic reviews of organisational-level and task restructuring interventions report a positive effect, on health, of increased control8 and the combination of decreased demands and increased control.9 These reviews also highlight the lack of information available on the implementation of interventions and the need to fill that gap.8 9
This is the third of a series of papers on an evaluative research intervention among healthcare professionals. The first paper described the development and implementation phases of a participatory intervention aimed at reducing four theory-grounded and empirically supported adverse psychosocial work factors (high psychological demands, low decision latitude, low social support and low reward) and their mental health effects.10 Four steps were used to develop and implement an intervention among healthcare professionals in an acute care hospital: a prior risk evaluation, observation within the care units, interviews with key informants and a participatory approach with an intervention team. The second paper assessed the effectiveness of the intervention 12 months after initiation.11 It reported a reduction in several adverse psychosocial factors in the experimental group, whereas no such reduction was found in the control group. There was also a significant reduction in sleeping problems and work-related burnout in the experimental hospital, whereas although sleeping problems decreased, both client-related and personal burnout increased in the control group. The present paper focuses on the effectiveness of the intervention 3 years after initiation and the implementation of changes in the hospital.
Study design and population
This research used a before-and-after quasi-experimental design with a control group. A participatory approach with an intervention team, including healthcare professionals from three targeted care units and other important stakeholders in the experimental hospital, aimed to determine what changes should be introduced to reduce adverse psychosocial work factors in these specific care units and throughout the hospital, and the best way to implement these changes. The intervention team met eight times for 3 h each time over a 4-month period. The intervention was defined as changes undertaken by the hospital to reduce adverse psychosocial factors in the workplace. Solutions proposed by the intervention team and adopted by the nursing department as well as any other objective change introduced with the explicit goal (or actual consequence) of improving one of the four targeted psychosocial factors were considered part of the intervention. The study population was composed of all care-providing personnel in the experimental (n=674) and control hospitals (n=894), both of which offer general and specialised short-term care. The population included all healthcare professionals in direct contact with patients (nurses and beneficiary attendants). Those on sick leave for more than 3 months and those working only 2 days per week or less over the 3 months preceding the pre-intervention measurement were excluded from the study.
Data compiled from the before and after intervention measures (M0–M2/effectiveness phase)
Exposure to psychosocial factors in the workplace and the health status of the healthcare professionals were measured in both experimental and control hospitals, before (M0 in 2000) and 3 years after the beginning of the intervention (M2 in 2004) to evaluate their long-term effects. Both measures were collected with the same validated instruments. The participants were contacted at home by a firm specialised in telephone surveys and informed of the study objectives and how they could contribute. If they agreed to participate, a 30 min telephone interview was conducted on the spot or at a later time agreed upon by the participant. Neither the firm nor the telephone interviewers knew which subjects were in the experimental or control hospital. A research assistant regularly monitored the interviews in real time throughout the M0 and M2 proceedings to verify whether the questions were asked as instructed to limit a potential bias from inter-interviewer variation and ensure the quality of the compiled data.
Psychosocial work factors
Psychological demands (nine items), decision latitude (nine items) and social support (eight items) were evaluated using Karasek's Job Content Questionnaire (JCQ).12 13 The validity of the JCQ has been demonstrated.4 14 15 Reward at work was measured by 11 items from Siegrist's original instrument, for which factorial validity and internal consistency have been documented.5 16 17 Psychological demands were used as a proxy for effort. The effort–reward imbalance was defined as a ratio of effort to reward greater than 1 as recommended by Siegrist.5 In our study, internal consistency based on Cronbach's α coefficient was 0.71 for job decision latitude, 0.76 for psychological demands, 0.81 for social support and 0.77 for reward. Because they are important specific factors in the work of healthcare professionals, three other work factors were considered and measured in this study: emotional demands from the Nursing Stress Scale,18 physical workload from the JCQ12 and a new question on the quality of work.
Psychological distress was measured with an abridged version (14 items) of the validated Psychiatric Symptom Index (PSI), which measures the frequency of symptoms of anxiety, depression, aggressiveness and cognitive problems.19–21 Burnout was measured using questions from the Copenhagen Burnout Inventory,22 which measures personal burnout, work-related burnout and client-related burnout. Sleeping problems were measured using five questions from the Nottingham Health Profile (NHP), an instrument used to estimate sleep disturbances associated with insomnia.23 24 In our data, Cronbach's α coefficients were 0.91 for the PSI, 0.88, 0.86 and 0.79 for personal burnout, work-related burnout and client-related burnout, respectively, and 0.78 for sleeping problems.
Data compiled to document changes (implementation phase)
The implementation phase of the intervention was concerned with the means put in place to produce changes to the work environment.25 These changes were documented through extensive follow-up throughout the research period. They included 56 suggestions proposed to the nursing department by the intervention team in its final report in 2001, as well as any change made in one or more units at the hospital independently of the research which may have influenced the psychosocial factors under study. Various solutions recommended by the intervention team had already been applied at M1, but others could only be implemented on a mid- or long-term basis.10 These changes were thoroughly monitored through the minutes of three intervention team meetings in 2002 and 16 follow-up interviews with key informants in 2003–2004. These informants were nurses and beneficiary attendants, chief nurses of the three experimental units, the nursing department coordinator, and the heads of nurses' and beneficiary attendants' unions. They were chosen on the basis of their participation in the intervention team or their position on the management team, giving them access to information relevant to the research. During each intervention team meeting, the minutes of the preceding meeting were validated. The 16 interviews were taped and then transcribed verbatim in full. Data were grouped chronologically by intervention target, care unit and key informant. All changes were classified by the intervention team according to the main targeted factor (psychological demands (PD), decision latitude (DL), support from supervisors (SSS) or from colleagues (SSC), and reward (RW)), implementation date or period, and personnel affected by the change (unit or hospital).
All information was classified by an assistant and the chief researcher into homogenous change categories, and then broken down into seven major emerging themes associated with the intervention context26: team work and team spirit, staffing processes, work organisation, training, communication, ergonomics and external context (outside the scope and independent of the research). The information was grouped according to comments related to any of the three targeted units or the hospital overall. Then the documents were synthesised and depersonalised to outline key points and identify the impact of the changes on care units. Since measurement of the effectiveness of the intervention encompassed the entire hospital, changes reported in this paper are mostly those affecting all care units in the hospital. However, changes affecting only certain units were also reported when they were likely to have on effect on overall results.
Statistical analyses were conducted using SAS.27 In all analyses, the significance level was fixed at 0.05. We examined two groups of dependent variables: psychosocial work factors and mental health. We compared the two hospitals before the intervention on both groups of variables using logistic regression analysis. We then made two comparisons for each variable: (1) before and after intra-group comparison and (2) post-intervention inter-group comparison. For these analyses, only healthcare professionals who responded to both interviews were included. Intra-group comparisons consisted of comparing the means of dependent variables before and after intervention within a single group (the experimental or the control hospital). ANOVA with repeated measures was conducted for each hospital.28 A 95% CI of the difference in means between M0 and M2 was constructed, for each hospital, to test whether the difference was statistically different from zero. Analyses were also adjusted for age, but since adjusting did not significantly change the crude effect measures (>10%), only crude measures are presented. Post-intervention inter-group comparisons consisted of an analysis of covariance (ANCOVA)29 comparing the experimental to the control hospital on the post-intervention measure for each dependent variable controlling for the pre-intervention measure in each group. Analyses were also adjusted for age and gender, but since adjusting did not significantly change the effect measures from the ANCOVA (>10%), only the measures adjusted for M0 are presented.
The results of this phase are given in table 1 which combines the problems identified by the intervention team in the development phase of the intervention and the solutions that were implemented before the second post-intervention measure (M2). These changes represent 80% of the recommended solutions suggested by the intervention team to the nursing department. The targeted problems and implemented solutions have been classified under six themes: team work and team spirit, staffing processes, work organisation, training, communication and ergonomics.
A seventh theme of implemented changes concerns the external context and some of the events that were independent of the research but may have influenced the effectiveness of the intervention.
Between the summer of 2002 and the second post-intervention measure (M2), there was significant turnover among management in the hospital, in the nursing department and in the care units (SSS−). A new hospital director was appointed, the post of director of the nursing department was not filled for over 1 year, and a nursing coordinator of care and many chief nurses left the hospital and were often replaced by temporary staff.
New epidemiological phenomena led to stressful situations. The danger of nosocomial infections had a major impact both on care provided and on healthcare professionals, who had to implement extremely stringent measures (PD+). For example, in 2003, they had to manage outbreaks of methicillin-resistant Staphylococcus aureus and some refused to provide care (SSC−). New technologies required constant adjustment, with training, adaptation and transfer of expertise (PD+ DL+). Hospitalisation time was decreasing and patients were increasingly ill with more pronounced health problems (PD+). In March 2004, at the time of M2, the transfer of a unit to another hospital was officially announced. All hospital staff members were affected, because some positions were abolished and people feared being bumped (RW−). In 2003–2004, the hospital was faced with a serious budget deficit that had to be absorbed. This translated into a drop in the number of positions, a decrease in occasional beds and a reduction in the number of work hours which led to difficulties recruiting nurses and beneficiary attendants because the hospital was unable to guarantee work hours (RW−). The difficulty in replacing or recruiting entailed extra hours for regular personnel or the hiring of agency personnel not familiar with the hospital (SSC− PD+). Given the client flow and uncontrolled consumption of services during the summer, planned bed reductions were not carried out due to a lack of available staff away on holiday (PD+). Moreover, in the summer of 2003, several beds were monopolised to reduce waiting lists in orthopaedics (PD+).
A project to improve working conditions was implemented in the hospital in the spring of 2001. Its purpose was to enhance physical and psychological wellbeing and retain staff. Several problems similar to those identified by the intervention team were raised (RW+ SSS+). Another work reorganisation project aimed at the optimal use of the skills of each staff member was carried out in one care unit in 2001 (DL+ PD+).
At baseline or the pre-intervention measurement (M0), 73% of the 674 eligible subjects in the experimental hospital, and 69% of the 894 eligible subjects in the control hospital, participated. At the second post-intervention measurement (M2), the response rate was 65% and 56%, respectively (figure 1). Participants and non-participants at M2 in each hospital were compared on their baseline characteristics (at M0). There was no significant difference between participants and non-participants in each hospital or between the experimental and control hospitals for gender, age, seniority, job status, occupation or work schedule (data not shown). The female:male ratio of eligible subjects at M0 was 4:1 and 5:1 in the experimental and control hospitals, respectively, and the ratio of participants at M2 was 5:1 in both hospitals.
In order to test further for a potential selection bias, the means of participants and non-participants at M2 were also compared according to psychosocial environment factors and health status at M0. They were comparable for psychological demands, decision latitude, co-worker support, psychological distress and client, work and personal burnout in both the experimental and control hospitals (table 2). However, the mean of supervisor support was higher among the non-participants in the control hospital, and the mean for rewards was lower for the non-participants in the experimental hospital.
Table 3 shows changes in the mean score of each psychosocial factor and each health problem between M0 and M2 in both hospitals. In the experimental hospital, all psychosocial work factors except social support improved and the improvement was statistically significant for 5/9 factors: psychological demands, effort–reward imbalance, quality of work, and physical and emotional demands. All health indicators improved and 2/5 significantly: work-related and personal burnout. A decrease in one other health indicator (client-related burnout) reached borderline significance (p<0.10). In the control hospital, 3/9 psychosocial work factors improved significantly: psychological demands, physical load and emotional demand. However, social support from supervisors and total support deteriorated significantly. Also, no health indicator improved significantly in this hospital.
When the means of each psychosocial factor at M2 were adjusted for the means at M0, except for emotional demand which was similar in both hospitals, the mean of all factors at M2 was more favourable in the experimental hospital (7/9 significantly so) (table 4). All health problems were also lower in the experimental hospital, and the difference was statistically significant for 3/5 of the health indicators (client, work and personal burnout) and borderline for a fourth indicator, psychological distress.
Summary of results and meaning
All adverse psychosocial work factors except one had improved in the experimental hospital 3 years after the intervention and the improvement was statistically significant for 5/9 factors: psychological demands, effort–reward imbalance, quality of work, and physical and emotional demands. In addition, all health indicators improved and 2/5 significantly: work-related and personal burnout. At M2, the mean of all psychosocial factors except for emotional demands, and all health indicators, were more favourable in the experimental hospital than in the control hospital. These results were more noteworthy than those measured at M1 and may be due to the fact that more changes had been implemented in the experimental hospital 3 years after the start of the intervention.
We sought to determine why social support from supervisors did not improve in the experimental hospital. Considerable turnover among chief nurses, and a new nursing coordinator and head of the nursing department during the study period may have influenced the report of social support at M2. Furthermore, based on qualitative information collected during interviews with the intervention team in three post-intervention meetings and with key informants, it appears that high expectations regarding the results of the intervention influenced attitudes towards changes and especially so for the social support dimension. Although many solutions proposed by the intervention team were implemented (80% as reported in table 1), other organisational changes also proposed by the intervention team had not yet been implemented at M2.
Positive changes were observed 3 years after the start of the intervention. It is difficult to identify which aspect of the intervention was responsible for the outcomes observed. In fact, even if the qualitative part of the study allowed to gather detailed information on what changes were made, still the design of the study was not experimental and as in most intervention research, changes in the workplace were beyond the researchers' control. In addition, many changes related to the economic context and management policies in the health network, some of which are reported under the seventh theme in the implementation phase results, affected both the experimental and control hospitals during the study period. Although changes were systematically followed up through interviews, it is impossible to attribute with certainty the results (the outcomes) to specific changes implemented in response to the intervention team recommendations. Still, the results support the whole process of the intervention given that significant improvements in psychosocial factors and health problems were observed in the experimental hospital but not in the control hospital.
Although the magnitude of the change in the mean between M0 and M2 may not be important for each individual psychosocial factor, this reduction in many of the targeted factors in the experimental hospital may have clinical significance. This is supported by the significant improvements observed in most health indicators in this hospital. These results support the long-term effectiveness of the intervention and are consistent with other intervention research on organisational job factors and stress-related emotional outcomes that showed shorter-term effects (6, 12 or 24 months)11 30–34 or the start of a beneficial change process.35 36 However, two prospective studies observed increased emotional exhaustion after interventions.37 38 In the first study, there is no explanation for the results but rather a focus on the qualitative data which show how constraints related to time and organisational practices impeded full involvement of the employees during implementation of the interventions. In the second study, the intervention mostly targeted individuals rather than the organisation and there is no evidence as to whether changes were implemented. Two others studies observed no effect of intervention.39 40 These studies had a small sample size which might have masked a potential effect (lack of statistical power).
Strengths of the study
This study has several strengths which were not present in many of the related studies. First, the development phase using a prior risk evaluation, observation and interviews allowed us to gather crucial background information to characterise the problem and its setting, and the relevance of the intervention in the experimental hospital. Second, both qualitative and quantitative methodology were used to document the three phases of intervention. Third, this research was based on sound theoretical models; the intervention targeted four well-defined and theory-grounded psychosocial job factors whose deleterious effects on health have been demonstrated in various work settings, and validated instruments were used to measure them. Fourth, the study used a quasi-experimental design with a comparable control group, a large number of subjects, several outcome measures and a 3-year follow-up period. The long-term evaluation of the effectiveness allowed a better understanding of conditions for success, the lasting nature of effects and the appropriation of the process by the worker. Fifth, the participatory process relied, from the start, on management and staff involvement, thereby ensuring the application of recognised conditions for successful preventive interventions.25 41
Limitations of the study
Some limitations of the study must also be mentioned. First, the use of an experimental design with a randomised experimental and control group was not feasible. The quasi-experimental design that we chose was the best alternative and the two participating hospitals were comparable. Second, changes in both hospitals were beyond the researchers' control, and it was impossible to prevent organisational changes over the study period which were not related to the research but could have influenced the targeted psychosocial work factors. In fact, many changes related to the economic context and management policies in the health network, some of which are reported under the seventh theme in the implementation phase results, affected both the experimental and control hospitals during the study period. Although changes were systematically followed up through interviews, it is impossible to attribute with certainty the results of the research (the outcomes) to specific changes implemented in response to the intervention team recommendations. Nevertheless, the whole process of the intervention research produced significant improvements in many of the targeted psychosocial factors and health problems in the experimental hospital, and these improvements did not occur in the control hospital. Third, the fact that the participatory intervention process could only include representatives from three care units in the hospital, made it difficult to determine the extent to which other units' problems were correctly identified and the extent of the intervention within these other units. In fact, the intervention was not limited to these three units, because many of the recommended changes were applicable to all care units and some of the changes adopted were implemented throughout the entire hospital. Therefore, the effect of the intervention was measured for the whole hospital. Fourth, despite consultation and communication activities inherent to the work of the intervention team, a problem was identified in terms of the level of transmission of information in one of the three targeted units, particularly during evening and night shifts. Also, gaps were identified in the transmission of information to healthcare personnel throughout the hospital of the implemented changes following the work of the intervention team.
The response rate was not optimal, and a selection bias may have occurred if participants at baseline were not representative of all eligible subjects. The description of participants at baseline showed they were comparable to non-participants on several characteristics. This limited the potential for selection bias. Another potential selection bias could have been introduced if participation at M2 were linked to changes in individual level of work psychosocial factors, which are in turn linked to mental health problems. However, a comparison between those who remained in the study at M2 and those who dropped out showed no significant difference in most of the psychosocial work factors and health indicators measured before the intervention, thus limiting the possibility of a selection bias by attrition. A possible Hawthorne effect may have caused an information bias, as employees in the experimental group knew they were part of an intervention to reduce adverse psychosocial work factors and their effects on health. This type of bias, which is liable to occur when the post-intervention measure takes place soon after the intervention, is less likely to occur in a long-term evaluation more than 3 years after the end of intervention team meetings. Therefore, a Hawthorne effect is unlikely to explain the results.
In addition, an information bias could have occurred since the work-related variables were actually based on self-reported rather than objective measures. However, no objective measures were available, and the perceived psychosocial factors are possibly more important in the development of mental health problems than objective factors that may not be perceived.42 In addition, Semmer et al reported that in studies which address this methodological issue, self-reports have been found to be “better than is often assumed”.43
This study suggests a perspective for future interventions. One of the six themes that emerged from the intervention team work, team work and team spirit, could become an integrating theme providing meaning to the group approach and empowerment by the milieu. Indeed, the team is a natural grouping suited to appropriation of the intervention approach. Teamwork on the identification of constraints and solutions would lead to better communication and improved exchange between the members. Head nurses would definitely have to play a leadership role in the development of this empowerment.
These results support the long-term effectiveness of the intervention. The reduction in many psychosocial factors in the experimental hospital may have clinical significance since most of the health indicators also improved in this hospital. These results support the whole process of the intervention given that significant improvements in psychosocial factors and health problems were observed in the experimental hospital but not in the control hospital.
The authors wish to thank all care professionals who participated in the research and all stakeholders in each hospital for their involvement. They are grateful to A Lower for coordination of the research, to G Guay and I Brisson for managing the data of the implementation process, to S Camden and Myrto Mondor for programming the analyses and to Mahée Gilbert-Ouimet and Nathalie Gauthier for revision of the manuscript.
Funding This research was funded by the Québec Council for Social Research, the Canadian Council of Humanities Research, the Canadian Institutes of Health Research (CIHR), the provincial Ministry of Health and Social Services and the Québec Regional Board of Health and Social Services. RB was a Research Scholar for the Québec Health Research Fund and CB was a Research Scientist Scholar for the CIHR.
Competing interests None.
Ethics approval This research was approved by the ethics committee of Laval University and the ethics committee of both the experimental and control hospitals.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.