Article Text
PDF
Abstract
Objective To evaluate the validity of widely used questionnaire items on work schedule using objective registry data as reference.
Method A cohort study of hospital employees who responded to a self-administered questionnaire on work schedule in 2008, 2012 and 2014 and were linked to individual-level pay-roll-based records on work shifts. For predictive validity, leisure-time fatigue was assessed.
Results According to the survey data in 2014 (n=8896), 55% of the day workers had at least 1 year of earlier shift work experience. 8% of the night shift workers changed to day work during the follow-up. Using pay-roll data as reference, questions on ‘shift work with night shifts’ and ‘permanent night work’ showed high sensitivity (96% and 90%) and specificity (92% and 97%). Self-reported ‘regular day work’ showed moderate sensitivity (73%), but high specificity (99%) and ‘shift work without night shifts’ showed low sensitivity (62%) and moderate specificity (87%). In multivariate logistic regression analysis, the age-adjusted, sex-adjusted and baseline fatigue-adjusted association between ‘shift work without night shifts’ and leisure-time fatigue was lower for self-reported compared with objective assessment (1.30, 95% CI 0.94 to 1.82, n=1707 vs 1.89, 95% CI 1.06 to 3.39, n=1627). In contrast, shift work with night shifts, compared with permanent day work, was similarly associated with fatigue in the two assessments (2.04, 95% CI 1.62 to 2.57, n=2311 vs 1.82, 95% CI 1.28 to 2.58, n=1804).
Conclusions The validity of self-reported assessment of shift work varies between work schedules. Exposure misclassification in self-reported data may contribute to bias towards the null in shift work without night shifts.
- Methodology
- speciality
Statistics from Altmetric.com
What this paper adds
Exposure misclassification in self-reported data may contribute to bias towards the null in shift work without night shifts.
The validity of self-reported assessment of shift work varied depending on the work schedule.
Self-reported assessment of shift work with night shifts and permanent night work had the highest validity and shift work without night shifts the lowest.
Exposure misclassification in self-reported shift work assessment can contribute to bias in epidemiological studies, emphasising the need for confirmatory studies using objective data on shift work.
Introduction
Observational studies suggest an association between shift work and various health problems, including circadian dysrhythmia, fatigue and insufficient recovery,1 ,2 breast cancer,3 cardiovascular disorders4 and type-II diabetes.5 Survey studies on shift work and health are sensitive for exposure misclassification.2 ,6 Changes between shift systems that are frequent in hospital work,7 can contribute to additional bias if earlier exposure to shift work is not known. Exposure assessment should preferably be based on repeated interviews or objective registry data.8 Nonetheless, self-reporting of work schedules is commonly used in studies on shift work and health.2–5 ,9 The aim of the current study was therefore to evaluate the validity of responses to widely used questions on employees' current work schedule, by comparing them to objective data.
Methods
We matched the questionnaire data on shift working to pay-roll-based registry data on work schedules in a study of Finnish hospital workers. The registry data were retrieved from years 2008, 2012 and 2014 corresponding to repeated questionnaire surveys of employees with a current work contract data (response rates 72%, 71% and 67%). Physicians (on-call work without shift work) were excluded. The included participants (n=8896 in 2014, 92% women) had at least 31 work shifts during the preceding 3 months of the questionnaire survey. The Finnish Public Sector Study has been approved by the Ethics Committee of the Hospital District of Helsinki and Uusimaa.
Work schedule was requested with the following questions: ‘What is your usual work schedule?’ (response alternatives: 1=regular day work; 2=shift work without night shifts; 3=shift work with night shifts; 4=regular night work and 5=other irregular work) and ‘How long have you been in shift work altogether?’ (response in years). Fatigue during leisure time, as a measure of insufficient recovery typical to shift work, was asked during the preceding 4 weeks (1=‘not at all’ to 6=‘every day’, as in Jenkins et al10).
The register-based assessment of work schedule was based on the average number of the daily morning (M), evening (E) and night (N) shifts during the 3 months preceding the survey. In the analysis of the validity of the survey questions, we used the following definitions for objectively assessed work schedules: day work: ≥1 M, <1 E and <1 N shifts/month, shift work (SW) without N shifts; ≥1 M, ≥1 E and <1 N shifts/month; SW with N shifts: ≥1 M, ≥1 E and ≥1 N shifts/month; Night work: <1 M, <1 E and ≥1 N shifts/month. The used method to retrieve the pay-roll-based registry data of working hours has been described before.8 N shifts were defined as ≥3 hours between 23:00 and 06:00 hours; E shifts as shifts with any time between 18:00 and 23:00 and not categorised to an N shift; M shifts as work starting not before 03:00 and ending no later than 18:00.
We computed sensitivity and specificity with 95% CIs and Cohen's κ coefficient values for the analysis of agreement of the two work schedule assessment methods. Differences in predictive validity were examined by studying the effect of shift work over time (as defined by the questionnaire or the registry data during each of the years 2008, 2012 and 2014) with fatigue in 2014 using logistic regression adjusted for age, sex and fatigue at baseline (2008). We included in the analyses only those participants who had the same work schedule in all three questionnaire surveys from 2008 to 2014.
Results
On the basis of the 2014 survey data, 2517 (55%) of the regular day workers reported to have been at least 1 year in shift work before. According to the registry-based cohort 2008–2014 (n=6038), 1247 (22%) changed work schedule between 2008 and 2014. In 2014, 8% of the shift workers with night shifts and 35% of the shift workers without night shifts in 2008 were in day work. Among the day workers in 2008, 2% moved to shift work with night shifts and 5% to shift work without night shifts by 2014.
The agreement between self-defined work schedule (questionnaire) and objectively assessed work schedule (pay-roll-based registry data) in 2014 is shown in table 1. Using registry data as the reference, the sensitivity (the proportion of true shift/night workers that are correctly identified) was 96% (95% CI 95% to 97%) for questions on ‘shift work with night shifts’ and 90% (95% CI 84% to 95%) for questions on ‘permanent night work’. The corresponding figures for specificity (the proportion of true non-shift/non-night workers that are correctly identified) were 92% (95% CI 92% to 93%) and 97% (95% CI 99% to 100%), respectively. Self-reported ‘regular day work’ showed lower sensitivity (73%, 95% CI 72% to 74%), but high specificity (99%, 95% CI 98% to 99%). The sensitivity and specificity of ‘shift work without night shifts’ was 62% (95% CI 59% to 65%) and 87% (95% CI 87% to 88%). The Cohen's κ coefficients indicated good agreement for day work (0.71), shift work with night shifts (0.86) and permanent night work (0.81), but only moderate/weak agreement for shift work without night shifts (0.42).
The association of self-defined work schedule (questionnaire) and objectively assessed work schedule (pay-roll-based registry data)
When precise registry-based exposure was used, shift work without night work (as exposed to all 3 times from 2008 to 2014), compared with permanent day work, was associated with increased fatigue during free time controlling for age, gender and fatigue at baseline (at least 3 M and E shifts/month: OR=1.89, 95% CI 1.06% to 3.39%, n=1627). This was not the case based on the self-reported shift system (OR=1.30, 95% CI 0.94 to 1.82, n=1707). Shift work with night shifts from 2008 to 2014 was associated with increased fatigue during free time in 2014 when both the registry-based (at least 3 M, E and N shifts/month: OR=1.82, 95% CI 1.28 to 2.58, n=1 804) and self-reported (OR=2.04, 95% CI 1.62 to 2.57, n=2 311) measures of shift work exposure were used.
Discussion
The validity of the self-reported assessment of shift work varied depending on the work schedule. While the self-reported permanent day and night work, as well as shift work with night work, were reasonably valid, the self-reported ‘shift work without night work’ was imprecise, resulting in bias towards the null when analysing the association with fatigue during free time.
The low sensitivity of the self-reported assessment of shift work without night shifts was probably due to the low number of evening shifts, as defined based on the objective data during the preceding 3 months. Having thus only few evening shifts (less than 3 during three months) in addition to morning shifts resulted in employees to be classified as day workers. An earlier survey-based study comparing self-defined and clock-derived work shifts showed that over a third of the objective (‘clock-defined’ survey questions) evening shift workers defined themselves as day workers in another question.11 However, we are not aware of earlier studies that have validated self-reported exposure to shift work with objective registry data.
On the basis of the current results of hospital workers, more than half of the day workers were exposed to shift work earlier in their career. From 8% to 35% of shift workers, depending on their type of shift work, changed to day work during 6 years. This suggests a remarkable selection, and a possibility for a ‘healthy worker effect’12 where shift workers with health problems or work–family conflict tend to change to day work. Our result is in agreement with an earlier case–control study of Danish nurses where the majority of both the cases and controls had earlier exposure to rotating night shift work,7 although that study did not provide information on the length of shift work exposure. The association between cardiovascular risk factors and staff turnover has been similar for both the day and shift workers,13 but leaving night or rotating shift work was associated with a decrease of sleepiness and insomnia among nurses.1 Changes in shift work schedule, as shown by this study, will contribute to the misclassification of the unexposed group if earlier exposure to shift work is not known.
In conclusion, our findings suggest low validity for self-reported assessment of shift work without night shifts, and substantial changes in work schedules over time. There is a need for an objective assessment of exposure to shift work in future studies. To increase the validity of questionnaire-based assessment of work schedules, additional information, such as average number of different type of shifts in a month, should be requested from the respondent. Only accurate information on shift work exposure, including also data on the intensity of different shifts, makes it possible to give evidence-based practical recommendations on the preference of different shift systems.
References
Footnotes
Funding This work was supported by a grant from the Finnish Work Environment Fund (number 114317) and NordForsk (74809). MK is supported by NordForsk (75021) and the Finnish Work Environment Fund (number 115421).
Competing interests None declared.
Ethics approval Ethics Committee of the Hospital District of Helsinki and Uusimaa.
Provenance and peer review Not commissioned; externally peer reviewed.