Aims: To investigate whether self reporting of psychological demands and control at work is as valid for psychologically distressed subjects as for subjects with psychological wellbeing.
Method: Self reported demands and control (according to the model of Karasek) were compared to expert assessments through direct observations of each subject’s work conditions concerning time pressure, hindrances, qualification for work tasks, and possibility of having influence. The comparison was made between respondents reporting and not reporting psychological distress as measured by the general health questionnaire with 12 questions (GHQ-12). The sample consisted of 203 men and women in 85 occupations.
Result: No systematic differences between self reported and externally assessed working conditions for respondents reporting different levels of psychological distress were found.
Conclusion: Over-reporting of work demands or under-reporting of work control is unlikely at the levels of psychological distress studied.
- demand-control model
- differential misclassification
- job analysis
Statistics from Altmetric.com
One problem with self reporting by questionnaire might be that workers with poor psychological wellbeing report more job stress than workers with good health at the same objective exposure levels. It is known from cognitive psychology that recollection of material is more efficient when it is congruent with current mood. For the methods used to investigate psychosocial factors in the causes of depression, this must be acknowledged as a fundamental issue.1,2 To avoid bias, methods that are more objective than self reports have been recommended.3 One is to assign to all workers with the same job title the average value of the self reported levels of the particular stressor. This method, however, creates another problem. Most occupational titles cover a broad range of specific jobs with different psychosocial working conditions. By using crude titles, the actual associations between job characteristics and health endpoints will be underestimated.3–6
One method of testing bias in risk factor studies relies on direct observation of the exposure. External assessments through observations are also based on the observer’s subjective appraisal of the work environment. Although these are not necessarily more accurate than incumbent measures, they are objective in that they are less influenced by the subject’s cognitive and emotional processing. Thus, there should be less measurement bias relating to individual psychological and behavioural conditions.4,5
The aim of this study was to investigate whether self reports of psychological demands and control at work are as valid for psychologically distressed subjects as for subjects with psychological wellbeing. This would be indicated by a lower concordance between self reported and externally assessed conditions for subjects reporting psychological distress than for subjects reporting psychological wellbeing.
The present study was performed within the MOA study: modern working and living conditions for women and men, which is aimed at developing and testing methods for epidemiological studies.7 In the MOA study, working conditions were reported by the subjects themselves by means of questionnaires, and also thoroughly assessed by the researchers through direct observation of each subject’s work.
Eighty work sites in five counties in Sweden were chosen for the study (for details see Härenstam et al).8 The work sites included traditional places of work and work sites in which new forms of organisation and production were being implemented, both in the public and in the private sector. The major aim with the selection of these work sites was to obtain as large a variation in working conditions as possible. The second principle was to achieve a distribution of such important characteristics as class, sex, ethnicity, age, and family situation as well as job contract, qualification level, and type of work object in occupation. The third principle was to achieve a gender matched sample. Of approximately 8000 employees at the selected work sites, 203 individuals (both women and men) were chosen who, according to data on the personnel structure at each work site, were typical representatives of staff. Dropout rates of work sites and individuals were 30% and 1% respectively. However, as the selection was performed successively, new work sites and individuals, similar to those selected initially, substituted the dropouts. The sample selected covered 85 occupational titles. Seventy five per cent of the study group were matched pairwise (woman-man) by type of work (working with people, things or data, according to Kohn and Schooler)9 and qualification level (according to the socioeconomic classification). Whenever possible the matched pairs were chosen at the same work sites. The remaining 25 per cent were chosen from within gender segregated work sites and occupations.
The aim of the job analysis was to describe each individual’s work as objectively as possible from a psychosocial perspective. Action theory integrates cognitive viewpoints and holds that human beings learn and develop through action.10,11 The job analysis instrument was based on action theory and another established observational instrument developed for industrial work.12 This instrument was adapted to be applicable to all types of occupation, and was used for interviews in the MUSIC-Norrtälje study.13
Previous studies have suggested and indicated bias in self reported working conditions because of psychological distress.
This question has been studied by comparing comprehensive observations of individual working conditions with self reported psychological demands and job control among those with and without self reported psychological distress.
Results indicate that the risk of over-reporting high demands and low control at work is low at commonly occurring levels of psychological distress.
Four well trained observers, who were following each study person for, usually, one typical workday, conducted the job analyses. A description of the organisation and interviews with supervisors provided information about the organisational context for the work to be assessed. Interviews with the subjects were conducted in order to gain knowledge concerning mental tasks (that is, work tasks that were not observable) as well as knowledge of the work on days other than the observation day. The group of observers had frequent meetings in which experiences, difficult estimations, and individual scores were discussed. This resulted in a common frame of reference for the group and also meant that each observer was familiar with all subjects. This should have minimised the differences in assessment criteria when applied to different types of occupation.
The work assignment comprised each subject’s work tasks and the proportion of time taken by each. Each work task was classified according to its qualification requirements into three categories: creativity or solving new problems, active use of occupational skills, and routine work or low mental demands. In this study, only the amount of creativity was used and divided into four classes according to proportion of the total work time (0%, 1–9%, 10–19%, and ⩾20%). Inter-rater reliability was as described by Waldenström and colleagues.13
Self reports of psychological demands and job control at work are as valid for psychologically distressed subjects (at commonly occurring level of distress) as for subjects with psychological wellbeing.
This is of importance not only in cross sectional studies but also in studies with several data collections over time, since in such studies each data collection will usually collect cross sectional information.
Several aspects of possibility of having influence were considered. To achieve a dimension that would be comparable from one type of occupation to another, possibility of having influence was categorised into four levels. Each level included ability to decide on work content and form—that is, to decide work tasks included in the work assignment as well as influence regarding how and when work tasks were conducted. (1) The lowest level did not allow the subject to choose between work tasks or how, where, and when they should be conducted; schedule, breaks, and pace were set and the work was fixed not only in time but also regarding place. (2) The second level was much like the first, but the subject had the opportunity to change some work tasks with workmates or to change the order of some work tasks. (3) The third level implied authority to choose between work tasks and to make decisions on how and when the work tasks should be done (at least during some part of the day). (4) The highest level implied authority to decide also on what to do—that is, which work tasks were included in the work assignment. The subject had control from a long term perspective, but not always over the short term schedule.
The quantitative demands in work were described by time pressure—that is, whether enough time was provided to conduct the work tasks. Time pressure was measured by assessing how long the subject could leave the work task without causing delay. If the work tasks could not be interrupted for more than agreed pauses, the work was considered to entail high time pressure. If time pressure varied according to day of the week or if only some of the work tasks were conducted with time pressure, the category variable/moderate was used. If the work tasks could be performed in 80% of the work time paid for, then time pressure was defined as being low.
The dimension of hindrances covered such aspects as unclear goals or work tasks, insufficient resources in terms of equipment, housing, and personnel, lack of support from supervisors or workmates necessary for good job performance, work tasks poorly adjusted to the competence of the subject, or hindrances outside the work organisation. The criteria for assessing hindrances were obvious loss of quality, considerable delay resulting in overtime work, work without breaks, or work done with an apparent risk of accident or illness. Hindrances were assessed as being at one of three levels; none, some, or severe.
Two indices of externally assessed conditions
In this study, three levels of time pressure were used together with three levels of hindrance, in an index measuring externally assessed demands. To calculate the relation between individual items on the scale, the correlation value gamma was used because the two items contained ordered values. The two items of demand correlated moderately (0.474).
The variable creativity, together with the variable possibility of having influence, constitutes the index of externally assessed control. The two items of control showed a moderate correlation (0.576).
Both for the demand and the control indices, the scale of the externally assessed index was adjusted to the range of the self reported index by multiplication. For the demand index, with five values varying originally from 2 to 6, multiplication by 3 gave the range 6–18. For the control index, with seven values varying originally from 2 to 8, multiplication by 3.5 gave the range 7–28.
There was a significant correlation (Pearson) in this study between externally assessed demands and the index of psychological distress (0.164). The external index for control was not associated with psychological distress (−0.005).
Self reported working conditions
Perceptions of working conditions according to Karasek’s demand-control model were obtained from the job content questionnaire, which included job demands (the demand dimension), skill discretion, and decision authority (the control dimension).16 The demand-control model generates a 2×2 table with high and low levels of psychological demands and decision latitude based on four quadrants with different kinds of psychological work characteristics. In this study, a Swedish version was used which included psychological demands (five items), skill utilisation (four items), and authority over decisions (two items). Four response categories for each question were available. The coefficient alpha for the index of demands (five items) was 0.760, and for the index of control (six items), 0.755. The mean value for the self reported demand index of 190 subjects was 13.2, with a range from 6 to 20, the 25th centile being at 11, the median at 13, and the 75th centile at 15. The mean value for the self reported control index of 195 subjects was 17.9, with a range from 10 to 24, the 25th centile being at 15, the median at 18, and the 75th centile at 20. There were no significant differences between men and women. These figures were very similar to those involving more than 1300 referents in a case-control study of musculoskeletal disorders.15 In the present study, there was a significant correlation (Pearson) between self-reported demands and the index of psychological distress (0.165). The self reported index for control was not associated with psychological distress (−0.096).
There were no missing data in the external assessments. Those subjects with missing self reports on demands (n = 13) had, according to the external assessment, clearer goals/work tasks than subjects with complete data. Subjects with missing self reports on the control index (n = 8) had more monotonous work tasks, according to the external assessment, than subjects with complete data.
The General Health Questionnaire with 12 questions (GHQ-12) measures psychiatric symptoms.16,17 Each response alternative contributes 1–4 points from each question to a summary index. The 12 items of the GHQ-12 were dichotomised and summed up. Among 202 subjects the mean value was 20.9, the median 20, and the range 14–39. Three or more symptoms were taken to indicate psychological distress. Eleven per cent of the men and 21% of the women had three or more symptoms and were regarded as “psychologically distressed”.
To determine whether differential misclassification was present, psychologically distressed subjects were compared to subjects with less than three symptoms concerning the relation between self reported and externally assessed working conditions. Since the external data were more ordered than continuous, the non-parametric Spearman rank correlation coefficients (rs) with 95% confidence intervals were calculated. The analyses were performed with SPSS for Windows, version 10.0.
Self reported demands showed a similar association with externally assessed demands among both non-psychologically distressed and psychologically distressed individuals (fig 1, table 1). The same applied to the relation between self reported control and externally assessed control (fig 2, table 1).
Concerning the association between self reports and external assessments, the correlation coefficients (rs) for the two dimensions (skill discretion and decision authority, separately) in the control index did not differ from the coefficient for the summary control index. For skill discretion, the rs was 0.504 for non-distressed and 0.675 for distressed subjects. For decision authority, rs was 0.576 for non-distressed and 0.704 for distressed subjects.
In this study, the concordance between self reported and externally assessed demands and control did not differ between psychologically distressed and non-psychologically distressed subjects.
Alternative methods of analysis
The choice of statistical method for the analysis was not obvious. There is a problem with construction of indices from both the external and the self assessed variables, since the distance between each step in the scale is unknown. To examine whether the results were stable over different methods, analyses were also made on single items. The results did not support any clear differences in prevalence of reported working conditions between subjects with psychological wellbeing and subjects with psychological distress.
The observations and the self reports in this study had in part different theoretical roots and were intended to measure partly different aspects of the same dimension. The criteria for the external measurements in this study were settled by the researchers and were the constant for all subjects. Self reports have a larger span of significance because each respondent can make a judgement as to what the question refers to. Hence, assessments from the two perspectives differ in the specificity of the exposure. The strength of the external assessments was that the observers had no knowledge of the subject’s self reported working conditions or any health outcomes.
External assessments most likely suffer from too little differentiation due to low observability, but approaches using observational and self report measures in parallel may overcome the limitation of each approach and allow analysis of the interplay between individual and work factors.18,19 However, one advantage with this study is that weaknesses in the external data should be the same for subjects with psychological distress and subjects without.
The threshold in GHQ-12 defining the group of psychological distressed subjects was not obvious, so the calculation was also made for a larger group of psychologically distressed subjects, who were defined as having two or more psychiatric symptoms. The results did not differ from those presented.
One measure of reliability in the external assessments would be similar correlations to self assessments for each observer. The associations between self reported and externally assessed demands (as in figs 1 and 2) were very similar for the different observers. With regard to control, the results were also similar for all observers.
One might expect a healthier group with better working conditions than in the general population because the workplaces and subjects had accepted extensive participation in the research project. However, when the study group was compared to a population based sample,10 neither self reported working conditions nor psychological distress or demographic aspects seemed to deviate from corresponding conditions in the general, employed, population.20 When assessing the external validity of the study, the main consideration would not be the general representativeness of the study, but rather whether there is sufficient contrast in exposure conditions as well as in psychological distress. This is since there is little reason to suspect that the study participants would react very differently from non-participants or the general population to the exposures under the study. In this regard the comprehensive observations for each subject limited the sample size and caused a somewhat limited contrast in psychological distress. Another possible limitation was the rather weak association between psychological distress and self reported demands and the lack of association between psychological distress and control. Other researchers have found an association between demands and also control and distress measured by GHQ.21–25 However, we have previously made similar findings in other research material—that is, that GHQ-12 is related to demands but not to control.26 Our conclusion is that the sample studied is similar to a representative sample from the general employed population. The results should be valid for the ranges of exposure conditions and psychological distress studied. A larger sample would have extended the external validity, particularly to more severe forms of psychological distress.
Comparison with previous studies
Studies indicating information bias in self reported working conditions, by measuring stressors independently of the exposed person, have investigated several outcomes such as perceived stress, absence due to sickness, and coronary heart disease (see examples in Kristensen6). These studies also used different methods to assess exposure, such as assigning the average value of self reported exposure in an occupation to each member of the occupational group, but no such study has assessed exposure by direct observation.
Several studies have controlled for possible over-reporting in cross sectional studies when only self reported data on psychosocial factors at work are available, by adjusting for negative affectivity (NA).27 However, controlling for affectivity may in fact reduce true variance (compare this with Spector et al).28
In a prospective study design (lasting one month), Daniels and Guppy studied the potential effect of psychological wellbeing, as measured by the GHQ-12, on three aspects of the stress process.29 They used self reported data mainly from men in one occupational group. There was a marginally significant association (r = 0.09, p < 0.06) between individuals with clinical or near clinical levels of psychological symptoms (first measurement) and subsequent (second measurement) reports of intensity of quantitative workload compared to non-distressed persons (controlled for levels of stressors at first measurement). Our study, with externally assessed data for each individual in several occupations, does not support an association regarding quantitative workload.
The advantage of the design of our study is that it takes into account both actual changes in working conditions due to a dynamic reciprocal stress process, and over-reporting due to psychological distress. If conditions actually change there will be high concordance, and if there is over-reporting there will be low concordance between self-report and externally assessed working conditions among psychologically distressed subjects.
The results indicate that self reports of psychological demands and job control at work are as valid for psychologically distressed subjects as for subjects with psychological wellbeing when psychological distress is measured by at least two or three symptoms in the GHQ-12. Accordingly, the risk of over-estimating the association between demands and control at work and psychological distress should be low in cross sectional studies using the GHQ-12 as the outcome.
This study was supported by the Swedish Council for Work Life Research and the Department of Occupational and Environmental Health, Stockholm County Council. The authors are grateful to Gun Johansson and Anna Rydbeck for skilful data collection
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.