Aims: To (1) evaluate the psychometric properties and (2) examine the ability to detect cases with anxiety disorder and depression in a population of employees absent from work because of mental health problems.
Methods: Internal consistency, construct validity, and criterion validity of the Depression Anxiety Stress Scales (DASS) were assessed. Furthermore, the ability to identify anxiety disorders or depression was evaluated by calculating posterior probabilities of these disorders following positive and negative test results for different cut off scores of the DASS-Depression and DASS-Anxiety subscales.
Results: Internal consistency of the DASS subscales was high, with Cronbach’s alphas of 0.94, 0.88, and 0.93 for depression, anxiety, and stress respectively. Factor analysis revealed a three factor solution, which corresponded well with the three subscales of the DASS. Construct validity was further supported by moderately high correlations of the DASS with indices of convergent validity (0.65 and 0.75), and lower correlations of the DASS with indices of divergent validity (range −0.22 to 0.07). Support for criterion validity was provided by a statistically significant difference in DASS scores between two diagnostic groups. A cut off score of 5 for anxiety and 12 for depression is recommended. The DASS showed probabilities of anxiety and depression after a negative test result of 0.05 and 0.06 respectively. Probabilities of 0.29 for anxiety disorder and 0.33 for depression after a positive test result reflect relatively low specificity of the DASS.
Conclusion: The psychometric properties of the DASS are suitable for use in an occupational health care setting. The DASS can be helpful in ruling out anxiety disorder and depression in employees with mental health problems.
- CIDI, Composite International Diagnostic Interview
- DASS, Depression Anxiety Stress Scales
Statistics from Altmetric.com
Occupational physicians in the Netherlands spend much of their time advising sick employees about return to work. Ideally, this management of the return to work process will consist of a diagnostic process and several interventions including the drawing up of a return to work plan.1 Approximately 30% of employees seen by their occupational physician are absent from work because of mental health problems.1 These problems encompass both common mental disorders, such as stress symptoms, as well as psychiatric disorders. In terms of the DSM-IV classification, the majority of these employees are suffering from the more common adjustment disorders, while a smaller yet substantial proportion suffers from depression or an anxiety disorder.2 In DSM-IV, an adjustment disorder diagnosis is not allowed when the severity and duration threshold for another disorder is reached.3
Anxiety disorders and depression are considered to be more severe disorders than adjustment disorders.4 The International Consensus Group on Depression and Anxiety underlines the necessity of treatment with antidepressant medication for patients suffering from either an anxiety disorder or depression.5–7 Furthermore, the level of depressive symptoms is related to the level of work impairment, with severe impairment when the threshold for depressive disorder is reached.8 However, work impairment decreases significantly as depression is treated.9,10 Compared to patients with anxiety disorders or depression, patients with adjustment disorders require less treatment and are able to return to work sooner.11
The psychometric properties of the DASS are suitable for use in a population of employees absent from work because of mental health problems.
The DASS can be helpful in ruling out cases with an anxiety disorder or depression in a population of employees with mental health problems.
The growing attention for the recognition of employees with an anxiety disorder or depression is not restricted to occupational health care alone. In primary care, recognition of anxiety disorders and depression by general practitioners is considered an important condition to ensure accurate treatment of these disorders. However, research12 has shown that the ability of general practitioners to recognise mental disorders is rather poor. Most studies find detection rates varying from 30 to 40%.13–15 Three factors may have contributed to such poor recognition rates. Firstly, recognition of depression and anxiety disorders is impeded by the presentation of multiple badly defined complaints by primary care patients.12,13,16 As a result, general practitioners tend to recognise only the more severe cases.17 In contrast, in- and outpatient psychiatric clinics treat only those patients referred to them by general practitioners. Therefore, psychiatric populations consist of patients with more severe and well defined symptoms than primary care populations. Secondly, the limited time available to general practitioners for assessment is deemed to contribute to low detection rates of depression and anxiety disorders.18 The third reason for low recognition rates may be that general practitioners lack comprehensive diagnostic knowledge concerning psychopathology.19
The DASS can be used to assist occupational physicians in a two-phase diagnostic process.
The DASS may be administered to all employees absent from work because of mental health problems (phase 1).
Occupational physicians should conduct a clinical interview with all employees who are identified by the DASS as possible cases of anxiety disorder or depression (phase 2).
It is likely that the same factors that impede the recognition of anxiety disorders and depression apply to occupational physicians since they see all employees who have been on sick leave for longer than about two weeks and there is no referral from another health care professional. Moreover, an average consultation with an occupational physician lasts approximately 20 minutes, which is not substantially longer than the time spent by general practitioners. Finally, occupational physicians do not receive any more training in diagnosing psychopathology than general practitioners do.
Considering these problems in identifying anxiety disorders and depression, it follows that a self administered instrument for case finding might prove helpful to occupational physicians. Such an instrument could be filled out by the employee prior to the consultation, thereby assisting occupational physicians in identifying employees with anxiety disorders and depression. One condition for implementation would be that this instrument is able to identify high risk cases within a group of patients similar to a primary care population. However, many of the validated instruments for use in primary health care either aim at reaching one specific diagnosis (for example, Goldberg screen for depression20) or are elaborate diagnostic instruments requiring specific training (for example, CIDI-PC,21 PRIME-MD22).
The Depression Anxiety Stress Scales (DASS)23 would seem to be a promising instrument for use in occupational health care. Theoretically, this instrument corresponds with the tripartite model of anxiety and depression.24 This model states that anxiety and depression possess unique features as well as common ones. Depression is uniquely characterised by low positive affect and anhedonia, while anxiety has physiological hyperarousal as a unique feature. Depression and anxiety have a non-specific factor of general distress in common. This general distress includes symptoms such as irritability and nervous tension, which are comparable to the symptoms reported by employees with adjustment disorders.25 Therefore, the structure of the DASS seems to support the view that both anxiety disorders and depression need to be distinguished from adjustment disorders in spite of their communality.
The psychometric properties of this instrument appear to be sound enough to be applied to both healthy and psychiatric populations. For these populations, the three factor solution has been determined by several authors.26–30 Internal consistency of the three subscales ranged from 0.81 to 0.97.27–29 Moreover, convergent and divergent validity have been shown to be satisfactory in these studies.26,27,29,30 However, the DASS has not yet been studied in either a primary care or an occupational health care population. The aim of this study is therefore to evaluate the psychometric properties of the DASS in an occupational health care population. This study examines internal consistency, construct validity, and criterion validity of the DASS. A further aim of this study is to evaluate its ability to identify cases with an anxiety disorder or depression in this population.
As part of a longitudinal study on determinants of recovery and return to work in employees with mental health problems, 30 occupational physicians from nine occupational health services provided data on patients seen over consecutive periods of one or more days a week. The following inclusion criteria needed to be met: the previous consultation with their occupational physician was longer ago than three months; a 100% absence from work; sickness absence because of mental health problems, defined as suffering from psychological symptoms that were not caused by a somatic disorder; and onset of sickness absence no longer than six weeks previously.
From March 2001 until February 2002, data on 326 employees with mental health problems were reported to us by the occupational physicians. Of these 326 employees, 32 were excluded because they did not meet the inclusion criteria. Another 17 were excluded because they were unable to read Dutch (n = 2), were fully recovered (n = 5), were to be treated by another occupational physician (n = 6), were unable to fill out the questionnaire because of severe psychiatric problems (n = 2), or could not be contacted by telephone (n = 2).
Of the remaining 277 patients eligible to participate in the study, 66 (24%) refused to participate. Of all 211 employees who signed an informed consent form, 198 filled out the questionnaire. Of these 198 employees included in this study, 192 were interviewed. In the other six cases, the interview was not conducted because the participant could not be contacted by telephone.
All participants were asked by their occupational physician to participate in the study. Each participant, after first having signed an informed consent form, was interviewed by the researchers via the telephone. Subsequently, questionnaires were sent to the participants by mail.
Participants were diagnosed by means of a short telephone version of the structured Composite International Diagnostic Interview (CIDI).31 An interview by telephone was used because of its convenience and its comparability with face to face interviews.32–34 The telephone interview included the following diagnostic groups: major depressive disorder, panic disorder, social phobia, somatoform disorder, bipolar disorder, obsessive-compulsive disorder, post-traumatic stress disorder, and psychotic disorder. For the first three diagnoses we administered the full CIDI scales, while we used only the stem (or screen) questions for the other categories. It was allowed for a participant to meet the criteria for more than one diagnosis. Anxiety disorder was operationalised as meeting the criteria for one or more of the following disorders: panic disorder, social phobia, somatoform disorder, obsessive-compulsive disorder, or post-traumatic stress disorder. Depression was operationalised as meeting the criteria for major depressive disorder. All interviews were conducted or supervised by a mental health professional. All interviews were tightly scripted, including the use of standardised introductory statements. The length of the telephone interview varied from 15 to 20 minutes.
Following the diagnostic interview, patients completed a self report questionnaire that comprised the DASS-42, the Hospital Anxiety and Depression Scale (HADS),35 and the Utrecht Coping List (UCL).36 Participants respectively take 3, 7, and 10 minutes to complete the HADS, DASS, and the UCL. The DASS-42 consists of 42 symptoms divided into three subscales of 14 items: depression scale, anxiety scale, and stress scale. Participants rated the extent to which they had experienced each symptom over the previous week on a four point Likert scale ranging from 0 (did not apply to me at all) to 3 (applied to me very much, or most of the time).
In order to assess concurrent validity of the DASS, participants also completed the HADS.35 The HADS is a 14 item screening scale that measures the presence of anxiety and depressive states. It contains two seven-item subscales: a depression subscale and an anxiety subscale, each item being scored on a four point scale (0–3). The HADS has been developed as a screen for detecting depressive and anxiety disorders in hospitalised patients. Items referring to symptoms that may have a physical cause (for example, weight loss or insomnia) are not included in the scale.
To measure divergent validity, participants also filled out the UCL.36 The UCL is a Dutch questionnaire that measures habitual coping styles. This questionnaire consists of 47 statements concerning ways of coping with problems. The UCL comprises seven subscales measuring seven coping styles. Correlations between the DASS subscales and the subscales of “active problem solving” (UCL-Active), “seeking social support” (UCL-Social Sup), and “comforting cognitions” (UCL-Comf) served as indices for divergent validity in the analysis.
Cronbach’s alphas were calculated for each of the DASS subscales in order to evaluate the internal consistency.
To examine construct validity of the DASS, exploratory factor analyses were performed first. A principal component extraction was used, after which the number of factors was determined by both eigenvalues (>1) and the Scree test.37 We applied a varimax rotation on this initial solution. To further examine construct validity, a correlational (Pearson’s) analysis of convergent and divergent validity was conducted by correlating the subscales of each of the three questionnaires. It was hypothesised that DASS-Depression would be moderately correlated to HADS-Anxiety and highly correlated to HADS-Depression. Furthermore, a high correlation between DASS-Anxiety and HADS-Anxiety was expected, while a moderate correlation between DASS-Anxiety and HADS-Depression was hypothesised. DASS-Stress was expected to correlate moderately high with both HADS subscales. It was hypothesised that all three DASS subscales would show low correlations with the UCL subscales.
In order to test concurrent validity of the DASS, a multivariate one way analysis of variance (MANOVA) was conducted. Employees were split into two diagnostic groups: one group with members suffering from an adjustment disorder and the other group with members suffering from a depression or anxiety disorder, as assessed by the CIDI interview. Employees with other disorders were excluded from this analysis. The analysis was conducted with group as between-subject factor and the DASS-subscales as within-subjects factor. Tukey post hoc analyses were carried out in order to test differences for each of the subscales.
Subsequently, we evaluated the ability to identify cases for the depression and anxiety subscales by using the CIDI interview as the gold standard. Sensitivity was determined by calculating the proportion of cases based on the DASS subscale among the cases according to the CIDI interview. Specificity was defined as the proportion of non-cases according to the DASS subscale among non-cases according to the CIDI. Furthermore, we calculated likelihood ratios for positive and negative test results.38 Finally, we assessed posterior probabilities of the disorders following a positive test result (positive predictive value) or a negative test result (complement of negative predictive value) in our population for a range of cut off values.
Considering the need for treatment of patients with an anxiety disorder or depression, false negative cases were regarded as more undesirable than false positive cases. Therefore, we considered a negative likelihood ratio of 0.19 to be sufficient, which is comparable to the negative likelihood ratios found in a review of validated instruments for detecting depression.39
Differences were tested at a significance level of p < 0.05. All data were analysed using the SPSS 10.0 software package.
Table 1 presents the characteristics of the participants. Of the 66 employees who refused to participate and the 49 that were excluded, 26 were male and 58 were female; the gender of 31 was unknown. With respect to this variable, no significant difference (t test, p = 0.21) between the participants and non-participants was observed.
Internal consistency of the DASS subscales was found to be high, with Cronbach’s alphas of 0.94, 0.88, and 0.93 for depression, anxiety, and stress subscales respectively.
The three factor solution accounted for 53% of all variance, with eigenvalues of 16.2, 3.3, and 2.8. Table 2 shows factor loadings for the 42 items. The first factor that emerged consisted of all items from the depression scale plus one item (item 22) from the stress scale. The range of factor loadings (after varimax rotation) was 0.44 to 0.82. None of these items loaded higher than 0.40 on another factor. The second factor comprised 12 items from the stress scale plus one item from the anxiety scale (item 19), with eigenvalues ranging from 0.38 to 0.83. Of these 13 items, one item (item 8) also loaded high (>0.40) on the depression factor and one item (item 39) loaded high (>0.40) on the anxiety factor. The final factor corresponded fairly well with the anxiety scale, with eigenvalues ranging from 0.39 to 0.78. All items from the anxiety scale, except item 19, loaded highest on this anxiety factor, while one item (item 33) from the stress scale loaded higher on this factor than on the stress factor (0.57 versus 0.41 respectively).
Convergent and divergent validity
Table 3 shows the correlations between the three DASS subscales on the one hand and the indices for convergent and divergent validity on the other. As expected, table 3 reveals high correlations between both DASS-Anxiety and HADS-Anxiety (r = 0.66) as well as between DASS-Depression and HADS-Depression (r = 0.75). Correlations between the DASS-Stress scale and the HADS scales were moderately high (0.58 and 0.60). This pattern of correlations confirms the hypothesis of good convergent validity. As can be seen from table 3, all three DASS scales showed low correlations (range −0.29 to 0.02) with the UCL subscales, indicating good divergent validity.
Multivariate analysis of variance (MANOVA) revealed a significant overall effect of group (F = 17.25, df = 3.171, p < 0.001). Table 4 presents the mean scores and standard deviations of the DASS subscales for both employees with adjustment disorders and employees with a depression or anxiety disorder. The post hoc analyses showed that employees with a depression or anxiety disorder scored significantly higher on DASS-Depression (p < 0.001), DASS-Anxiety (p < 0.001), and DASS-Stress (p < 0.01) compared to employees with adjustment disorders.
The ability to identify employees with either a depression or an anxiety disorder was assessed for DASS-Depression and DASS-Anxiety respectively. Table 5 shows sensitivity and specificity rates for different cut off scores of these two subscales. Furthermore, table 5 shows both likelihood ratios for positive (LR+) and negative (LR−) test results as well as posterior probabilities of the disorders following positive or negative test results. As can be seen from this table, high sensitivity in identifying anxiety and depressive disorders results in relatively low specificity rates. If LR− is set at >0.19, then, for depression, the LR+ is 1.69, the posterior probability of a depression after a positive test result is 0.33. The posterior probability of a depression after a negative test result is 0.06. Given the same LR−, for anxiety, the LR+ is 1.53, the posterior probability of an anxiety disorder following a positive test result is 0.29, and 0.05 following a negative test result. Corresponding cut off scores would be 12 for DASS-Depression and 5 for DASS-Anxiety.
The results of our study suggest that the psychometric properties of DASS are suited for use in a population of employees who are off work because of mental health problems. Internal consistency of the three DASS scales was high. Furthermore, the factor analysis revealed three factors, which yielded support for the construct validity of the DASS. In addition, all correlations of the DASS with indices of convergent and divergent validity were as predicted. The DASS showed good criterion validity since employees from different diagnostic groups differed in their score on the DASS. Low posterior probabilities of the disorders following a negative test result indicates that the DASS can be helpful in ruling out cases with an anxiety disorder or a depression. However, the relatively low probabilities of the disorders following a positive test result indicates a considerable rate of false positives.
The psychometric properties of the DASS in our population are consistent with DASS studies in other populations. High internal consistency was observed in both student and clinical populations.26,27,29 Moreover, both criterion and construct validity proved to be adequate in previous studies with student, clinical, and community samples.26–30 This indicates that the DASS can provide occupational physicians with detailed information on the level of depression, anxiety, and stress symptoms in their patients. Ours was the first study to evaluate the quality of the DASS in case findings of employees with an anxiety disorder or depression. Recently, Williams and colleagues published a review of case finding studies for depression39 in primary care. They found a median likelihood for a positive test result of 3.3 and a median likelihood for a negative test result of 0.19. Our study revealed similar likelihood ratios for a negative test result, but lower likelihood ratios for a positive test result for both anxiety disorder and depression.
A unique feature of this study is that it evaluates the ability to detect anxiety disorder and depression among employees with mental health problems. Until now, most DASS studies addressed the distinction between subjects with and without an anxiety disorder or depression, while this study addressed the distinction between employees with adjustment disorders and those with an anxiety disorder or depression. This distinction is less clear cut, but is highly relevant for practice, while it has important consequences for the treatment of these employees. The DASS appears to be able to detect anxiety disorders and depression despite the communality in symptoms between these disorders and adjustment disorders.
Caution is required when generalising these results across the entire occupational health care population. This study encompassed a population of employees with mental health problems who were absent from work. This entails that the prevalence of anxiety disorder and depression is substantially higher than in a general occupational health care population. Whereas in primary care settings around the world the prevalence of both major depressive disorder and anxiety disorder was found to be 10% for each disorder,18 this study revealed a prevalence of 21% for anxiety disorder and 23% for depression.
Two methodological aspects of this study require further consideration. The first aspect concerns the use of a structured interview administered by a mental health professional (CIDI interview) as a gold standard. Although a true gold standard for depression and anxiety is not available, the diagnosis of a psychiatrist might have been more accurate than the CIDI interview. In this respect it should be noted that a recent study identified several articles in which semistructured interviews were conducted by mental health professionals in order to diagnose major depressive disorders.39 These studies revealed interrater agreements (Cohen’s kappa) between the semistructured interview and psychiatrists of 0.64 to 0.93, indicating that mental health professionals are capable of diagnosing major depression reliably. A second methodological aspect of the present study concerns possible selection bias as a result of the large number of employees who refused to participate. The reasons for refusing to participate were recorded. Further examination revealed that these reasons were diverse. Reasons for refusing to participate were that employees were “too busy”, “did not feel like it”, “were too tired”, or “were okay now and did not want to be reminded of those bad times”. Selection could also have led to a population with missing extremes. This may have led to an underestimation of the diagnostic qualities of the DASS. Furthermore, the gender ratio of the non-participants did not differ from that of the participants.
An important aim of this study was to evaluate the quality of the DASS as a case finding instrument for use in occupational health care. Treatment with medication and specific psychological treatment has been proven to be effective in treating both anxiety disorders and depression.3 Because adequate treatment is available, we consider high sensitivity the most important asset of the DASS. In line with this preference, a cut off point of 12 for the depression subscale and 5 for the anxiety subscale is recommended. The prevalence of anxiety disorder and depression was 21% and 23% respectively. Given the prevalence, sensitivity, and specificity rates of anxiety disorders and depression, this means that an occupational physician seeing 100 patients per month who are absent from work because of mental health problems, can expect 66 patients to screen positive for anxiety disorder and 63 to screen positive for depression. After a more elaborate and detailed clinical interview with these employees, 19 employees will prove to have an anxiety disorder and 21 will meet the criteria for a depression. Of the employees who screen negative on the DASS, two will still have an anxiety disorder and two will still have a depression. Taking these considerations into account, occupational physicians can be advised to use the DASS in a two phase diagnostic process. First, the DASS should be used to alert the occupational physician to all possible cases of anxiety disorder and depression. The second step would be to conduct an elaborate clinical interview. Occupational physicians can either perform this second step themselves, or refer the employee to a mental health professional for that purpose.
Three aspects of quality assessment of occupational health care instruments can be distinguished.40 The first aspect is technical quality, which addresses issues of reliability and validity of an instrument. The second characteristic of quality can be described as process quality. Features such as acceptability for employees and logistical aspects determine the process quality of an instrument. Finally, the strategic quality of an instrument should be assessed. The presumed utility of the instrument defines this aspect of quality.
The present study established the technical quality of the DASS. The process and strategic quality, however, still remain to be assessed. All employees in this study filled out the DASS, which is a tentative indication that the DASS is user friendly. This assumption should, however, be tested in a less motivated population—that is, not in a population of employees who agreed to participate in a cohort study. Whether the DASS can be implemented in occupational health care depends for a large part on logistic aspects of administering the DASS. One procedure could be that the DASS is administered to all employees with mental health problems prior to the consultation with the occupational physician. The occupational physician is then able to use the information from the DASS as an aid to his own diagnostic process. These aspects of process quality and the utility of the DASS should be addressed in future implementation research. An important question is whether routinely administering the DASS prior to the consultation leads to a more accurate diagnosis of occupational physicians. Furthermore, the effect of the DASS on patient outcome also needs to be evaluated. Recognition of anxiety disorder and depression is a necessary, though not sufficient condition to improve outcome.
In conclusion, the results of the present study suggest that the DASS is a valid instrument for use in occupational health care. It can be helpful in ruling out anxiety disorder and depression in employees with mental health problems. Furthermore, the DASS can be used to select employees in need of a more elaborate and accurate diagnostic process.
Grants were obtained from: The Netherlands Organization for Scientific Research (NWO): Netherlands Concerted Research Action “Fatigue at Work”; and the Foundation for Replacement and Occupational Health in Education (Stichting Vf/BGZ).
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.