Occupational factors are important contributors to adult asthma and rhinitis.1 2 Nonetheless, no single, standardised approach is used to systematically assess relevant work-related exposures in survey-based research. This presents an important methodological challenge for researchers who wish to analyse the work-related burden of respiratory disease. Various approaches have been used in exposure classification. These include stratification by broad exposure groups (eg, blue collar versus white collar job titles),3 use of a single expert or expert panels,4^–6 use of a job-exposure matrix (JEM) based on the likelihood of exposure within certain job/industry categories,7^–10 and self-reported exposure, typically based on structured interview responses.

For self-reported exposure, one of the most common strategies has been to query respondents using a single, dichotomous item ascertaining any work-related exposure to vapours, gas, dust or fumes (VGDF). This is often asked in reference to a long working period or even an entire career.11^–15 Because self-reported exposure to VGDF is a very widely used measure, delineating its performance characteristics in survey research applications is critical. Test-retest consistency is one important performance characteristic. Assessing the validity of the VGDF tool is less straightforward, given that direct industrial hygiene measures of workplace exposures are unlikely to be available, especially in studies spanning multiple work sites over many years of employment. It is possible, however, to assess the specific exposures that may be understood by respondents as subsumed under the broad VGDF rubric, for example, through survey administration of both the single broad item followed by a specific exposure checklist. Moreover, assessing the sensitivity, specificity and sources of misclassification of the VGDF item against a JEM approach based on occupation (sometimes treated as the de facto survey research gold standard in assigning exposure risk) provides multiple measures of the criterion validity for VGDF.

We focused on these methodological questions using longitudinal data that allowed the study of both JEM and VGDF measures of exposure. The analysis presented here includes responses from surveys administered to the same cohort of adults with asthma and chronic rhinitis interviewed twice over an approximately 24-month period. Using these data, we were able to compare reported VGDF exposure to a checklist of specific exposures, evaluate the sensitivity and specificity against exposure likelihood based on JEM assignment, compare test-retest responses to the VGDF item among persons who maintained the same jobs over the study period, and test the association between selected demographic, job and health condition variables against inconsistency in VGDF self-report and discordance between such self-report and JEM assignment.

METHODS

Survey recruitment

The data used in this analysis are derived from questionnaire responses to a survey administered during two series of interviews carried out as part of an ongoing longitudinal study of adult asthma and allergic rhinitis. The survey methodology, including initial subject recruitment, has been reported in detail previously.16^–21 In brief, the study cohort consists of adults with asthma and/or allergic rhinitis, recruited in several phases, between the ages of 18 and 55 at the time of recruitment, living in northern California at the time of recruitment, and having a physician’s diagnosis of either asthma, rhinitis or both. The first recruitment (1993–1994) was carried out through random sampling of pulmonary and allergy subspecialists. This was later supplemented (1996–1997) with similar recruitment through a sample of family medicine specialist physicians. A third group of participants was added through random digit telephone dialing that identified persons reporting a physician’s diagnosis of asthma and/or rhinitis (1999).

Cohort follow-up

All groups were merged into a single cohort and were all surveyed together in 2000–2001 and then at approximately 2-year intervals thereafter. For the current analysis, we used data from two study waves: interviews that took place in 2000–2001 and interviews that took place in 2003–2004. In 2000–2001, in the baseline interview series for this analysis, 548 subjects successfully completed the structured telephone survey using computer-assisted telephone interview (CATI) software. Re-interviews approximately 2 years later (mean (SD) time elapsed: 22.3 (2.7) months) were successfully completed for 416 (76%) of the 548 subjects interviewed 2 years previously.

Employment status

We assessed employment status based on subjects’ responses to standard labour market items. Of the 548 subjects interviewed initially, 408 (74%) were employed at the time of the first survey, but only 287 of these were included among the 416 subjects interviewed a second time. A total of 314 (75%) subjects were currently employed at the time of the second survey, including 27 not employed at baseline. We limited this analysis to the 348 subjects who were employed at the time of either of the interviews.

Other survey data

Subject interviews also included items covering demographics, cigarette smoking, socioeconomic status and health conditions. Dichotomous variables were created for race (white, non-Hispanic versus all others), education (high school graduate or less versus some college or more) and cigarette smoking status (never versus former or current smoker). The subjects were divided into groups by health condition, categorised as asthma and concomitant rhinitis, asthma alone without rhinitis, and chronic rhinitis alone without asthma.

Definitions of exposure

At both interviews, subjects were asked about their current job, type of business, position and usual work activities. These items were asked as open-ended questions with responses noted in text format that could be reviewed later for coding purposes. We used the 2000 Census codes to assign codes for each subject’s occupation and industry.22 The initial assignment of codes was carried out by one of the study staff and was later reviewed by the project’s industrial hygienist (PJQ), based on open-ended job descriptions and reported duties as stated above. This expert review resulted in changes to the originally assigned job codes when indicated.

In addition to occupation, subjects were also asked a general question regarding exposure to VGDF specifically worded as follows: “At your current job, do you come in regular contact with any dusts, fumes, gases, vapours, chemical or pharmaceutical products?”. Only those responding positively to the general exposure question were asked to respond to a checklist of 19 specific exposures. Subjects who denied exposure to VGDF in response to the global item were not questioned about these 19 specific substances, a strategy designed to minimise respondent burden. For those who held the same job at both interviews we used the checklist responses reported at baseline for the analysis. This checklist was developed for this study, with exposures categorised for analysis such that exposure to one or more constitutes exposure to that group. The specific exposures, grouped by type, are: plant dust or pollen; natural rubber or latex; cotton or saw dust; wheat flour or grain dust; hair dander or fur; enzymes (biological materials); paints, dyes or inks; pharmaceuticals; formaldehyde; sealant or shellac; urethane or polyurethane; epoxy or glues; photo chemicals (synthetic organic chemicals); cleaners or bleach; irritant chemicals; combustion products; metal dusts or fumes; refrigerants; and flux or solder (irritant gases or fumes). This selection was based, in part, on a checklist approach used to elicit exposures in a study of chronic obstructive pulmonary disease (COPD).13 23 24 Seven of those items were adapted for this checklist, contributing five separate items of the 19; the remaining 12 were items new to this study. The original checklist had featured dusts and fumes with known or suspected COPD risk, whereas the checklist used in the current study was comprised of substances known to cause or exacerbate asthma or rhinitis, based on the general literature in this field and not targeted for any specific occupations or industries as reported by the cohort at hand.

A total of 150 different 2000 Census occupational codes were assigned to 436 distinct jobs held by the 348 subjects in the analysis. Census code numerical grouping differentiated among major categories of occupations, allowing dichotomisation to professional, managerial, administrative and sales occupations (by the most frequent as a group) as opposed to technicians, service, manufacturing, construction and agriculture. We constructed the JEM by assigning each of the 150 jobs codes present in the dataset a likelihood ranking of exposure, modifying the initial assignment if indicated after review of any narrative job details altering the default assignment. The JEM was adapted from a job-exposure classification system originally used in the European Community Respiratory Survey and then further adapted for use in studies of airway diseases.12 All of the subjects’ jobs were coded as having low, moderate or high likelihood of exposure to sensitising or irritant agents known to cause or exacerbate asthma.

The initial assignment of exposure likelihood based on job code was then followed by review of the narrative (open-ended) job description elicited in the interviews consistent with the methodology of Kennedy.9 Using this methodology, in cases where the job descriptions did not match the initially assigned classification based on job code alone, the exposure likelihood was modified. For example, the occupational code “22” (supervisor) would typically be classified as low exposure likelihood, but if an individual indicated specific exposures in their job description narrative, the exposure JEM was re-assigned as moderate (eg, working as a supervisor onsite in construction areas, a construction trade worker would automatically have been assigned a JEM ranking of moderate exposure likelihood). Another example is the code “290” (audio-video engineer), which would also be classified as low exposure likelihood barring additional information; however, in a case in which the narrative described the direct use of chemicals in repairing equipment, the exposure likelihood would also have been raised to moderate. Such narrative reviews resulted in changes to 61 (14%) of the 431 JEM assignments in this analysis. One key aspect of our JEM approach is that the assignment was not modified by subjects’ responses to the global VGDF item or to any of the specific exposure in the 19-item checklist. For the current analysis, we took a dichotomous approach, combining the moderate and high exposure likelihood assignments into one category and compared that with the jobs where the assignment was rated as low likelihood.

Data analysis

We employed two principal strategies in the data analysis. In the first approach, the unit of analysis was the reported job; in the second, the unit of analysis was the study subject. In the first strategy, each subject could contribute up to two observations (one for the job reported at baseline and one for the job reported at the follow-up interview) if the jobs differed between interviews. There were 436 such distinct jobs among the 348 subjects included in this analysis. Thus, exposure frequencies with a total n = 436 could be calculated for JEM status, each exposure to VGDF, and, among the VGDF exposures, the prevalence of each of the 19 specific materials and their parent groups (biological materials, synthetic organic chemicals, and irritant gases and fumes). In the second analytical strategy, as noted above, the interview respondent was the unit of analysis. Although we ascertained job status and demographics for all 348 subjects included in the study, the subject-based analysis focused on the 199 subjects interviewed at both baseline and follow-up who reported no job change or substantive change in duties (in narrative text) over the interval.

The baseline demographic and health characteristics of the 348 subjects with or without VGDF exposure at baseline were compared. If subjects were not working at the time of the first interview, they were considered part of the non-VGDF group. The χ² test was used to test for differences between these categorical variables. For all 436 jobs, simple κ coefficients and their 95% confidence intervals (CIs) were calculated for concordance between self-reported VGDF and JEM classification. We also calculated the sensitivity and specificity of VGDF self-report against JEM assignment, treating the later as the gold standard. We repeated the κ and sensitivity/specificity calculations testing each of three groups of specific exposures (biological materials, synthetic organic chemicals, and irritant gases and fumes) against the JEM assignment. The simple κ coefficient and 95% CIs for test-retest concordance between baseline and second interview were calculated for the 199 subjects with the same job at both interviews. We used multiple logistic regression analysis to estimate the associations between demographics, health condition and occupational grouping as independent variables and three dependent variables: VGDF discordance in self-report comparing the two interviews; VGDF reported, but JEM assignment categorised the job as being of low exposure likelihood (over-report of VGDF); and VGDF not reported, but JEM assignment categorised the job as being of moderate or high exposure likelihood (under-report of VGDF). We limited the latter three analyses to subjects interviewed at both sets of interviews who were employed in the same job at both time points (n = 199). For the analysis of VGDF over-report, we excluded subjects who were classified as under-reporting exposure (n = 17); for under-report, we excluded those classified as over-reporting (n = 49). Consequently, the referent outcome for those models was comprised of subjects both reporting VGDF and assigned a moderate–high exposure likelihood JEM. Analyses used SAS 9.1 (SAS Institute, Cary, SC).

RESULTS

The 348 study subjects included 226 (65%) with asthma and concomitant rhinitis, 56 (16%) with asthma alone, and 66 (19%) with rhinitis alone. Study subjects were mainly female (68%), white, non-Hispanic (75%) and well educated (89% some college education or more), with a mean (SD) age of 43.9 (8.6) years. Overall, 188 (34%) were former or current cigarette smokers. Comparing the participants with reported baseline VGDF exposure (n = 136) with those reporting no exposure to VGDF (n = 212), the non-VGDF group was more likely report a higher educational level (p = 0.04) and less likely to report rhinitis alone (p = 0.04). Although the non-VDGF group was also less likely to be male, this difference is not statistically significant (p = 0.06). Age, race-ethnicity and smoking status were similar by VGDF status.

Exposure by JEM and VGDF

When analysed at the level of jobs held (n = 436), as shown in table 1, four in five subjects held a professional, managerial, administrative or sales position. Consistent with this, most (72%) were assigned a low likelihood of exposure JEM. Of 426 jobs, 193 (44%) were reported by the respondents to involve regular exposure to VGDF. For each of these jobs, a 19-item checklist of specific exposures was elicited (see Methods).

For the checklist responses, the most common exposures were biological materials; the least common, but still reported for over half of the occupations with VGDF exposure, were synthetic organics (table 1). There were 10 jobs (5%) which, despite the self-report of VGDF, did not elicit a positive response for any of the 19 checklist items.

Figure 1 provides a graphic representation of the frequencies for self-reported exposures in response to the 19-item checklist grouped in the three exposure categories.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1 Frequency of reports for 19 specific exposures, grouped by type, among 199 subjects with the same job at both interviews.

Concordances among exposure measures

The performance of self-reported exposure to VGDF (the single questionnaire item) against JEM assignment is shown in table 2. Self-report of exposure to VGDF had a sensitivity of 71% and a specificity of 66% when compared with moderate to high exposure likelihood by JEM assignment. Based on the specific item checklist, the subset of biological materials (sensitivity 64%, specificity 73%) and irritant gases (sensitivity 56%, specificity 83%) performed similarly. Reported exposure to at least one of seven synthetic organics, however, manifested the poorest sensitivity (44%), with a corresponding increase in specificity (85%).

Although 287 subjects were employed at the time of both interviews, only 199 held the same job and job duties at baseline and at follow-up. There were 29 subjects who reported exposure to VGDF at baseline and not at follow-up, while 35 reported such exposure at follow-up but not at baseline. Altogether, this represents 64 (32%) discordant responses among 199 repeat interviews and a corresponding κ statistic of 0.35 (95% CI 0.22 to 0.48). When we re-estimated reporting of exposure to VGDF across interview series stratified by condition, κ was higher and similar for those with asthma and rhinitis (0.49; 95% CI 0.26 to 0.56) and for those with asthma alone (0.43; 95% CI 0.11 to 0.74), but was poorer for those with rhinitis alone (0.12; 95% CI −0.14 to 0.38).

Among the same 199 subjects without job change, we studied predictors of test-retest discordance as well as predictors of VGDF-JEM mismatch (the latter based on VGDF report in the first interview, see Methods). In the multivariate analysis (table 3), the factors associated with discordance in self-report between interviews were increasing age (OR 2.2, 95% CI 1.4 to 3.6; p = 0.01), female sex (OR 2.5, 95% CI 1.2 to 5.5; p = 0.02) and disease status of rhinitis only (OR 3.6, 95% CI 1.4 to 9.0; p = 0.01). The only diagnostic group that was associated with VGDF-JEM mismatch (over-reporting exposure to VGDF) was the rhinitis only group (OR 3.9, 95% CI 1.6 to 9.4; p = 0.003). Under-reporting of exposure was associated with technicians, service jobs, manufacturing, construction and agriculture (OR 4.0, 95% CI 1.2 to 13.5; p = 0.02).

DISCUSSION

This analysis focused on central methodological questions relating to the performance of self-reported VGDF as a measure of occupational exposure, particularly relative to a JEM approach. As with any other survey technique, the performance characteristics of the VGDF method indicate that it does have limitations that should be borne in mind. It is moderately sensitive against JEM assignment when the latter uses an expert review approach as recommended by Kennedy.9 However, its specificity, based on that standard, is less and κ was low. Moreover, inconsistency (test-retest discordance) and inaccuracy (presuming JEM classification to be correct) differed systematically by diagnostic group. Those with chronic rhinitis alone were more likely to report VGDF status differently despite a lack of job change and to report VGDF (based on their baseline interview) in a job for which the JEM assignment was low exposure likelihood, consistent with a confounding effect. Future studies should take this into account with appropriate methods, including stratified analyses where possible.

Self-reported exposure, whether based on a single item, group of items, or a more detailed questionnaire, has limitations. Potential misclassification, due to either under- or over-reporting exposure, presents a serious challenge. Some subjects may be unaware of the substances they have worked with or, conversely, if they have a disease that they associate with workplace factors, may over-report exposures. In our study, this morbidity-driven bias may have been a relatively greater factor among subjects with chronic rhinitis alone, leading them to over-report exposure even though rhinitis is typically a less severe condition than asthma. Differential self-reported exposure misclassification linked to health status can lead to over- or underestimation of effects, as opposed to non-differential misclassification which would tend to bias toward the null.25 Although we do not have an obvious explanation for the differential rhinitis effect, we have previously reported that those with rhinitis alone, compared to asthma, have decreased work effectiveness on the job, even though persons with asthma have lower work force participation.18

Another limitation of our study, as well as most community-based studies, is the lack of independent quantification of exposures. This is a typical problem, given that industrial hygiene measurements or biological monitoring data are rarely available. Other potential limitations of this particular study should be acknowledged as well. Because all the subjects had airway disease, these findings may not be applicable to survey methods of exposure assessment in other conditions, such as cancer, or among those with no chronic health problems. Studying those currently working may have introduced selection effects if former workers would have systematically differed in performance measures. Our data collection method (telephone interviews) may have introduced measurement error not present with other methods, for example, face to face interviews with flash cards. Another source of error may have been subjects who considered VGDF as including relatively minor exposures such as perfumes or the equivalent of house dust on surfaces. This may account for the low percentage reporting VGDF but denying exposure to any of the items on the checklist and is consistent with what we observed in a comparable survey in a different cohort.24 Finally, treating as discordant a change in VGDF report over two interviews, when there was no change in occupation or industry or a narrative indication of change, could have overestimated inconsistency, if working conditions did indeed vary.

In this analysis, we used a JEM modified by expert review as the gold standard against which to measure the performance of a VGDF item. We did not compare this to performance of the JEM prior to such review, although 14% of the assignments did change as a result of this. Furthermore, we did not study a hybrid JEM-self report methodology, for example, using selected data from the 19-item checklist to modify the JEM classifications as a supplement to the expert review step. The data shown in table 2 suggest that, for subgroups of checklist items (in particular, irritant gases and fumes), specificity and overall agreement with the JEM are higher than for the single VGDF item. Of interest, a recent case–control study of occupational risk for COPD employed just such a hybrid method of expert review-modified JEM based not on open-ended text as used in our study but rather on a multi-tem checklist.26

The JEM approach has been used extensively since the 1980s as an objective method to assess occupational exposures. The use of JEMs based on both European and US occupational coding systems (often called generic JEMs) has been central to a number of case–control studies, primarily in analyses of occupationally-related cancer. In an extensive review of occupational exposure assessment for case–control studies, the sensitivity of a generic JEM was low when compared to self-report or expert assessment.27

A major problem with the generic JEM approach is that it fails to take into consideration the variability of exposures within the same job classification. Expert review can theoretically reduce this problem by modifying the JEM assignment based on additional data on actual job duties and/or exposures reported by the individual subjects.9 28 It is also important to acknowledge that the JEM approach is probabilistic, thus a low likelihood of exposure is not the same a “no” exposure.

Expert assessment of exposure and generic and modified JEM assignment have been used as the de facto gold standard to assess self-reported VGDF, as shown in table 4. In this tabular presentation, there is range of sensitivity and specificity for self-reported exposure tested against expert interview or JEM or for agreement between self-reported exposure and JEM assessed with the κ statistic. In analyses other than the current study, among persons with airway disease, sensitivity estimates have ranged from 48% to 65% and specificity from 80% to 83%; among general samples or among those without lung disease, the sensitivity has ranged from 42% to 64% and the specificity from 74% to 91%. κ Values have ranged between 0.30 and 0.71. The values from the current study fall within the range of these other reports, with the exception of the very low κ (0.12) for self-report against JEM assignment among those with rhinitis alone.

In multivariate analysis, rhinitis was indeed a risk factor for over-report of exposure against the JEM, which would contribute to a poor κ. Indeed, it was the only factor in that analysis that did manifest a statistically significant association with over-reporting. Moreover, in multivariate analysis rhinitis was also associated with increased odds of a discordant self-reported exposure on repeated survey, as were increased age and female gender. The test-retest performance characteristics of self-reported occupational exposure have not been well studied, although self-assessed generic “dustiness” has been reported to have better test-retest performance than querying a more specific exposure (asbestos).29

Our study is uniquely placed to address the methodological uncertainties of survey-based exposure assessment in airway disease in that it used systematic survey data including multiple exposure assessment measures and also afforded longitudinal re-assessment. In summary, based on this analysis, we found that the VGDF single-item approach does have limitations but that, overall, it performed in a manner indicating that it can be a useful exposure assessment method. Attention should always be given to potential biases; our analysis suggests that rhinitis is a health condition that could lead to differential misclassification. However, no single method is likely to meet all needs. Applying multiple approaches to exposure assessment within the same analysis is likely to provide greater confidence in interpreting the results.