Article Text

Download PDFPDF

Original article
A comparison of exposure assessment approaches: lung cancer and occupational asbestos exposure in a population-based case–control study
  1. Jill S Hardt1,
  2. Roel Vermeulen2,
  3. Susan Peters2,3,
  4. Hans Kromhout2,
  5. John R McLaughlin1,4,
  6. Paul A Demers1
  1. 1Occupational Cancer Research Centre, Cancer Care Ontario, Toronto, Ontario, Canada
  2. 2Environmental Epidemiology Division, Institute for Risk Assessment Sciences, Utrecht University, Utrecht, The Netherlands
  3. 3Epidemiology Group, Centre for Medical Research, University of Western Australia, Perth, Australia
  4. 4Public Health Ontario, and Lunenfeld-Tannenbaum Research Institute, Toronto, Ontario, Canada
  1. Correspondence to Jill S Hardt, Occupational Cancer Research Centre, Cancer Care Ontario, 505 University Avenue, 17th Floor, Toronto, ON, Canada M5G 1X3; jill.hardt{at}


Objectives In attempts to overcome the limitations of self-reported data in occupational health research, job-exposure matrices, which assign exposure by occupation, have emerged as an objective approach for assessing occupational exposures. On the basis of a lung cancer case–control study conducted in the Greater Toronto Area, 1997–2002, assessment of occupational exposure to asbestos was compared using self-reports and a general population job-exposure matrix (DOM-JEM).

Methods Cases and frequency matched controls provided life-time job histories and self-reported exposures to potential lung carcinogens including asbestos through a detailed questionnaire. Exposure to asbestos was also assigned to each job by linking occupational histories with DOM-JEM. Agreement in classification of exposed and unexposed jobs according to self-reports and DOM-JEM was evaluated using Cohen's κ. Risks for lung cancer were estimated using unconditional logistic regression for each exposure assessment approach.

Results The prevalence of occupational asbestos exposure was greater when based on DOM-JEM than when based on self-reports. Agreement in classifying exposure to jobs between the two assessment approaches was poor. The risk of lung cancer was not elevated among workers who self-reported asbestos exposure, whereas workers considered exposed on the basis of DOM-JEM were almost twice as likely as unexposed workers to be diagnosed with lung cancer (OR 1.9, 95% CI 1.3 to 2.7).

Conclusions It is generally assumed by epidemiologists that self-reported exposure assessments result in inflated risk estimates. In this study, self-reports found no association with a well-established risk factor, whereas a high-quality job-exposure matrix revealed relative risk estimates that are more consistent with previous findings.

Statistics from

What this paper adds

  • Studies examining health impacts of occupational carcinogens often collect exposure information directly from workers through questionnaires, but workers’ inability to accurately assess, recall and report exposures to specific carcinogenic agents can obscure the ability to detect associations between exposures and adverse health outcomes.

  • Job-exposure matrices, developed by exposure scientists and which assign exposure by occupation, are an efficient alternative for assessing occupational exposures.

  • A comparison of self-reports and a job-exposure matrix assessment of occupational asbestos exposure revealed poor agreement in classifying exposed and unexposed workers in a lung cancer case–control study conducted in the Greater Toronto Area, 1997–2002.

  • Only assessment using the job-exposure matrix identified the expected associations between occupational asbestos exposure and increased lung cancer risk, which indicated that the job-exposure matrix assessment approach was more accurate in classifying exposure than self-reports.

  • This study demonstrates the significant challenge posed by retrospective assessments of occupational exposures and highlights the diverging adequateness of assessment approaches to support effective risk identification.


Obtaining accurate retrospective information about occupational exposures can be a challenge because of the extent and variable quality of available data. In occupational epidemiology, case-by-case expert assessment has emerged as the preferred approach for retrospective exposure assessment for population-based studies.1 By reviewing work histories on a case-by-case basis, experts including occupational hygienists can take into account subject-specific differences such as types of industry and local differences in production, safety measures and variations over time. However, this approach is time and labour intensive, and validity and reliability are dependent on the exposures being assessed, assessor expertise, and the clarity of information. Less time-consuming and expensive methods include applying a job-exposure matrix (JEM) or assessing exposure using self-reports. The latter option is the least costly and does not require the involvement of exposure experts, although the ability of workers to accurately assess, recall and report exposure to specific agents is a potential limitation.2 The complexity of an individual's occupational history, duration of employment, and the length of recall period have been found to influence the validity and reliability of self-reports.1 Recall bias is a particular threat in case–control studies, since cases are thought to be more likely than controls to recall prior exposures.3 Despite these limitations, Teschke et al1 concluded that, in comparison with quantitative exposure measurements, case-by-case expert assessments performed only slightly better than self-reports in case–control studies.

JEMs, first developed in the 1980s, are a relatively inexpensive alternative approach to retrospective occupational exposure assessment.4–7 With occupations and/or industries on one axis and exposure agents on the other, the cells of the matrix indicate the presence, intensity and/or probability of exposure to a particular substance in a specific job.7 ,8 JEMs are based on expert judgement of occupational hygienists, published literature, and/or databases of direct measures. Although it first requires applying standardised occupation and/or industry codes to job histories, the use of existing JEMs to retrospectively assess occupational exposures is much less resource intensive than case-by-case expert assessment.9 ,10 In contrast with self-reports, exposure assessments based on JEMs are not affected by potential differential recall between cases and controls.11 However, JEMs do not take into account local conditions and potential differences in exposure among individuals with the same job title.

DOM-JEM was recently developed for application in population-based studies.12 It can be used to assess exposure to a selection of agents for each five-digit International Standard Classification of Occupations 1968 (ISCO-68) code. DOM-JEM does not include a time axis, and exposure levels are based on a combination of intensity and probability of exposure within a job.13 It has previously been applied in studies of lung cancer risk and exposure to asbestos, diesel motor exhaust, chromium, nickel, organic dust, polycyclic aromatic hydrocarbons and silica.12–17 In the Netherlands Cohort Study,13 DOM-JEM showed the highest level of agreement with case-by-case expert assessment (κ 0.29) compared with two other JEMs (FINJEM and Asbestos JEM).

Recently, Brenner et al17 studied the risk of lung cancer associated with selected occupational exposures based on self-reports in a case–control study in the Greater Toronto Area, but no association was found with asbestos (OR 1.1, 95% CI 0.6 to 2.0). We hypothesise that the null result reported by Brenner et al could be due to limitations of self-reported exposure, which may have contributed to exposure misclassification, obscuring exposure–disease associations. Although the association between lung cancer and asbestos exposure is well known,18 a wide range of risk estimates have been reported in occupational studies.19 Some of this heterogeneity can be attributed to true differences resulting from variation in circumstances of exposure such as fibre type, processes, regulations and practices with respect to personal protective equipment and the work environment, and temporality and duration of exposure. However, on the basis of a recent review and meta-analysis, Lenters et al19 concluded that differences across studies in lung cancer risk estimates associated with asbestos exposure were at least partly explained by variations in exposure assessment quality. Furthermore, this meta-analysis revealed that, when the analyses were limited to studies with higher-quality exposure assessment methods, lung cancer risk estimates increased. Previous comparisons of self-reported occupational exposure with JEMs found that exposure tended to be more prevalent according to self-reports.4 ,5 ,20–22 Asbestos exposure was previously found to be associated with increased lung cancer risk when based on self-reports, but not when based on JEM assessment.20

Our study used data from the study reported by Brenner et al17 to measure agreement between approaches in classifying jobs as exposed or unexposed, and to examine the characteristics of individuals and jobs where assessments were discordant. Furthermore, this study examined the risk of lung cancer associated with occupational asbestos exposure according to self-reports versus DOM-JEM and to determine the risk of lung cancer according to duration of exposure for each approach.

Materials and methods

Study design

Cases of incident cancer of the trachea, bronchus or lung diagnosed among men and women between 20 and 84 years old were identified between 1997 and 2002 through four major tertiary care hospitals in metropolitan Toronto.17 Cancer diagnoses were classified according to the International classification of diseases for oncology, 3rd edition.23 Cancer-free population-based controls randomly selected from property tax assessment files (n=425) and hospital controls selected from patients at Mount Sinai Hospital Family Medicine Clinic, Toronto (n=523) were frequency matched to cases based on age, sex and ethnicity. Study design was previously described by Brenner et al.17

Data collection and exposure assessment

Lifetime work histories and demographic information were collected through a detailed questionnaire that was administered in person or by phone. Participants were asked about workplace exposure to potential lung carcinogens (including asbestos, paints and/or solvents), welding equipment, pesticides, grain elevator dust, wood dust and smoke, soot or exhaust. Self-reported asbestos exposure was assessed with the question: ‘In any of your jobs have you ever worked with asbestos?’ On the basis of work histories, each job was coded according to ISCO-68 and the International standard industrial classification of all economic activities, Rev.2. For stratified analyses, jobs were grouped according to ISCO-68 codes as follows: Professional/technical, 010–199; Construction, 870–874, 950–959; Blue collar excluding construction, 700–862, 880–949, 960–999; Other, 200–649 (this group includes administrative, managerial, clerical, sales, service, agricultural and related workers). Occupational histories were linked, blinded to case status, with a general population JEM (DOM-JEM) according to five-digit ISCO-68 codes to assign no, low or high exposure to asbestos to each job based on a combination of intensity and probability of exposure in a job. DOM-JEM was created by three occupational exposure experts (HK, RV and SP) independently of information from any study population and has been described previously.12 Classification of exposed and unexposed jobs was determined according to each method. Each study participant was evaluated using a dichotomous exposure indicator (never/ever) according to each assessment approach. Duration of exposure was defined as the number of years employed in a job classified as ever exposed according to each approach. Each year of part-time employment was considered to be 0.5 years. On the basis of DOM-JEM assessment, individuals were further categorised using an ordinal intensity variable (no/only low/ever high).

Statistical analysis

Differences in demographics, smoking history and exposure between cases and controls were assessed using χ2 and Student's t tests. Chance-corrected agreement in exposure assessment (exposed or unexposed) according to self-reports and DOM-JEM was evaluated using Cohen's κ for cases and controls by sex, occupation group, duration of employment, and years since job end. Cohen's κ statistics were interpreted as follows: <0.4, poor; 0.4–0.75, fair to good; >0.75, excellent agreement.24 Characteristics of study participants (and jobs) for whom there was discordance in exposure assessments were compared with those with concordant assessments using χ2 and t tests. Lung cancer risks were estimated using unconditional logistic regression adjusted for age group (<35 years, 35–44 years, 45–54 years, 55–64 years, 65–74, >75 years), sex and cumulative tobacco consumption (cigarette pack-years). Risks associated with duration of exposure using predefined categories (1–10 years, >10 years) were also estimated. Analyses were stratified by source of controls. A 10-year lag was introduced from the date of diagnosis for cases or the date of interview for controls. Analyses were conducted using SAS V.9.2.


Subject characteristics

Participation was 62% for cases, 60% for population controls, and 84% for hospital controls. Of the 716 eligible cases, 116 refused to participate and 155 died before study entry and/or complete data collection. Owing to missing work history information, 20 cases and 38 controls were excluded, resulting in a sample of 425 cases and 910 controls. Nearly 85% (n=1139) of participants reported two or more jobs in their lifetime work history. A total of 5083 work history observations were collected from 1335 individuals. Of these, 4406 (87%) were full-time jobs. Cases were significantly older and less educated than controls. Compared with controls, a greater proportion of cases were employed in blue collar jobs and were current or former smokers. The distribution of demographic and smoking variables for cases and controls is detailed in Brenner et al.17

Exposure prevalence

According to self-reports, 82 (6%) study participants had been exposed to asbestos in one or more jobs. Overall, more than twice as many were considered exposed according to DOM-JEM (194, 15%) (table 1). The difference between DOM-JEM assessment and self-reports was twofold for workers employed in construction and fourfold for workers employed in jobs classified as ‘blue collar excluding construction’. Differences in assigned exposure between DOM-JEM and self-reports were greater for jobs worked less recently, with nearly eight times more workers considered exposed according to DOM-JEM compared with self-reports when considering jobs worked before 1940, and more than 12 times for jobs that ended during the 1940s. Most individuals classified as exposed according to DOM-JEM were in the ‘only low’ group based on exposure probability and intensity (n=176, 13%). Only 18 (1%) individuals were ever employed in a job in the ‘high’ exposure group. While the likelihood of self-reporting exposure did not differ significantly between cases and controls (7% vs 6%, p=0.23), cases were much more likely than controls to be classified as ever exposed according to DOM-JEM (23% vs 11%, p<0.0001). The proportion self-reporting exposure to asbestos was similar between population (6%) and hospital controls (5%), while prevalence of exposure according to DOM-JEM was greater among population controls (13% vs 9%).

Table 1

Numbers and percentages of workers ever exposed to asbestos, and agreement (Cohen's κ) between self-reports and DOM-JEM assessment approach, Greater Toronto Area, Ontario, 1997–2001

Agreement in exposure assessment

Agreement was poor between self-reports and DOM-JEM in assigning exposure (exposed or unexposed) to each job (κ 0.19) (table 1). When restricted to only the longest job held by each individual, the estimate of agreement was unchanged (results not shown). Agreement was higher among controls than cases (κ 0.25 vs 0.12), and did not vary by source of controls. This higher agreement observed in controls compared with cases was consistent for all analyses of agreement (results not shown). The greatest agreement between the assessment approaches was observed for jobs in construction (κ 0.30). When stratified by the last decade in which an individual was employed in a particular job, there was a weak positive trend of increasing agreement associated with recentness of employment.

Characteristics of individuals where discordance between assessments occurred

Discordance in exposure assessments for at least one job occurred among 17% of study participants for whom assessments according to both approaches were available (table 2). Of those with discordant observations, 22% over-reported and 78% under-reported exposure relative to DOM-JEM in one or more jobs. Discordance was more likely to occur among cases than controls (26% vs 13%). Population controls were slightly more likely than hospital controls to under-report exposure relative to DOM-JEM (11% vs 8%). Likelihood of discordance was not statistically significantly associated with age for over-reporting (p=0.21), but approached statistical significance for under-reporting (p=0.06) relative to DOM-JEM. The large majority of discordant assessments were observed among older study participants, which parallels the results indicating an increase in agreement between assessments for more recently worked jobs (table 1). Education was not a determinant of agreement between approaches, but those with the least education were most likely to under-report exposure (24%) (table 2). Likelihood of discordance was significantly associated with occupation group (p<0.0001). Discordance in assessments occurred often for jobs in the ‘blue collar excluding construction’ group, with under-reporting more common than over-reporting (36% vs 5%). Under-reporting was most common for jobs in ‘construction’. Of the 69 individuals who were ever employed in construction, over 40% under-reported exposure to asbestos relative to DOM-JEM in one or more construction jobs. Under-reporting occurred most often in jobs where individuals were employed for 10 years or fewer. The proportion of study participants who under-reported exposure appeared to decline with recentness of employment.

Table 2

Characteristics of individuals where there was discordance between self-reports relative to DOM-JEM assessment approach, Greater Toronto Area, Ontario, 1997–2001

Lung cancer risk

Self-reported asbestos exposure was not associated with an increased risk of lung cancer (OR 0.9, 95% CI 0.5 to 1.6) (table 3). However, when assessed using DOM-JEM, a nearly twofold increase in risk was observed (OR 1.9, 95% CI 1.3 to 2.7). Compared with those not exposed according to DOM-JEM, those ever employed in a job classified as ‘high’ exposure were at slightly greater risk of lung cancer than those employed only in jobs classified as ‘low’ exposure (OR 2.3, 95% CI 0.8 to 6.5 vs OR 1.9, 95% CI 1.2 to 2.6). Individuals employed for more than 10 years in one or more occupations classified as asbestos-exposed according to DOM-JEM were at significantly greater risk of lung cancer than those exposed for 10 years or fewer compared with the unexposed group (OR 2.6, 95% CI 1.5 to 4.5 vs OR 1.5, 95% CI 0.9 to 2.4). Risk estimates did not differ significantly when restricted to population or hospital controls (results not shown).

Table 3

Risk estimates for asbestos exposure and lung cancer according to self-reports and DOM-JEM assessment, Greater Toronto Area, Ontario, 1997–2001


This study compared two retrospective occupational exposure assessment approaches, self-reports and DOM-JEM, in a Toronto-area lung cancer case–control study. The identification of risks to support interventions to protect workers depends on the accuracy of exposure assessment approaches. Self-reports are commonly used to assess occupational exposure, but perception of exposure among individuals and the correspondence between potential and actual exposure are not well understood. There is strong evidence that workers are neither aware2 nor able to accurately report25 ,26 occupational exposure to agents when compared with more direct measures of exposure.27 This study found substantial differences in prevalence of asbestos exposure and associated risks of lung cancer when exposure was assessed by both self-reports and DOM-JEM.

First, this study found that DOM-JEM estimated a greater prevalence of asbestos exposure than self-reports. Previous studies examining asbestos exposure and lung cancer risk similarly found that JEMs identified a greater prevalence of exposure than self-reports.12 ,28 ,29 In our study, prevalence of asbestos exposure was nearly 2.5 times greater according to DOM-JEM compared with self-reports. While cases and controls were equally likely to report occupational exposure to asbestos, exposure based on DOM-JEM was significantly greater among cases. This elevated exposure prevalence among cases was expected if asbestos exposure were indeed associated with excess lung cancer risk in this study population.

Second, agreement was remarkably poor between assessments, which is consistent with previous comparisons of self-reports and JEM assessments.28 ,30 As might be expected,26 cases were more likely than controls to over-report exposure relative to DOM-JEM, which may be evidence of differential recall bias. This differential recall bias is usually assumed to result in inflated risk estimates.31 However, cases were also more likely than controls to under-report exposure relative to DOM-JEM. Thus, while differential recall bias should be considered, the typical assumption regarding the direction of misclassification and the resulting inflated risk estimates was not supported by this study.

Third, risk analyses based on DOM-JEM, but not self-reports, revealed the expected associations between asbestos exposure and lung cancer risk. Those who self-reported occupational asbestos exposure in one or more jobs were not observed to be at increased risk of lung cancer. However, when DOM-JEM was used to assess exposure, exposed individuals were 90% more likely to be diagnosed with lung cancer. The excess risk observed using the JEM is consistent with previous findings in population-based studies of occupational asbestos exposure and lung cancer risk.12 ,16 ,32–37 Furthermore, workers who were classified by DOM-JEM as exposed for more than 10 years were at increased risk of lung cancer compared with those classified as exposed for 10 or fewer years.

While the positive association between lung cancer and asbestos exposure as measured by DOM-JEM, but not self-reports, led to the conclusion that DOM-JEM may have provided a more accurate assessment approach, the extent and effect of any misclassification resulting from DOM-JEM assessment is unknown. Since non-differential exposure misclassification is expected to bias the risk estimate towards the null,7 ,38 the positive effect estimate observed in this study may nevertheless indicate that the extent of misclassification according to DOM-JEM was smaller than that resulting from self-reports. To further examine the reliability of assessment approaches, it would be of interest to conduct a comparison of DOM-JEM and self-reports with case-by-case expert assessment. In the recent study by Offermans et al,13 DOM-JEM showed the highest level of agreement with case-by-case expert assessment of asbestos exposure in the Netherlands Cohort Study compared with two other JEMs (FINJEM and Asbestos JEM), although agreement was nonetheless moderately poor (κ 0.29, 95% CI 0.23 to 0.32). DOM-JEM has been compared with case-by-case expert exposure assessment (Cohen's κ 0.17) and a population-specific JEM (κ 0.23) in a multi-centre European lung cancer case–control study.12 In this study, Peters et al concluded that DOM-JEM and expert assessment of asbestos exposure resulted in similar lung cancer risk estimates (OR 1.19, 95% CI 1.05 to 1.36 and OR 1.12, 95% CI 0.93 to 1.37, respectively), while the latter approach was more time-consuming and costly.

The primary limitation of this study is the absence of a true gold standard approach for retrospective asbestos exposure assessment in population-based studies with which to compare the performance of self-reports and DOM-JEM. While case-by-case expert assessment is often favoured for retrospective exposure assessment, studies examining comparisons of expert assessment with objective measures indicate that case-by-case expert assessment should not be presumed to be the gold standard.1 ,12 ,30 Additional limitations of this study should be taken into account. Participants with incomplete data collection were excluded and next-of-kin were not interviewed. If exposure increased mortality, this exclusion would have biased the effect estimates towards the null. Since this study was limited to asbestos, the observed differences between approaches may not extend to other exposures. We suspect, however, that the discordance of exposure assessments would be even greater for substances that are, or have been historically, used at lower levels, that are components of mixtures, and that are less familiar to the general working population (eg, 1,1,1-trichloroethylene). The validity of any assessment method is subject to the nature of the exposure of interest. Experts more accurately estimate exposures for commonly used agents and classes of agents compared with individual substances,1 while workers more accurately self-report exposure to substances that are easily perceptible, such as occupational noise,39 vibrations40 or odour.1 ,4 The utility of JEMs is optimised for exposures that are prevalent and occur with increased uniformity within occupational categories, since heterogeneity contributes to misclassification.41 The validity of assessments is also influenced by the questions posed to case–control study participants. Fletcher et al29 determined that, in interviews of Swedish construction workers, the question posed returned more accurate responses for the use of asbestos cement products than for the use of asbestos insulation products. Similarly, the question posed to participants in our study, ‘In any of your jobs have you ever worked with asbestos?’, may have failed to capture occupational exposures such as working in the vicinity of, but not directly with, asbestos. Workers may have been unaware of, or failed to recall or report, such exposures. Accuracy of JEMs depends on the validity of both occupation and exposure classification. While self-reports did not capture information regarding the intensity of exposure, DOM-JEM did distinguish between jobs with low and high levels of potential exposure. In a validation study of a JEM used to assess utility worker exposure to magnetic fields, the JEM effectively distinguished between workers with low and very high exposure, but introduced misclassification when intermediate categories of exposure were used.42 DOM-JEM assumes homogeneous exposure within occupational groups, which, while practical, fails to capture variations within those groups. DOM-JEM further did not capture historical changes in the prevalence of asbestos use, production processes, and the introduction and enforcement of safety measures and regulations. The discordance between approaches before 1960 may be partially attributable to poor recall and reporting by workers because of the long duration since employment and/or overestimation of exposure by DOM-JEM, although Canada was, by 1920, a large consumer and producer of 84% of the world's asbestos supply.43

Retrospective assessments of occupational exposures pose a significant challenge to researchers, who must carefully consider the implications of methods used to determine exposure. Our study revealed markedly different lung cancer risks associated with asbestos exposure according to each method of assessment. In contrast with self-reports, DOM-JEM assessment resulted in the expected excess risk estimates. When the well-established link between occupational exposure to asbestos and risk of lung cancer is considered,18 the findings of this study indicated that DOM-JEM assessment provided a more accurate measure of exposure. This study serves as a cautionary note to researchers to carefully consider the implications of the methods used to evaluate historical occupational exposures. The substantive disagreement revealed between assessment methods in this study, and the resulting impact on risk estimates indicates that results of studies using only self-reports to measure exposure should be interpreted with care, and null results should instead be considered ‘inconclusive’. The findings of this study have important implications for exposure assessment in occupational health research. Detection of occupational risks, particularly at lower levels, requires valid, precise and unbiased methods of occupational exposure assessment. This is even more important in exploratory investigations of potential exposure to agents that are not yet identified or well understood.


The authors gratefully acknowledge Dr Darren Brenner for his role in preparing the lung cancer case–control dataset for analysis and for providing his expertise relating to this study.



  • Contributors JSH and PAD were responsible for the conception and design of the study. RV, SP and HK developed the job-exposure matrix used in this analysis and conducted the linkage of the DOM-JEM to the work histories in the lung cancer case-control dataset. JRM provided the lung cancer case-control dataset. JSH carried out the data managment, analysis and preparation of the first draft of the manuscript. PAD, RV, SP, HK and JRM contributed to the interpretation of data and provided critical revisions to the intellectual content of the manuscript. All authors provided final approval of the version to be published.

  • Competing interests None.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.