MATERIALS AND METHODS

The mixed effects models evaluated in this paper have been described in detail elsewhere¹² and are summarised in table 1. These models were aimed at identifying the factors predicting changes in exposure levels of road paving workers to exposures to bitumen fume and benzo(a)pyrene. They revealed a declining trend in exposures to bitumen fume and benzo(a)pyrene with time, 6% and 11% per year, respectively. Furthermore, differences in exposure levels were observed between different methods of paving. Coal tar use was shown to be the most important predictor of benzo(a)pyrene exposure, but the magnitude of this effect was somewhat less than that expected on the basis of laboratory studies. The differences in sampling and analytical methods, and in applied measurement strategies, were accounted for. There were no differences between comparable paving operations among countries.

The general model used to study the fixed and random effects is described by the following expression:where:

Y_{ij|β1 . . . βn} = natural logarithm of the exposure concentration measured on the jth day of the ith worker in presence of the β₁. . . β_n determinants of exposure;

μ = true underlying mean of log transformed exposure averaged over all determinants of exposure;

β₁. . . β_n = fixed effects of the determinants of exposure;

χ_i = random effect of the ith worker;

ε_ij = random within worker variation.

The restricted maximum likelihood algorithm estimated variance components derived from the mixed effect models. The algorithm assumes that χ_i and ε_ij are normally distributed with zero means and variances σ²_BW (between worker logarithmic exposure variance) and σ²_WW (within worker logarithmic exposure variance), respectively, which are mutually independent.

The models were validated against external data obtained from the United States,^14–19 Germany (measurements were only partially described in this publication),²⁰ and Italy (unpublished). These data were made available to us after the original statistical models were constructed. Table 2 summarises the key features of these newly acquired data. Three US benzene soluble matter measurements from 1982 were excluded from the current analysis because they were collected during an experimental application of sulphur containing asphalt. All bitumen fume and Italian benzo(a)pyrene measurements were collected using personal samplers. However, because of the limited number of benzo(a)pyrene measurements, we also used stationary samples collected in Germany.²⁰

We computed Pearson correlation of the predicted and observed values. We also compared 95% confidence intervals of the geometric means of survey specific predicted and measured exposure levels. Bias and precision of the models were estimated using a procedure similar to that proposed by Hornung²¹ (see equations 2, 3, and 4). Bias was defined as the mean difference between predicted and measured values on logarithmic scale; precision was defined as the standard deviation of bias. These were calculated on a logarithmic scale, because bias followed a left skewed distribution that approximated normal distribution after logarithmic transformation. The effect of the German sampling method on benzo(a)pyrene exposure could not be estimated using data that the original model was based on. Therefore, we recalculated predicted values for the German study, taking into account: (1) benzo(a)pyrene is present mostly in fume phase in asphalt work; and (2) the GGP sampler used in Germany tends to collect 3.3 (= exp(1.20)) times more dust than the sampler used in the original benzo(a)pyrene exposure models (table 1).where n = number of pairs of measured and predicted values being compared.

Analyses were carried out using SAS version 6.12 (SAS Institute, Cary, NC), Microsoft Excel 7.0 (Microsoft Corporation, Seattle, WA), and SigmaPlot 4.01 (SPSS Inc., Chicago, IL). Data management and acquisition were facilitated by the use of Microsoft Access 2.0 (Microsoft Corporation, Seattle, WA).

RESULTS

The correlation between observed and predicted bitumen fume exposures for the US data was weak, but statistically significant (Pearson correlation coefficient (r) = 0.28, p = 0.004; n = 98). For the data obtained from Germany and Italy, the relation between observed and predicted benzo(a)pyrene exposure levels was much stronger (r = 0.45, p = 0.0001; n = 339). Figures 1 and 2 illustrate the relation between medians, estimated as geometric means, of measured and predicted values of bitumen fume and benzo(a)pyrene exposure. In fig 1, measured mean values correspond to the results of each Health Hazard Evaluation conducted by US NIOSH. The reason why some years in fig 1 have only one predicted and two measured values is because the predicted value is the same for each of the two surveys conducted in that year. The figure indicates that the bitumen fume model tends to underestimate exposure levels, even though for five out of six surveys these differences can be expected to be caused by chance (95% confidence intervals of observed and estimated values overlap).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Measured versus predicted median bitumen fume exposures for each one of six NIOSH Health Hazard Evaluations used in external validation; all measurements are from hot mix paving in the USA (n = number of observations/measurements).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Measured versus predicted median benzo(a)pyrene exposures; measurements from Italy and Germany. Whiskers: 95% CI. All samples collected during tar use were stationary one hour measurements at the expected location of a worker, obtained in Germany. Predictions for German data were not corrected for the GGP sampler use in this figure. All samples from tar free paving were collected in Italy.

In fig 2, measured mean values correspond to different exposure scenarios, as indicted by the legend. It would appear that there is a reasonable degree of agreement between observed and predicted medians for recent measurements in tar free environments. However, the benzo(a)pyrene exposure model tended to underestimate exposure levels for the circumstances in which tar was used in Germany. The observed effect of in situ recycling on benzo(a)pyrene exposure was similar to that expected on the basis of the previously developed statistical model.

Table 3 illustrates bias and precision of estimated exposures to bitumen fume and benzo(a)pyrene relative to external measurement data. Bitumen fume and benzo(a)pyrene models showed negative bias, −70 and −82% respectively. Correction for the sampling method used in Germany further decreased the estimate of relative bias to −51%.

DISCUSSION

Overall comparison of the individual predictions of bitumen fume and benzo(a)pyrene exposure models to external data revealed a significant, but weak correlation. This arose from the fact that measured data had a much wider range, implying that absolute values predicted by our models for each individual observation may be inaccurate. Furthermore, discrepancies in ranges suggest that our models may underestimate any contrasts that exist between different groups of subjects, leading to reduction in power to detect quantitative exposure–response relations. The models had relatively poor precision, probably resulting mostly from large day to day variances. This is adequate because our goal was to model between worker differences in exposure. The original models did explain a substantial 54–79% of the estimated between worker variance, implying that the between worker variance is reflected by the variables used in the calculation of exposure intensity estimates for the exposure matrix to be employed in epidemiological analyses.¹² Bias estimates indicated that our models can underestimate bitumen fume and benzo(a)pyrene concentrations by 50–70%. Underestimating exposures, especially those occurring further in the past and lying at the upper edge of exposure distribution, as suggested by fig 2, would lead to overestimation of dose–response relations based on quantitative indices of exposure (for example, cumulative exposure or career average exposure).

Data used to construct the original exposure models was based primarily on measurements collected in Scandinavia. USA and Italy are not participating in the cohort study, therefore they did not contribute any data to the original models. Some bitumen fume exposure data from Germany was used in constructing the original bitumen fume exposure model. However, this German data was from a different survey than that used in validation.

It is possible that negative bias with respect to bitumen fume measurements from the USA is caused by either chance or systematic differences between road paving practices in Western Europe and the USA. Unfortunately, comparable data on bitumen fume exposure was not available in Western Europe, and we had to resort to US data for assessment of external validity. The two to four times greater pace of paving in the USA compared to Western Europe and differences in the types of asphalt mixes used (Max von Devivere, personal communications) may well explain observed higher exposures in the USA. If that is the case, apparent bias in our bitumen fume models probably does not impede applicability of our models to the European situation.

In validation of benzo(a)pyrene model, differences in sampling methods, such as use of short term stationary samples and differences in method of extraction of organic matter may also contribute to the observed discrepancies between predicted and observed values.^12,22 An alternative reason for this discrepancy may arise from the fact that the coal tar content of asphalt binder, an important predictor of benzo(a)pyrene exposure,²³ was not taken into account in the original models. Even if a model that takes this factor into account could be constructed, the coal tar content of asphalt binder would be impossible to estimate with any precision in an industry wide cohort study. Thus, our models may underestimate benzo(a)pyrene exposure under circumstances similar to those monitored in the German data (25–30% coal tar in asphalt binder). Nonetheless, the estimate of the effect of coal tar use derived from the validation data set is within a range that can be expected on the basis of the results of laboratory studies.¹²

In the IARC study of asphalt industry we resorted to empirical modelling of exposures for paving workers. Model based exposure assessment might have numerous advantages over exposure assessment based solely on the experts' opinions. Quantitative exposure assessment is important for both identification of “weak associations”¹ and establishment of scientifically based exposure limits. One of their other advantages is that the use of statistical models of exposure in epidemiological studies allows data driven sensitivity analysis of exposure–response relations.^5,6,11,24 We have also shown previously that interpretable empirical models can be derived despite limitations of the data.^12,23 In this paper we have further shown that these models can be expected to produce reasonably accurate predictions—that is, have inaccuracy on the order of day to day variability in exposure. Such examination of patterns of exposure within the road paving industry would not have been possible on the basis of previously published reports, which would have formed the basis of subjective expert evaluations as a method for exposure assessment.²² Therefore, when measurement data are available, their use for exposure assessment should be considered as the primary basis of assessing exposure intensity. Subjective evaluation of exposures should be used only as the last resort (for example, few or no measurements, well known errors in analytical procedures used to obtain available measurements). This is especially relevant for large multicentre international studies in which calibration of different assessors may prove to be very difficult. If expert evaluation of exposures is the only available option in exposure assessment, the penalty is paid in the form of uncertainty about where the weaknesses of the exposure assessment protocol lie. Subjective evaluations, just like any other exposure assessment procedure, must be validated.⁸ The most direct method of validating any exposure model is through workplace measurements,^25,26 but other validation methods are also available.^8,21,27,28

We have shown that previously developed models of bitumen fume and benzo(a)pyrene can underestimate exposures under certain circumstances. Model estimates can be expected to be imprecise, making them most suitable for group based exposure predictions in which all members of a group are assigned the same average exposure, instead of individual based exposure predictions. Limited validation against external measurement data revealed that the patterns described by the original bitumen fume and benzo(a)pyrene models were also present in the validation data sets. This provided an additional guarantee that the originally derived models provided useful estimates of bitumen fume and benzo(a)pyrene exposure. Despite these encouraging findings, we should note that the evaluation of the external validity was possible only to a limited extent, since validation data sets lacked the diversity of exposure scenarios needed for a more comprehensive evaluation. Furthermore, models do seem to contain bias, but its magnitude appears reasonable given the nature of cohort design and uncertainties usually encountered in retrospective exposure assessment.^29–31 Overall, results indicated that further improvements could not presently be made to the two original models, given the quality of data available on determinants of exposure and retrospective design of the epidemiological study. However, the consequences of assigning a higher multiplier to the effect of coal tar on benzo(a)pyrene will be explored in sensitivity analysis. Validation of the models increased our confidence in their applicability to exposure assessment in the historical cohort study of cancer risk among asphalt workers.