Validity and reliability of self reported exposures
| Authors, year | Study population | Self report measure | Comparison measure | Method of assessing validity or reliability | Results |
|---|---|---|---|---|---|
| Samet et al, 19784 | 70 shipyard insulation workers | Years of self reported exposure to asbestos, radiation, and fibreglass in interviewer administered questionnaire in 1976 | Years of self reported exposures, by same method, 1 year later | - R2 for regression of two measures of years of exposure | - R2 = 0.84 for asbestos, = 0.82 for radiation, = 0.73 for fiberglass |
| doPico, 198246 | 209 US grain workers | Self reported estimates of dust exposure, as heavy versus not heavy, and as less than average, average, or more than average | Measured levels of total dust concentration, one sample per person | - Comparisons of mean dust concentrations for self reported exposure categories | - Mean dust concentration for “heavy dust” = 10.1 mg/m3 vs. 1.58 mg/m3 for not heavy, p<0.001 |
| - Mean dust concentration for “more than average” = 13.9 mg/m3 vs. 4.21 mg/m3 for “average”, and 1.84 mg/m3 for “less than average”, p<0.01 | |||||
| Kromhout et al, 198747 | Employees of a paint factory (n=29), a food processing facility (n=58), a nonwoven materials factory (n=164), and 2 coach works (n=144) in the Netherlands | Self reported rankings of exposure to either dust or solvents, using a 4-point scale, for the task they were performing at the time of exposure measurement | 58 solvent measurements in the paint factory; and 421 dust measurements in the other plants | - Proportion of variance in exposure explained (adjusted R2) by the employees' rankings | - R2 ranged from 0.03 to 0.23 for dusts, median = 0.14 |
| - R2 = 0.56 for solvents | |||||
| Järvolm and Sandén, 198748 | 951 males employed in Swedish shipyards | Self reported intensity of exposure to asbestos, in four ordinal categories (very low =1, low =2, heavy =3, very heavy =4) | Ratings by employee experts with long experience in the industry (4 production workers, 4 safety engineers, and 7 safety stewards), of each job's asbestos exposure intensity, in four ordinal categories | - Exposure-reponse relation between pleural plaques and estimated asbestos exposure intensity | - Pleural plaque prevalence with self-reported asbestos intensity: 1=33%, 2=34%, 3=41%, 4=48%; With expert estimated asbestos intensity: 1=40%, 2=37%, 3=35%, 4=42%; |
| Bond et al, 198810 | 143 males employed for at least 1 year at a US chemical plant on or after 1940 | Self reported exposures, based on open ended questioning, as reported in a telephone interview in 1984 | 171 agents catalogued by company hygienists | - % agreement | - Highest agreement for chlorine (11%) and asbestos (9%), lowest for sulfur dioxide and heat (<1%) |
| Eskenazi and Pearson, 19889 | 57 women working during pregnancy, recruited in a US prenatal clinic | Self reported exposures to heat, cold, noise, poor ventilation, radiation, video display terminals, biological agents, fumes, gases, and dusts, as reported in a clinical interview | Industrial hygiene review of self reported exposures to heat, cold, noise, poor ventilation, radiation, video display terminals, biological agents, fumes, gases, and dusts, as reported in a self administered questionnaire | - Sensitivity and positive predictive value, using the industrial hygiene review as the gold standard; kappa | - Sensitivities ranged from 0.5 to 0.9, median = 0.67 |
| - Positive predictive values ranged from 0.56 to 1.0, median = 0.75 | |||||
| - Kappas ranged from 0.42 to 0.94, median = 0.63 | |||||
| Hertzman et al, 198849 | 172 Canadian sawmill workers | Self reported hours of exposure to chlorophenate fungicides, per year | Concentration of total chlorophenate in urine samples, measured in 150 workers in the summer, and 154 workers in the fall | - Pearson correlation coefficient | - Pearson r = 0.67 for summer samples; = 0.58 for the fall samples |
| Pron et al, 198850 | 117 controls randomly selected from a municipal population in Canada | Self reported continuous exposure to second hand tobacco smoke, and number of worksites where exposed | Reinterview 6 months later | - Kappa | - Kappa = 0.46 for ever vs. never exposed to second-hand smoke at work; |
| - Weighted kappa = 0.37 for number of worksites exposed | |||||
| Teschke et al, 198951 | 225 Canadian sawmill workers | Self reported hours of exposure to chlorophenate fungicides, per year | Concentration of total chlorophenate in one urine sample from each worker | - R2, proportion of variance in urinary chlorophenate concentrations explained by self-reported hours of exposure | - R2 = 0.15; with self-reported skin exposure also included in the model, R2 = 0.17 |
| Ahlborg, 199052 | From a cohort of Swedish drycleaners, 48 women with a pregnancy ending in a miscarriage requiring hospitalisation, perinatal death, low birthweight or malformed child, and 110 control women | Self reported presence of drycleaning operations in worksite and exposure to tetrachloroethylene | Employer reports on type of production and drycleaning agents used, including tetrachloroethylene, in various employment periods | - Sensitivity and specificity in comparison to employer reports | - Sensitivity and specificity of drycleaning operation for cases = 0.97 and 0.75; for controls = 0.96 and 0.69 respectively |
| - Sensitivity and specificity of tetrachloroethylene exposure for cases = 1.0; for controls = 0.93 and 0.94 respectively | |||||
| Bachmann and Myers, 199153 | 224 South African grain mill employees | Self reported classification of their work as not dusty, slightly dusty or dustry, and very dusty | Investigators' classification of work areas into 4 dustiness scores: 0, 1, 3, 18; scores verified by exposure measurements | - R2, proportion of variance in investigators scores explained by self-reported classifications | - R2 = 0.13 |
| - % agreement of dustiness classifications | - % agreement = 54% | ||||
| Holmes and Garshick, 199154 | 116 US male veterans | Self reported work in a dusty job and exposure to asbestos, reported in a mailed questionnaire | Self reported work in a dusty job and exposure to asbestos, reported in a clinic based interview, an average of 7 months later | - % agreement | - 60% agreement on asbestos exposure |
| - 71% agreement on dust exposure | |||||
| Joffe, 199255 | 420 employees of 5 factories in the printing or plastics industries in a study of fertility and miscarriage in the UK | Self reported exposures in the most recent job to imidazoline, carbon black, diazo dyes, resins, varnishes, oils and greases, solvents/degreasers, coloured inks | Department head and other company data on the use of these chemicals in each department, including changes over time | - Sensitivity and specificity using the company data as the the gold standard | - Sensitivities from 0.70 to 0.85 for oils and greases, solvents/degreasers, and colored inks, from 0.24 to 0.45 for remainder |
| - Specificities at least 0.95 for imidazoline, carbon black, and diazo dyes, from 0.48 to 0.78 for remainder | |||||
| Walter et al, 199256 | 103 subjects of a case-control study of melanoma in Canada | Self reported exposure to fluorescent lights at work, as reported in an in-person interview | - Self reported exposure to fluorescent light, as reported on a mailed questionnaire, several weeks later | - % agreement and kappa | - % agreement = 80%, kappa = 0.57 |
| - Employer reported exposures to fluorescent lights at work, for 25 jobs | - % agreement overall = 68%; kappa for jobs where both employers and subjects able to classify fluorescent light exposure = 0.79 (n=19) | ||||
| Blair and Zahm, 199357 | 69 cases and 41 controls in a case-control study of soft tissue sarcoma and lymphoma in US farmers | Self reported use of herbicides and insecticides | Suppliers' reports of farmers' herbicide and insecticide use | - % agreement | - 59% agreement on use of both herbicides and insecticides |
| Fonn et al, 199358 | 305 South African grain mill employees | Self reported dustiness in 4 ordinal categories (very high, high, medium, low), as reported in an interviewer administered questionnaire | Measured exposure to total dust, used to categorise work areas into the same 4 dustiness categories | - Kendall's tau | - tau = 0.45 |
| - Contingency coefficient | - cont. coef = 0.48 | ||||
| van der Gulden et al, 199359 | 209 subjects of a case-control study of prostate cancer in the Netherlands | Self reported exposure to pesticides, fertilisers, iron and steel, non-ferrous metals, welding fumes, solvents, paints, and lubricating oils, as reported in a mailed questionnaire | Self reported exposure to the same substances, as reported in an telephone interview 3 to 5 weeks later | - % agreement and kappa | - % agreement from 75 to 88% for iron and steel, welding fumes, and fertilizers, and from 64 to 70% for remainder; median = 73% |
| - Kappas from 0.55 to 0.70 for iron and steel, welding fumes, and fertilizers, and from 0.36 to 0.48 for remainder; median = 0.52 | |||||
| Halpin et al, 199460 | 90 current sawmill workers, 14 former sawmill workers, and 58 light engineering factory employees, in the UK | Self reported dustiness, on an ordinal scale from 0 to 3 | Personal dust samples of random sample of workers within certain mill/factory areas | - Comparisons of mean dust concentrations and median dustiness ratings for each area | - Mean dust concentrations for “0” dustiness areas = 0.24 mg/m3; for “1” = 0.71 to 1.13 mg/m3; for “2” = 1.32 to 6.25 mg/m3 |
| Savitz et al, 199461 | 161 mothers who worked in the US textile industry, selected from subjects in a case-control study of miscarriage, preterm delivery and low birth weight | Self reported exposure (yes/no) to vibration, solvents, heat, and noise | Expert review of subjects' job histories, type of machinery, work methods, and environmental conditions, to assign exposure to each job as unlikely, possible, or likely | - Kappa | - Kappas for vibration = 0.08 for females, 0.23 for males; for solvents = 0.26 and 0.02; for narrow heat classification = 0.15 and 0.17; for narrow noise classification = 0.20 and 0.24. |
| Teschke et al, 199462 | 78 sawfilers in 8 Canadian sawmills | Self reported exposures to 8 individual metal components of saws and 5 composite materials (coolant, babbitt, tungsten carbide, stellite, grinding dust), as reported in an interviewer administered questionnaire using either partial or detailed prompting | - Measured air concentrations of 8 specific metals above detection limits | - Sensitivity and specificity using measured concentrations and observations of work as the gold standards | - Sensitivities for metals ranged from 0 to 0.58 (median = 0.22), specificities from 0.69 to 1.0 (median = 0.88) |
| - Observations by hygienists of sawfilers' proximity to machines where the 5 composite materials | - Sensitivities for composite materials ranged from 0.80 to 1.0 (median = 0.83), specificities from 0.62 to 0.86 for all materials except grinding dust (0.18), (median = 0.78) | ||||
| Fritschi et al, 199663 | 1657 cases and 253 population based controls from a study among men with cancer at any of 19 tumor sites in Montreal, Canada | Self reported exposure to 11 groups of substances (fur or leather, wood products, glues, paints, pharmaceuticals, pesticides/fertilisers, insulation, oils/greases, fuels, solvents, plastics/rubber), as reported in a self administered questionnaire | Review by a team of industrial hygienists and chemists of subjects' job and exposure histories to assess exposure to 42 substances falling within these 11 groups of substances | - Sensitivity and specificity, using the industrial hygiene review as the gold standard; kappa | - Sensitivities ranged from 0.39 to 0.91 (median = 0.61) |
| - Specificities ranged from 0.83 to 0.97 (median = 0.90) | |||||
| - Kappas ranged from 0.33 to 0.64 (median = 0.51) | |||||
| Rodvall et al, 199664 | 151 glioma cases and 343 population based controls in Sweden | Self reported exposures to pesticides; oil or coal products; paints, pigments or glues; plastic materials; radiation; and solvents, degreasers, or cleaning agents | Industrial hygienist's review of self reported exposures to classify probability and level of exposure in the corresponding job | - Kappa | - Kappas for pesticides = 0.88 and 0.46 for cases and controls respectively; for oil and coal products = 0.72 and 0.74; for paints, pigments and glues = 0.59 and 0.51; for plastic materials = 0.61 and 0.80; for radiation = 0.89 and 0.78; for solvents, degreasers, or cleaning agents = 0.69 and 0.58 |
| Wiktorin et al, 199665 | 343 Swedish workers in a study of musculoskeletal disorders | Self reported duration of exposure to vibrating floors, and vibrating hand tools, as reported in a self administered questionnaire | Self reported duration of exposure, as reported 2 weeks later | - Intraclass correlation coefficient | - ICC for vibrating floors = 0.70 |
| - ICC for vibrating hand tools = 0.84 | |||||
| Calvert et al, 199766 | 32 employees of 15 US structural fumigation companies | Self reported information on fumigation industry employment, % of jobs using methyl bromide and sulfuryl fluoride, and pounds of sulfuryl fluoride used in previous two weeks | Company personnel records and daily work records, fumigant use logs | - Pearson correlation coefficient | - Pearson r = 0.97 for years employed in structural fumigation; = 0.66 to 0.88 for percent of jobs using specific fumigants; = 0.68 for pounds of sulfuryl fluoride used on job |
| Ising et al, 199767 | 80 employed men from a German population based case-control study of myocardial infarction | Self reported categorisation of noise in current workplace: 1=refrigerator; 2=typewriter; 3=electric lawnmower; 4=electric drill; 5=pneumatic drill | Measured one minute average noise level at each worksite | - Median noise levels for each self reported noise category | - Median noise level for category “1” = 53 dBA; for category “2” = 53 dBA; for category “3” = 75 dBA; for category “4” = 88 dBA; for category “5” = 100dBA |
| - Spearman rank correlation coefficient between self reported ordinal category and measured noise level | - Spearman r = 0.84 | ||||
| Nieuwenhuijsen et al, 199768 | 104 workers from 10 US farms | Self reported dust exposure during a single sampling period of about 2 to 3 hours, on ordinal scale from “0” = “no dust exposure at all” to “10” = “dust exposure that severely restricted your view” | Measured inhalable and respirable dust concentrations | - Spearman rank correlation coefficient | -Spearman r for inhalable dust levels and self-reports = 0.67 |
| -Spearman r for respirable dust levels and self-reports = 0.36 | |||||
| Rybicki et al, 199742 | 188 subjects in a case-control study of neurologic disease, all with some occupational history in manufacturing in the US | Self reported exposures to copper, lead and iron, in 544 jobs reported in an interviewer-administered questionnaire | Expert review by an industrial hygienist of self reported exposures | - Sensitivity and specificity in comparison to expert review | - Mean sensitivities for iron = 0.65, for lead = 0.73, for copper = 0.84 |
| - Mean specificities for iron = 0.88, for lead = 0.94, for copper = 0.96 | |||||
| Willemsen et al, 199769 | 107 non-smokers from 36 offices in the Netherlands | Self reports of how often office mates smoke; how often bothered by the stench of tobacco smoke (both never, sometimes, regularly); and how much tobacco smoke there is on average in the office (7 categories from “no smoke” to “very much smoke”) | Nicotine concentrations measured in each office for one full shift | - Pearson correlation coefficients between average self reports for each office and the measured nicotine levels in the office | - Pearson r = 0.41 for frequency of office mates smoking; = 0.33 for frequency of being bothered by stench; = 0.65 for average amount of tobacco smoke on average |
| Nordstrom et al, 199870 | 28 carpal tunnel syndrome cases and 33 controls in a US case-control study | Self reported work in a noisy area where plugs or muffs used, and in cold temperatures in the winter | Observations of the subject's work for 1 hour by an ergonomist | - Kappa and Spearman rank correlation | - Work in a noisy area: for cases kappa = 0.44, Spearman r = 0.53; for controls kappa = 0.31, Spearman r = 0.40 |
| - Work in a cold environment: for cases kappa = 0.31, Spearman r = 0.55; for controls kappa = 0.68, Spearman r = 0.74 | |||||
| Tielemans et al, 199943 | Subjects of 2 case-control studies of male infertility in the Netherlands | Self reported exposure to solvents (as indicated by indicating contact with any of the following: industrial cleaning products, degreasers, paint, glue, printing inks, paint removers, other solvents); or to chromium (as indicated by contact with welding fumes) | - Urine samples analysed for metabolites of toluene and xylene (n=267) and for chromium (n=156) | - Sensitivity and specificity in comparison to urine samples | - Sensitivities = 0.85 for toluene/xylene, 0.41 for chromium |
| - Specificities = 0.34 for toluene/xylene, 0.68 for chromium | |||||
| - Kappa in comparison to urine samples | - Kappas = 0.08 for toluene/xylene, 0.08 for chromium | ||||
| Neale et al, 200071 | 243 subjects of a colorectal cancer screening program among pattern and model makers in the US | Self reported exposure to 13 substances (cutting oils, epoxies, fiberglass, wood dusts, fibreglass, plaster dust, polyesters, solvents, welding fumes) for each job in their work history, as reported in a self administered questionnaire in 1985 | Self reported exposure to same 13 substances as reported in a self administered questionnaire in 1988 | - Pearson r for % of time exposed for job held in 1982 and 1985 | - r for jobs held in 1982 ranged from 0.54 to 0.74, median = 0.60 |
| - r for jobs held in 1982 ranged from 0.40 to 0.72, median = 0.57 | |||||
| Palmer et al, 200072 | 179 workers in various jobs involving exposure to hand transmitted or whole body vibration in the UK | Self reported exposure to hand transmitted or whole body vibration, in 1-hour period, including name of vibrating equipment and duration of exposure | Observations of workers during one-hour period | - Sensitivity and specificity | - Sensitivity = 0.96 for hand-transmitted vibration, 0.97 for whole body vibration |
| - Specificity = 0.98 and 0.91, respectively | |||||
| - Median ratio of self-reported to observed exposure time = 2.5, range 1.2 to 6.3 for hand-transmitted vibration; Median = 1.1, range 1.0 to 1.2 for whole body vibration |