Previous studies in various countries have described an increased occurrence of asthma among specific groups of healthcare workers (HCWs), including nurses and respiratory therapists.^{1,2,3,4,5,6,7,8,9,10,11} In the USA, the health services industry was second only to the transportation equipment manufacturing sector in total number of reported asthma cases (16% of the total), and five of the top 11 industries and nine of the 22 leading occupations associated with significantly increased asthma mortality were related to healthcare services.¹²

Validation of asthma questionnaires to date has largely focused on their ability to predict asthma in populations.^13–19 Recent studies have underscored the importance, when conducting aetiological research on asthma rather than screening, of developing instruments that favour specificity over sensitivity, both for the definition of asthma as well as for exposure assessment.^20–22 To our knowledge few or no asthma questionnaires, designed for aetiological research, have undergone formal validation in a putative high risk population such as that of healthcare workers.^20,23,24

The purpose of this study was to develop, validate, measure reliability, and refine a new asthma survey instrument for subsequent use in epidemiological studies of healthcare workers. The survey instrument (formatted for use with the Cardiff Teleform software (Cardiff Software, Inc., Vista, CA), to facilitate direct data entry) can be viewed on the OEM website (http://www.occenvmed.com/supplemental).

METHODS

Questionnaire development

The initial draft questionnaire was designed to be completed in under 30 minutes and consisted of four sections: (a) asthma and asthma symptoms (12 questions, with subquestions); (b) occupational exposures and job history (17 questions, with subquestions); (c) non-occupational exposures and asthma risk factors (9 questions); and (d) demographics (8 questions). The survey development team was multidisciplinary and included industrial hygienists, occupational/pulmonary physicians, epidemiologists, and survey design experts. A preliminary version of the questionnaire was first tested for language clarity, ease of completion, timing, and cognition in an initial small pilot study of volunteer HCWs from the Houston area.

Asthma related questions were originally derived from the International Union Against Tuberculosis and Lung Diseases (IUATLD) bronchial symptom questionnaire, and included a cluster of five questions from that instrument that had exhibited the best combination of sensitivity and specificity for the detection of bronchial hyperresponsiveness (collectively referred to as the “discriminant function predictor” or DFP).¹⁵ A separate question on physician diagnosed asthma was also included, as well as questions on age or year of asthma diagnosis and on work absences due to asthma or respiratory symptoms.

The occupational exposure section focused on current and longest jobs held, job titles, practice setting, duration and frequency of exposure to a list of specific chemicals, and a history of exposure to accidental chemical spills or gas releases. Lists and descriptions of chemical agents present in healthcare settings were initially identified from the literature and collapsed into specific sections for development of individual questionnaire items, with the input of three industrial hygienists (TS, ES, LW) and two occupational physicians (GD, AC).^9,23,25 To update these lists, this team also conducted a series of walk-through surveys and review of material safety data sheets in three large Houston hospitals: a 350 bed paediatric hospital (3200 employees); a 450 bed specialty cancer referral centre (10 000 employees); and a 1200 bed tertiary referral and general hospital (4600 employees). This process resulted in the development and inclusion of two separate chemical lists in the initial draft questionnaire: (a) a list of 39 chemical agents, for which respondents were asked to indicate any exposure on at least one occasion per month for six months or longer; and (b) a separate set of questions regarding frequency of exposure (never, at least once a month, at least once a week, every day, or more than once a day) to nine general classes of agents (disinfectants, cleaning agents, latex products, microorganisms, aerosolised medications, mildew, adhesives/glues, gases/vapours, and paints/craft materials).

The non-occupational exposure and asthma risk factors section of the draft survey instrument contained questions related to common environmental aeroallergens and allergies, family history of atopy and asthma, household pets, smoking habits, residential housing characteristics, and recreational exposures, derived where possible from previously developed questionnaires.^14,23

Validation study

A cross-sectional study was conducted in a convenience sample of non-smoking, currently employed HCWs between 18 and 65 years of age, both with and without asthma, recruited via widespread advertisement in the Houston metropolitan area. Exclusion criteria included pregnancy, a prior diagnosis of chronic obstructive pulmonary disease, emphysema, and/or chronic bronchitis. Sample size calculations were based on information obtained from the initial pilot study; calculations were made separately for the asthma (median sensitivity, 55%; specificity, 80%) and non-occupational exposure (“allergy”) (median sensitivity, 41%; specificity, 50%) sections of the questionnaire.²⁶ The resulting minimum sample size was approximately 96 persons.

The study protocol was approved by the University of Texas–Houston Committee for the Protection of Human Subjects. Study participants completed the draft questionnaire, a non-specific bronchial challenge test with methacholine, and a detailed occupational exposure interview with an industrial hygienist, and provided a blood sample for measurement of RAST specific IgE antibodies to a panel of indoor and outdoor aeroallergens common in the southwestern USA. The order in which these various tests were performed was random, and the research team was blinded to the medical histories and questionnaire responses of the study participants. Two weeks after this session, participants were asked to complete a second, abbreviated questionnaire to assess test–retest reliability of responses, measured by the kappa (κ) statistic.²⁷

Internal consistency reliability for item groups in the asthma and non-occupational exposure sections of the questionnaire was measured by Cronbach’s α values.²⁸ Internal consistency was also assessed through exploratory principal factor analysis, applied to the asthma section of the questionnaire, as reported recently.¹⁸ The same procedure was used to identify groupings of chemical agents in the occupational exposure section, in order to shorten the original list of 39 chemicals.²⁹

Asthma related items were validated against two measures, the provocative concentration of methacholine that produced a 20% or greater decrease in forced expired volume in one second (FEV₁) from the baseline (PC₂₀) and a previous physician diagnosis of asthma (MD asthma). Two separate cut-off points for PC₂₀ were evaluated, ⩽8 mg/ml and ⩽4 mg/ml. Previous studies have shown that a “cut point” of 8 mg/ml is clinically practical, as virtually 100% of symptomatic asthmatics and only 4.5% of non-asthmatic subjects will have values at or below this concentration of either methacholine or histamine. The ⩽4 mg/ml level was added as a second cut-off level that could add greater specificity.³⁰ Performance of these questions was also compared to that of the 5-question DFP.^15,16 Prediction equations were developed using PC₂₀ ⩽8 mg/ml, PC₂₀ ⩽4 mg/ml, and MD asthma as dichotomous outcome variables. Variables related to asthma symptoms were then added into the logistic model and sensitivity, specificity, and percentage correctly classified were computed, based on an analysis of receiver operating curve characteristics (ROC). To evaluate construct validity, these questionnaire items were tested for associations with two known non-occupational asthma risk factors (atopy defined by RAST panel results and a family history of hay fever) and one established occupational risk factor (latex sensitisation, defined as an elevated anti-latex IgE antibody). Strength of these associations was expressed as the crude odds ratios (OR) and corresponding 95% confidence intervals.

For the occupational exposure section, criterion validity was assessed by comparing the level of agreement (κ) between the industrial hygienists (taken as the “gold standard”) and the study subject’s self-reported exposure history for selected questions on job/industry classification, as well as for exposure (type, frequency within ± one category level) to the nine classes of agents. Construct validity was evaluated, in a limited fashion, by testing the known association between latex allergy (as self-reported on the questionnaire) and both MD asthma and PC₂₀ ⩽8 mg/ml.

Criterion validity for the non-occupational and asthma risk factor section of the questionnaire was determined by comparison of the allergy related questions to RAST panel serum titres. A serum titre of ⩾0.35 kU/l (class I) for one or more allergens was considered indicative of atopy. Those items that offered the best combination of sensitivity and specificity were retained in the final version of the questionnaire. Construct validity was then examined by testing these latter items for expected associations with asthma and bronchial hyperresponsiveness, as defined by MD asthma and PC₂₀ ⩽8 mg/ml respectively, using simple logistic regression.

RESULTS

One hundred eighteen subjects participated in the validation study. Descriptive statistics of the study population are presented in table 1. Time to completion of the questionnaire ranged from 13 to 25 minutes.

Reliability

Internal consistency for respiratory symptoms (Cronbach’s α = 0.86), allergic symptoms when near animals or trees (Cronbach’s α = 0.86), and allergy questions (Cronbach’s α = 0.89) was excellent. Exploratory principal factor analysis produced a 12-item model that separated asthma related questions into three domains: “wheezing” (4 items: wheezing, wheezing at home, nocturnal wheezing, and nocturnal cough), “shortness of breath” (5 items: shortness of breath, shortness of breath with activity, shortness of breath at home, nocturnal chest tightness, and trouble breathing), and “no asthma” (3 items: absence of wheezing with a cold, absence of a history of asthma, and absence of a prior physician diagnosis of asthma). Each domain was then tested for associations with PC₂₀ ⩽8 mg/ml; all yielded significant associations in the expected direction (“wheezing”, OR = 1.52, p = 0.047; “shortness of breath”, OR = 1.68, p = 0.017; “no asthma”, OR = 0.53, p = 0.008). Similarly, when tested for associations with PC₂₀ ⩽4 mg/ml, all three domains yielded significant odds ratios (“wheezing”, OR = 1.49, p = 0.049; “shortness of breath”, OR = 2.00, p = 0.002; “no asthma”, OR = 0.54, p = 0.006).

Principal factor analysis conducted on the list of 39 chemical agents produced a 28-item model, collapsed into five domains: “cleaning agents”, “sterilising agents/disinfectants”, “strong odours”, “anaesthetics/nebulised medications”, and “miscellaneous” (table 2).

Test–retest reliability ranged from 75% to 95% for both asthma and allergy related questionnaire items (overall κ = 0.70).

Validity

Analysis of ROC characteristics identified a subset of eight asthma related items that offered the best combination of sensitivity and specificity, while retaining good internal consistency (Cronbach’s α = 0.75), when tested against PC₂₀ and MD asthma. Table 3 lists the individual questionnaire items for both the 8-question predictor and the DFP.

Table 4 summarises the results of the 8-question predictor and the 5-item DFP when applied to this study population. Use of the 8-question predictor resulted in 70–94% of study participants being correctly classified with regard to asthma and bronchial hyperresponsiveness, versus 65–93% for the DFP.

Construct validity testing for associations between each of the asthma definitions (8-item predictor, MD asthma, and DFP) and known non-occupational (atopy, family history of hay fever) and occupational (latex sensitisation) factors yielded elevated odds ratios in the expected direction, although some of the confidence intervals included the null (table 5).

In the occupational exposure history section of the questionnaire, when self-reported exposures were compared to the industrial hygienist’s assignment of individual exposures, greater agreement was observed with respect to job titles (κ = 0.57 and 0.67, for longest held and current/most recent job, respectively) than for practice setting (κ = 0.46 and 0.51, for longest held and current/most recent job, respectively). Agreement on type and duration of exposure to agents varied, depending on the agent class. For current/most recent job, agreement was greatest for exposure to latex products (κ = 0.60), disinfectants/sterilising agents (κ = 0.59), cleaning agents (κ = 0.56), aerosolised medications (κ = 0.53), and gases/vapours (κ = 0.48), and marginal for exposure to bacteria/viruses (κ = 0.43) and adhesives/glues (κ = 0.41). Agreement was poor (κ<0.40) for exposure to mildew/fungi and paints/crafts materials. For longest held job, agreement was greatest for exposure to latex products (κ = 0.63), disinfectants/sterilising agents (κ = 0.60), bacteria/viruses (κ = 0.53), and aerosolised medications (κ = 0.45), and marginal for exposure to cleaning agents (κ = 0.44) and gases/vapours (κ = 0.42). Agreement was poor (κ<0.40) for exposure to mildew/fungi, adhesives/glues, and paints/crafts materials. The odds ratios for an association between self-reported latex allergy and both MD asthma or PC₂₀ ⩽8 mg/ml were elevated in the expected direction, although confidence intervals included the null (table 5).

In the non-occupational and asthma risk factor section, items regarding a personal history of allergic conditions and family history of allergic conditions exhibited a wide range of sensitivity (19–74%), but high specificity (71–89%) when compared to RAST panel results. Sensitivity was highest for a history of hay fever (74%) and “dust” allergy (68%), and lowest for allergy to chemicals (19%) and a family history of skin allergies (28%). Specificity was highest for a history of allergy to chemicals (89%), animals (86%), dust mite (86%), and medications (82%), and lowest for hay fever (45%). The best combinations of sensitivity and specificity were obtained for a history of allergies to dust, dust mite, and animals. In the evaluation of construct validity, the odds for an association with either MD asthma or PC₂₀ ⩽8 mg/ml were elevated for all three of these items (table 5).

DISCUSSION

It is well recognised that questionnaire based definitions of asthma may not necessarily correspond to the clinical definition of asthma, and that there is no universally accepted “gold standard” definition of asthma for use in epidemiology studies.¹⁸ Prior validation studies of asthma questionnaires have generally relied on comparison of questionnaire items on asthma and asthma-like symptoms to putative gold standards, including physiological measures of non-specific bronchial hyperresponsiveness, previously validated questionnaires, or physician diagnosed asthma.^15,31,32 Depending on the standard used, as well as on the nature of the questionnaire items, sensitivity and specificity have varied.

The present study used an approach that compared the performance of the asthma section of the questionnaire to all three of these standards (PC₂₀, MD asthma, and the previously validated DFP). The DFP exhibited a specificity of 81% and sensitivity of 52%, virtually identical to those obtained by Burney et al in their original validation studies.¹⁵ In contrast, the 8-item predictor in this study showed a higher sensitivity (71%) and lower specificity (70%), but resulted in a slightly higher percentage of “correctly classified” cases than the DFP (70% versus 65%). As a predictor of bronchial hyperresponsiveness, therefore, there was little measurable difference between the 8-item predictor and the DFP. Kongerud et al used a modified MRC questionnaire to test 296 workers in a Norwegian aluminium plant, and compared questionnaire responses to the clinical judgement of a chest physician.³² Questions on wheezing and dyspnoea showed sensitivities of 77% and 75%, and specificities of 82% and 88%, respectively. The sensitivity of 79% and specificity of 98% found with our 8-item predictor (94% of cases correctly classified), therefore, compares favourably with these results and slightly better than the DFP, supporting its suitability for use in future asthma epidemiology studies. The combination of several symptom based questions to define asthma has been found to perform better, and is less conducive to misclassification, than reliance on a single question or questions that include the term “asthma”.¹⁸

An important limitation of many occupational asthma surveys is the inability to distinguish between pre-existing asthma and work related asthma. Although not specifically validated in this study, this questionnaire also includes items regarding time of asthma onset (relative to entry into the healthcare profession), worsening of asthma and/or respiratory symptoms with work, amelioration when away from work, and work absences due to asthma and/or respiratory symptoms. Combining these questions with the validated asthma and bronchial hyperresponsiveness predictors should allow a better approximation to these asthma–workplace relationships.^5,33

Various methods are used to retrospectively assess occupational exposures in epidemiology studies, including self-reported exposure, detailed interviews with workers or relatives, expert industrial hygiene assessment of occupational histories, and use of a priori developed job-exposure matrices; each of these methods has its limitations. In this study, the occupational exposure section of the questionnaire is based on self-reported exposures to a list of agents, established through a detailed process that included review of previous lists, hospital walk-through surveys, and exploratory principal factor analysis, to identify a fairly comprehensive checklist of agents that is still brief enough to be answered in a short period of time. Agreement between study participants and the industrial hygienists (median κ = 0.45) was similar to a previous study that examined this issue in a case-control study of cancer and occupational exposures in Canada (median κ = 0.51).³⁴ Self-reported exposure to checklists of chemical agents has been reported to have high specificity (ranging from 83% to 97%), but low sensitivity (median 61%; range 39–91%) when compared to expert assessment by hygienists and chemists, which could lead to misclassification of exposure when used in population based studies.³⁴ On the other hand, detailed industrial hygiene interviews with workers or their proxy require a large time commitment, are costly and often logistically difficult, and have exhibited suboptimal validity.^35–38 Fritschi and colleagues also questioned whether use of expensive, time consuming expert assessment was an acceptable “gold standard”, acknowledging that few data exist on the validity of this method.³⁴ Louik et al further note that expert assessment is also limited by a scarcity of qualified experts.³⁸

The similarity of findings between this validation study and previous studies underscores the need to incorporate more reliable methods of exposure assessment that go beyond self-report. Conceivably, in the case of our questionnaire, some advantage might be gained by combining both methods sequentially—that is, the self-reported exposures could be subjected to subsequent additional review by one or more industrial hygiene experts, and this is being explored. However, we have also developed a healthcare worker specific job-exposure matrix, focused on asthmagens, that emphasises high specificity, and which is undergoing detailed validation and field testing.^20,21,33

Certain limitations of this study should be noted. Although methacholine challenge testing was selected as the “gold standard” for asthma, it is well known that airway hyperresponsiveness is present in a certain proportion of asymptomatic persons without asthma, which could affect the specificity of certain questionnaire items.^39,40 Use of more than one “gold standard” for the definition of asthma in this study probably offset this effect, by providing a range of sensitivity and specificity values for the 8-item predictor that may allow a broader characterisation of susceptible subgroups. A similar issue arises for the RAST antibody panel, where some asymptomatic persons may have significantly elevated titres of these antibodies.⁴¹ However, the good specificity (86%) shown by the questionnaire items for allergens known to be strongly related to asthma (dust mite and animals) suggests that this effect was small. Differences in opinions and judgement among professionals are a fact of life, and using an industrial hygienist review and classification of occupational exposures as the “gold standard” for questionnaire items in the occupational exposure section can introduce misclassification bias. The development of the previously mentioned highly specific job-exposure matrix, with multiple levels of expert input, should decrease this effect.

In this convenience sample, the large number of persons with previously diagnosed asthma, bronchial hyperresponsiveness, and atopy provided a sufficient number of cases to test questionnaire validity. In this regard, the study population was not likely to be strictly representative of the target population. On the other hand, the study population would be expected to be representative of the general healthcare worker population by virtue of profession (all were healthcare workers), educational level and language, providing confidence on the relevance, understanding, ease of completion, and applicability of the various questionnaire items to this worker population.

All study participants were non-smokers, in order to reduce confounding from chronic obstructive pulmonary disease in the validation study. It may be more difficult to control this effect when the questionnaire is applied in a population based study. A section on smoking history is part of the final questionnaire, which will allow control for this confounder.

Based on findings from this validation study, the questionnaire was reduced to 43 main questions and appeared to be easily completed by participants, making it applicable for use in other healthcare worker groups, including housekeeping personnel, security, facilities maintenance, etc. The education level of the study population was quite high, however; thus care should be taken before using this questionnaire in a broader cross-section without further cognitive testing and validation.

In summary, initial evaluation of the performance of this new questionnaire for the evaluation of asthma in healthcare workers indicates good validity and reliability for the detection of asthma and for the characterisation of non-occupational exposures and other asthma risk factors. Although occupational exposure assessment was shown to have a reliability similar to previous studies, it would be preferable to develop additional approaches that go beyond self-report, providing a separate measure of exposure to workplace risk factors. Further field testing and cross-validation of this instrument are currently being undertaken by our group in a large cross-sectional study of licensed healthcare professionals in Texas. Although the instrument was specifically designed for use in the healthcare sector, if the field studies support this validation study, then this methodology could also be adapted to studies of other worker populations.