Systematic review of methods used to assess exposure to pesticides in occupational epidemiology studies, 1993–2017
====================================================================================================================

* Johan Ohlander
* Samuel Fuhrimann
* Ioannis Basinas
* John W Cherrie
* Karen S Galea
* Andrew C Povey
* Martie van Tongeren
* Anne-Helen Harding
* Kate Jones
* Roel Vermeulen
* Hans Kromhout

## Abstract

**Objective** Numerous exposure assessment methods (EAM) exist for investigating health effects of occupational exposure to pesticides. Direct (eg, biomonitoring) and indirect methods (eg, self-reported exposures) are however associated with degrees of exposure misclassification. We systematically reviewed EAM in studies of occupational pesticide exposure.

**Methods** We searched for articles reporting observational epidemiological studies in MEDLINE and Embase published 1993 to 2017. The relative frequency of EAM was analysed according to EAM type (direct and indirect methods), health outcome, study design, study location (country) and specificity of assessment. Temporal trends in EAM were analysed.

**Results** In 1298 included articles 1521 EAM occurrences were documented. Indirect EAM (78.3%), primarily self-reported exposures (39.3%) and job titles assessments (9.5%), were mainly applied in case-control studies (95.0%), in high-income countries (85.0%) and in studies of doctor-diagnosed health outcomes (>85%). Direct EAM (20.8%), primarily biomonitoring of blood (15.6%) or urine (4.7%), were predominantly applied in cross-sectional studies (29.8%), in lower middle-income countries (40.9%) and in studies of neurological (50.0%) outcomes. Between 1993 to 2017 no distinct time trends regarding the ratio indirect to direct methods was seen. Within the category of indirect methods use of self-reported exposures and job exposure matrices increased while assessments by job titles and registers decreased. The use of algorithms showed no trend. The specificity of pesticide assessment increased since studies assessing exposure by using job title as a proxy declined. Assessments of type of pesticide increased.

**Conclusion** Over the last 25 years, the ratio (5:1) of indirect to direct EAM applied in articles on occupational pesticide epidemiology stayed relatively constant; changes were mainly attributable to increasing use of self-reported exposures and job exposure matrices. This review, combined with studies assessing EAM validity, will inform on magnitudes of exposure misclassification and help improve the quality of studies on occupational pesticides exposure.

*   pesticides
*   exposure assessment
*   epidemiology
*   systematic review

### Key messages

#### What is already known about this subject?

*   Numerous exposure assessment methods (EAM) in occupational epidemiology have been developed for classifying workers’ pesticide exposure. Both direct (eg, biomonitoring) and indirect (eg, self-reported exposures) EAM are however intrinsically associated with degrees of exposure misclassification, which might lead to conflicting study results.

#### What are the new findings?

*   In this systematic review of articles on occupational exposure to pesticides published between 1993 to 2017, the majority of documented EAM applied were indirect, particularly based on self-reported exposure. The use of self-reported exposures and job exposure matrices increased throughout the study period, whereas the use of expert-assessments and job title assessments decreased. The use of algorithms and predictive models showed no trend. Likewise, no temporal trends in the use of different direct EAM were observed. The specificity of pesticide assessment increased since studies assessing exposure by using job title as a proxy declined, while assessment of type of pesticide increased. For assessments at the level of active ingredients only a temporary increase was seen.

#### How might this impact on policy or clinical practice in the foreseeable future?

*   This review, combined with future studies assessing the validity of applied EAM, such as the IMPRESS project (*Improving exposure assessment methodologies for epidemiological studies on pesticides*), will provide information on the magnitude of exposure misclassification and suggest ways to improve the quality of human observational studies on pesticides exposure.

## Introduction

Occupational exposure to pesticides has been associated with several health outcomes including cancer,1 2 neurological diseases,3 4 mental disorders,2 rheumatoid arthritis,5 respiratory diseases,6–8 various genetic biomarkers9 10 and reproductive and developmental disorders.3 11 Occupationally exposed populations comprise for example, agricultural workers, gardeners, flower growers, forestry workers and hygiene and pest control workers.12 The assessment of occupational pesticide exposure is however methodologically challenging. In agricultural populations — the most frequently studied — living and working conditions are often hard to separate,3 with take-home exposures potentially biassing occupational exposure assessments.13 Exposure in farm workers may vary according to treated crop, (micro) climate, professional training, task performed, application method, use of personal protection equipment (PPE) and control measures, clothing and hygiene.14 Additionally, an increasing number and quantity of different active ingredients are being applied15; globally the average use of pesticides per area of cropland increased from 1.5 kg/ha in 1990 to 2.57 kg/ha in 2016.16 These conditions generate heterogeneous exposure patterns within working populations with variations between workers, between exposed body parts, over time17 18 and space.19 

Numerous direct and indirect exposure assessment methods (EAM) have been developed for classifying workers’ pesticide exposure. Direct methods rely on direct measurements by any type of biomonitoring, primarily through sampling of biomarkers or metabolites in blood, urine or skin in the target population,14 or by personal sampling of the workers’ breathing zone (inhalation exposure assessment) or skin (dermal exposure assessment). However, direct measurements of the exposure are generally feasible only in prospective cohort and cross-sectional studies.2 20 Consequently, indirect methods for pesticide exposure assessment have been developed and applied. These include assessments based on job title, self-reported exposures and job histories by self-administered or interview-administered questionnaires, records and registers (eg, the NIOSH dioxin register),21 expert case-by-case assessments and environmental monitoring of exposure levels.22 Exposure data collected in indirect EAM are often further used for semi-quantitative methods such as job exposure matrices (JEM)8 23–25 and crop exposure matrices (CEM).26 More recently, exposure algorithms and empirical models have been developed to estimate exposure-intensity scores based on various exposure-modifying factors such as application equipment, PPE use or hygienic behaviour.27 28 

Existing EAM are however intrinsically associated with degrees of exposure misclassification, which might lead to conflicting study results. In a systematic review and meta-analysis of studies on occupational pesticide exposure and prostate cancer, significant associations were found in studies using exposure assessments based on group level, but not in studies which sampled serum pesticide levels.1 Moreover, Daniels *et al* 20 analysed the effect of applying different EAM in workers in relation to neuroblastoma risk in offspring. Exposure classifications based on various degrees of self-reports, data based on industry type and on industrial hygiene review revealed ORs ranging from 1.0 to 1.9 in men and 0.7 to 3.2 in women.

Consequently, the choice of EAM may account for considerable variations in workers’ assigned exposure and resulting disease risk estimates. The IMPRESS project (*Improving exposure assessment methodologies for epidemiological studies on pesticides), [www.impress-project.org](http://www.impress-project.org),* aims to better understand the performance of existing methods of occupational exposure assessment to pesticides used in epidemiological studies, and to use this information to recommend improvements in scientific practice for the future. Within the IMPRESS project, we performed a systematic review of EAM used in epidemiological studies of occupational exposure to pesticides.

## Methods

We searched for EAM in peer-reviewed literature on occupational exposure to pesticides in relation to any health outcome published from 1st January 1993 to 31st December 2017 in English, French, German, Spanish, Dutch, Swedish, Danish and Norwegian. Searches were made in the two largest databases for life sciences and biomedical literatures, Embase and MEDLINE, enabling a comprehensive search.29 30 The searches were made unrestricted regarding type of pesticide, health outcome or study design using both subject headings, and keywords including relevant synonyms qualified by titles/abstracts.

The search syntax (online supplementary file 1) included the following subject-heading based search terms: ‘pesticide’ AND (‘occupational exposure’ OR ‘dermal exposure’ OR ‘dietary exposure’ OR ‘inhalation exposure’ OR ‘ingestion exposure’ OR ‘environmental monitoring’ OR ‘paternal exposure’ OR ‘maternal exposure’). The syntax also included keyword searches based on the subject headings and terms reflecting relevant exposure types not listed in the database-specific structure of subject headings for MEDLINE and Embase, the MeSH-tree and the Emtree, respectively — for example, ‘eye exposure’, and ‘nasal exposure’. The syntax further included keywords reflecting work tasks associated with pesticide exposure (eg, handling crops or animals, or forestry work) and professions which traditionally, typically or potentially are exposed to pesticides (eg, mixers, irrigators, veterinarians and farmers).

### Supplementary data

[[oemed-2019-105880supp001.pdf]](pending:yes) 

### Article selection

Title and abstract screening of retrieved articles was performed by the lead author of this manuscript (JO) using the screening tool Rayyan (rayyan.qcri.org).

The following inclusion criteria were applied: peer-reviewed original observational epidemiological studies (including conference abstracts) describing analyses of occupational exposure to pesticides in association to any health outcome. Review articles, methodological articles, toxicological studies, clinical trials, case reports, case series and descriptive studies without health endpoints were excluded. Included articles were screened for eligibility a second time by JO. Furthermore, a second independent reviewer (HK) assessed a random selection of 5% of included articles for eligibility and extracted data, and a random selection of 1% of excluded articles for eligibility. The final selection of articles was stored in EndNote (V.X8.2) for further analysis.

### Data extraction

The following data were extracted from each included article: type of EAM, specificity of EAM assessment, study design, study location (country), health outcome, authors of article, year of publication and journal. When possible EAM, study design, study location and health outcomes were extracted in abstracts, otherwise in full-text versions.

Based on the extracted data, two EAM categories were created, these being indirect and direct EAM. Indirect EAM included the following methods: job title, expert case-by-case assessment, self-reported exposure (by self-administered questionnaire or by interview administered questionnaires), self-reported job history (by self-administered questionnaire or by interview administered questionnaires), register data, job, task exposure and crop exposure matrices, algorithms/empirical models, indexes, scores, metrics, environmental air monitoring and geographical information systems. Direct EAM included the following methods: biomonitoring (by blood, urine, skin, hair or adipose tissue), and personal air and dermal sampling. Some articles considered cholinesterase activity as a measure of exposure, others a biomarker of (health) effect. The former were classified into the EAM category ‘biomonitoring by blood’’, which most frequently included measures of cholinesterase inhibition, dichlorodiphenyldichloroethylene, and 2,3,7,8-Tetrachlorodibenzo-*p-*dioxin. The category ‘biomonitoring by urine’ comprised most frequently of dialkyl phosphate, ethylene thiourea and 3,5,6-trichloro-2-pyridinol. Eligible articles that did not report on applied EAM were grouped into the EAM category ‘EAM not reported’.

We also classified at which level of specificity the applied EAM assessed exposure: job title, pesticides in general, type of pesticide (eg, insecticides, herbicides, fungicides), chemical class (eg, organochlorines, organophosphates, carbamates) and specific active ingredient (eg, captan, chlordane, permethrin).

Furthermore, the articles were classified by study design: cross-sectional studies, prospective cohort studies, retrospective cohort studies and case-control studies. Articles describing analyses of cross-sectional data originating from cohort studies were classified as cross-sectional studies.

Using the study location data we classified the articles according to the World Bank Atlas method31 : low-income countries (LIC), lower middle-income countries (LMIC), upper middle-income countries (UMIC) and high-income countries (HIC). Articles performed at multiple study locations were classified as either all high-income countries (AHIC), or some high-income countries (SHIC).

Finally, the articles were classified based on the study health outcome in three ways:

1.  By type of health outcome: biochemical, cancer, dermatological, endocrine/nutritional/metabolic, genetic biomarkers (eg, genotoxicity, DNA damage, micronuclei frequency), genitourinary, haematological, immunological, mental disorders, morbidity, general (all cause) mortality, neurological, outcomes in offspring, pesticide-related symptoms, pesticide-related illness, reproductive, respiratory and other (miscellaneous outcomes with very low frequencies, for example, musculoskeletal disorders, suicide and diseases of the circulatory system).

2.  By health outcome assessment type: self-reported outcomes, doctor-diagnosed outcomes (based on assessments in individual studies or via outcome registers) and diagnostic/medical test results (from here referred to as test results) (eg, analyses of DNA damage, lung function or neurobehavioral tests).

3.  A final health outcome category was created by combining the most frequently analysed health outcome types in 1 and 2 with resulting in the following categories: self-reported outcomes, cancer (doctor-diagnosed), genetic biomarkers (test results), neurological (doctor-diagnosed or test results), reproductive and offspring (doctor-diagnosed or test results), respiratory (doctor-diagnosed or test results), general mortality (doctor-diagnosed) and other (miscellaneous outcomes with very low frequencies)

### Analysis

Using R V.3.5.032 we estimated the relative frequency of EAM occurrences in included articles according to individual EAM, indirect EAM and direct EAM over the whole 25 year period. To detect temporal trends relative frequencies of EAM (individual, indirect, direct), study design, outcome categories, study location and specificity of EAM were analysed by 5 year periods. We further estimated the relative frequency of types of EAM occurrences by study design, outcome category and study location. Study types and health outcomes were also analysed by study location. We additionally constructed a map plot showing number of articles according to country of study location. For this purpose (only) articles reporting international multicentre studies were counted multiple times according to the number of individual countries serving as study location in the article.

## Results

The searches in MEDLINE and Embase resulted in 8945 articles of which 252 were duplicates, leaving 8693 unique articles for title and abstract screening (figure 1).

![Figure 1](http://oem.bmj.com/https://oem.bmj.com/content/oemed/77/6/357/F1.medium.gif)

[Figure 1](http://oem.bmj.com/content/77/6/357/F1)

Figure 1 
Screening and selection of articles resulting from searches in MEDLINE and Embase.

During title screening we excluded 4512 articles considered off topic (did not fulfil inclusion criteria) and another 241 reviews. Of the remaining 3940 articles subjected to abstract screening, the following were excluded: 776 review articles, 243 articles describing methodologies of exposure assessment only, 695 articles without any association between pesticide exposure and a health outcome presented, 344 articles analysing non-occupational populations (eg, children, animals or cells) and 376 articles analysing non-occupational exposures (eg, residential pesticide exposure, para-occupational exposure). In total 1506 articles fulfilled the stated eligibility criteria. Of these, 235 articles could not be included for data extraction and analysis, as they did not report on the EAM applied in the abstract and could not be accessed in full-text by the researchers. This left a total of 1271 articles with extracted data (online supplementary file 2).

### Supplementary data

[[oemed-2019-105880supp002.pdf]](pending:yes) 

### Number of articles over time

Over the 25 year period we observed an increasing number of articles and more than a doubling in the number of relevant articles, with 152 articles published between 1993 to 1997, 224 articles between 1998 to 2002, 274 articles between 2003 to 2007, 289 articles between 2008 to 2012 and 332 between 2013 to 2017 (table 1).

View this table:
[Table 1](http://oem.bmj.com/content/77/6/357/T1)

Table 1 
Total and 5 year interval relative frequencies of exposure assessment methods (EAM), and related study types, study locations, health outcomes and specificity of assessment in studies of occupational pesticide exposures and various health outcomes

### Number of articles by country of study location

The 1271 articles concerned studies performed in 97 countries (figure 2). The USA served as most frequent study location for a total of 407 articles. Other frequent study locations (in 30 to 84 articles) were Canada, Mexico, Brazil, India, China, Sweden, Denmark, the UK, Spain, France, Italy, Germany and the Netherlands. With the exception of Egypt (in 15 to 29 articles), South Africa and Tunisia (both in 5 to 14 articles), few articles described studies performed in Africa.

![Figure 2](http://oem.bmj.com/https://oem.bmj.com/content/oemed/77/6/357/F2.medium.gif)

[Figure 2](http://oem.bmj.com/content/77/6/357/F2)

Figure 2 
Frequency strata showing number of reviewed and included articles on occupational exposure to pesticides and various health outcomes published between 1993 to 2017 per country serving as study location. Numbers in brackets represent the number of different study locations (countries) for each article frequency strata. Articles reporting multicentre studies are in the plot (only) counted multiple times according to the number of individual countries serving as study location in the article.

### Relative frequencies of EAM, study designs, outcomes, study locations and specificity of assessment during 1993 to 2017

From the 1271 articles we extracted a total of 1483 EAM occurrences; 203 articles (16.0%) reported using more than one EAM (table 1). Of all EAM occurrences documented 1993 to 2017, indirect methods were most commonly applied (82.2%) of which the most frequent were self-reported exposures by self-administered or interview-administered questionnaires (39.9%), followed by assessment by job title (12.3%). Direct EAM (17.1%) were primarily based on biomonitoring of workers’ blood (11.8%) or urine (4.7%).

Documented EAM were applied almost equally in cross-sectional studies (31.4%), prospective cohort studies (34.1%) and case-control studies (30.1%) (table 1). The use of retrospective cohort studies was rare (1.3%).

Regarding health outcomes, the majority of EAM were documented in studies of doctor-diagnosed cancer outcomes (26.6%), followed by self-reported outcomes (13.8%) and genetic biomarkers outcomes (12.4%) (table 1).

The majority of documented EAM originated from studies performed in HIC (67.9%), followed by studies in UMIC (15.0%) (table 1). Very few EAM resulted from studies in LIC (1.1%), and relatively few articles from international multicentre studies (3.5%).

The specificity of assessment of the used EAMs was most frequently pesticides in general (32.1%), followed by exposure to specific active ingredient (25.6%), chemical class (18.4%) and job title (15.9%), and least frequently according to type of pesticide (7.6%) (table 1).

### Time trends in EAM, outcomes, study designs, study locations and specificity of assessment during 1993 to 2017

Although fluctuating around the average of 82.2%, the use of indirect EAM remained relatively constant throughout the 25 years considered (table 1). However, within the category of indirect EAM, the percentage of job title declined from 14.2% during 1993 to 1997 to 9.6% during 2013 to 2017. Further, the percentage of expert case-by-case assessments declined from 9.8% to 2.6% while JEM increased from 2.2% during 1993 to 1997 to 8.1% during 2013 to 2017. The use of exposure data in registers declined from 10.9% to 3.9%, while the use of self-reported exposures assessed by self-administered or interview-administered questionnaires showed a steady increase, comprising 26.8% of EAM occurrences during 1993 to 1997 and 44.8% during 2013 to 2017. The use of algorithms or predictive models, or job histories showed no temporal trend.

With regard to time trends in the relative frequency of direct EAM, no large changes were seen; biomonitoring by blood fluctuated between 9.6% and 13.1% with an exceptional increase to 17.6% during 1998 to 2002, and biomonitoring by urine varied between 2.5% and 6.0%.

Over the 25 year time-period cross-sectional, prospective cohort and case-control studies accounted each for about one-third of the articles analysed; a ratio which remained relatively constant over time. (table 1). However, the use of cross-sectional designs increased slightly from 25.7% during 1993 to 1997 to 33.6% 2013 to 2017.

The percentage EAM occurrences documented in studies of general mortality as outcome declined from 16.9% to 1.3% (table 1) and increased in studies analysing genetic biomarkers test results (from 6.6% to 14.1%) and doctor-diagnosed neurological outcomes (from 4.4% to 7.0%).

A consistent decline in the percentage of EAM occurrences in HIC was seen (80.3% during 1993 to 1997 vs 54.7% during 2013 to 2017) (table 1). In contrast, an increase was seen in LMIC (6.0% during 1993 to 1997 vs 14.6% during 2013 to 2017) and UMIC (7.7% during 1993 to 1997 vs 22.1% during 2013 to 2017).

Concerning the specificity of pesticide assessment a decline in use of job title as a proxy was seen from 21.3% during 1992 to 1997 to 15.1% during 2012 to 2017 (table 1). Concurrently, assessment by type of pesticide more than doubled from 4.4% during 1992 to 1997 to 9.6% during 2012 to 2017. Assessment of active ingredient showed an increase from approximately 22% to approximately 30%, but fell back to the approximately 22% in the most recent period.

### EAM by study design

Stratifying EAM occurrences by study design, the percentage of indirect EAM was highest in case-control (96.2%) and retrospective cohort studies (95%), whereas the percentage of direct methods was highest in cross-sectional studies (23.8%) and in prospective cohort studies (23.2%) (table 2). The percentage of job title as EAM was highest in retrospective cohort studies (40.0%) and cross-sectional studies (18.5%). Moreover, retrospective cohort studies showed the largest percentage of exposure assessment via register data (25.0%). The percentage of self-reported exposures was highest in case-control studies (47.4%) and cross-sectional studies (44.8%). Further, case-control studies showed the highest percentage of expert case-by-case assessments of workers’ exposure (11.2%) and JEM (10.1%). JEM comprised approximately 3% to 10% of reported EAM in cohort studies (retrospective and prospective) and case-control studies, however less than 1% in cross-sectional studies. The percentage of biomonitoring by blood sampling and urine sampling was highest in cross-sectional studies (17.2% and 6.2%) and prospective cohort studies (15.2% and 6.9%) (≤5% in the other study designs).

View this table:
[Table 2](http://oem.bmj.com/content/77/6/357/T2)

Table 2 
Relative frequencies of exposure assessment methods (EAM) according to study design extracted from articles on occupational pesticide exposures and various health outcomes

### EAM by outcomes

Stratifying by outcome category, the percentage of indirect EAM was highest in articles analysing doctor-diagnosed outcomes; reproductive and offspring outcomes (97.9%), neurological outcomes (96.8%) and cancer (95.4%) (table 3). By contrast, direct EAM comprised 40.3% of EAM occurrences in articles with neurological test results as health outcome. Self-reported exposures were the preferred method in the majority of articles on respiratory outcomes (doctor-diagnosed=83.3%) and self-reported outcomes (63.9%). The corresponding percentage for cancer (42.6%), doctor-diagnosed neurological outcomes (46.8%), genetic biomarker outcomes (35.9%) and reproductive and offspring-related outcomes (doctor-diagnosed=29.9%, test results=32.9%) was considerably lower. The percentage of blood sampling was highest in articles analysing neurological test results (23.4%), for example, cholinesterase activity, and not applied in articles analysing doctor-diagnosed respiratory outcomes, when comparing across study designs.

View this table:
[Table 3](http://oem.bmj.com/content/77/6/357/T3)

Table 3 
Relative frequencies of exposure assessment methods (EAM) according to outcomes categories in articles on occupational pesticide exposure and various health outcomes

### EAM by study location

The percentage of direct methods was higher in studies in LMIC and UMIC (30.3% and 24.4%) compared with in HIC (13.0%) (table 4). In contrast, the percentage of indirect methods was highest in HIC (86.8%) and AHIC (91.7%), however relatively high in LIC (82.4%), UMIC (73.9%) and LMIC (67.7%). Self-reported exposures were the most frequently applied EAM regardless study location, however the highest percentage was documented in LIC (58.8%) and HIC (40.3%). The percentage of job title as EAM was highest in UMIC (22.1%), more than twice that registered in HIC (9.8%). Use of JEM and expert assessments was highest in AHIC (22.2% and 13.9%) and HIC (5.6% and 7.1%), and hardly applied in LIC, LMIC and UMIC (≤2.0%). Algorithms and predictive models were almost exclusively used in HIC (7.5%). Biomonitoring by blood sampling was most commonly applied in LMIC (25.5%), and biomonitoring by urine samples in SHIC (12.5%) and UMIC (10.0%).

View this table:
[Table 4](http://oem.bmj.com/content/77/6/357/T4)

Table 4 
Relative frequencies of exposure assessment methods (EAM) according to study location extracted from articles on occupational pesticide exposures and various health outcomes

### Health outcomes and study designs by study locations

Types of health outcomes by study location are presented in online supplementary file 3. The percentage of cancer studies in HIC (34.0%) and AHIC (36.2%) was approximately three times higher than in LMIC and UMIC, and the percentage of self-reported health outcomes was at least three times higher in LIC (70.6%) compared with other study locations.

### Supplementary data

[[oemed-2019-105880supp003.pdf]](pending:yes) 

Types of study designs by study location are presented in online supplementary file 4. The percentage of case-control studies in HIC (33.5%), SHIC (43.8%) and AHIC (66.7%) was at least 15% higher compared with LMIC and UMIC, and the percentage of prospective cohort studies in HIC (40.6%) was about twice as high compared with lower income countries.

### Supplementary data

[[oemed-2019-105880supp004.pdf]](pending:yes) 

## Discussion

We performed a systematic review of EAM used in articles on occupational exposure to pesticides published 1993 to 2017. The majority of analysed EAM comprised indirect methods, particularly methods making use of self-reports, which accounted for almost 40% of the total number of documented EAM. The percentage of indirect methods was highest in retrospective cohort studies and case-control studies, in analyses of doctor-diagnosed health outcomes, and in articles covering studies in HICs. For direct EAM, blood sampling showed the highest percentage accounting for approximately every tenth EAM occurrence. The percentage of direct exposure assessment methods was highest in cross-sectional and prospective cohort studies, in analyses of neurological test results and in LMICs.

Throughout 1993 to 2017 we saw a trend towards decreasing use of assessments by job titles, register data and expert case-by-case evaluations. Concurrently, the use of self-reported exposures and JEM increased. Cross-sectional, prospective cohort and case-control studies accounted each for about one-third of the articles analysed; a ratio which remained relatively constant throughout the 25 year period. Meanwhile, we observed a decline in the percentage of EAM applied in articles analysing general (all cause) mortality and an increase in those analysing genetic biomarker test results and doctor-diagnosed neurological outcomes. Regarding the specificity of assessment of reviewed EAM no consistent time trends were seen, apart from a decrease in assessment by job titles and an increase in assessments by type of pesticide. There was also a temporary increase in assessments of the level of active ingredients, but in later years this increase disappeared. Articles describing health effects from occupational exposure to pesticides with documented EAM have become more prevalent in LMIC and UMIC, whereas a decrease was seen in HIC the last 10 years. The former increases might be related to the relocation of certain industries to LMIC and UMIC, for example, flower production in countries like Ecuador, Mexico and Kenya,33 34 in combination with a higher awareness of the potential negative effects of occupational pesticide exposure and resulting increased funding for studies of such health effects in working populations.

Our review of EAM used in studies of occupational pesticide exposure is to our knowledge the most comprehensive to date. By searching MEDLINE and Embase we retrieved articles from two of the largest databases of life sciences and biomedical literature. The review process covered almost 9000 articles, of which almost half were removed during title screening. Additionally, through using combinations of subject headings and relevant keywords, we further tailored our search syntax to target potentially relevant articles not indexed by subject headings as ‘occupational exposure’. A large part of included articles originated from the agricultural health study (AHS),28 in which applicators at enrolment self-reported their use of pesticides. A sensitivity analysis showed however that the exclusion of AHS articles merely weakened the reported positive time trend for use of self-reported exposures (data not shown in tables).

This review naturally has its limitations. Due to time and budget constraints it was not possible for a second reviewer to independently assess the eligibility and extracted data of all included articles. Nevertheless, a second independent review of 5% randomly selected included articles showed agreement of 95% regarding eligibility and extracted items, and a random selection of 1% of excluded articles showed 100% agreement on eligibility. Moreover, during the data extraction procedure all included articles were assessed twice for eligibility. Still, combinations of EAM in some articles might remain undetected, as EAM for the majority of articles were extracted from abstracts; EAM were searched for in full-text versions when none were reported in abstracts. The frequency of articles using more than one EAM is thus possibly higher than reported in our review as the abstract may not have referred to all those used. Finally, the EAM of 235 relevant articles remains unknown, as these were not accessible in full-text. These publications were however equally distributed over time, reducing a potential bias of reported trends in EAM (data not shown in tables).

The decline in the use of job title as an EAM might be associated with reduced exposure misclassification. Assessments by job title may introduce bias as, for example, not all farmers apply pesticides, and work tasks may vary during a working life.2 35 Yet a large percentage of documented EAM comprised self-reported exposures, to a high degree applied in studies on self-reported outcomes and cancer, which, particularly in case-control settings, might indicate a large potential of differential misclassification and responder bias. In agreement with Ge *et al* 36 we registered an increase in the use of self-reports by questionnaires as of the 1990s until end 2017. Although self-reports usually offer a convenient approach for assessing exposure at the individual level,36 the validity of self-reported exposures regarding lifelong use of pesticides is questionable and variable. As the validity of self-reports per se depends on the applicators’ knowledge of the applied pesticides, self-reported exposures by farmers knowing which active ingredients were used15 37 will be less prone to misclassification compared with hired farm workers or re-entry workers reporting on their pesticide exposure. Our review further showed a low relative frequency of exposure algorithms and deterministic models, with no distinct time trend regarding the use of these. This result contradicts the work by Ge *et al*, who saw an increase in the use of algorithms, although they only considered cancer case-control studies.36 

We further saw a decline in the use of expert case-by-case assessments, which often is considered the ‘golden standard’ in indirect EAM.22 Concurrently, we saw an increase in the use of JEM. It is possible that these trends are interrelated and partly driven by the advantage of JEM incorporating expert knowledge, thus reducing the necessity for the often time-consuming and costly expert case-by-case assessment. Expert assessment is additionally a subjective method, hard to reproduce.2 38 In contrast, JEMs assign workers’ exposure through combining expert knowledge and job histories in a systematic, transparent manner, with high reproducibility. Despite such advantages, comparisons of the performance between JEMs and self-reported data or expert assessment have shown inconsistent results.2 JEMs, however, performed better when job history data were scarce,2 39 and when the prevalence of exposure in populations is common (>10%).40 JEM should be more stringent in assigning exposure when applied in the general population (eg, case-control studies where exposure is most often less than 10%).17 The validity and reliability of self-reported occupational histories (including employment duration, job title and industry), which informs the JEM, is however considered reasonably good.2 22 

Biomonitoring by collection of blood samples to detect acetylcholinesterase activity or relevant metabolites was the second most frequently applied EAM. When comparing biomarkers with JEM, reasonable agreement has been documented.26 41 Still, a review of the performance of surrogate exposures measures concluded that validation studies comparing surrogate measures with biological and environmental exposure monitoring data are needed.42 Additionally, the relevance of biomonitoring is sensitive to the half-life of the monitored substance, making the timing of assessment important.43 

Evidently, the choice of EAM is affected by multiple factors of which some are difficult to control or take into account. Precise monitoring and documentation of pesticide exposure is difficult, partly due to large temporal and spatial variance, and cost, time and logistical issues. Registration of pesticide use is compulsory in some countries, for example, for farmers in the UK, but is not the case in most developing regions of the world.44 Mobile phone technology might provide a cost-efficient way of overcoming some of these problems through monitoring pesticide application and associated exposure with automatic determination of time and location.45 

In our review 16.0% of studies applied multiple EAM. Combining EAM might be a recommendable approach for counterbalancing the weaknesses of one EAM with the strengths of another. Our review provides an overview of the majority of EAM applied during the last 25 years and points out how these are related to epidemiological study design, study location, pesticide-related health outcomes and specificity of assessment. To reduce exposure misclassification and the generation of spurious associations in occupational exposure to pesticides epidemiology, the strengths and limitations of each EAM should be thoroughly considered before the most appropriate methods are chosen.

## Conclusion

In studies of occupational exposure to pesticides the majority of EAM applied were indirect, particularly based on self-reported exposure. Over the analysed 25 years the ratio of indirect to direct EAM (5:1) was relatively constant, although with changes within the category of indirect EAM, particularly attributable to increasing use of self-reported exposures and a declining use of exposure assignment by job title and expert case-by-case assessment. Modelled quantitative exposure data, exposure algorithms and their use in combination with more traditional methods are still very scarce in studies on pesticide-related health effects. These methods should be further developed and implemented. This review combined with future studies assessing the validity of applied EAM, such as the IMPRESS project, will inform researchers on the magnitude of exposure misclassification when applying a certain method, and provide ways to improve the quality of human observational studies on pesticides exposure leading to more informative study results.

## Acknowledgments

The authors would like to thank the IMPRESS Advisory Board members Professor Aron Blair (USA), Professor Len Levy (UK), Dr Mark Montforts (NL) and Professor Silvia Fustinoni (IT), for their helpful comments on earlier versions of this manuscript.

## Footnotes

*   Contributors The undersigned author warrants that the article is original, is not under consideration by another journal and has not been published previously. The authors have read and approved this manuscript for publication. All authors meet the criteria for authorship stated in the Uniform Requirements for Manuscripts Submitted to Biomedical Journals. JO had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. JO was involved in the conception and design, acquisition of data, analysis and interpretation of the data and drafting of the manuscript. HK was involved in the conception and design, analysis and interpretation of the data and review of the manuscript. All other authors (SF, IB, JWC, KG, AP, MvT, AHH, KJ, RV) were involved in the interpretation of the data and review of the manuscript. SF additionally created the figure depicting the study locations of the analysed articles.

*   Funding This study was supported by the European Crop Protection Association (ECPA). The sponsor had no role in the design and conduct of the study, collection, management, analysis and interpretation of the data, nor in preparation, review or approval of the manuscript.

*   Map disclaimer The depiction of boundaries on the map(s) in this article do not imply the expression of any opinion whatsoever on the part of BMJ (or any member of its group) concerning the legal status of any country, territory, jurisdiction or area or of its authorities. The map(s) are provided without any warranty of any kind, either express or implied.

*   Competing interests None declared.

*   Patient consent for publication Not required.

*   Provenance and peer review Not commissioned; externally peer reviewed.

[http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/)
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/).

## References

1.   Lewis-Mikhael A-M , Bueno-Cavanillas A , Ofir Giron T , et al . Occupational exposure to pesticides and prostate cancer: a systematic review and meta-analysis. Occup Environ Med 2016;73:134–44.[doi:10.1136/oemed-2014-102692](http://dx.doi.org/10.1136/oemed-2014-102692) 
    
    [Abstract/FREE Full Text](http://oem.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToib2VtZWQiO3M6NToicmVzaWQiO3M6ODoiNzMvMi8xMzQiO3M6NDoiYXRvbSI7czoyMDoiL29lbWVkLzc3LzYvMzU3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

2.   Carles C , Bouvier G , Lebailly P , et al . Use of job-exposure matrices to estimate occupational exposure to pesticides: a review. J Expo Sci Environ Epidemiol 2017;27:125–40.[doi:10.1038/jes.2016.25](http://dx.doi.org/10.1038/jes.2016.25) 
    
    

3.   Gangemi S , Miozzi E , Teodoro M , et al . Occupational exposure to pesticides as a possible risk factor for the development of chronic diseases in humans (Review). Mol Med Rep 2016;14:4475–88.[doi:10.3892/mmr.2016.5817](http://dx.doi.org/10.3892/mmr.2016.5817) 
    
    

4.   Gunnarsson L-G , Bodin L . Parkinson's disease and occupational exposures: a systematic literature review and meta-analyses. Scand J Work Environ Health 2017;43:197–209.[doi:10.5271/sjweh.3641](http://dx.doi.org/10.5271/sjweh.3641) 
    
    

5.   Meyer A , Sandler DP , Beane Freeman LE , et al . Pesticide exposure and risk of rheumatoid arthritis among licensed male pesticide applicators in the agricultural health study. Environ Health Perspect 2017;125:077010.[doi:10.1289/EHP1013](http://dx.doi.org/10.1289/EHP1013) 
    
    

6.   Negatu B , Kromhout H , Mekonnen Y , et al . Occupational pesticide exposure and respiratory health: a large-scale cross-sectional study in three commercial farming systems in Ethiopia. Thorax 2017;72:498.1–9.[doi:10.1136/thoraxjnl-2016-208924](http://dx.doi.org/10.1136/thoraxjnl-2016-208924) 
    
    

7.   Le Moual N , Zock J-P , Dumas O , et al . Update of an occupational asthma-specific job exposure matrix to assess exposure to 30 specific agents. Occup Environ Med 2018;75:507–14.[doi:10.1136/oemed-2017-104866](http://dx.doi.org/10.1136/oemed-2017-104866) 
    
    [Abstract/FREE Full Text](http://oem.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToib2VtZWQiO3M6NToicmVzaWQiO3M6ODoiNzUvNy81MDciO3M6NDoiYXRvbSI7czoyMDoiL29lbWVkLzc3LzYvMzU3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

8.   de Jong K , Boezen HM , Kromhout H , et al . Association of occupational pesticide exposure with accelerated longitudinal decline in lung function. Am J Epidemiol 2014;179:1323–30.[doi:10.1093/aje/kwu053](http://dx.doi.org/10.1093/aje/kwu053) 
    
    [CrossRef](http://oem.bmj.com/lookup/external-ref?access_num=10.1093/aje/kwu053&link_type=DOI) 
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=24780843&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 

9.   van der Plaat DA , de Jong K , de Vries M , et al . Occupational exposure to pesticides is associated with differential DNA methylation. Occup Environ Med 2018;75:427–35.[doi:10.1136/oemed-2017-104787](http://dx.doi.org/10.1136/oemed-2017-104787) 
    
    [Abstract/FREE Full Text](http://oem.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToib2VtZWQiO3M6NToicmVzaWQiO3M6ODoiNzUvNi80MjciO3M6NDoiYXRvbSI7czoyMDoiL29lbWVkLzc3LzYvMzU3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

10.  Bolognesi C , Creus A , Ostrosky-Wegman P , et al . Micronuclei and pesticide exposure. Mutagenesis 2011;26:19–26.[doi:10.1093/mutage/geq070](http://dx.doi.org/10.1093/mutage/geq070) 
    
    [CrossRef](http://oem.bmj.com/lookup/external-ref?access_num=10.1093/mutage/geq070&link_type=DOI) 
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=21164178&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 
    
    [Web of Science](http://oem.bmj.com/lookup/external-ref?access_num=000285627700004&link_type=ISI) 

11.  Hanke W , Jurewicz J . The risk of adverse reproductive and developmental disorders due to occupational pesticide exposure: an overview of current epidemiological evidence. Int J Occup Med Environ Health 2004;17:223–43.
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=15387079&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 

12.  Carles C , Bouvier G , Esquirol Y , et al . Occupational exposure to pesticides: development of a job-exposure matrix for use in population-based studies (PESTIPOP). J Expo Sci Environ Epidemiol 2018;28:281–8.[doi:10.1038/jes.2017.26](http://dx.doi.org/10.1038/jes.2017.26) 
    
    

13.  Curwin BD , Hein MJ , Sanderson WT , et al . Pesticide contamination inside farm and nonfarm homes. J Occup Environ Hyg 2005;2:357–67.[doi:10.1080/15459620591001606](http://dx.doi.org/10.1080/15459620591001606) 
    
    [CrossRef](http://oem.bmj.com/lookup/external-ref?access_num=10.1080/15459620591001606&link_type=DOI) 
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=16020099&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 
    
    [Web of Science](http://oem.bmj.com/lookup/external-ref?access_num=000230408000005&link_type=ISI) 

14.  Aprea MC . Environmental and biological monitoring in the estimation of absorbed doses of pesticides. Toxicol Lett 2012;210:110–8.[doi:10.1016/j.toxlet.2011.08.008](http://dx.doi.org/10.1016/j.toxlet.2011.08.008) 
    
    [CrossRef](http://oem.bmj.com/lookup/external-ref?access_num=10.1016/j.toxlet.2011.08.008&link_type=DOI) 
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=21925579&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 

15.  Tielemans E , Bretveld R , Schinkel J , et al . Exposure profiles of pesticides among greenhouse workers: implications for epidemiological studies. J Expo Sci Environ Epidemiol 2007;17:501–9.[doi:10.1038/sj.jes.7500544](http://dx.doi.org/10.1038/sj.jes.7500544) 
    
    [CrossRef](http://oem.bmj.com/lookup/external-ref?access_num=10.1038/sj.jes.7500544&link_type=DOI) 
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=17299530&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 
    
    [Web of Science](http://oem.bmj.com/lookup/external-ref?access_num=000249400900002&link_type=ISI) 

16. Food and Agriculture Organization of the United Nations (FAO). FAOSTAT, 2019. Available: [fao.org/faostat](http://fao.org/faostat) [Accessed 16 Feb 2019].
    
    

17.  Kromhout H , Vermeulen R . Temporal, personal and spatial variability in dermal exposure. Ann Occup Hyg 2001;45:257–73.[doi:10.1016/S0003-4878(00)00093-4](http://dx.doi.org/10.1016/S0003-4878(00)00093-4) 
    
    [CrossRef](http://oem.bmj.com/lookup/external-ref?access_num=10.1093/annhyg/45.4.257&link_type=DOI) 
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=11378147&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 
    
    [Web of Science](http://oem.bmj.com/lookup/external-ref?access_num=000169070700002&link_type=ISI) 

18.  Kromhout H , Symanski E , Rappaport SM . A comprehensive evaluation of within- and between-worker components of occupational exposure to chemical agents. Ann Occup Hyg 1993;37:253–70.[doi:10.1093/annhyg/37.3.253](http://dx.doi.org/10.1093/annhyg/37.3.253) 
    
    [CrossRef](http://oem.bmj.com/lookup/external-ref?access_num=10.1093/annhyg/37.3.253&link_type=DOI) 
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=8346874&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 
    
    [Web of Science](http://oem.bmj.com/lookup/external-ref?access_num=A1993LK52600001&link_type=ISI) 

19.  Zhan Y , Zhang M . Spatial and temporal patterns of pesticide use on California almonds and associated risks to the surrounding environment. Sci Total Environ 2014;472:517–29.[doi:10.1016/j.scitotenv.2013.11.022](http://dx.doi.org/10.1016/j.scitotenv.2013.11.022) 
    
    

20.  Daniels JL , Olshan AF , Teschke K , et al . Comparison of assessment methods for pesticide exposure in a case-control interview study. Am J Epidemiol 2001;153:1227–32.[doi:10.1093/aje/153.12.1227](http://dx.doi.org/10.1093/aje/153.12.1227) 
    
    [CrossRef](http://oem.bmj.com/lookup/external-ref?access_num=10.1093/aje/153.12.1227&link_type=DOI) 
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=11415959&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 
    
    [Web of Science](http://oem.bmj.com/lookup/external-ref?access_num=000169337900015&link_type=ISI) 

21. 1.  
    2.   Lowrance W  
    
     Fingerhut MA , Marlow DA , Honchar PA . The NIOSH occupational dioxin registry. In:  Lowrance W  , ed. Public health risks of the dioxins. Los Angeles, CA, 1984: 361–6.
    
    

22.  Teschke K , Olshan AF , Daniels JL , et al . Occupational exposure assessment in case-control studies: opportunities for improvement. Occup Environ Med 2002;59:575–94. discussion 94.[doi:10.1136/oem.59.9.575](http://dx.doi.org/10.1136/oem.59.9.575) 
    
    [Abstract/FREE Full Text](http://oem.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToib2VtZWQiO3M6NToicmVzaWQiO3M6ODoiNTkvOS81NzUiO3M6NDoiYXRvbSI7czoyMDoiL29lbWVkLzc3LzYvMzU3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

23.  Matheson MC , Benke G , Raven J , et al . Biological dust exposure in the workplace is a risk factor for chronic obstructive pulmonary disease. Thorax 2005;60:645–51.[doi:10.1136/thx.2004.035170](http://dx.doi.org/10.1136/thx.2004.035170) 
    
    [Abstract/FREE Full Text](http://oem.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToidGhvcmF4am5sIjtzOjU6InJlc2lkIjtzOjg6IjYwLzgvNjQ1IjtzOjQ6ImF0b20iO3M6MjA6Ii9vZW1lZC83Ny82LzM1Ny5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

24.  Sunyer J , Zock JP , Kromhout H , et al . Lung function decline, chronic bronchitis, and occupational exposures in young adults. Am J Respir Crit Care Med 2005;172:1139–45.[doi:10.1164/rccm.200504-648OC](http://dx.doi.org/10.1164/rccm.200504-648OC) 
    
    [CrossRef](http://oem.bmj.com/lookup/external-ref?access_num=10.1164/rccm.200504-648OC&link_type=DOI) 
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=16040784&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 
    
    [Web of Science](http://oem.bmj.com/lookup/external-ref?access_num=000232971400013&link_type=ISI) 

25.  van der Mark M , Vermeulen R , Nijssen PCG , et al . Occupational exposure to pesticides and endotoxin and Parkinson disease in the Netherlands. Occup Environ Med 2014;71:757–64.[doi:10.1136/oemed-2014-102170](http://dx.doi.org/10.1136/oemed-2014-102170) 
    
    [Abstract/FREE Full Text](http://oem.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToib2VtZWQiO3M6NToicmVzaWQiO3M6OToiNzEvMTEvNzU3IjtzOjQ6ImF0b20iO3M6MjA6Ii9vZW1lZC83Ny82LzM1Ny5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

26.  London L , Myers JE . Use of a crop and job specific exposure matrix for retrospective assessment of long-term exposure in studies of chronic neurotoxic effects of agrichemicals. Occup Environ Med 1998;55:194–201.[doi:10.1136/oem.55.3.194](http://dx.doi.org/10.1136/oem.55.3.194) 
    
    [Abstract/FREE Full Text](http://oem.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToib2VtZWQiO3M6NToicmVzaWQiO3M6ODoiNTUvMy8xOTQiO3M6NDoiYXRvbSI7czoyMDoiL29lbWVkLzc3LzYvMzU3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

27.  Negatu B , Vermeulen R , Mekonnen Y , et al . A method for semi-quantitative assessment of exposure to pesticides of applicators and re-entry workers: an application in three farming systems in Ethiopia. ANNHYG 2016;60:669–83.[doi:10.1093/annhyg/mew022](http://dx.doi.org/10.1093/annhyg/mew022) 
    
    

28.  Dosemeci M , Alavanja MCR , Rowland AS , et al . A quantitative approach for estimating exposure to pesticides in the agricultural health study. Ann Occup Hyg 2002;46:245–60.[doi:10.1093/annhyg/mef011](http://dx.doi.org/10.1093/annhyg/mef011) 
    
    [CrossRef](http://oem.bmj.com/lookup/external-ref?access_num=10.1093/annhyg/mef011&link_type=DOI) 
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=12074034&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 
    
    [Web of Science](http://oem.bmj.com/lookup/external-ref?access_num=000174505600013&link_type=ISI) 

29.  Lam MT , De Longhi C , Turnbull J , et al . Has Embase replaced Medline since coverage expansion? J Med Libr Assoc 2018;106:227–34.[doi:10.5195/JMLA.2018.281](http://dx.doi.org/10.5195/JMLA.2018.281) 
    
    

30. 1.  
    2.   Higgins JPT , 
    3.   Green S  
    
    , eds. Cochrane handbook for systematic reviews of interventions. Version 5.2. Cochrane Collaboration, 2011. [http://handbook.cochrane.org](http://handbook.cochrane.org) 
    
    

31. World Bank. World bank atlas method, 2019. Available: [datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups](http://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups) [Accessed 16 Feb 2019].
    
    

32. R Core Team. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2018. [R-project.org](http://R-project.org) 
    
    

33.  Ribeiro MG , Colasso CG , Monteiro PP , et al . Occupational safety and health practices among flower greenhouses workers from Alto Tietê region (Brazil). Sci Total Environ 2012;416:121–6.[doi:10.1016/j.scitotenv.2011.11.002](http://dx.doi.org/10.1016/j.scitotenv.2011.11.002) 
    
    [CrossRef](http://oem.bmj.com/lookup/external-ref?access_num=10.1016/j.scitotenv.2011.11.002&link_type=DOI) 
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=22134030&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 
    
    [Web of Science](http://oem.bmj.com/lookup/external-ref?access_num=000301155200014&link_type=ISI) 

34.  Xia Y , Deng X , Zhou P , et al . Chapter: 35 The world floriculture industry: dynamics of production and markets. In: Floriculture, ornamental and plant biotechnology: advances and topical issues vol IV. Edition: 1, 2006: 336–47.
    
    

35.  MacFarlane E , Glass D , Fritschi L . Is farm-related job title an adequate surrogate for pesticide exposure in occupational cancer epidemiology? Occup Environ Med 2009;66:497–501.[doi:10.1136/oem.2008.041566](http://dx.doi.org/10.1136/oem.2008.041566) 
    
    [Abstract/FREE Full Text](http://oem.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToib2VtZWQiO3M6NToicmVzaWQiO3M6ODoiNjYvOC80OTciO3M6NDoiYXRvbSI7czoyMDoiL29lbWVkLzc3LzYvMzU3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

36.  Ge CB , Friesen MC , Kromhout H , et al . Use and reliability of exposure assessment methods in occupational case-control studies in the general population: past, present, and future. Ann Work Expo Health 2018;62:1047–63.[doi:10.1093/annweh/wxy080](http://dx.doi.org/10.1093/annweh/wxy080) 
    
    

37.  Blair A , Tarone R , Sandler D , et al . Reliability of reporting on life-style and agricultural factors by a sample of participants in the agricultural health study from Iowa. Epidemiology 2002;13:94–9.[doi:10.1097/00001648-200201000-00015](http://dx.doi.org/10.1097/00001648-200201000-00015) 
    
    [CrossRef](http://oem.bmj.com/lookup/external-ref?access_num=10.1097/00001648-200201000-00015&link_type=DOI) 
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=11805592&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 
    
    [Web of Science](http://oem.bmj.com/lookup/external-ref?access_num=000172908400015&link_type=ISI) 

38.  Peters S , Glass DC , Milne E , et al . Rule-based exposure assessment versus case-by-case expert assessment using the same information in a community-based study. Occup Environ Med 2014;71:215–9.[doi:10.1136/oemed-2013-101699](http://dx.doi.org/10.1136/oemed-2013-101699) 
    
    [Abstract/FREE Full Text](http://oem.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToib2VtZWQiO3M6NToicmVzaWQiO3M6ODoiNzEvMy8yMTUiO3M6NDoiYXRvbSI7czoyMDoiL29lbWVkLzc3LzYvMzU3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

39.  Benke G , Sim M , Fritschi L , et al . Comparison of occupational exposure using three different methods: hygiene panel, job exposure matrix (JEM), and self reports. Appl Occup Environ Hyg 2001;16:84–91.[doi:10.1080/104732201456168](http://dx.doi.org/10.1080/104732201456168) 
    
    [CrossRef](http://oem.bmj.com/lookup/external-ref?access_num=10.1080/104732201456168&link_type=DOI) 
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=11202032&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 

40.  Kauppinen TP , Mutanen PO , Seitsamo JT . Magnitude of misclassification bias when using a job-exposure matrix. Scand J Work Environ Health 1992;18:105–12.[doi:10.5271/sjweh.1604](http://dx.doi.org/10.5271/sjweh.1604) 
    
    [CrossRef](http://oem.bmj.com/lookup/external-ref?access_num=10.5271/sjweh.1604&link_type=DOI) 
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=1604270&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 
    
    [Web of Science](http://oem.bmj.com/lookup/external-ref?access_num=A1992HR37100006&link_type=ISI) 

41.  Dalvie MA , Africa A , Solomons A , et al . Pesticide exposure and blood endosulfan levels after first season spray amongst farm workers in the Western Cape, South Africa. J Environ Sci Health B 2009;44:271–7.[doi:10.1080/03601230902728351](http://dx.doi.org/10.1080/03601230902728351) 
    
    [CrossRef](http://oem.bmj.com/lookup/external-ref?access_num=10.1080/03601230902728351&link_type=DOI) 
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=19280480&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 
    
    [Web of Science](http://oem.bmj.com/lookup/external-ref?access_num=000264112700010&link_type=ISI) 

42.  Basinas I , van Tongeren M , Dixon K , et al . What is the validity of surrogate measures of occupational pesticide exposure used in epidemiological studies? A review of the published literature. Ann Work Expo Health 2019. (under review).
    
    

43.  Kapka-Skrzypczak L , Cyranka M , Skrzypczak M , et al . Biomonitoring and biomarkers of organophosphate pesticides exposure - state of the art. Ann Agric Environ Med 2011;18:294–303.
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=22216802&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 

44.  Ecobichon DJ . Pesticide use in developing countries. Toxicology 2001;160:27–33.[doi:10.1016/S0300-483X(00)00452-2](http://dx.doi.org/10.1016/S0300-483X(00)00452-2) 
    
    [CrossRef](http://oem.bmj.com/lookup/external-ref?access_num=10.1016/S0300-483X(00)00452-2&link_type=DOI) 
    
    [PubMed](http://oem.bmj.com/lookup/external-ref?access_num=11246121&link_type=MED&atom=%2Foemed%2F77%2F6%2F357.atom) 
    
    [Web of Science](http://oem.bmj.com/lookup/external-ref?access_num=000167688800005&link_type=ISI) 

45.  Eskenazi B , Quirós-Alcalá L , Lipsitt JM , et al . mSpray: a mobile phone technology to improve malaria control efforts and monitor human exposure to malaria control pesticides in Limpopo, South Africa. Environ Int 2014;68:219–26.[doi:10.1016/j.envint.2014.03.003](http://dx.doi.org/10.1016/j.envint.2014.03.003)