Article Text


Mortality in New Zealand workers exposed to phenoxy herbicides and dioxins
  1. A ’t Mannetje1,
  2. D McLean1,
  3. S Cheng1,
  4. P Boffetta2,
  5. D Colin2,
  6. N Pearce1
  1. 1Centre for Public Health Research, Research School of Public Health, Massey University, Wellington Campus, New Zealand
  2. 2International Agency for Research on Cancer, Unit of Environmental Cancer Epidemiology, Lyon, France
  1. Correspondence to:
 Dr A ’t Mannetje
 Centre for Public Health Research, Research School of Public Health, Massey University Wellington Campus, PO Box 756, Wellington, New Zealand;


Aims: To evaluate mortality in New Zealand phenoxy herbicide producers and sprayers exposed to dioxins.

Methods: Phenoxy herbicide producers (n = 1025) and sprayers (n = 703) were followed up from 1 January 1969 and 1 January 1973 respectively to 31 December 2000. A total of 813 producers and 699 sprayers were classified as exposed to dioxin and phenoxy herbicides. Standardised mortality ratios (SMR) were calculated using national mortality rates.

Results: At the end of follow up, 164 producers and 91 sprayers had died. Cancer mortality was reduced for sprayers (SMR = 0.82, 95% CI 0.57 to 1.14) and increased in exposed production workers (SMR = 1.24, 95% CI 0.90 to 1.67), especially for synthesis workers (SMR = 1.69), formulation and lab workers (SMR = 1.64), and maintenance/waste treatment/cleaning workers (SMR = 1.46). Lymphohaematopoietic cancer mortality was increased in exposed production workers (SMR = 1.65, 95% CI 0.53 to 3.85), especially for multiple myeloma (SMR = 5.51, 95% CI 1.14 to 16.1). Among sprayers, colon cancer (SMR = 1.94, 95% CI 0.84 to 3.83) showed increased mortality.

Conclusions: Results showed 24% non-significant excess cancer mortality in phenoxy herbicide producers, with a significant excess for multiple myeloma. Associations were stronger for those exposed to multiple agents including dioxin during production. Overall cancer mortality was not increased for producers and sprayers mainly handling final technical products, although they were likely to have been exposed to TCDD levels far higher than those currently in the general New Zealand population.

  • 2,4,5-T, 2,4,5-trichlorophenoxyacetic acid
  • CI, confidence interval
  • IARC, International Agency for Research on Cancer
  • ICD, International Classification of Diseases
  • MM, multiple myeloma
  • NHL, non-Hodgkin’s lymphoma
  • NIOSH, National Institute for Occupational Safety and Health
  • NZHIS, New Zealand Health Information Service
  • PCLTAS, personal computer life table analysis system
  • RR, rate ratio
  • SIA, small intestinal adenocarcinoma
  • SMR, standardised mortality ratio
  • TCDD, 2,3,7,8-tetrachlorodibenzo-p-dioxin
  • cancer
  • cohort
  • dioxin
  • mortality
  • phenoxy herbicides

Statistics from

Chlorophenoxy herbicides have been produced and used extensively in New Zealand from the late 1950s until 1987. In the 1970s and 1980s, 2,4,5-trichlorophenoxyacetic acid (2,4,5-T) represented the most widely used herbicide in New Zealand; it was mainly used to control gorse and scrub. With over 500 tonnes of the active ingredient sprayed annually,1 New Zealand was described as the “heaviest user of 2,4,5-T in the world”.2

Since the late 1960s it has been known3 that 2,4,5-T and its intermediates (for example, chlorophenols), are contaminated with the highly toxic 2,3,7,8-tetrachlorodibenzo-p-dioxin (TCDD) during production. Studies from different countries (for example, Sweden, United States, Italy, Germany, Netherlands) in occupationally or accidentally exposed populations, have reported an increased risk of cancer, notably soft tissue sarcomas and non-Hodgkin’s lymphomas, related to chlorophenoxy herbicides including those contaminated with TCDD. Excess risks were not consistently observed, although the cohorts with highest exposures to TCDD did show a consistent 40% increased risk for all cancers combined.4

In 1980, the International Agency for Research on Cancer (IARC) established an international cohort of producers and sprayers of phenoxy herbicides.5,6 The collaboration consisted of 36 cohorts from 12 countries, including two New Zealand cohorts of 1026 phenoxy herbicide producers and 703 sprayers. The IARC international study found a 29% overall increased risk of cancer in workers exposed to dioxin compared to unexposed workers.6 The New Zealand findings were not published separately at that time because the short follow up period (22 years for producers and 18 years for sprayers) meant that the numbers of deaths were relatively small (133 in total).

While the production of phenoxy herbicides in New Zealand ceased in the mid-1980s, public concern regarding exposure to TCDD experienced by residents in the vicinity of the plant and its workers has continued. We have therefore extended follow up of the two New Zealand cohorts with an additional 10 years, in order to examine the long term effects on mortality in the New Zealand workers exposed to phenoxy herbicides and dioxin contaminants.


Selection of study subjects

The “producers” cohort included workers from a plant in New Plymouth where phenoxy herbicides and chlorophenols were produced between the late 1950s and mid-1980s. Employment records were complete for the period from January 1969 to December 1984, and workers who had worked at the plant for at least one month during this period were included. Also included were storemen working for the company in other parts of New Zealand, being frequently involved in repacking or refilling containers. Excluded were kitchen staff, clerical staff, and field research staff. The total number of workers included in the “producers” cohort was 1025, excluding 12 workers with missing dates of birth or first exposure, and one with incorrect code of death.

The “sprayers” cohort included chemical applicators who were registered at any time between January 1973 and December 1984 on the register of New Zealand chemical applicators. They were studied previously in a survey of congenital malformations conducted in 1980/82.7 All workers were ground sprayers, mainly using backpack spraying and boom spraying from vehicles, and most of them were agricultural workers, spraying mainly during summer months using 2,4,5-T and other phenoxy herbicides (as well as paraquat, organophosphate insecticides, and other pesticides). The total number of sprayers included was 703, excluding 77 sprayers with missing dates of birth or missing dates of first exposure.

Main messages

  • New Zealand has been described as the heaviest user in the world of 2,4,5-T, and produced phenoxy herbicides until the mid-1980s. The mortality patterns of phenoxy herbicide producers and sprayers in New Zealand have not been studied in detail previously.

  • Among the phenoxy herbicide producers exposed to dioxins, a 24% non-significant excess cancer mortality was observed. All cancer mortality was highest for synthesis workers for whom also a strong duration response association was observed.

  • For phenoxy herbicide sprayers, total cancer mortality was reduced, while increased mortality was observed for colon and other gastrointestinal cancers which are not usually associated with dioxin exposure.

Exposure assessment

For the “producers” cohort, individual employment records included department and dates of employment. Department was coded using 21 job codes defined for the IARC international cohort.5 No direct data on levels of exposure were available. Information from a company exposure questionnaire, completed with the assistance of industrial hygienists and factory personnel, was used to classify the job codes according to their exposure to TCDD, higher chlorinated dioxins, and phenoxy herbicides. Exposed job codes included those in synthesis, formulation, packing, maintenance, laboratory, chemical effluent/waste, cleaning, shipping/transportation/stores/warehouse, plant supervision, and other/unclassified exposure. A total of 813 workers had worked in one or more of these departments and were thus classified as exposed to TCDD, higher chlorinated dioxins, and phenoxy herbicides. Job codes classified as not exposed to these agents included manual jobs in parts of the company not involved in the production of chlorophenoxy herbicides or chlorophenols, comprising 212 workers. We do not have employment data after 1984. However, production of phenoxy herbicides stopped in the mid-1980s (about 1987), and we therefore have virtually complete information on occupational TCDD exposure.

For the “sprayers” cohort, no detailed employment records were available. As part of the previous study on congenital malformations,7 548 registered chemical applicators completed an exposure history questionnaire (response rate 89%) in 1980, and an additional 232 completed questionnaires in 1982. Respondents were asked to indicate 2,4,5-T-containing product use for each year between 1969 and 1980 (or 1982), as well as the use of any pesticide. Of the 703 sprayers, 699 were classified as exposed to TCDD, higher chlorinated dioxins, and phenoxy herbicides. For the sprayers, we did not have any employment information after 1980 (or 1982). The use of 2,4,5-T was however discontinued during the 1980s, and we therefore have almost complete information on occupational TCDD exposure.

Policy implications

  • While the production of phenoxy herbicides ceased in the mid-1980s, public concern regarding dioxin exposure has continued in New Zealand. This study in New Zealand workers with relatively high dioxin exposure, is therefore important for risk communication.

  • Although the numbers were small, this study shows increased all cancer mortality for workers exposed to multiple agents including dioxin during production, but not for producers and sprayers mainly handling final technical products.

Follow up

The study period started on 1 January 1969 for the “producers” and on 1 January 1973 for the “sprayers” cohort, and ended on 31 December 2000 for both. For each worker, follow up started on the first date of employment in the industry (reported employment for sprayers) or the study starting date, whichever came later. Vital status and cause of death were determined for each cohort member by searching national records for death registrations through the New Zealand Health Information Service (NZHIS). This was done using matching methods (allowing for minor errors in spelling of names and in dates of birth) that were developed and used in similar studies previously.8–11 To confirm that subjects not registered as having died during the study period were actually alive and resident in New Zealand, the New Zealand Electoral Roll, drivers’ licence, and social security records were used. Underlying causes of death were coded into the International Classification of Disease (ICD revision 8 before 1979, 9 from 1979 onwards). For cohort members who had died during the study period, the last date of follow up was the date of death, and for all others follow up ended on the earliest of date last observed and study end date.

Statistical analyses

Standardised mortality ratios (SMR) for each cause of death were calculated using the New Zealand mortality rates as an external comparison.12 Life table analysis using the NIOSH developed PCLTAS program13,14 was applied to calculate person-time at risk for intervals for age, calendar time, sex, time since first exposure, and exposure duration categories. Observed deaths, person-time at risk, and expected deaths were calculated for each stratum, and expected deaths were computed by multiplying the national New Zealand mortality rates by the observed person-years at risk in each stratum. The observed and expected deaths were then summed across all strata to calculate the SMR for each cause of death. Probability values and 95% confidence intervals were calculated, assuming the observed deaths following a Poisson distribution. Unless stated otherwise, analyses were restricted to those producers and sprayers classified as exposed to TCDD, higher chlorinated dioxins, and phenoxy herbicides (813 producers and 699 sprayers). Analyses were stratified by sex and department if numbers permitted. Because TCDD levels in New Zealand produced 2,4,5-T dropped dramatically in 1973 (from 470 to 47 μg/kg),4,15 those who started working as sprayers before 1973 were analysed separately.



Table 1 summarises the characteristics of both cohorts. By the end of the year 2000 there were 122 deaths observed in the exposed “producers” cohort (15.0%) and 91 in the exposed “sprayers” cohort (12.9%). The median age of death was 62 for the exposed “producers” and 64 for the exposed “sprayers”. Both cohorts were predominantly male. The two cohorts differed considerably in their mean duration of recorded employment, which was 3½ years for the “producers” and 16 years for the “sprayers”. The total person-years accumulated to 17 613 for the exposed “producers” (87.2% of the 20 185 total possible person-years if all those lost to follow up had been alive and followed to 31 December 2000) and 20 044 for the “sprayers” (97.3% of the 20 592 total possible person-years). Analysed separately were 212 production workers with no exposure to TCDD, higher chlorinated dioxins, and phenoxy herbicides, adding up to 4712 person-years and including 42 deaths. Only four sprayers were designated “unexposed” and were excluded from all analyses. This update added respectively 7075 and 8510 person-years and 66 and 55 deaths to the “producers” and “sprayers” cohorts of 1990 that were included as part of the IARC international study.6

Table 1

 Characteristics of the New Zealand phenoxy herbicide producers and sprayers cohort

Cancer mortality

Mortality in the exposed production workers was close to expected (SMR = 0.99, 95% CI 0.82 to 1.18). Cancer mortality (table 2) was non-significantly greater than expected (SMR = 1.24, 95% CI 0.90 to 1.67). Longer duration of employment was not associated with higher cancer mortality. The small subgroup of exposed female production workers showed a reduction in both overall mortality (two deaths, 8.16 expected) and cancer mortality (table 2).

Table 2

 Standardised mortality ratios for all cause mortality and cancer mortality in the phenoxy herbicide producers and sprayers

Cancer mortality was particularly increased for synthesis workers (SMR = 1.69, 95% CI 0.85 to 3.03), formulation and laboratory workers (SMR = 1.64, 95% CI 0.75 to 3.12), and maintenance, waste treatment, and cleaning workers (SMR = 1.46, 95% CI 0.70 to 2.68). For synthesis workers a statistically significant duration-response association was observed (p = 0.04), with a more than threefold risk for cancer mortality among those with more than five years of exposure. Cancer deaths among synthesis workers included five lung cancer deaths, and one death each from cancer of the mouth, liver, prostate, kidney, brain, and non-Hodgkin’s lymphoma. All five lung cancer deaths occurred in workers with more than five years of exposure. On the other hand, formulation and laboratory workers showed a statistically significant negative duration-response association. Cancer mortality was not increased among packing and transport workers (SMR = 0.83). Production workers classified as unexposed to TCDD, higher chlorinated dioxins, and phenoxy herbicides, showed a small increase in cancer mortality (SMR = 1.10, 95% CI 0.59 to 1.88).

The number of deaths in the pesticide sprayers cohort was below expected (SMR = 0.61, 95% CI 0.49 to 0.75). A deficit, albeit less pronounced, was also observed for cancer mortality (see table 2) (SMR = 0.82, 95% CI 0.57 to 1.14). Longer duration of sprayer employment was not associated with higher cancer mortality. There was no indication of a higher all cause mortality (SMR = 0.61, 95% CI 0.49 to 0.76) or cancer mortality (SMR = 0.75, 95% CI 0.50 to 1.07) for those who started working as sprayers before 1973.

Mortality for primary cancer sites

Tale 3 lists the mortality for specific causes of death among producers and sprayers exposed to TCDD, higher chlorinated dioxins, and phenoxy herbicides. Three multiple myeloma deaths that occurred in the producers cohort resulted in an SMR of 5.51 (95% CI 1.14 to 16.1). Two cases had worked in packing and transport and one had worked in an unspecified exposed department. Including these three cases, in total five lymphohaematopoietic cancer deaths occurred in the producers cohort (SMR = 1.65, 95% CI 0.53 to 3.85). Many other cancer sites had a statistically non-significant excess mortality in the producers cohort, including head and neck cancers (SMR = 2.75), cancers of the digestive organs (SMR = 1.38), cancers of the respiratory organs (SMR = 1.39), cancers of other/unspecified sites (SMR = 1.25), and male breast cancer based on one death (0.03 expected). In the producers cohort, mortality was slightly below expected for some smoking related causes of death, including diseases of the circulatory systems (SMR = 0.96), and diseases of the respiratory system (SMR = 0.93). One soft tissue sarcoma (ICD9 171) death occurred in the production workers (0.05 expected), but this was in an unexposed production worker (therefore not listed in table 3).

Table 3

 Cause specific standardised mortality ratios for the workers exposed to TCDD, higher chlorinated dioxins, and phenoxy herbicides, in the producers and sprayers cohorts

For the sprayers cohort, a statistically significant deficit in deaths of diseases of the circulatory system was apparent (SMR = 0.52, 95% CI 0.36 to 0.73), as well as a deficit in lung cancer mortality (SMR = 0.45, 95% CI 0.15 to 1.05). Also overall cancer mortality was reduced, although some specific cancers showed excess mortality, notably for different sites of the digestive system: stomach (SMR = 1.36, 95% CI 0.28 to 3.98), colon (SMR = 1.94, 95% CI 0.84 to 3.83), and rectum (SMR = 1.47, 95% CI 0.40 to 3.76). The SMR for kidney cancer was 2.73 (3 observed, 1.1 expected). One soft tissue sarcoma (ICD9 171) death occurred in the sprayers cohort (0.23 expected, SMR = 4.28), having worked for 28 years as a sprayer.


In New Zealand, as in other countries, the long term health effects related to exposure to dioxins remain a topic of controversy and public concern. High dioxin exposure levels have occurred in New Zealand from the use and production of 2,4,5-T and other agricultural chemicals, although current serum TCDD levels of the general New Zealand population are at the low end of the scale of levels reported internationally.16 We have studied mortality in two New Zealand populations likely to have been exposed to high levels of TCDD and higher chlorinated dioxins from the production and spraying of phenoxy herbicides.

It is a limitation of this study that there is little information on the actual exposure levels experienced by the workers described here. Availability of individual TCDD measurements or estimates could have facilitated dose-response analyses for TCDD exposure and cancer such as done for other cohorts.17,18 Nine New Zealand pesticide applicators19 studied in 1988 had an average TCDD serum level of 53.3 ng/kg lipid (3.0–131), almost 10 times that of nine matched control subjects (5.6 ng/kg lipid). Back extrapolation of this level to 1970 gave a value of around 300 ng/kg lipid. For the New Zealand herbicide producers no measurements are available, but data from other countries indicate that phenoxy herbicide production workers generally have higher serum TCDD levels than sprayers. Mean serum TCDD levels between 3 and 400 ng/kg lipid have been measured in production workers in other countries,6 with much higher levels measured in chloracne cases. TCDD exposure has likely reduced over time for the New Zealand sprayers, given that TCDD concentration in 2,4,5-T had dropped from 950 μg/kg in 1971 to 4.7 μg/kg in 1985,4,15 with the largest reduction occurring in the early 1970s. This reduction in TCDD concentration was established through solvent extraction in the production plant; how this could have affected TCDD exposure in production workers is not known.

Our results for mortality from all malignant neoplasms in production workers (SMR = 1.24, 95% CI 0.90 to 1.67) are generally consistent with the results of the IARC international study,6 and other industrial cohorts with high TCDD exposure and long latency.4,20–23 The IARC international study found mortality from all malignant neoplasms slightly increased (SMR = 1.12, 95% CI 1.04 to 1.21), and in Poisson regression comparing workers exposed to phenoxy herbicides contaminated with dioxins to workers exposed to phenoxy herbicides without dioxin contaminants, they found a rate ratio of 1.29 (95% CI 0.94 to 1.76). These results were not reported separately for phenoxy herbicide producers and sprayers. For the New Zealand cohorts we however found considerable difference in all cause and all cancer mortality between producers and sprayers. The observed deficit in all cause mortality among the sprayers is likely due to the healthy worker effect24 and to lifestyle factors including diet and physical activity related to employment in agriculture. The absence of an “unexposed” comparison group within the sprayers cohort poses limitations on the interpretation of results. Also, we did not observe an increased risk for longer duration of employment, nor for those sprayers working before 1973 when the TCDD content in 2,4,5-T was considerably higher. The results for all cancer mortality found for the sprayers appear to be consistent with the international literature where to date, no individual cohort study of commercial sprayers or applicators has reported increased all cancer mortality,4,25 except for one study from 1980 in 348 Swedish railroad herbicide sprayers.26

The associations found for cancer mortality were considerably stronger for the production workers. Analyses by department showed increased cancer mortality, that increased by duration of exposure, for synthesis workers who were involved nearly all of their working time in the manufacture of phenoxy acids or 2,4,5-TCP. It is during this process when the dioxin contaminants are produced, and in addition to TCDD these workers could have been exposed to the whole range of raw materials, processing chemicals, technical products, intermediates, and by-products.27 Other departments expected to entail multiple exposures (formulation and lab, maintenance and waste) also showed increased SMRs for total cancer, while the department mainly handling final products (packing and transport) did not show increased SMRs for cancer.

Producers and sprayers also show differences in site specific cancer mortality. In the producers cohort, lymphohaematopoietic cancer mortality was found to be non-significantly increased (SMR = 1.65, 95% CI 0.53 to 3.85). Lymphohaematopoietic cancers have been evaluated frequently in relation with phenoxy herbicides, with especially non-Hodgkin lymphoma (NHL) showing excesses in cohort as well as case-control studies.4,28 In the producers cohort only one NHL death occurred (1.1 expected). A case-control study in New Zealand did not find an increased risk for NHL in those exposed to phenoxy herbicides (OR = 1.0, 95% CI 0.7 to 1.5) either.29 We observed a strong association for multiple myeloma (MM) (SMR = 5.51, 95% CI 1.14 to 16.1). Blair and Hoar Zahm28 discuss studies from before 1990 and find consistent increased risks for MM, although numbers were small. Lynge found excess MM in female production workers of phenoxy herbicides (not 2,4,5-T),30 but no male cases of MM were observed. In the Seveso population exposed to dioxins after an industrial accident,31 an increased RR was observed for exposed women (zone A+B) (RR = 3.2, 95% CI 1.2 to 8.8), but not for exposed men. An MM case-control study in New Zealand found a weak association for spraying phenoxy herbicides for at least six days 10 years before cancer registration (OR = 1.4, 95% CI 0.7 to 2.2),32 and farmers were at increased risk of MM (OR = 1.7, 95% CI 1.0 to 2.9). An association between MM and farming was also found in a study based on the occupations recorded on the New Zealand cancer registry.33 Here we did not observe any MM deaths in the sprayers (farmers) cohort, and the association was only observed in the producers.

Among the sprayers, different sites within the digestive system showed increased cancer mortality, including colon cancer (SMR = 1.94, 95% CI 0.84 to 3.83). Colon cancer is generally found to be decreased in farmers,34 and is not usually associated with dioxin. However, a recent update of the Seveso cohort31 showed an excess of selected digestive cancers for males and females in the high exposure zone. We also found increased cancer mortality for other sites within the digestive system, including stomach (SMR = 1.36, 95% CI 0.28 to 3.98), and rectum (SMR = 1.47, 95% CI 0.40 to 3.76), although numbers were small and confidence intervals wide. For the Seveso population in dioxin exposure zones A and B,31 mortality for rectal cancer was also increased. In the light of our findings, a small intestinal adenocarcinoma (SIA) study in New Zealand sheep35 is also of interest: 125 cases of SIA were found at slaughter of 20 678 female sheep aged 5.5–7.5 years and cases were age and sex matched with non-cases. Exposure to phenoxy and picolinic acid herbicides from sprayed feed stuffs was associated with significant increases in tumour rate, but no association was found for TCDD exposure. Kidney cancer mortality was also increased in the New Zealand sprayers (SMR = 2.73); an association also observed in the IARC international study6 for workers exposed to TCDD or higher chlorinated dioxins (SMR = 1.60, 95% CI 1.05 to 2.35), although no association with duration was observed.

Soft tissue sarcoma (STS) has repeatedly been associated with exposure to phenoxy herbicides and TCDD. Our study included one STS death occurring in a production worker not exposed to phenoxy herbicides (0.05 expected in the non-exposed), and one in a sprayer with 28 years of spraying (0.23 expected). A previous STS case-control study in New Zealand36 found weak increased risk for those with probable expose to phenoxy herbicides (OR = 1.6, 95% CI 0.7 to 3.3). In an update including more cases and controls,15 no increased risk was observed (OR = 1.1, 95% CI 0.7 to 1.8). A case-control study nested in the IARC international study37 showed a strong association (OR = 10.3, 95% CI 1.2 to 91) for exposure to phenoxy herbicides and STS.

We did not have information on smoking habits for either cohort and could only indirectly study the possible confounding effect of smoking. Although in the producers cohort lung cancer mortality was increased (SMR 1.37, 95% CI 0.71 to 2.39), we did observe that the SMRs for other typically smoking related causes of death were slightly below expected (SMR diseases of circulatory system = 0.96; SMR diseases of respiratory system = 0.93) and strongly reduced for the sprayers (SMR = 0.52 and 0.55). Also, the associations found for synthesis workers are not likely due to confounding by smoking,38 since the relative risks are too strong.24 Chance can however not be excluded as an explanation of our results, which mainly could have effected those associations that were based on small numbers.

In conclusion, this historical cohort in New Zealand phenoxy pesticide producers exposed to chlorophenoxy herbicides and TCDD showed increased cancer mortality risks for departments with complex exposure profiles occurring during the production process, while we did not observe increased all cancer mortality for those mainly handling final technical products. There was little evidence of an increased cancer risk in the cohort of sprayers, despite the fact that the workers studied here were likely to have been exposed to TCDD levels far higher than those currently in the general New Zealand population.


View Abstract


  • Funding: The original work on the cohort was funded by the International Agency for Research on Cancer (IARC), and the further follow up was funded by the New Zealand Lottery Grant Board. The Centre for Public Health Research is supported by a Programme Grant from the Health Research Council of New Zealand. The sprayers cohort was originally based on a survey of registered chemical applicators conducted by Professor Allan Smith7 which was funded by grants from the New Zealand Department of Health, the Medical Research Council, and the War Pensions Board.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.