Article Text
PDF
Statistics from Altmetric.com
The article of Dr Goodman and coworkers on “Neurobehavioural testing in workers occupationally exposed to lead”1 covers an interesting approach with a surprising main message: “None of the individual studies is conclusive or adequate in providing information on the subclinical neurobehavioural effects ...”. Such a sentence astonishes a reader since the studies used were selected from established journals.
A long section of the discussion deals with an article of Meyer-Baron and Seeber,2 the beforehand published meta-analysis on the topic. We agree that prospective studies are the best basis to receive a stable knowledge about exposure effects, also in neurobehavioural studies. However, the repeated information on cross sectional studies should also be accepted as source for conclusions on (neurobehavioural) effects due to exposures. Meta-analyses are one approach to search such summarising information.
Taking into account that the extended study selection in the article of Goodman et al may lead to different results we do not agree with several arguments. For example, they refer to the bias problem, the exposure range, the interpretation in terms of age related changes, and the results for the digit symbol test. On these problems an exchange of opinions has been published in Archives of Toxicology.3,4 Without making reference to this discussion, several arguments and conclusions were presented again. They are identical with the main conclusions in an anonymous “expert opinion” for the German Battery Association.5
From our point of view it makes no sense to repeat the same details of argumentation for a second time. However, we believe that the readers of your journal should be informed that the conclusions of the article of Goodman et al have been discussed in other places. In the meantime an additional article on the subject has been published.6 In this article the data of the original “expert opinion”—the basis of the article in Occupational and Environmental Medicine—and the data of our first meta-analysis were comparatively evaluated. We hope that the critical readers of your journal pick up the full information on the matter. Thereupon they may draw their own conclusions regarding meta-analyses of neurobehavioural effects due to occupational exposure to inorganic lead.
References
Authors’ reply
We thank Drs Seeber and Meyer-Baron for their comments on our paper,1 and also Drs Schwartz, Stewart, and Hu for comments published in the September 2002 issue of OEM.2
The following is our response to the specific criticisms made by Schwartz and colleagues:
“No evaluation of the quality of the evidence from available studies based on study design and analytical method” Study quality assessment was the first task we completed. As discussed in our methods section, our quality criteria included evaluating pre-exposure status, use of blinding procedures, and adjustments for age, other occupational exposures, alcohol use, and socioeconomic factors (income level, education, etc).
“Data were combined from poorly done studies with data from well done studies.” Table 1 shows that no study satisfied all of the above quality criteria. Schwartz et al did not provide criteria to distinguish a “poorly done” from a “well done” study. However, we conducted an additional analysis of the five relatively well designed studies that adjusted for age, education, and alcohol use (Baker and colleagues,3 Campara and colleagues,4 Chia and colleagues,5 Maizlish and colleagues,6 and Williamson and Teo7). These five allowed us to conduct a meta-analysis for only three tests. For the Santa Ana preferred hand test, the effect size changed from non-significant negative to non-significant positive. For the Santa Ana non-preferred hand the result changed slightly towards the null and remained statistically non-significant. For the digit symbol test, the result changed away from the null and remained statistically significant in the fixed effects model, but changed slightly towards the null and was no longer statistically significant in the two random effects models.
“Inclusion of studies that did not control for age and education.” Schwartz et al do not provide evidence that age and education are “the two most important predictors”. One could argue that alcohol use or the presence of pre-existing neuropsychiatric conditions could also act as powerful confounders. The studies in our meta-analysis had non-overlapping strengths and limitations and further inclusion or exclusion based on quality would be a matter of judgement. However, an additional analysis based on the 13 studies that adjusted for age and education revealed that, as opposed to our original findings based on all 22 studies, none of the tests showed a statistically significant difference in all three models. (See OEM website for results table.)
“No adjustment for age, education, or lead dose differences across studies.” This criticism appears to be somewhat theoretical, as the data did not allow such adjustment.
“Reliance on exposed versus control comparisons” rather than “only including studies that reported beta coefficients for the blood lead versus test score relation, or adjusting for mean blood lead levels in exposed and non-exposed groups.” We used the same definition of exposure as the previously published meta-analysis by Meyer-Baron and Seeber8 (less than 70 μg/dl) to find out if the results of our two studies were reproducible. The direct comparison of the two analyses in the discussion section was important in explaining our position with regards to meta-analysis as a research technique. We agree that other approaches could also be informative. The statement “The authors conclude that blood lead levels, that are described as ‘moderate’ in one location in the manuscript and ‘low’ in another, are not associated with neurobehavioral test scores” misrepresents our conclusions listed on page 222 of our paper.
“Reliance on a small number of unspecified studies for effect estimates. Table 2 of the study reports the number of studies that were combined to derive effect estimates, but does not specify which studies were combined.” The original version of the paper included information on each individual study; however, based on the reviewers’ and editor’s comments, we had to shorten the manuscript substantially. We will make this information available on request. With respect to the purported omission from our meta-analysis of the May 2001 article by Schwartz and colleagues,9 this article was unavailable when our manuscript was submitted for publication in December 2000. The other two studies they cite did not meet our inclusion criteria. While we have not had an opportunity to evaluate the association between cumulative exposure to lead and neurobehavioural testing results, we did note that the 2001 article by Schwartz and colleagues9 found no association between tibia lead levels and test scores.
With regard to Seeber and Meyer-Baron’s statements that “the repeated information on cross-sectional studies should also be accepted as source for conclusions on (neurobehavioural) effects due to exposures” and that “meta-analyses are one approach to search such summarising information”, after having reviewed the results of five meta-analyses on the subject (two presented in the recent article by Seeber and colleagues,10 our paper,1 and the two additional re-analyses discussed here), we found five different sets of results with no evidence of consistency to qualify these results as “repeated”. Therefore, we have to adhere to our original conclusions.