• Epidemiology

In September 2009, the European Centre for Ecotoxicology and Toxicology of Chemical (ECETOC) held a workshop to discuss how the transparency of the design, conduct and reporting of observational studies could be improved.1 The main ‘solution’ discussed at the workshop was to increase the registration of observational studies, together with their a priori protocols (and any subsequent changes) in a recognised international database. Benefits of registration discussed at the workshop included increased transparency of design and conduct, the ability to distinguish hypothesis driven from exploratory analyses, improved peer review and ethical processes, reduction in publication bias through knowledge of the total body of research as opposed to only that that reaches the peer reviewed literature, and avoidance of duplication of effort.

So why are observational studies not registered as a matter of course? Clinical trial registration is now required under US law,2 the Declaration of Helsinki 3 and the International Committee of Medical Journal Editors (ICMJE).4 Has registration worked for clinical trials in regard to the benefits outlined above? It appears not entirely. Several scandals drew attention to the selective suppression of trial data, for example, in the use of selective serotonin reuptake inhibitors to treat depression in children where published data suggested a favourable risk–benefit profile, but the addition of unpublished data suggested risks could outweigh the benefits.5 One wonders how many inappropriate decisions have been made based on incomplete evaluation of epidemiological studies. Although one might argue that any damage cannot be as serious as that relating to clinical trials, the uses of epidemiological studies include making regulatory decisions on the setting of standards and limits, and the prioritisation of practical risk-reduction measures.

Following the clinical trial furore, several pharmaceutical companies began to set up their own registries which varied in completeness and content. Indeed, one of the main problems appears to be the plethora of trial registration databases including ClinicalTrials.gov, the WHO International Clinical Trials Registry Platform (ICTRP) and the International Standard Randomised Controlled Trial Number (ISRCTN) register. There have now been calls for legislation to ensure that researchers register their trials and disseminate their results and a call for a global network to ‘ensure uniform, international consistency in policy and enforcement of trial registration and data transparency.’6 The ICTRP has taken the initiative to develop a network of registers that meet WHO-specified criteria, and the ICMJE also now accept registration of any registers that meet this.7

At the ECETOC workshop, there was general agreement that existing registers should be used and adapted for registration of observational studies, and indeed by September 2009 approximately 16% of studies registered in ClinicalTrial.gov were observational. Data elements specific to observational studies have been added to this register following the Strengthening the Reporting of Observational Studies guidelines.8 Does registration of studies and their protocols prior to their conduct have special relevance to epidemiologists working in the area of occupational and environmental health, and the journals publishing their papers? Authors and editors involved in this discipline will already be particularly aware of the need for openness about potential conflicts of interest, for example, regarding sources of funding. The successful conduct of many occupational cohort and case–control studies depends on collaborative effort between researchers, industry management, employees and employee representatives, and regulatory bodies such as those involved with health and safety. Without this cooperation, access to all the information needed to carry out a study could be difficult. The ongoing debate concerning the design and analysis of the studies carried out to investigate the carcinogenicity of beryllium is an illustration of the controversy that can arise between researchers and industry, leaving the regulator to try and make decisions from seemingly conflicting evidence.9–12 Encouragement to register studies might potentially help reduce these problems.

Most observation studies will have had to produce a detailed study plan in order to obtain funding and/or ethical approval, but how many funders and ethics committees then require development of a formal protocol? Maybe it is time to renew interest in Good Epidemiological Practice (GEP). Unlike the field of toxicology where Good Laboratory Practice is the norm at least for studies conducted for regulatory if not purely academic purposes, the use of formal GEP guidelines for epidemiology is rarely mentioned in published papers. The universal adoption of an agreed GEP could enhance the quality of epidemiological studies, help optimise research resources and improve confidence of both the public and scientific bodies in the conduct of and results from studies.13

The ECETOC workshop acknowledged that epidemiologists have been reluctant to embrace voluntary GEP and the registration of studies, and that requirement by ethical committees, funders and journals would encourage this. Following the workshop, editorials in the Lancet14 and the BMJ15 encouraged the registration of observational studies welcoming the inclusion of protocols when submitting a paper reporting an observational study. However, both journals also publish related commentaries that express concerns at the insistence of journals on protocol and registration for observational studies.16 17 They argue that this might discourage publication of important secondary analyses and that data exploration is an essential element of epidemiology, a sentiment echoed by many of the ECETOC workshop participants. The editors of the journal Epidemiology concur with these sentiments and feel that new ‘bureaucratic structures’ such as registration have no bearing on good epidemiology which should be the synthesis of data in the context of biological plausibility and with ‘a sceptic’s eye for alternative explanations.'18 However, not all of the authors of the five commentaries in Epidemiology agree with the Editors.19–23 One of the main issues is the conflict between registering a priori hypothesis and the freedom to carry out exploratory analysis. Poole believes that resolution of this could be reduced by complete disclosure of data collection, full access to data and comprehensive reporting of analyses.22 Certainly in the environmental field, some of the much publicised controversy over climate change could have been avoided if there had been fuller open access to data.

A fundamental issue is how to convince stakeholders of the need to register epidemiological studies and their protocols. The Lancet and BMJ articles offer encouragement but not compulsion. Should all journals publishing observational studies, including those publishing occupational and environmental studies, encourage this? At the very least, surely the relatively few intervention studies in the occupational and environmental field should follow the same procedure as clinical trials? The question then arises as to whether all occupational and environmental observational studies should be included. The participants at the ECETOC workshop felt that in principle any observational study on human subjects that requires ethical review should be registered; the increasing emphasis on ethical approval and data confidentiality means that this will apply to a majority of observational studies. A practical suggestion from the workshop was the use of a pilot project, for example using new prospective studies in select specialities such as environmental air pollution or occupational chemical manufacture to test registration and the use of existing registers, adapting these if necessary, and including protocols. One rather rueful conclusion from the workshop was the general agreement that the bureaucratic hurdle of registering an observational study is probably minor in comparison with obtaining ethical approval.

Would registration of studies and their protocols reduce publication bias, both of studies per se and of the totality of results? At the very least, it would increase knowledge of what data might be available, even if the results from analysis never ‘achieve’ publication in the peer-reviewed literature. There has been an emphasis in the literature on the bias from over-reporting of false-positive outcomes if an a priori hypothesis is not available.24 Boffetta et al25 also feel that epidemiology has been criticised for producing results that are often sensationalised in the media but then fail to be upheld in subsequent studies. In contrast, Crosignani26 suggest that false-negative results are equally damaging, if not more so, because they discourage further research on potentially harmful associations. Registration of epidemiological studies and encouragement to researchers to develop clear protocols that are also registered would address both these points of view.

However, registration of epidemiological studies is not in itself enough to ensure wrong decisions are avoided. These studies will be only one part of the totality of available evidence that will be reviewed that will include laboratory and other mechanistic evidence. Encouragement for study registration has the potential to improve the transparency of the design, analysis reporting and interpretation of studies which would increase the weight given to evidence from epidemiological studies and the standing of the discipline generally.