Analysis of cluster randomized trials with repeated cross-sectional binary measurements

Analytical techniques appropriate for cluster randomized trials that utilize a repeated cross-sectional design have not been extensively evaluated. This paper compares methods that can be used to evaluate the impact of an intervention on dichotomous outcomes. The methods are applied to data from a study on the implementation of Cochrane review evidence, in which 25 hospital obstetric units were randomized. Assessments were made for 30 pregnancies in each obstetric unit at baseline, and for 30 separate pregnancies at follow-up. The principal issues addressed are how best to take clustering into account and to allow for baseline imbalance. We compare cluster level analyses, the clustered Woolf method, marginal models based on generalized estimating equations, multilevel models, and methods based on random effects meta-analysis. Analyses which ignored the baseline assessments showed no effect of the intervention. There was substantial baseline imbalance, however, so that analyses taking into account the baseline were necessary. Yet, while analyses of change from baseline showed evidence of an effect of intervention, adjusting for baseline using analysis of covariance did not. Analysis of covariance required the use of cluster level rather than individual level responses, since different pregnancies were evaluated at baseline and follow-up. Also, when analysing change from baseline, we show it is important to allow for variation in the effect of secular trend between clusters in a multilevel model, or use robust variance estimates in a marginal model, for otherwise confidence intervals for the effect of intervention will be too narrow. We conclude however that analyses of change from baseline can be misleading since they are affected too much by baseline results, and that analysis of covariance approaches are preferable. To prevent difficulties in interpreting the results from repeated cross-sectional cluster trial designs, one should either attempt to achieve baseline balance by careful stratification of the clusters prior to randomization, or have sufficiently large samples for precise estimation of the effects of imbalance.