Study quality assessed by the modified Downs and Black checklist
| Item | Criteria | Score | |||||||||||
| Bantoft et al 21 | Botter et al 13 | Commissaris et al 25 | Cox et al 14 | Gilson et al 18 | Kruse et al 17 | Ohlinger et al 20 | Mullane et al 19 | Sliter and Yuan10 | Straker et al 15 | Tronarp et al 16 | Zeigler et al 8 | ||
| Reporting | |||||||||||||
| 1 | Is the hypothesis/aim/objective of the study clearly described? | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 2 |
Are the main outcomes to be measured clearly described in the introduction? or Methods section? | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 3 | Are the characteristics of the patients included in the study clearly described? | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 4 | Are the interventions of interest clearly described? | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 5 | Are the distributions of principal confounders in each group of subjects to be compared clearly described? | 2 | 0 | 2 | 0 | 1 | 2 | 2 | 2 | 0 | 1 | 1 | 2 |
| 6 | Are the main findings of the study clearly described? | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 7 | Does the study provide estimates of the random variability in the data for the main outcomes? | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 8 | Have all important adverse events that may be a consequence of the intervention been reported? | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 |
| 9 | Have the characteristics of patients lost to follow-up been described? | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 1 | 0 | 1 | 1 |
| 10 | Have actual probability values been reported (eg, 0.035 rather than <0.05) for the main outcomes except where the probability value is less than 0.001? | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| External validity | |||||||||||||
| 11 | Were the subjects asked to participate in the study representative of the entire population from which they were recruited? | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 1 | 1 |
| 12 | Were those subjects who were prepared to participate representative of the entire population from which they were recruited? | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 1 |
| 13 | Were the staff, places and facilities where the patients were treated representative of the treatment the majority of patients receive? | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Internal validity—bias | |||||||||||||
| 14 | Was an attempt made to blind study subjects to the intervention they have received? | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 15 | Was an attempt made to blind those measuring the main outcomes of the intervention? | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 16 | If any of the results of the study were based on ‘data dredging’, was this made clear? | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 17 | In trials and cohort studies, do the analyses adjust for different lengths of follow-up of patients, or in case–control studies, is the time period between the intervention and outcome the same for cases and controls? | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 18 | Were the statistical tests used to assess the main outcomes appropriate? | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 19 | Was compliance with the intervention/s reliable? | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 20 | Were the main outcome measures used accurate (valid and reliable)? | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| Internal validity—confounding (selection bias) | |||||||||||||
| 21 | Were the patients in different intervention groups (trials and cohort studies) or were the cases and controls (case–control studies) recruited from the same population? | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 1 | 0 |
| 22 | Were study subjects in different intervention groups (trials and cohort studies) or were the cases and controls (case–control studies) recruited over the same period of time? | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 23 | Were study subjects randomised to intervention groups? | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 1 | 1 |
| 24 | Was the randomised intervention assignment concealed from both patients and healthcare staff until recruitment was complete and irrevocable? | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 25 | Was there adequate adjustment for confounding in the analyses from which the main findings were drawn? | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 1 | 0 |
| 26 | Were losses of patients to follow-up taken into account? | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 1 | 1 | 0 | 1 | 1 |
| Power | |||||||||||||
| 27* | Did the study have sufficient power to detect a clinically important effect where the probability value for a difference being due to chance is less than 5%? Sample sizes have been calculated to detect a difference of x% and y%. | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 1 | 0 |
| Total score | 19/28 | 16/28 | 17/28 | 14/28 | 16/28 | 16/28 | 20/28 | 19/28 | 17/28 | 15/28 | 21/28 | 19/28 | |
*Item has been modified ‘yes’=1; ‘no’=0.