Article Text
PDF
Abstract
Objectives We evaluated a military resilience intervention which aimed to help UK military recruits to manage their personal health and well-being more effectively.
Methods Trainers within six pre-existing training teams were randomly allocated by team to deliver a resilience-based intervention (SPEAR) or usual training (control) during recruit training. 23 trainers delivered SPEAR; 18 delivered the control training. 707 recruits participated (n=358 SPEAR and n=349 controls). Outcome measures were obtained before and after recruit training and 3 months later. Measures of post-traumatic stress disorder (PTSD), common mental disorder (CMD) symptoms, alcohol use, homesickness and mental health stigmatisation were obtained at baseline. Repeat baseline scales plus measures of help-seeking, cohesion, leadership and training impact were obtained at the two follow-up points.
Results Response rates were 91.7% (baseline), 98.1% (post) and 73.6% (follow-up). Following adjustment for potential confounders, levels of PTSD, CMD symptoms, alcohol misuse, help-seeking and homesickness were not significantly different between groups at any measurement point. Stigmatisation was significantly lower among SPEAR recipients at baseline but was not significantly different at the two follow-up points. Following adjustment for mental health confounders, there were no significant between-group differences in perceptions of leadership and cohesion and in ratings of six training outcomes at the two follow-up points.
Conclusions We found no evidence that resilience-based training had any specific benefit to the health and well-being of UK military recruits.
- mental health
- occupational health practice
- intervention studies
- training and education
- defence force personnel
Statistics from Altmetric.com
- mental health
- occupational health practice
- intervention studies
- training and education
- defence force personnel
Key messages
What is already known about this subject?
Resilience-based workplace health and well-being interventions are becoming increasingly popular and are thought to have positive financial and health effects.
Recent well-constructed studies suggest that the benefits of these interventions may not be as substantial as has been claimed.
What are the new findings?
We tested a resilience-based health and well-being intervention using a randomised controlled trial design among military recruits participating in their initial period of training.
We found no evidence that receipt of the intervention had any beneficial or negative impact on mental health and well-being, attitudes to mental illness and help-seeking, cohesion, perceptions of leadership and military training effects.
How might this impact on policy or clinical practice in the foreseeable future?
We suggest that novel resilience, health and well-being interventions be evaluated for effectiveness.
Although the current study found no benefit for a specific intervention, this is an important finding as a great deal of time and expenditure is spent implementing such interventions without establishing whether they are effective or not.
Doing no harm is not a reasonable defence of an ineffective intervention as time spent in delivery effectively reduces the time available for engaging in more meaningful activity.
Introduction
Interventions have been developed that purportedly promote psychological resilience. These interventions are rarely robustly evaluated, often lack a common resilience definition and fail to operationalise key resilience components.1 Potential resilience definitions include ‘the preservation/recovery of pre-morbid functioning after exposure to a physical or psychological trauma’, ‘a stable equilibrium in spite of trauma’2 or ‘a malleable attribute which supports personal wellbeing in the presence of adversity over time’.3 Returning to a previous level of functioning and use of mental health services have been used as proxy measures of resilience.
Popular support has led to a proliferation of workplace wellness programmes which reportedly have a range of positive resilience and financial effects4; however, evaluations frequently use observational methodology5 and a recent randomised controlled trial (RCT) found no evidence of effectiveness when inherent recruitment bias was eliminated.6
Given the lack of a clear resilience definition, in the current study, we viewed good health and well-being as consequences, and hence indicators, of being resilient. Not wanting to fall foul of ‘implementation prior to evaluation’ we assessed a military resilience intervention delivered by military recruit training teams.
The intervention
SPEAR is a novel military health and well-being approach specifically developed for UK Royal Air Force (RAF) recruits. It is predicated on a model that emphasises key activities; participating in Social networks, capitalising on Personal strengths and weaknesses, managing Emotions, enhancing Awareness of psychological symptoms and learning methods to promote Resilience.
Military recruit trainers were provided with tools to frame their interactions with recruits. The NATO mental health continuum was used to grade mental health using a traffic light system, where green represented better and red represented poorer mental health. Education for trainers used video material and hypothetical case studies which were health-rated using the traffic light system; green represented problem-free, amber potential problems and red definite problems requiring help. A CARE acronym was rehearsed which related to Coping with uncertainty and self-image, self-Awareness in different situations, maintaining healthy Relationships and managing Emotions. Trainers used the ‘HARDFACTS’ model to organise support for recruits. HARDFACTS is a monitoring tool designed to enable commanders to periodically monitor individual planning and personal development in order to support military careers and assess preparation for transition to civilian life. It relates to Health (mental and physical), Accommodation, Relocation (geographical), Drugs & alcohol, Finance & benefits, Attitudes, Children & family, Training, education and employment and access to Supporting agencies. The NATO continuum was integrated with ‘HARDFACTS’ to guide either self-help or signposting to professional support.
SPEAR trainers encouraged individuals to help themselves by incorporating SPEAR concepts into their routine interactions with recruits. Attempts were made to reduce mental health-related stigmatisation, promote parity of mental and physical health and facilitate communication and awareness of mental health and well-being. A structured workbook was provided and trainers were instructed in positive interpersonal skills to manage recruit interactions.
Study aims
We assessed whether the provision of SPEAR improved the mental health and well-being of UK RAF recruits and improved their attitudes to mental disorder during the initial stages of their military career. We hypothesised that better mental health, less alcohol use and lower stigmatisation levels would be present when trainers were SPEAR-aware compared with a control condition. In addition, we examined perceptions of leadership, cohesion and help-seeking for mental health problems.
Method
Trainers within six pre-existing training teams were allocated by the training team manager by alternate number sequence to receive either SPEAR training or to continue with standard recruit training (control condition) independent of the research team. The first training team received SPEAR training, the next team formed a control group, repeating the process until all were recruited. As there were no circumstances attached to being in one training team as opposed to another, random assignment of recruits to conditions was achieved.
To mitigate potential placebo effects of providing additional training (SPEAR) among intervention trainers, control trainers received supplementary training too. To isolate any potential effects of SPEAR training and to ensure that both groups had awareness that a new form of training was being delivered, control and intervention groups both received the first stage of neurodiversity training (NDT), a UK government cascade project to update institutions about learning difficulties.7 After initial NDT training for both groups, controls received additional NDT training at the same time as SPEAR training took place for the intervention group but did not receive the SPEAR training component. All training was observed for consistency by the research team.
Participants
No exclusion criteria were applied. All participants were adults aged ≥18 years.
Trainers
Trainers were given a full explanation of the trial and participated in a question and answer session. Forty-one trainers participated in the study (23 SPEAR and 18 controls).
Recruits
Recruits in the first two days of military training were provided with a full explanation of the study with a question and answer session to follow. Recruits were told that a newly designed form of health and well-being support was being assessed. Recruits were unaware of which study arm they were in. Baseline self-report measures were then completed. Nine weeks later, following initial military training and then 3 months later within the next specialist training environment, recruits provided repeat baseline and additional measures.
Measures
Symptoms of common mental disorder were measured using the 12-item General Health Questionnaire8 using ≥4 scale item endorsements to indicate symptoms of common mental disorder; symptoms of post-traumatic stress disorder (PTSD) were measured using the five item Primary Care PTSD scale using ≥3 symptom endorsements to indicate possible caseness9; alcohol use was measured using the three-item Alcohol Use Disorders Identification Test (AUDIT-C).10 Cut-off scores for possible alcohol use were ≥4 for women and ≥5 for men.11
Recruits were asked about experiences of significant stressful, emotional or family problems and associated help-seeking; response categories were ‘yes’ or ‘no’. As it could theoretically impact on health and well-being in young people leaving their homes and local support networks for the first time, homesickness was assessed; responses were ‘no homesickness’, ‘a little’ or ‘a lot’. A binary measure was created by combining ‘a little’ and ‘a lot’ categories.
Mental health-related stigmatisation and perceived barriers to care were assessed at baseline, post-training and at 3-month follow-up using a 15-item scale adapted from a measure designed for use among military personnel and widely employed in military research studies.12 13 Clusters of questions asked about awareness of help sources, attitudes to help-seeking, practical barriers to care and attitudes to mental illness. Responses were rated using a four-item Likert scale ranging from strongly disagree through disagree, agree and strongly agree. Binary variables were derived by combining agree or strongly agree responses and combining disagree and strongly disagree responses. Endorsing five or more stigmatisation/barrier items was used as a cut-off score. There are no established cut-off scores for this scale; ≥5 endorsements fell above the upper tertile.
Perceptions of leadership and cohesion
Subjective impressions of leadership were measured on completion of recruit training and 3 months later using a four-item scale that has been used in UK military samples.14 Questions asked: how often leaders (1) embarrassed juniors in front of other unit members, (2) treated all members of the unit fairly, (3) showed concern about the safety of unit members and (4) accepted extra duties or tasks for the unit to impress seniors. Responses were graded using a five-item Likert scale (never, seldom, sometimes, often or always); the first and fourth items were reverse-scored. A binary variable was derived for each scale item by combining the seldom and never responses (poorer leadership) and combining sometimes, often and always responses (better leadership). A count variable of 0–4 items was generated, and a binary variable of 3 or 4 endorsements versus fewer represented positive perceptions of leadership (approximately 50% of respondents in each category).
Perceptions of unit cohesion were measured using a four-item scale.15 16 Questions assessed whether recruits felt a sense of comradeship (or closeness) with other people in the unit, whether they could go to most people in the unit about a personal problem, whether seniors were interested in them and whether they felt well informed about what was going on. Responses were rated using a four-item Likert scale ranging from strongly disagree through disagree, agree and strongly agree. Binary variables were generated by combining agree or strongly agree responses and combining disagree and strongly disagree responses. A count variable of 0–4 items was generated, and a binary cohesion variable was represented by 3 or 4 endorsements versus fewer (approximately 50% of respondents).
In order to assess any effects on different aspects of training, a six-item measure was administered; measure items are detailed in table 5. Response categories were strongly disagree, disagree, agree and strongly agree. Disagree and strongly disagree responses were combined as were agree and strongly agree responses to provide a binary variable indicating rejection or endorsement for each of the six training outcomes.
Phase I training ratings, post-training and follow-up
Statistical analyses
Analyses were conducted in the Statistical Package for Social Sciences V.24 for Windows. Stigma reduction is a central aim of SPEAR. Research suggests that around 70% of UK military personnel are likely to endorse three or more items on a stigmatisation scale. The study was therefore powered to demonstrate a 10% reduction in stigma in the intervention compared with the control arm. A sample size of 712 personnel was required (356 in each arm) to detect a 10% reduction in stigma with 80% power with an alpha of 0.05 and a beta of 0.2. Logistic regression examined associations between receipt of SPEAR or the control condition and a range of outcomes at baseline, post-phase one training and follow-up; ORs with 95% CIs were generated. Where appropriate, univariable ORs were adjusted for confounding variables: for mental health-related variables, adjustments were made for baseline stigmatisation level as this varied significantly between intervention and control participants; help-seeking was also adjusted for baseline stigmatisation. Statistical significance was p<0.05. Sensitivity analyses were applied to continuous measures of mental health disorder symptoms, alcohol use and stigmatisation. All continuous outcomes were abnormally distributed and could not be transformed; outcomes were therefore compared using Mann-Whitney U tests for independent samples. Non-responses varied at each stage of the study, therefore numbers and percentages shown in the results section may not sum to the total sample n (%). Missing data in completed key outcome scales varied between 0.3% and 1%; as most scales were largely complete we chose not to impute missing data. The exception was the second AUDIT-C item relating to number of units consumed which had 7% missing data; this was mostly explained by respondents reporting that they did not drink alcohol on the first scale item and understandably leaving the second question (relating to units consumed) blank; these responses were allocated zero scores.
Results
In total, 771 recruits were offered the opportunity to participate in the trial; 707 provided consent and baseline measures (baseline response rate 91.7%). At the end of the 9-week phase I (initial) recruit training period, of 663 available recruits (44 voluntarily left the Services), 655 provided post-training measures (98.1% response rate); 99.4% for SPEAR, 95.4% for controls. At the 3-month follow-up point, of 654 available recruits 481 provided measures (73.6% response rate); 74.9% for SPEAR participants (261 of 325 possible) and 66.9% for controls (220 of 329 possible) (χ2 15.2, d.f. 1, p<0.001). There were no delays in obtaining data at post-training and 3-month follow-up.
Response at follow-up
Although more SPEAR participants than controls responded at 3-month follow-up, there were no significant differences between responders and non-responders in all measures of mental health, alcohol misuse, stigmatisation and homesickness obtained post-phase one training (data shown online in supplementary table 1).
Supplementary file 1
Socio-demographic factors
At baseline, males predominated; 84.9% (n=304) of SPEAR recipients were male compared with 81.4% of controls (n=284) (χ2 1.6, d.f.1, p=0.21). 29.0% (n=102) of SPEAR participants were in a long-term relationship compared with 32.8% of control participants (n=114) (χ2 1.17, d.f.1, p=0.28). Despite the loss of 25% of the initial sample at follow-up, the proportions of men and women and the two relationship categories were not significantly different at follow-up (table 1).
Socio-demographic characteristics
Mental health and related outcomes
PTSD, CMD and alcohol misuse caseness was not significantly different between groups at baseline, post-phase I training or at 3 months post training. Mental health-related stigmatisation was significantly lower among SPEAR recipients at baseline compared with controls (26.6% vs 34.3%) (OR 0.70, 95% CI 0.50 to 0.96, p<0.05) but stigma levels were not significantly different at the two follow-up points. Levels of homesickness were not significantly different between trial arms at the three measurement points. When adjusted for baseline stigmatisation level, there was no significant difference between intervention group members and controls for any of the mental health-related outcomes (table 2).
Mental health and related factors outcome measures
Help-seeking
SPEAR and control recruits entered phase I training having previously experienced similar levels of past significant stressful, emotional or family problems prior to joining. They experienced similar levels of such problems during phase I training and at 3 months post training. Levels of help-seeking for these problems both during and after phase I training were not significantly different between groups. Adjusting for baseline mental health-related stigmatisation had no impact on significance levels (table 3).
Subjective significant stressful, emotional or family problems and help-seeking
Cohesion and leadership
There were no significant differences in cohesion and leadership ratings between SPEAR recipients and controls at either follow-up assessment point. Adjusting for mental health factors had no impact on significance levels (table 4).
Cohesion and leadership
Training outcomes
There were no significant differences between SPEAR recipients and controls in their ratings of the six training outcomes on completion of phase I training and at follow-up (table 5).
Sensitivity analyses
Continuous measures for the four key outcome variables were compared. At baseline, controls had statistically significantly lower mean scores on the GHQ-12 (p<0.05); however, the scores were only marginally different and not clinically meaningful (mean score 1.27 vs 1.01). At baseline, mean stigma scores were significantly higher among controls (p<0.05); however, the differences were again marginal and not clinically meaningful (mean score 3.11 vs 3.49). At all follow-up points, there were no statistically significant differences between arms in mean scores on the mental health, alcohol and stigmatisation scales (data shown online in supplementary table 2).
Supplementary file 2
Discussion
In this study of a resilience-based intervention delivered by military trainers, we found no evidence that the SPEAR intervention had any significant positive or negative effects on mental health and well-being, stigmatisation and help-seeking for mental health reasons and alcohol use compared with training as usual. We examined perceptions of leadership and cohesion as these factors were an additional focus of the SPEAR intervention and six training effects/outcomes; again, we found no significant between-group effects.
Strengths and limitations
There are a number of potential limitations to this study. First, although the follow-up response rate was overall high, around 25% of participants did not provide data. This was related to non-availability rather than refusal to participate; some participants were engaged in training or duties when the research team undertook base visits. Although significantly more SPEAR participants responded at follow-up, socio-demographic, mental health and related variables collected on completion of phase I training did not predict response at follow-up and the proportions of men and women and those in or not in a relationship did not become significantly different at either follow-up point; therefore we do not believe that the loss of participants introduced bias. Although we tested a resilience-based intervention, we did not take any direct measures of resilience (such as the Connor-Davidson Resilience Scale (CD-RISC17); however, we consider that the various measures of mental health and help-seeking represent key aspects of psychological resilience. The reason for not embedding the CD-RISC was to keep the study measures as brief as possible, to maximise participation, to focus on mental health and well-being outcomes and to minimise the impact of the study on training time. It may be that the effects of SPEAR training, particularly on factors such as stigma, may not have been apparent in the relatively short study time frame; however, we feel that it would be unusual for an intervention to have no measurable effect initially and then exert a substantial effect in the longer term. Indeed, stigma levels were initially significantly lower at baseline in the SPEAR recipients and not significantly different to controls at the follow-up points. It is possible that training teams became aware of whether they were in the control or SPEAR condition as they frequently interacted. All recruit trainers are employed in the role as they meet exceptional standards of military integrity and are highly skilled.
The absence of effect seen in this study could be related to a number of factors, foremost among which was the good mental health of RAF recruits on entry to service, which might limit the opportunity to generate further substantial improvement at a group level. However, on entry to the military, RAF recruits were drinking, and continued to drink substantial amounts of alcohol despite receiving the SPEAR intervention even though it contained a component focusing specifically on substance and alcohol use. The lack of effect may reflect the relative ineffectiveness of group and education-based interventions for alcohol use reduction in young people.18 A second observation relates to the positive endorsement of leader behaviours by the majority of study participants. Good leadership and cohesion were present in both study arms. Effective leadership is known to be supportive of mental health,19 furthermore, unit cohesion is known to positively influence the discussion of mental health matters and to reduce stigmatisation.20 The margin for SPEAR to have any additional positive benefit over and above effective leadership and cohesion may have been small. We therefore suggest that key resilience interventions are probably about effective leadership and fostering cohesion rather than stand-alone interventions. Many organisations search for a ‘silver bullet’ intervention that can be used to improve the mental health and well-being of their employees when time might be better spent refining leadership and building strong cohesion.
Resilience interventions are proliferating in both civilian and military sectors; however, interventions with apparent face validity may be problematic when their effects are subject to critical evaluation. A recent example of such a programme is the US Army’s ambitious Comprehensive Soldier and Family Fitness (CSFF) programme21 which has been delivered to over 1 million US soldiers. Other than programme evaluations, few robust outcome measures were gathered during roll-out.22 One of the core themes of both CSFF and SPEAR is the promotion of optimism and positive reframing of challenging situations. The use of positive psychology interventions has been criticised by academics who argue that an emphasis on positivity and optimism could be counterproductive and could ultimately stigmatise those who struggle with mental illness. Frequently, such programmes offer no coherent account of how a focus on wellness helps to reduce either the burden of psychological symptoms or the risk of developing mental disorder.23 There is a growing call on both scientific and ethical grounds to cease mass programme dissemination until preliminary evidence for effectiveness has been demonstrated.24
Although we did not encounter any negative effects in the SPEAR trial, we found no evidence of any beneficial outcomes and a possible enhancement of stigma in the intervention group. This is a potentially important finding as the UK military is currently seeking to introduce health and well-being interventions often without a robust outcome evaluation plan. In previous studies, we have evaluated a number of interventions which seek to improve military mental health and well-being. RCTs have evaluated UK BATTLEMIND, a resilience promotion intervention25; Trauma Risk Management, a peer support intervention26; and a further comparative study using propensity score matching methodology of decompression, a form of structured post-deployment rest, which seeks to improve readjustment and well-being.27 On each occasion we have demonstrated limited benefit or specific small effects, particularly among those with arguably the greatest need, such as combat personnel and those with substantial operational exposure. Given the apparent marginal benefit in many health and well-being support programmes and the lack of any measurable positive effect in the SPEAR intervention, careful thought should be given when considering introducing such interventions. At the very least care should be taken to ensure that they do not cause unintended consequences. Any novel intervention should be accompanied by a robust evaluation. We also consider that more work is needed to understand the best methods to measure the impact of resilience-based interventions as it is possible that focusing on the absence or presence of mental disorder symptoms may possibly miss important effects that any novel intervention might have.
In conclusion, we tested a resilience-based intervention among military recruits delivered by training teams in their initial phase of training and found no evidence that it had any specific benefit to health and well-being. Given that many health and well-being strategies incorporate interventions without an accompanying evaluation strategy to test their effectiveness, this study provides a cautionary example of why the latter is important.
References
Footnotes
Contributors NJ wrote the draft manuscript and participated in all stages of the research project. NG contributed to the design of the study and writing the manuscript. HB contributed to the design of the study and writing the manuscript. LH contributed to all stages of the study and writing the manuscript. CW contributed to all stages of the study and writing the manuscript. AM provided logistic support at all stages of the study and contributed to writing the manuscript.
Funding This work was supported and funded by the Ministry of Defence.
Disclaimer The views expressed are those of the author(s) and not necessarily those of the UK MoD.
Competing interests At the time of the study NJ, NG, LH and CW were based at King’s College London which, for the purpose of the study and other military-related studies, receives funding from the UK Ministry of Defence (MoD). AM was based with the military organisation that provided the training element of the intervention tested in the study and participated in training delivery. AM was blind to the collection and analysis of the study outcomes. NJ, NG, LH, CW and AM’s salaries were totally or partially paid by the UK MoD. The UK MoD provides support to the Academic Department of Military Mental Health, and the salary of NJ is covered totally by this contribution. NG is the Royal College of Psychiatrists’ Lead for Military and Veterans’ Health, a trustee of Walking with the Wounded, and an independent director at the Forces in Mind Trust; however, he was not directed by these organisations in any way in relation to his contribution to this study. NJ is a full-time member of the UK armed forces seconded to King’s College London.
Patient consent Obtained.
Ethics approval The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. The study was approved by the Ministry of Defence Research Ethics Committee (766/MODREC/16 dated 24 November 2016).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement We have unpublished data that did not have a direct relationship with the study outcomes. We will further explore these data and publish further studies in due course.