Article Text
PDF
Abstract
Introduction Previous research comparing veteran and civilian mental health (MH) outcomes often assumes stable rates of MH service use over time and relies on standardisation or restriction to adjust for differences in baseline characteristics. We aimed to explore the stability of MH service use in the first 5 years following release from the Canadian Armed Forces and the Royal Canadian Mounted Police, and to demonstrate the impact of using increasingly stringent matching criteria on effect estimates when comparing veterans with civilians, using incident outpatient MH encounters as an example.
Methods We used administrative healthcare data from veterans and civilians residing in Ontario, Canada to create three hard-matched civilian cohorts: (1) age and sex; (2) age, sex and region of residence; and (3) age, sex, region of residence and median neighbourhood income quintile, while excluding civilians with a history of long-term care or rehabilitation stay or receipt of disability/income support payments. Extended Cox models were used to estimate time-dependent HRs.
Results Across all cohorts, time-dependent analyses suggested that veterans had a significantly higher hazard of an outpatient MH encounter within the first 3 years of follow-up than civilians, but differences were attenuated in years 4–5. More stringent matching decreased baseline differences in unmatched variables and shifted the effect estimates, while sex-stratified analyses revealed stronger effects among women compared with men.
Conclusions This methods-focused study demonstrates the implications of several study design decisions that should be considered when conducting comparative veteran and civilian health research.
- mental health
- veterans
- health services research
Data availability statement
Data may be obtained from a third party and are not publicly available. The data set from this study is held securely in coded form at ICES. While legal data sharing agreements between ICES and data providers (eg, healthcare organisations and government) prohibit ICES from making the data set publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at www.ices.on.ca/DAS (email: das@ices.on.ca).
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Canadian veterans are more likely to self-report a mental health condition than members of the Canadian general population and are more likely to seek mental healthcare than civilians.
Most existing research has assumed a constant risk or rate over time and often does not account for the potential healthy soldier effect.
WHAT THIS STUDY ADDS
We demonstrated variability in the hazard of an outpatient mental health encounter over the first 5 years of release from the Canadian Armed Forces and the Royal Canadian Mounted Police, with veterans having a significantly higher hazard within the first 3 years of release compared with civilians, with no notable differences observed for years 4–5.
Effect estimates were further influenced by sex and matching approach.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Our study uses outpatient mental health encounters as an example of how methodological decisions, such as matching strategy and analytic approaches, influence effect estimates in comparative veteran health research, and provides guidance for future research in this field.
Introduction
Previous research suggests that military veterans are more likely to report mental health (MH) conditions and increased MH service use than their civilian counterparts. Research from male US veterans1 and male and female Canadian Armed Forces (CAF) veterans2 found that veterans were more likely to report a mood or anxiety disorder than civilians. Another study comparing health service use of Canadian veterans with members of the general population found that male and female veterans were more likely to seek mental healthcare from a family doctor, psychiatrist, psychologist, nurse or social worker than civilians.3 Taken together, these findings may suggest that veterans may have a greater need for MH services, greater access to MH services or both.
To date, research has been limited in two important areas, precluding a complete understanding of the differences in MH service use between veteran and civilian populations. First, most studies report a single effect estimate of health outcomes over a period of follow-up (eg, ref 4), which assumes a constant risk or rate over time.5 Given that previous research suggests that the military-to-civilian transition can be stressful for some individuals,6 7 it is possible that MH needs are greater in the period immediately following release from the forces. As such, research comparing veteran and civilian MH service use should strive to consider potential differences over time. Second, when making comparisons between veterans and the Canadian general population, it is important to consider their sociodemographic differences. Approximately 14% of Canadian veterans are female8 versus approximately 50% of Canadian civilians,9 while 2021 census data indicated that 42% of veterans were over the age of 6510 versus approximately 19% of civilians.11 Previous work also demonstrates that veterans tend to reside in higher-income communities and particular geographical regions,12 which could determine local access to MH services.13 14 Some research comparing veteran and civilian health outcomes has attempted to account for differences in the age and sex distribution of veteran populations using standardisation (eg, ref 15), stratification (eg, ref 2) or restriction (eg, refs 16 17). Matching is another strategy that can prevent confounding of crude effect estimates by the matching variables and increase efficiency when the matched variables are related to the exposure and outcome of interest.18 19 Matching may be particularly advantageous in veteran–civilian comparisons using administration healthcare data, as it does not significantly increase costs or time. To date, the creation of a veteran cohort in routinely collected administrative data holdings in Ontario, Canada20 has allowed veteran health researchers to conduct matched cohort studies directly comparing non-fatal self-harm emergency department visits among male veterans and civilians21 and death by suicide among veterans and civilians.22
While the availability of administrative healthcare data facilitates direct comparisons between veterans and civilians, comparisons may be complicated by the presence of the ‘healthy soldier effect’ (HSE).23 The HSE is akin to the healthy worker effect and, when present, can result in selection bias or confounding.24 25 Accounting for the HSE becomes increasingly complex when working with veteran populations, as it may vary according to a number of factors, including length of time between military service and timing of the study, and the health outcomes of interest.26 Further, not all members of the general population are well enough to be employed, and including these individuals in a comparator cohort could therefore exacerbate the HSE. Some research has aimed to reduce the HSE by excluding civilians who report health conditions that would typically render them ineligible for recruitment into the CAF (eg, ref 16); however, the feasibility of this approach using administrative healthcare data has not been widely explored. As such, using an outpatient MH encounter in the first 5 years following release from active service as an example, the objectives of this methods paper are to (1) examine whether the rate of an outpatient MH encounter varies over the first 5 years of follow-up; (2) explore whether sex-based differences in the estimated rates between veterans and civilians exist; and (3) demonstrate the impact of increasingly stringent matching criteria of veterans to the general population on effect estimates.
Methods
Study design
This study employed a matched, retrospective cohort design of veterans and civilians residing in Ontario, Canada using administrative healthcare data held by ICES (formerly known as the Institute for Clinical Evaluative Sciences), a not-for-profit institute whose legal status under Ontario’s health information privacy law allows it to collect and analyse healthcare and demographic data, without consent, for health system evaluation and improvement.
Study population
The study population consisted of CAF or Royal Canadian Mounted Police (RCMP) veterans and civilians residing in Ontario. In Canada, health insurance coverage is overseen by the federal government for active-duty members of the CAF and RCMP and by the provinces for civilians and veterans. The standard 3-month waiting period for Ontario Health Insurance Plan (OHIP) coverage is waived when evidence of military service is provided and a veteran administrative code is linked to the provincial health card, along with military service start dates. For this study, the Ministry of Health and Long-Term Care (MOHLTC) provided an anonymised list of individuals with a veteran administrative code, which was linked to ICES key number (IKN), a unique encoded identifier.
Veteran inclusion/exclusion criteria
Veterans were included in the current study if they registered for OHIP between 1 April 1991 and 31 March 2020 (for CAF veterans), or between 1 April 1991 and 31 December 2013 (for RCMP veterans). Previous research indicates that ≥85% of the cohort consists of CAF veterans.12 The date of OHIP registration was used to approximate CAF/RCMP release dates.12 Veterans who held OHIP coverage while serving or who were ≤16 years of age at the time of recruitment into the forces were excluded from this study (see figure 1 for the flow diagram of exclusions). A total of 31 759 veterans were identified for inclusion in this study.
Flow diagram of exclusions in the creation of initial veteran cohort. OHIP, Ontario Health Insurance Plan.
Civilian matching approaches
Civilians were eligible for inclusion in this study if they were alive at the time of the study index date (ie, the date of OHIP registration for the matched veteran). A total of three civilian comparator cohorts were created, with the matching approach for each cohort building on the previous one. The first comparator cohort was hard-matched on age at index (using birth year) and sex; the second additionally matched on region of residence (one of Ontario’s 14 local health integration networks; assigned using census postal code data); and the third additionally matched on median neighbourhood income quintile, which was also determined using census postal code data. This third cohort also excluded civilians who (1) had a long-term care or rehabilitation facility stay; or (2) received disability or income support payments prior to their assigned index date, in efforts to reduce the potential of an HSE.
Data sources
The current study relied on provincial healthcare administrative databases, which were linked using IKNs and analysed at ICES. The Registered Persons Database was used to determine demographic information, while the OHIP database provided physician billing data for outpatient MH encounters.
Study variables
Exposure variable
The primary exposure was a dichotomous variable for veteran status (eg, veteran/civilian), which was ascertained using the veteran administrative code supplied by the MOHLTC.
Outcome variable
The outcome variable was the first outpatient MH encounter (primary care or psychiatric visit) within the first 5 years of follow-up after the index date. Outpatient MH-related primary care visits were identified using an existing ICES validated algorithm that uses physician specialty, OHIP diagnostic code and service location to identify outpatient MH-related visits;27 all visits with a psychiatrist were included.
Covariates
In addition to the matching variables, we considered three additional covariates: rurality and the number of major and minor comorbidities. Rurality was assessed at the index date using Rurality Index for Ontario (RIO) 2008 scores, which take into consideration the population size, density and access to healthcare services of communities in Ontario.28 RIO scores range from 0 to 100, with lower scores indicating more urban communities and higher scores representing more rural communities. We employed the same categories used by Lucas et al,29 which group communities using the following RIO scores: 0–3 (large, urban centres with access to advanced referral centres; eg, Toronto), 4–14 (large, urban centres with access to basic referral centres; eg, Kitchener), 15–39 (smaller urban centres; eg, Brockville), 40–74 (small, rural communities located primarily in southern Ontario; eg, Goderich) and 75+ (remote, rural communities located primarily in northern Ontario; eg, Kapuskasing). The number of major and minor comorbidities was assessed at 1 year following index date using the Johns Hopkins Adjusted Clinical Groups (ACG) system for categorising illnesses using International Classification of Disease (ICD) codes. ICD codes are assigned to one of 32 ACG system’s aggregated diagnosis groups (ADGs), based on the aetiology, duration and severity of the condition, diagnostic certainty, and involvement of specialty care.30 31 For this study, we excluded three psychosocial ADGs to reduce artificial inflation with the outcome and created two count variables: one for the number of major comorbidities in the first year of follow-up (range=0–7) and one for the number of minor comorbidities within the first year of follow-up (range=0–22).
Analytic approach
To describe the baseline characteristics of the cohorts, means with SD were estimated for continuous variables, while frequencies and proportions were used for categorical variables. Standardised differences, which assess the balance of covariate distribution between two groups without being influenced by sample size,32 33 were used to compare the veteran and civilian cohorts.
For each matched cohort, we obtained survival and hazard curves using the actuarial method due to the large number of distinct event times. The proportional hazards (PH) assumption was assessed by graphically examining the survival, hazard and log(-log) survival curves, including a time*veteran status interaction term in the regression model, and statistical testing using Schoenfeld residuals. Visual inspection of the survival and log(-log) survival curves did not suggest that the PH assumption had been violated. However, the statistical testing approaches, which are highly sensitive to sample size, did indicate a potential deviation from the PH assumption. As such and to account for right censoring (ie, individuals who did not have an outpatient MH encounter within the first 5 years of follow-up), we used extended Cox models with a Heaviside function34 to estimate the HRs of an outpatient MH encounter for two distinct intervals: one for the first 3 years of follow-up (HR1) and one for the fourth and fifth years of follow-up (HR2). This time point was selected based on the appearance of the hazard functions (see figure 2). We used the word ‘hazard’ throughout the manuscript as a statistical term, which should not necessarily impart a negative connotation.
Hazard functions, by cohort and veteran status. Cohort 1 is matched on age and sex; cohort 2 on age, sex and region of residence; and cohort 3 on age, sex and region of residence and median neighbourhood income quintile. Cohort 3 also excludes civilians with a history of long-term care/rehabilation stay and/or receipt of disability/ income support payments.
For each matched cohort, we first ran unadjusted regression models. The second model adjusted for the matching variables (ie, cohort 1 models were adjusted for age and sex, while cohort 2 models were adjusted for age, sex, region of residence, etc). The third model adjusted for all remaining matching variables. The fourth model additionally adjusted for rurality, while the fifth model adjusted for all matching variables and the number of major and minor comorbidities. The final model for each cohort adjusted for all matching variables, rurality and the number of major and minor comorbidities. We also included an interaction term between veteran status and sex to assess for potential effect measure modification by sex. For all analyses, we used two-sided hypothesis tests, and p values of less than 0.05 were considered statistically significant. Analyses were carried out using SAS V.9.4 statistical software.35
Results
Participant characteristics
Online supplemental table 1 compares the baseline characteristics of each matched veteran–civilian cohort. Only one veteran could not be matched in the creation of the first cohort (see figure 1), and the use of increasingly stringent matching criteria did not result in a significant loss of veteran participants. For unmatched variables, differences in the distribution of major and minor comorbidities decreased with more stringent matching criteria, although minor differences remained for rurality.
Supplemental material
Outpatient MH service use
Approximately one-third of veterans had at least one outpatient MH visit within the first 5 years of follow-up (range=32.5%–33.2% across cohorts), compared with 30.3%–32.8% of civilians. Table 1 shows the actuarial life table output for each matched cohort. Across all cohorts, the cumulative survival probability (ie, the probability of not having an outpatient MH encounter within the first 5 years of follow-up) was higher for civilians than veterans.
Actuarial life table, by cohort and veteran status
Regression models
The hazard functions and fully adjusted effect estimates for each matched cohort are presented in figure 2 and table 2, respectively (see online supplemental table 2 for comparison with the unadjusted effect estimates). Across all cohorts, the overall (average) HR and the HR for the first 3 years of follow-up (HR1) were higher for veterans than civilians; these differences were statistically significant. The HRs for years 4 and 5 of follow-up (HR2) suggested no significant differences in the hazard of an outpatient MH encounter within the first 5 years of follow-up, provided they had not had an encounter prior to this time period. Additionally, the interaction term between veteran status and sex was statistically significant, suggesting effect measure modification by sex (see table 2 for output from the sex-stratified models). χ2 tests of equality indicated that the time-dependent HRs (HR1 and HR2) were significantly different overall and for women in cohort 1, women only in cohort 2, and overall and for men and women in cohort 3 (p<0.001). Together, these findings suggest that the HR of an outpatient MH encounter is not constant over time; rather, the hazard is significantly greater among veterans compared with civilians in the first 3 years of follow-up compared with years 4 and 5, and this difference is further influenced by sex and matching approach.
Supplemental material
Fully adjusted HR of an outpatient mental health encounter within the first 5 years of follow-up for the three cohorts, overall, and at 3 or fewer years of follow-up and at 4–5 years of follow-up, by sex and veteran status
Impact of matching approaches on effect estimates
Including additional matching variables beyond age and sex resulted in slight differences in the effect estimates produced. The inclusion of region of residence as a matching variable shifted the magnitude of the association between veteran status and an outpatient MH encounter within the first 5 years of follow-up towards the null, although the overall HR and HR for the first 3 years of follow-up remained statistically significant. Additionally including median neighbourhood income quintile as a matching variable resulted in the effect estimate moving away from the null, above and beyond what was observed for the cohort matched on only age and sex (eg, overall fully adjusted HRcohort 3=1.18 (95% CI 1.15 to 1.20) vs HRcohort 1=1.11 (95% CI 1.08 to 1.14)). The effects of matching were noticeable overall and for the first 3 years of follow-up but not on the effect estimates for years 4 and 5.
Discussion
This methods paper offers novel contributions to our understanding of the implications of design choices, including matching and analytic approaches, on comparative veteran–civilian health research. Importantly, we found that the HR of an outpatient MH encounter within the first 5 years of follow-up after release varies over time, which may have important implications for future veteran health outcomes research. Specifically, we found that the average adjusted HR of an outpatient MH encounter, which can be interpreted similarly to a relative rate,36 was 9%–22% higher for veterans than civilians within the first 3 years of follow-up, and only 2%–4% higher than civilians at 4 and 5 years of follow-up. It is possible that the much higher rates among veterans earlier in the follow-up period are a cultural reflection of proactive healthcare-seeking behaviours in military populations that then attenuate over time. Importantly, the use of Heaviside functions to estimate time-dependent health outcomes has not been widely used in existing veteran–civilian comparisons, which have instead used Cox PH models and a single period of follow-up (eg, refs 4 37). The differences observed in the example used here demonstrate the importance of considering the timing of visits when examining patterns of MH service use between veterans and civilians.
We also found similar trends in MH service use between male and female veterans and civilians, although the difference between veterans and civilians was more pronounced among women overall and for the first 3 years of follow-up. Among men, the overall adjusted HR was between 7% and 19% higher for veterans compared with civilians during the first 3 years of follow-up, and 4%–5% higher for veterans during years 4–5. Among women, the overall adjusted HR was 13%–26% higher during the first 3 years of follow-up, with no notable difference in the HR observed for women during years 4–5. This expands on findings from the Life After Service Survey, which demonstrated that male and female veterans were more likely to self-report accessing MH care from a family doctor or psychiatrist than their civilian counterparts,3 by providing additional data about how these differences vary over time in the 5-year period immediately following release from the forces and by highlighting the heterogeneity that exists between male and female veterans and civilians.
Examination of the overall and time-dependent HRs indicates some variability across the cohorts as a result of differing matching approaches. Matching on age and sex alone likely produces biased estimates, given the known differences in region of residence between veterans and civilians. Indeed, veteran health research from Scotland has consistently matched on age, sex and region of residence (eg, refs 4 38 39). In this study, the inclusion of region of residence as a matching variable resulted in an unanticipated shift of the effect estimate towards the null, while the addition of median neighbourhood income quintile as a matching variable and exclusion of civilians who were less likely to be employed due to health conditions resulted in a correction of the previous effect estimate. These findings suggest that the use of more stringent matching criteria combined with efforts to mitigate the HSE results in more accurate effect estimates. However, if researchers face constraints in matching beyond age and sex, the inclusion of other sociodemographic characteristics as covariates will help minimise bias in the effect estimates.
This study had a number of strengths. First, it used population-level data and likely includes most veterans residing in Ontario.12 This offers an advantage over existing Canadian research, which has generally relied on smaller, nationally representative surveys (eg, refs 3 40) to draw inferences about patterns of MH service use, as it allows for direct comparisons between veterans and civilians. It also provides guidance about creating comparable civilian cohorts for use in veteran health outcome and health service use research. Importantly, the finding that rates of MH service use vary within the period of time following transition from the forces could be used to help inform healthcare planning and policy at the Veterans Affairs Canada (VAC) and provincial healthcare system levels. However, there are also some limitations worth noting. Due to limitations with administrative healthcare data, we were unable to exclude some civilians with health conditions that may preclude service in the CAF (eg, Crohn’s disease, chronic bronchitis, etc), meaning we were unable to create a truly equivalent occupational cohort of civilians. Despite this limitation, the standardised differences for the number of major and minor comorbidities indicate that the groups were well balanced, particularly for the third matched cohort. Further, the administrative healthcare data capture only provincially funded healthcare visits, meaning that any services accessed via private insurance or VAC benefits are not included in these data and our effect estimates are likely underestimations of the true rate of MH service use in the first 5 years of follow-up. It is also possible that unmeasured variables, such as adverse childhood experiences, could partially explain the associations observed in this study. Additionally, the inability to exclude RCMP veterans may limit comparability with research from other countries, as their occupational experiences may vary from those of military veterans. Finally, we acknowledge that the index date used in this study carries importance for veterans, but is relatively meaningless for civilians. For many of these individuals, and indeed for some veterans, the first outpatient MH encounter following the index date represents a prevalent episode of care, versus an incident one, due to the left-censored nature of the data.
Conclusions
This research provides valuable methodological contributions for conducting comparative veteran health outcome research. In the example presented here, we demonstrated variability in the rates of MH service use over time, indicating the importance of considering timing of MH visits in future research. We also observed differences based on sex, which highlights the importance of treating sex as a potential effect measure modifier in veteran MH service use research. Additionally, our findings provide a demonstration of how varying approaches to creating civilian comparator cohorts affects the magnitude of effect estimates. Interestingly, we found that matching on region of residence resulted in a shift of the effect estimate towards the null, while additionally including median neighbourhood income quintile as a matching variable shifted the effect estimate away from the null. We also found that matching approach impacted the overall effect estimate and for the first 3 years of follow-up, but not for years 4 and 5, which further emphasises the importance of considering time when making design choices and interpreting findings.
Data availability statement
Data may be obtained from a third party and are not publicly available. The data set from this study is held securely in coded form at ICES. While legal data sharing agreements between ICES and data providers (eg, healthcare organisations and government) prohibit ICES from making the data set publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at www.ices.on.ca/DAS (email: das@ices.on.ca).
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by the University of Toronto Research Ethics Board (protocol #00039511). ICES is an independent, non-profit research institute funded by an annual grant from the Ontario Ministry of Health (MOH) and the Ministry of Long-Term Care (MLTC). As a prescribed entity under Ontario’s privacy legislation, ICES is authorised to collect and use healthcare data, without consent, for the purposes of health system analysis, evaluation and decision support. Secure access to these data is governed by policies and procedures that are approved by the Information and Privacy Commissioner of Ontario.
Acknowledgments
This document used data adapted from the Statistics Canada Postal CodeOM Conversion File, which is based on data licensed from Canada Post Corporation, and/or data adapted from the Ontario Ministry of Health Postal Code Conversion File, which contains data copied under licence from Canada Post Corporation and Statistics Canada. Parts of this material are based on data and/or information compiled and provided by the MOH and CIHI. The analyses, conclusions, opinions and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred. We thank IQVIA Solutions Canada for use of their Drug Information File.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors KS, PK, ALM and PMS conceptualised the study and contributed to the study design, analytical plan and interpretation. ALM, PK and KS contributed to data collection. KS conducted the statistical analyses. KS drafted the manuscript. All authors reviewed and approved the final version of the manuscript. KS acts as the guarantor of the manuscript.
Funding KS received funding for her doctoral research from a Canadian Institutes of Health Research Doctoral Research Award, the Wounded Warriors Canada Doctoral Scholarship and a PEO International Scholar Award. This research is part of a larger project that received funding from True Patriot Love and the Canadian Institute for Military and Veteran Health Research through the CIMVHR TPL Research Initiative. This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health (MOH) and the Ministry of Long-Term Care (MLTC).
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.