Article Text

Original research
Multilevel approach to individual and organisational predictors of stress and fatigue among healthcare workers of a university hospital: a longitudinal study
  1. Oumou Salama Daouda1,
  2. René Sosata Bun2,
  3. Karim Ait Bouziad1,
  4. Katiuska Miliani3,
  5. Anastasia Essa-Eworo3,
  6. Florence Espinasse4,
  7. Delphine Seytre5,
  8. Anne Casetta6,
  9. Simone Nérome7,
  10. Adelaide Nascimento8,
  11. Pascal Astagneau2,3,
  12. Laura Temime1,
  13. Mounia N Hocine1
  1. 1MESuRS Laboratory, Conservatoire National des Arts et Metiers, Paris, Île-de-France, France
  2. 2iPLESP, INSERM, Sorbonne Universite, Paris, Île-de-France, France
  3. 3CPIAS Ile de France, Paris, Île-de-France, France
  4. 4Hôpital Ambroise Paré, AP-HP, Paris, Île-de-France, France
  5. 5Hôpital Avicenne, AP-HP, Paris, Île-de-France, France
  6. 6Hôpital Cochin, AP-HP, Paris, Île-de-France, France
  7. 7Hôpital Beaujon, AP-HP, Paris, Île-de-France, France
  8. 8CRTD Laboratory, Conservatoire National des Arts et Metiers, Paris, Île-de-France, France
  1. Correspondence to Dr Oumou Salama Daouda, MESuRS Laboratory, Conservatoire National des Arts et Metiers, Paris, France; oumou-salama.daouda{at}lecnam.net

Abstract

Objective Healthcare workers (HCWs) are at high risk of experiencing stress and fatigue due to the demands of their work within hospitals. Improving their physical and mental health and, in turn, the quality and safety of care requires considering factors at both individual and organisational/ward levels. Using a multicentre prospective cohort, this study aims to identify the individual and organisational predictors of stress and fatigue of HCWs in several wards from university hospitals.

Methods Our cohort consists of 695 HCWs from 32 hospital wards drawn at random within four volunteer hospital centres in Paris-area. Three-level longitudinal analyses, accounting for repeated measures (level 1) across participants (level 2) nested within wards (level 3) and adjusted for relevant fixed and time-varying confounders, were performed.

Results At baseline, the sample was composed by 384 registered nurses, 300 auxiliary nurses and 11 midwives. According to the three-level longitudinal models, some predictors were found in common for both stress and fatigue (low social support from supervisors, work overcommitment, sickness presenteeism and number of beds per ward). However, specific predictors for high level of stress (negative life events, low social support from colleagues and breaks frequently cancelled due to work overload) and fatigue (longer commuting duration, frequent use of interim staff in the ward) were also found.

Conclusion Our results may help identify at-risk HCWs and wards, where interventions to reduce stress and fatigue should be focused. These interventions could include manager training to favour better staff support and overall safety culture of HCWs.

  • mental health
  • occupational health
  • statistics
  • fatigue
  • longitudinal studies

Data availability statement

Data are available upon reasonable request. The data that support the findings of this study are available from the authors, upon reasonable request. It can be addressed to Pr Pascal Astagneau (pascal.astagneau@aphp.fr), the project leader.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

  • Healthcare workers (HCWs) have high levels of perceived stress and fatigue, particularly in medical fields highly exposed to infectious risks.

  • Previous studies show that the high physical and mental demands, as well as unfavourable work environment and organisation, can increase stress and fatigue levels of HCWs.

  • However, there is a lack of studies exploring the complex dynamics of the link between working conditions and HCW health, using multilevel design (HCWs nested within wards), including ward/organisational level predictors.

WHAT THIS STUDY ADDS

  • The ward-level environment significantly influences the stress and fatigue of HCWs in addition to individual factors and time variations.

  • Supervisors providing low social support and with low safety culture, work overcommitment, sickness presenteeism and working in smaller wards were identified as predictors of both high stress and fatigue of HCWs.

  • Negative life events (whether occurring in personal or in professional life), low support from colleagues and breaks frequently cancelled due to work overload are specific predictors of high level of stress. While commuting duration, frequent use of interim staff and working in a medical ward were associated with high level of fatigue of HCWs.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

  • In this study, we can identify some areas for improvement to better prevent stress and fatigue for HCWs. High stress and fatigue can be reduced through mutual and specific organisational intervention strategies.

Introduction

There is a growing research interest about stress in healthcare workers (HCWs), as the prevalence of nurses affected by negative mental states is high.1 2 A recent meta-analysis including 45 539 nurses worldwide in 49 countries across multiple specialties estimated 11.2% prevalence of burnout among global nurses.3

There is a vast literature exploring occupational predictors of both stress and fatigue among HCWs, including studies with cross-sectional, longitudinal and experimental designs. Low social support from supervisors and colleagues, as well as younger age and being female, was found to be predictors of stress among HCWs.4–6 In addition, several studies highlight the importance of job satisfaction and favourable work environment in order to prevent stress of HCWs. Indeed, unacceptable work schedule, work overcommitment and shift work appeared to lead to higher stress of HCWs.7–9 Also, being highly committed to the organisation was associated with both role and work overload and work-related stress.9

Similarly, several studies explored the relationship between HCWs fatigue and some aspects of working conditions such as low social support from supervisor and colleagues, older age as well as carrying for both children and elders.10–12 Psychological job demands and unfavourable work environment (including shift work organisation and work schedule characteristics) also emerged as significant predictors of fatigue among HCWs.10 13 In addition, HCWs with higher levels of overcommitment were more prone to experience chronic fatigue syndrome.14

As HCWs are responsible for playing their role in keeping patient safety, observing safety principles while delivering daily care can lead to a decrease in adverse events and damages.15 Negative HCW perception of patient safety can also have a detrimental effect on stress and fatigue, as suggested by recent studies reporting negative correlations between patient safety culture and the level of stress and fatigue.16 17

HCWs fatigue and mental health are important elements to monitor for continued HCWs well-being and patient safety.18 Stress and fatigue in the workplace negatively impact productivity and increase absenteeism of HCWs19 and may result in non-optimal quality of care for patients in healthcare settings.20 For instance in 2020, the average absenteeism rate in French public hospitals was estimated 9.5% among a representative sample of 300 hospitals21 versus 5.1% nation wide (https://www.ayming.fr/wp-content/uploads/sites/11/2020/09/Barometre-absenteisme-2020-Ayming.pdf).

Still, a better and more global understanding of how individual-level and organisational-level factors interacts with stress and fatigue of HCWs in hospitals is needed, especially in the French context where studies on hospital staff stress and fatigue are scarce. The originality of our study comes from its multilevel design. We designed the study longitudinally to observe the possible evolutions of fatigue and stress regarding the organisational factors that may vary over time in order to better identify the levers of organisational control that could reduce stress and fatigue.

Here, we propose a three-level longitudinal analysis using individual and organisational/ward factors related to stress and fatigue of HCWs, using a 1-year longitudinal data collected from 730 individuals in 32 French hospital wards.

Material and methods

Study design and data collection

We designed a multicentre study on the individual and organisational predictors of stress and fatigue among HCWs at the Hospitals of Paris, the STRIPPS (Stress au Travail et Risque Infectieux chez les Patients et le Personnel Soignant) study.22 The study was conducted between February 2018 and July 2019 and data were collected on midwives, registered and auxiliary nurses. An auxiliary nurse, also known as a healthcare assistant, works closely with healthcare professionals (nurses and physicians,). He/she performs duties such as washing and dressing patients, or making beds, in healthcare facilities. HCWs were recruited from four voluntary French University general care hospitals. Eight wards per participating hospital were drawn at random among wards employing at least 30 HCWs.

Data were collected longitudinally every 4 months during 1 year by two different interviewers for all included participants as follows: t0, corresponding to the first collection during the HCWs inclusion visit; t1, t2, t3, corresponding to follow-up visits at 4 months, 8 months and 12 months. For the first data collection (t0), dates and times of visits by an interviewer were drawn randomly for each participating ward. For later data collections, individual appointments were made with each included HCWs. Data were collected through questionnaire-based interviews at both levels (ward and individual levels).

Individual-level variables

To collect individual-level variables, HCWs of included wards were interviewed individually at 4 times of visits (t0, t1, t2 and t3). The individual questionnaire collected demographic and occupational characteristics, including age, sex, professional status, contractual situation, experience in the hospital and daily working hours. Individual data related to work organisation, including schedule, nightshift, extra hours, mealtimes and rest periods, were also collected. Social support from colleagues and from supervisor was also assessed using the social support dimension of the Job Content Questionnaire (JCQ), which was developed and validated by Karasek et al.23 Social support dimension includes eight 4-points Likert scale items (from 1 to 4), with a resulting score ranging from 8 to 32 (from very low to very high social support). In addition, work overcommitment was measured using the work overcommitment dimension of the Effort-Reward Imbalance questionnaire (ERI).24 This dimension includes six 4-points Likert scale items (from 1 to 4), with a resulting score ranging from 6 to 24. Low total score indicates low work overcommitment. The reliability statistics results showed that Cronbach’s alphas for social support and work overcommitment were 0.79 and 0.76, respectively, indicating acceptable scale reliability. Finally, the management support for patient safety dimension was measured using the French version of the hospital survey on patient safety culture.25

Ward-level variables

To collect data for each of the 32 wards, the hospital health executives (nurse managers) were interviewed at t0 only regarding the following characteristics: medical specialty, number of beds per ward, proportion of double rooms, frequency of tasks performed outside the ward, patient/physician ratio, patient/paramedics ratio and use of external healthcare service providers (ie, interim staff).

Outcome variables

Two primary outcomes were considered:

  1. Perceived stress, assessed with the Perceived Stress Scale 10-item scale (PSS-10), was developed by Cohen et al26 and validated in French.27 The PSS-10 questionnaire includes 10 5-point Likert items (from 0 to 4), with a resulting score ranging from 0 to 40 (from very low to very high perceived stress). The reliability statistics results showed that Cronbach’s alpha for PSS-10 scale was 0.85, indicating acceptable scale reliability.

  2. Fatigue, assessed with the Pichot Fatigue Scale.28 The Pichot Scale includes eight 4-point Likert items (from 1 to 4), with a score ranging from 8 to 32 (from very low to very high fatigue). Cronbach’s alpha for Pichot scale was 0.91, indicating acceptable scale reliability.

Why study stress and fatigue together

Although stress and fatigue are distinct phenomena, both can be considered as mediating factors of behavioural consequences, including absenteeism, turnover and productivity loss of HCWs.29 30 Earlier studies considered both stress and fatigue together as outcomes.31 Here, we assumed that both may result from the high physical and mental demands to which HCWs are exposed, with possible consequences in terms of patient safety. Identifying factors associated with both high stress and fatigue may, therefore, help improve HCW physical and mental health and patient care.

Missing data imputation

For missing data, multiple imputations were performed on validated questionnaire items only (JCQ, PSS-10, Pichot and ERI questionnaires). Data were imputed using multiple imputation, using the R mice package.32 The mice package allows performing imputation of continuous and categorical variables in a context of multilevel and longitudinal data. For all questionnaire items included in the imputation model, missing data were assumed to be missing at random.

Statistical analysis and modelling

We conducted analyses to identify predictors of stress and fatigue levels of participating HCWs. First, in order to validate the use of a three-level longitudinal model, we built two unconditional models (ie, null models, with no independent variables) with two levels (ie, time and individual levels) and three levels (ie, time, individual and ward levels), for each outcome. Indeed, before conducting multivariate multilevel models, performing null models is strongly encouraged.29 We then compared the two unconditional models using the Akaike’s Information Criterion (AIC).30 Lower AIC for three-level unconditional models validated the using of three-level models to predict stress and fatigue. In addition, using null models, intraclass correlations were computed, in order to quantify how much response variable variance is shared, across different combinations of levels.

Due to the large number of explanatory variables and to avoid collinearity with some of them, we proceeded to two steps of selection. Specifically:

  1. First, univariate analyses were performed for all individual-level variables in order to reduce the number of variables to include in the multivariate model. Then, any variables with univariate associations with p values ≤0.20 were included in the multivariate model. This cut-off was chosen to exclude variables of questionable importance.33 Despite the fact that the study was longitudinal, univariate analysis was performed at baseline. This technique is often used and allows to capture the maximum of information on the variables and the outcome.34 Indeed, the response rate at the baseline is naturally higher than at follow-ups.

  2. Second, we developed a three-level multivariate model on each outcome, using the AIC for variable selection, using only variable retained in the first step. All data analyses were conducted using the R software.35

Results

Response rate

Overall, the response rate for all included HCWs who answered the questionnaires at the four visits was 73.5% (510 out of 694), corresponding to 2040 responses in total for all visits. In total, 695 HCWs were included at t0, 644 at t1, 578 at t2 and 556 at t3, with an overall of 2473 observations. A general response rate at t0 is impossible to compute due to the voluntary participation of HCWs present at the time of inclusion in the study. However, at the time of inclusion, the participation rate based on the total number of staff working in the participating wards was 695/2473=28.1%. In addition, the mean number of observations per ward decreases across time points. We observed 22.8 observations at t0, 21.25 at t1, 19.1 at t2 and 18.5 at t3.

An auxiliary nurse failed to answer any of the four questions of social support from supervisor of the Karasek questionnaire and was, thus, excluded from all analyses.

HCW characteristics

The final study sample consisted of 694 HCWs as follows: registered nurses (n=384) (55.3%), auxiliary nurses (n=299) (43.1%) and midwives (n=11) (1.6%). Overall, the female/male gender ratio was 5.5, with 588 (84.7%) female respondents. The majority of HCWs were permanent staff members (n=616) (88.9%) compared with temporary (n=58) (8.4%) and contractual (n=19) (2.7%) staff members. The average number of years of experience was 9 years (9.6), and more than a half of the respondents had supervising responsibilities (n=365) (52.6%) (table 1).

Table 1

Characteristics of individuals at the time of inclusion t0

Ward characteristics

In total, 32 wards were included from various medical fields as follows: 14 (43.8%) in surgery and obstetrics, 11 (34.4%) in ‘other medical specialty wards’ and 7 (21.9%) in intensive care unit (ICU) and reanimation. The average number of beds per ward was 35.5 (SD=18.5), and the proportion of double rooms was approximately 20% (table 2).

Table 2

Characteristics of included wards

In participating wards, the average patient/physician ratio was 2.9 (4.0), whereas the patient/paramedics ratio was 0.8 (0.3). In the vast majority of participating wards, work was organised in 3 to 8-hour shifts, while 16% of wards worked in 2 to 12-hour shifts.

Mean-level change across 1 year of survey

The trajectories for each participant across the total sample from baseline to the last time point are presented in online supplemental table 1 for time-varying individual-level factors. Significant differences among the four times of visits were observed for schedule assignment frequency, number of nightshifts on duty over the last months, irregularity of mealtimes, number of breaks cancelled due to work overload, number of visits to the occupational safety and health department, sickness presenteeism and social support from supervisor.

Supplemental material

Outcome characteristics

The distribution of PSS-10 (stress) and Pichot (fatigue) scores in the whole sample are presented in figure 1, respectively, in (A) and (B). The overall mean score is equal to 16.5 (7.0) out of 40 points for stress, and 11.0 (7.9) out of 32 points for fatigue. For fatigue only, we observed an increasing trend (p=0.028) of the means across the time of visits (online supplemental table 2). For both stress and fatigue, significant differences were observed between the four hospitals (p<0.001) (online supplemental table 3). Figure 2A describes the trend of stress and fatigue level across time of visits and hospital. Additional figures on the distribution of stress and fatigue by hospital are available at online supplemental figures 1 and 2, respectively.

Figure 1

Distribution of PSS-10 and Pichot scores, respectively in (A) and (B) among the whole sample. The vertical dotted lines represent the mean of the PSS-10 and Pichot scores. PSS-10, Perceived Stress Scale 10-item scale.

Figure 2

PSS-10 and Pichot scores means, respectively in (A) and (B), by hospital and time of visits. PSS-10, Perceived Stress Scale 10-item scale.

According to unconditional models, AICs from three-level unconditional models are lower than those from two-level unconditional models for both stress and fatigue (online supplemental table 4). Hence, the results obtained validate the use of the three-level models to analyse both outcomes. Additional information on intraclass correlations is available under online supplemental table 4). A summary of missing values according to the four validated scales, by time of visits, is available in online supplemental table 5).

The best three-level multivariate models identified after model selection using AIC are described in table 3 along with model fit measures.

Table 3

Final three-level models for outcomes (perceived stress and fatigue) using a stepwise approach with AIC criterion

Individual and organisational-level predictors of stress

At individual level, negative life events (whether occurring in personal or in professional life), frequent cancellation of breaks, low social support from supervisor and colleagues, low perception of hospital management attitude towards patient safety culture, work overcommitment, as well as sickness presenteeism, were significantly associated to high perceived stress of HCWs. At ward-level, medical specialty (with less stress in ICU than in surgical and obstetrical units or wards with other medical specialties), and number of beds (with more stress in smaller wards) emerged as significant predictors of perceived stress.

Individual and organisational-level predictors of fatigue

At individual level, gender (with more fatigue in female participants), longer commuting time to work, low social support from supervisor, low perception of hospital management attitude towards patient safety culture, work overcommitment, as well as sickness presenteeism, emerged as significant predictors of high level of fatigue of HCWs. Significant predictors of fatigue at ward level were medical specialty (with more fatigue in wards with other medical specialties), higher rates of interim use and number of beds in the ward (with more fatigue in smaller wards). High fatigue level was also associated to time-level variable (time of visit, with an increasing trend over 12-month period).

Discussion

Main findings

In this longitudinal study, we identified various individual and organisational risk factors of stress and fatigue of HCWs. In particular, a supervisor providing low social support, low safety culture, work overcommitment, sickness presenteeism and working in smaller wards were identified as predictors of both high stress and fatigue.

In addition, breaks frequently cancelled due to work overload, negative life events (whether occurring in personal or in professional life), low social support from colleagues and working in non-surgical/obstetrical wards are specific predictors of high stress. However, longer commuting duration and frequent use of interim staff were associated with high level of fatigue.

Comparison with the literature

Many of our findings are consistent with those reported in previous studies investigating the determinants of stress or fatigue of HCWs.

In particular, the influence of the low social support from supervisor on both stress and fatigue was underlined in an earlier French study.31 Consistently with our results, this same study also showed higher fatigue in small to medium hospital wards, and in work environments where staff frequently had to go outside the ward as well as lower energy levels and more frequent sleep difficulties when use of interim staff was frequent.31 Safety climate perceptions were found to be significantly related to HCW stress and fatigue in several recent studies.36

Moreover, our finding that work overcommitment and sickness presenteeism, another indicator of overcommitment, were significant predictors of stress and fatigue, is supported by a recent French study in which overcommitment was found to favour emotional exhaustion and increase the risk of burnout in French HCWs.37

However, some other factors including younger age, being female, shift work as well as unacceptable work schedule were previously reported in the literature as associated with stress, but not in our results.5 7 38 Regarding fatigue, factors previously reported in the literature but not found in our results were support from colleagues, and work schedule characteristics such as total working hours working, overtime number of monthly night (if at least half of the working hours are between midnight and 8:00) and evening (if more than half of the shift hours are between the hours of 16:00 and midnight) shifts and shift length (12 hours vs 8 hours).31 This latter could be explained by a potential lack of power due to the moderate sample size, there were a few wards with 12-hour shifts compared with those with an 8-hour shifts.

Interestingly, our results showed some factors associated with stress and/or fatigue not previously reported in the literature. Regarding stress, associated factors were negative life events, breaks cancelled due to work overload, in line with role overload, which a well-known factor of stress. Regarding fatigue, associated factors were longer commuting duration and frequent use of interim staff in the ward. Indeed, temporary contracts require dedicated time for staff training for managers, leading to fatigue. Another factor related to stress and fatigue is the number of beds (higher stress and fatigue in smaller wards). This could be explained by the fact that strain of working on understaffed wards is harder to manage in smaller wards. In addition, smaller wards are associated to lower level of management and HCWs staff, who are required to perform a wide variety of tasks.

Strengths

First, one strength of this study is its longitudinal nature. A few studies explored stress and fatigue longitudinally; however, the majority of currently available studies are cross-sectional. Furthermore, the high response rate of wards and HCWs, as well as the large sample size and the inclusion of wards of different size and activity, represent strength of this study. In addition, the large panel of sociodemographics, health and occupational characteristics of HCWs collected over time allow performing robust and well-adjusted multivariate analysis.

Second, stress and fatigue were explored together. To our knowledge, no previous study had proposed a single model to identify factors associated with high levels of combined stress and fatigue while accounting for time in hospital setting. In fact, a previous study conducted in French ICU attempted to predict stress and fatigue using demographic and occupational was based on cross-sectional survey.31

Finally, the power of the model used, which takes into account the complexity of the data, namely, the longitudinal design and the multiwards collection of the data. In recent years, these types of models have been frequently used39 considering the idea that longitudinal data could be analysed at three levels of nesting (eg, repeated measures (level 1), collected across individuals (level 2) and within different wards (level 3)).

Limitations

However, our study has some limitations. First, only four hospitals https://www.sciencedirect.com/topics/nursing-and-health-professions/tertiary-care-center in Paris-area were included, which is not representative for other areas of France. In fact, only services in public Paris-area hospitals were included in the study, so results may not be generalisable to wards in private hospitals or outside the Paris region, Future studies including HCWs and more hospitals from other cities in France are needed to verify the results generated in this study.

Second, due to the in-person interview, a risk of bias could be present due to the discomfort from having to reply face-to-face to some sensitive questions; however, ensuring anonymity of the participants was used to minimise such bias. Another possible source of bias in data collection is the presence of two different interviewers assigned to two hospitals each. However, we were able to consider this bias as we included this interviewer-related variable into the multilevel model. In the stress model, there was a significant interviewer effect, with higher stress in hospitals B and D. However, interviewer effect was not significant in the fatigue model.

Third, we were not able to investigate stress and fatigue outcomes for the physicians given low response rates, as questions regarding work organisation were less adequate than for nurses. Therefore, physicians were excluded from our sample of HCWs.

Insights for designing potential interventions

From these models, we can identify some areas for improvement to better prevent stress and fatigue of HCWs: (1) perception of the hierarchy (low social support from supervisor, low perception of hospital management attitude towards patient safety culture), (2) work overcommitment and (3) sickness presenteeism. Breaks frequently cancelled due to work overload and low social support from colleagues were also found as significant as specific predictors for stress level. For fatigue specifically, long commuting duration and use of external staff are also identified as predictors. Mutual and specifics preventive programmes for reducing stress and fatigue of HCWs could be implemented in order to reduce this burden, targeting on the most at-risk groups.

Conclusion

This research question is important given the influence on quality of patient care of high stress work environments.18 Our results may (1) help identify at-risk HCWs and wards, where interventions to reduce stress and fatigue could be focused. (2) These interventions could include manager training to favour better staff support and overall safety culture among HCWs.

Data availability statement

Data are available upon reasonable request. The data that support the findings of this study are available from the authors, upon reasonable request. It can be addressed to Pr Pascal Astagneau (pascal.astagneau@aphp.fr), the project leader.

Ethics statements

Patient consent for publication

Ethics approval

The study protocol was elaborated in collaboration with the AP-HP Department of medical policy and the Department of care and of paramedical activities and was approved after presentation to the Directorate General and the Committee on hygiene, safety and working conditions. It obtained both an agreement from the French Committee for the Protection of Persons (CPP) on 11/14/2017 and clearance from the French Data Protection Authority (CNIL) on 12/14/2017 (IDRCB N° 2017-A02939-44). Potential participants were informed of the study through an information letter. Verbal consent was obtained by the interviewer at the beginning of each interview. Participants were guaranteed confidentiality and anonymity of responses. exempted this study. Participants gave informed consent to participate in the study before taking part.

Acknowledgments

The sponsor was Assistance Publique—Hôpitaux de Paris (Délégation à la Recherche Clinique et à l’Innovation). The authors would like to thank Zahoua Tina Habbal, the AP-HP and all participating wards and HCWs for their time and their contribution to this study.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • OSD and RSB contributed equally.

  • Contributors OSD is the study guarantor. OSD: methodology, software, formal analysis, writing—original draft, writing—review and editing, visualisation, validation; RSB: methodology, formal analysis, writing—original draft, writing—review and editing, visualisation, validation; KAB: methodology, formal analysis, writing—review and editing, visualisation, validation; KM: conceptualisation, writing—review and editing, visualisation, validation; AE-E: investigation. FE: conceptualisation, supervision, project administration; DS: conceptualisation, supervision, project administration; AC: conceptualisation, supervision, project administration; SN: conceptualisation, supervision, project administration; AN: conceptualisation, visualisation, funding acquisition, validation; PA: conceptualisation, methodology, formal analysis, writing—review and editing, visualisation, supervision, funding acquisition, validation; LT: conceptualisation, methodology, formal analysis, writing—review and editing, visualisation, supervision, funding acquisition, validation; MNH: conceptualisation, methodology, formal analysis, writing—review and editing, visualisation, supervision, funding acquisition, validation.

  • Funding The study was publicly funded through the Research on the Evaluation of the Healthcare System Performance Programme, Ministry of Health, France (grant number PREPS-16-261).

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.