Article Text
PDF
Abstract
Background The Fatality Assessment and Control Evaluation (FACE) programme was established by the National Institute for Occupational Safety and Health to help prevent occupational traumatic fatalities by funding states to conduct targeted fatality investigations within cause-specific focus areas and associated prevention efforts.
Purpose To investigate the impact of the state-based FACE programme on two previous focus areas.
Methods A longitudinal time-series analysis spanning 22 years compared state fatality rates for occupational falls and electrocutions before and after FACE programme funding with states not receiving FACE programme funding. Lag periods were utilised to allow time for the programme to have an effect, and rates were adjusted for a variety of covariates. Separate analyses were conducted for each injury outcome.
Results A reduction in fall fatality rates that was of borderline significance (1-year lag adjRR=0.92 (0.84 to 1.00)) and a non-significant reduction in electrocution fatality rates (3-year lag adjRR=0.92 (0.82 to 1.03)) were observed in states with FACE programme funding, Best-fit models presented two separate lag periods.
Conclusions While it is challenging to quantitatively evaluate effectiveness of programmes such as FACE, the data suggest the FACE programme may be effective in preventing occupational injury deaths within its outcome focus areas throughout the state. It is important to look for ways to measure intermediate effects more precisely, as well as ways to maintain effects over time.
Statistics from Altmetric.com
What this paper adds
-
There are few evaluations of occupational safety programmes focused on preventing deaths from falls and electrocutions.
-
Statewide preventive efforts can be effective in preventing occupational injury deaths.
Introduction
In 2009, 4541 occupational injury fatalities were reported in the USA.1 A variety of public and private organisations are actively engaged in preventing these deaths. These include federal and state governments, trade groups and unions, and academic institutions. Prevention efforts include promulgation and enforcement of regulations, surveillance and research to guide prevention efforts, changes in equipment and personal protective equipment, and education and training.
Occupational traumatic injury fatality investigations are one of the prevention measures carried out by Occupational Safety and Health Administration (OSHA), National Institute for Occupational Safety and Health (NIOSH) and NIOSH-funded organisations. The purpose of the fatality investigations conducted by OSHA is to determine the circumstances surrounding the fatality and what, if any, standards and rules were violated. Fines are levied in an effort to enforce and encourage safety compliance. Selected fatalities targeted by the NIOSH Fatality Assessment and Control Evaluation (FACE) programme are investigated by occupational safety and health professionals who formulate prevention strategies for dissemination by states and NIOSH.2 The FACE programme is able to identify and evaluate, through site visits and personal interviews, contributing factors not detected through review of population-based administrative data. Examples of possible contributing factors include the level of supervision, extent of safety training, equipment design and malfunctions, and presence of employer safety programmes at the time of the fatality. A FACE report is written summarising the event and providing safety recommendations for preventing future similar events. The FACE programme has no connection with penalties levied.
The FACE programme consists of the national component managed by NIOSH which began in 1983, and programmes funded at the state level beginning in 1990. NIOSH has funded 3–16 state programmes in any given year. The NIOSH and state programmes conduct their own fatality investigations emphasising several programme focus areas determined from fatality data and anticipated opportunities for prevention efforts. The findings from the NIOSH and state programmes are widely disseminated to be referenced and used for training programmes, policy development and standards setting. The state-based FACE programme designated falls as a programme focus area from 1990 through 1998, and electrocutions as a programme focus area from 1990 through 1994. During the time period when falls and electrocutions were focus areas, they were leading causes of death among workers who suffered a traumatic injury at work.3
The purpose of this study was to assess the impact of the FACE programme on reducing work-related fatalities. Specifically, there were two objectives: (1) evaluate fall fatality rates among states participating in the FACE programme compared with states not participating in the FACE programme and (2) evaluate electrocution fatality rates among states participating in the FACE programme compared with states not participating in the FACE programme.
Methods
FACE programme framework
The framework guiding the effect of the FACE programme relies upon the conduct of FACE investigations, development of products, and the related actions taken by stakeholders who can directly or indirectly influence safety in the workplace (figure 1).4
Framework for effect of Fatality Assessment and Control Evaluation programme on reducing incidence of traumatic injury fatalities.
Short-term actions affecting the impact of the FACE programme involves delivery of FACE investigation reports to safety professionals, industry leaders, union representatives and other target audiences aiming to enhance their safety knowledge and attitudes, as well as dissemination of FACE investigation findings to the public through electronic and printed media. Longer-term actions involve informing stakeholders regarding recommended occupational safety practices, policies and procedures, and interventions which may yield a sustained effect of the FACE programme. Such interventions to improve worker safety may include engineering changes, hazard analysis, supervision, safety training and establishing safer work procedures. Policy changes and research efforts may also be logical and necessary consequences of FACE investigations in reaching the ultimate goal of reducing the incidence of fatal occupational injuries.
States interested in receiving FACE programme funding must apply by submitting a grant proposal. States are funded through a competitive process based on the merits of their proposal. States are funded independent of size, fatality rates, geographical location or political/economic climates.
Study design
A retrospective longitudinal time series analysis was employed to evaluate the association of states participating in the FACE programme and fatality rates for two outcomes: falls and electrocutions. The two main outcome variables, fall fatality rates and electrocution fatality rates, were constructed using two data sources. Fall and electrocution fatality count data by year and state were provided by the National Traumatic Occupational Fatalities (NTOF) surveillance system, which was chosen for its longevity (1980–2001).5 ,6 The NTOF system is comprised of death certificates from all 50 states, New York City and the District of Columbia. The NTOF system encompasses all deaths of persons aged 16 years or older that included injuries (Ecodes 800-999) and a positive response to the ‘Injury at Work?’ box on the death certificate. NTOF is the only reliable source of occupational injury deaths prior to 1992, and when compared with a subsequently developed multi-source data system, the Bureau of Labor Statistics’ Census of Fatal Occupational Injuries, was found to identify, on average, 84% of occupational injury deaths.6 The Current Population Survey (CPS), conducted by the Bureau of Census for the Department of Labor, was used to determine the number of employed workforce by state and year.7 The CPS is a rotating monthly survey of approximately 50 000 households of the civilian non-institutionalised population aged 16 years and older. The yearly fatality rates were calculated as the number of fatalities (using NTOF) per 100 000 employed (from CPS).
The main-effect variable was state participation in the FACE programme, a dichotomous variable (1=participation for a state for a year, 0=non-participation for a state for a year). Funding status was extracted from a NIOSH FACE programme document summarising funding periods for participating states since funding began in 1990.
Data on potential covariates were identified through summary statistics using NTOF data, published literature and input from state FACE programmes. Based on trends in work-related fatality rates among high-risk groups identified by NTOF,3 data on the following covariates were obtained for each state and year: the number of workforce older than 65 years, number of men in the workforce, number of workers belonging to a minority group, and number of workers in the construction industry. The OSHA provided summary statistics of its fall and electrocution investigations by state and year. Additionally, the number of NIOSH investigations conducted independent of the state-based programmes were retrieved from the NIOSH FACE investigation team.
Finally, macroeconomic factors found to be associated with work-related fatality rates in the USA were included as potential covariates with values collected by state and year.8 Specifically, these factors were proxy measures for statewide unionisation, fiscal capacity of states, and statewide social welfare policies. Union density was measured as the proportion of the non-agricultural workforce belonging to labour unions.9 The labour grievance rate was measured by the number of labour grievances per 1000 union employees.10 States were dichotomised by the presence of a Right-to-Work law (time-varying for two states), which gives employees the right to choose to participate in a union.11 The unemployment rate was the proportion of the civilian labour force unemployed.12 Fiscal capacity was measured as state debt per capita.13 Social welfare was measured by public welfare expenditure per capita.13 State debt, union density, labour grievance rate, social wage and unemployment were dichotomised at the 20th percentile for each year to identify the 10 states with workers employed under the following statewide conditions—high state debt, low union density, low labour grievance rate, high unemployment rate and high social wage.
Statistical analysis
The fatality rates for falls and electrocutions were separate dependent variables in statistical modelling, and separate sets of statistical analyses were conducted for each outcome. The main-effect independent variable was state participation in the FACE programme, expressed as lag-year variables at 1, 2, 3 and 5 years after FACE programme funding started to allow for a delayed effect of the FACE programme as it was not practical to expect an immediate decrease in fatality rates when funding was received. Additionally, a variable for calendar year was included to control for background fatality rates as overall rates were decreasing over the time period. Statistical models were constructed for both unadjusted and adjusted rate ratios for comparison purposes. Poisson models employing Generalised Estimating Equations14 were used to account for the serial correlation of the time series, and the clustering of data within states. All analyses were conducted using SAS V.9.2.15 The natural logarithm of the number of employed workers by state each year was used as an offset variable so that rates were modelled. For each lag period, unadjusted models were constructed with the main effect, FACE programme funding status, linked to a specific lag time and the variable year. For each lag period, adjusted models were constructed using a three-stage model-building process. First, the selection of potential covariates for the adjusted models was started by testing each covariate in a model with the main effect and the variable year. Covariates associated with the outcome by a p value <0.25 were included in the next stage. Covariates meeting selection criteria were modelled altogether without the main effect (FACE program status) and entered into a forced manual stepwise backward process, and eliminated until all remaining covariates were significant at p<0.25 level. The final stage consisted of forced manual stepwise backward modelling with the remaining covariates, the lag-time-linked main effect, and the variable year. Covariates not significant at p≤0.05 level were eliminated starting with the least significant covariate until all covariates were statistically significant. The final model selected was the one with the best fit considering both the main effect and lag period, determined by a combination of model goodness-of-fit testing (quasi-likelihood information criteria, or QIC), and main-effect rate ratio and p value.16 Comparisons across lag years for each outcome are available from the corresponding author.
Results
Fall fatality rates
There were 12 781 fall-related deaths from 1980 through 2001 identified through the NTOF data. Twenty states participated at some point in time in the FACE programme during the falls focus area time period spanning 1990 through 1998. Years of participation by states ranged from 2 to 9 years, with 15 states participating five or more years.
Programme funding was associated with a reduction in fall fatality rates at the 1-, 2- and 3-year lag periods, with association diminished at the 5-year lag period (1-year lag: RR=0.80 (0.71 to 0.90); 2-years lag: RR=0.84 (0.75 to 0.95); 3-years lag: RR=0.87 (0.79 to 0.96); 5-years lag: RR=0.93 (0.84 to 1.03)). After adjusting for number of OSHA investigations, number of federal FACE programme fall reports, number of male employees, high state debt, high unemployment, right to work state laws and year, the association was attenuated for all the four lag periods (1-year lag: RR=0.92 (0.84 to 1.00); 2-year lag: RR=0.96 (0.89 to 1.05); 3-year lag: RR=0.98 (0.89 to 1.08); 5-year lag: RR=1.01 (0.94 to 1.09)). The final model describing the impact of the state-based FACE programme on falls was chosen for a 1-year lag and is presented in table 1.
Final model describing effect of FACE programme with a 1-year lag on fall fatality rates, adjusted for covariates
A graphical depiction of fall fatality rates comparing states participating in the FACE programme with states not participating in the FACE programme was constructed (figure 2). From 1980 through 1990, the fatality rates for FACE and non-FACE states decreased in a similar pattern. After 1990, the rate changes begin to differ, with FACE states continuing to decrease and plateau, while non-FACE states collectively experienced an increase and plateau. Finally, by 2000, the rate patterns are similar between the two groups.
Fall fatality rates by Fatality Assessment and Control Evaluation programme funding status from 1980 through 2001. This figure is only reproduced in colour in the online version.
Electrocution fatality rates
There were 7709 electrocution-related deaths from 1980 through 2001 identified through the NTOF data. Fourteen states participated in the FACE programme at some point in time during the electrocutions focus area time period spanning 1990 through 1994. Years of participation ranged from 2 to 5, with 12 states participating for three or more years.
FACE programme funding was associated with reduced rates of electrocution injuries at 2-, 3- and 5-year lag periods (1-year lag: RR=1.01 (0.92 to 1.11); 2-year lag: RR=0.84 (0.76 to 0.93); 3-year lag: RR=0.82 (0.73 to 0.92); 5-year lag: RR=0.75 (0.68 to 0.83)). After adjusting for the number of OSHA investigations, number of male employees, number of employees belonging to a minority group, year and high state debt, the effect was attenuated (1-year lag: RR=0.98 (0.90 to 1.08); 2-year lag: RR=0.94 (0.85 to 1.05); 3-year lag: RR=0.92 (0.82 to 1.03); 5-year lag: RR=0.89 (0.76 to 1.06)). The final model describing the impact of the state-based FACE programme on electrocutions was chosen for a 3-year lag and is presented in table 2.
Final model describing effect of FACE programme with a 3-year lag on electrocutions fatality rates, adjusted for covariates
A graphical depiction of electrocution fatality rates comparing states participating in the FACE programme with states not participating in the FACE programme was constructed (figure 3). From 1980 through 1990, the fatality rates for FACE and non-FACE states decreased in a similar pattern. After 1990, the rates continued to decrease in a similar pattern at a slower rate. There is no discernible difference between FACE states and non-FACE states during the years of the FACE programme focus regarding electrocution fatality rates.
Electrocution fatality rates by Fatality Assessment and Control Evaluation programme funding status from 1980 through 2001. This figure is only reproduced in colour in the online version.
Discussion
Because it was considered implausible that the FACE programme would have an immediate effect on fall and electrocution fatality rates, lag times were modelled for 1, 2, 3 and 5 years post-FACE funding. For falls, the 1-year lag period (RR=0.92 (0.84 to 1.00)) revealed the best overall model fit and lowest rate ratio. Over the next two lag periods, the rate ratios for falls grew closer to 1, while their statistical significance weakened further. For electrocutions, the effect was different: adjusted rate ratios decreased over the 5 years of FACE funding. The 3-year lag period (RR=0.92 (0.82 to 1.03)) was characterised by the best overall model fit in addition to one of the lowest rate ratios and p value for the main-effect variable.
Recently, Loomis et al8 examined the association between political economic indicators of US states and occupational injury fatality rates. They reported that states least favourable to labour were associated with higher rates of occupational injury. We found states with high unemployment and the presence of a Right to Work law to be associated with increased fall fatality rates. This finding is consistent with previous analyses examining state fatality rates overall8 for two time points within the current study. Additionally, high state debt was associated with a decrease in both fall and electrocution fatality rates, also consistent with previous analyses.8 It is possible that high state debt represents a negative economic situation that accompanies decreased business activity that may result in decreased injury rates. There was no association observed between low union density, low grievance rate and low social wage with either fall or electrocution fatality rates. Our study is the first to examine these socio-political economic indicators at multiple time points within the context of a time series analyses, and to include them in the evaluation of a national programme designed to improve safety. Including these important measures reinforces that the FACE programme may have had a potential impact by measuring the effect of the FACE programme even in the presence of important socio-political economic predictors.
There are very few peer-reviewed publications available that evaluate the effect of fall safety standards or programmes on fall injury rates. One well-structured review17 revealed two international studies and one evaluation of the Washington State fall protection regulations.18–,21 While the two international studies did not have data on injuries or any denominator data available, the evaluation of the Washington State fall protection regulations demonstrated a reduction in workers’ compensation injury claims for falls in multivariate analyses.18–,20 A subsequent, more comprehensive study evaluating the Washington State fall arrest standard found the regulations to be effective in reducing fall rates between 3 and 3.5 years after the standard went into effect, even after adjusting for already declining rates.21 In the past decade, at least three national safety standards related to falls were promulgated.22 Although an evaluation of the accuracy of OSHA's estimates of the injury benefits found them to be overestimated, it is not clear to what extent the fall standards played a role, if any, in reducing fall fatality rates.22 The present study was the first to use a national database and construct a time-series analysis with comparison group to evaluate the effect of a state-funded programme focused on falls prevention.
There are no peer-reviewed publications available that evaluate the effect of electrocution safety standards or prevention programmes on electrocution injury rates. A recent analysis of the projected benefits of safety standards promulgated since 1990 (two were focused on electrocutions) found that training was a large component in the standards, and the projected deaths prevented by compliance with the standards were overestimated.22 The lack of an effect of the FACE programme on electrocution fatalities compared with fall fatalities could be due to a greater need for effective training that allows for better compliance with the OSHA standards relevant to industries in which electrocutions occur.
Quantitative evaluations of the effect of a programme are distinct from process evaluations. A qualitative process evaluation of the FACE programme was previously conducted and found select outputs of the FACE programme to be technically accurate and further disseminated among safety professionals.23 The current analysis was a logical continuation in the evaluation process of the FACE programme for the purpose of bridging the implementation of the project in states (the beginning of the process) to the final outcome, state-wide injury rates (end of the process). Between implementation of the programme and injury rates are several significant steps. Implementation of the project involves fatality investigations conducted by the funded state FACE programme. Due to finite resources (limited number of safety specialists hired to conduct investigations coupled with travel costs), only a small proportion of fall and electrocution fatalities were investigated. Following the process from figure 1, if every fall and electrocution fatality was investigated, and effective educational materials and training were produced and FACE reports were read, a significant burden would still rest on the short-term and long-term actions of knowledge changes, policy changes and other action items being carried out. In addition, findings from NIOSH and state investigations were also disseminated in non-FACE states, and may have lowered death rates in the non-FACE states. Accordingly, it would be very difficult to see a statistically significant effect when evaluating the beginning of the programme, with endpoints that are results of multiple factors, and where the reference states are not isolated from the FACE states. The current analysis is not meant to be the definitive answer on the effectiveness of the programme, but, rather, an evaluation of the FACE programme intended to complement the earlier process evaluation.
There are a number of limitations to these analyses. Medical examiners in FACE states may have been aware of the funding and been more likely to correctly identify falls and electrocutions as being work-related. This would have biased the effect of the FACE programme funding towards the null making it more difficult to detect an effect. Another limitation is states that applied for and received FACE funding are likely to have more of an infrastructure and orientation for preventing occupational fatalities, making it difficult to isolate an effect in these states as being solely a consequence of the FACE programme. Finally, selecting an appropriate lag period was a difficult task. There was scant guidance for deciding a priori what an appropriate time period was for a detectable effect for each outcome, so the decision had to be guided by statistical analyses rather than previous research conducted in this field. A summary of the process is available as an appendix.
The objective of the current study was to present a quantitative evaluation of a programme designed to reduce occupational fatalities. From 1990 through 1998, the FACE programme included fall fatalities as a focus area; from 1990 through 1994, the FACE programme also included electrocution fatalities as a focus area. The quantitative analysis included 22 years of time-points, pre- and post-funding data, comparison groups and a rigorous statistical analysis that accounted for the nested data structure (repeated measurements). This quantitative evaluation was meant to provide one aspect of the effectiveness of FACE in reducing fatality rates of focus areas.
References
Footnotes
-
Competing interests There are no conflicts of interest to report.
-
Provenance and peer review Not commissioned; externally peer reviewed.