Article Text

PDF

Assessing and intervening on OSH programmes: effectiveness evaluation of the Wellworks-2 intervention in 15 manufacturing worksites
  1. A D LaMontagne1,
  2. E Barbeau2,
  3. R A Youngstrom2,
  4. M Lewiton2,
  5. A M Stoddard3,
  6. D McLellan2,
  7. L M Wallace2,
  8. G Sorensen2
  1. 1Centre for the Study of Health & Society, School of Population Health, University of Melbourne, 207 Bouverie St, Level 4, Melbourne, Victoria 3010, Australia
  2. 2Center for Community-based Research, Dana-Farber Cancer Institute, 44 Binney St, Boston, MA 02115-6084, USA
  3. 3Department of Biostatistics & Epidemiology, School of Public Health & Health Sciences, University of Massachusetts, Amherst, MA 01003 USA
  1. Correspondence to:
 Associate Professor A D LaMontagne
 Centre for the Study of Health & Society, School of Population Health, University of Melbourne, 207 Bouverie St, Level 4, Melbourne, Victoria 3010, Australia; alamontaunimelb.edu.au

Abstract

Aims: (1) To develop a transparent and broadly applicable method for assessing occupational safety and health (OSH) programmes or management systems; (2) to assess OSH programmes in a sample of manufacturing worksites; and (3) to determine whether a management focused occupational health intervention results in greater improvement in OSH programmes compared to minimal intervention controls.

Methods: OSH programmes were assessed using an adaptation of the US Occupational Safety & Health Administration’s 1995 Program Evaluation Profile. Scores were generated from 91 binary indicator variables grouped under four “Essential Elements”. Essential Element scores were weighted to contribute to an overall programme score on a 100 point scale. Seventeen large manufacturing worksites were assessed at baseline; 15 sites completed the 16 month intervention and follow up assessments.

Results: There was considerable variation in Essential Element scores across sites at baseline as judged by our instrument, particularly in “management commitment and employee participation” and “workplace analysis”. Most sites scored highly on “hazard prevention and control” and “training and education”. For overall OSH programme scores, most sites scored in the 60–80% range at baseline, with four sites scoring below 60%, suggesting weak programmes. Intervention sites showed greater improvements than controls in the four programme elements and in overall programme scores, with significantly greater improvements in “management commitment and employee participation”.

Conclusions: The OSH programme assessment method used is broadly applicable to manufacturing work settings, and baseline profiles suggest needs for improvement in OSH programmes in most such worksites. Despite a small sample size, results showed that sustained management focused intervention can result in improvement in these OSH programme measures.

  • OSH programs
  • OSH management systems
  • intervention research
  • intervention effectiveness research
  • randomised controlled trial

Statistics from Altmetric.com

Programmatic or systematic approaches to occupational safety and health (OSH) have long been acknowledged as essential to the prevention and control of occupational injury and illness. In the past two decades, OSH programmes or management systems have emerged internationally as a major strategy for addressing workplace safety and health.1 OSH programme regulations and voluntary guidelines have been developed or are under development in numerous countries.2 In the USA, the Occupational Safety & Health Administration (OSHA) first developed voluntary guidelines on OSH programmes in 1989.3 Despite widespread praise for these guidelines, OSHA determined more recently that voluntary implementation has not been adequate to realise the full potential of OSH programmes in stemming the continuing high levels of job related deaths, injuries, and illnesses.4,5 Accordingly, an OSHA rule on OSH programmes is in development, with an initial draft published in 1998.6,7

Given the prominence of OSH programmes as an international phenomenon, there is surprisingly little published empirical research in this area. In addition to regulatory bodies, various models of what constitutes an OSH programme or management system have been put forth by industry, professional groups, and voluntary standards bodies.3,6,8–14 However, there is a dearth of published reports in which these models have been translated into measurement instruments and applied in the field. Thus there is little information available on the current state of OSH programmes or systematic OSH management efforts in industry. Much of the available literature is not peer reviewed, and even peer reviewed reports sometimes do not fully report on assessment methods, ostensibly due to proprietary interests.15,16

Empirical research on OSH programmes is urgently needed in several areas to better inform practice and policy development in this area.1,17 These include how to assess or evaluate existing OSH programmes, the current state of OSH programme implementation in various work contexts, how to intervene to improve OSH programmes, and the effectiveness of OSH programmes in reducing occupational injury and illness. The present report explores the first three of these questions in the context of a recent workplace intervention trial.

Wellworks-2 was a randomised, controlled workplace intervention trial targeting the reduction of lifestyle and occupational health risks in blue collar workers in the manufacturing sector. The primary hypothesis was that workers would be more likely to make lifestyle behaviour changes if occupational health risks were addressed simultaneously.18,19 Thus, Wellworks-2 as a whole addressed both health promotion and occupational health: intervention sites received an integrated intervention containing both, versus a “standard care” health promotion only intervention for control sites. A secondary hypothesis was that integrated intervention sites would show greater improvements in OSH programmes at the organisational level in comparison to non-OSH intervention controls. Because at the time of the study there were no standard measures for OSH programmes or management systems, we undertook this as a Phase II Methods Development study,20,21 aiming to develop and apply a new OSH programme measurement method.

The Wellworks-2 OSH intervention was designed around a three level social ecological framework,22 with specific intervention activities at the level of the worker (for example, OSH training and education), the organisation (for example, management consultation on OSH), and the physical work environment (for example, tailored efforts to improve the prevention and control of hazardous substance exposures).18,23,24 This report presents the organisational level occupational health intervention and evaluation, including the Wellworks-2 OSH programme assessment methodology, pre-intervention OSH programme characteristics, and an evaluation of intervention effectiveness in changing OSH programme scores.

Main messages

  • OSH programme/management system assessment methods should gauge the degree to which an organisation systematically manages OSH, rather than the presence or absence of a programme/management system.

  • An assessment instrument based on US OSHA’s conception of OSH programmes/management systems is broadly applicable, transparent, and simple to administer.

  • Results from field application of this assessment instrument indicate that it has reasonable discriminatory power and is well matched to the range of prevalent OSH programmes in the US manufacturing sector.

  • Most sites in a small positively skewed sample scored in the 60–80% range on a 100 point OSH programme scale, with roughly one fourth of sites scoring below 60%.

  • Management focused intervention led to significant improvements in a measure of “management commitment and employee participation”.

METHODS

Study design and population

The Wellworks-2 study used a randomised, controlled design with the worksite as the unit of assignment and intervention. After baseline assessments, worksites were randomly assigned to one of two conditions: worksite health promotion integrated with occupational health protection (HP+), and worksite health promotion only (HP). The HP-only condition is described as a non-intervention control in this report because OSH was not addressed at all by the Wellworks-2 intervention in control worksites. To mitigate baseline imbalances between intervention and control groups, worksites were randomised within blocks: unionised versus non-unionised; single versus multiple buildings; and three worksites that were part of a single large company. Sites were randomly assigned by the study biostatistician using a process conducted independently from study staff involved in baseline field assessments and intervention development.

With respect to OSH outcomes, Wellworks-2 is a Phase II Methods Development study.20,21 We are presenting new measures of OSH programmes as well as the first intervention effect size estimates in this area (explained further in “Measures” section below). Health behaviour changes (smoking and dietary habits) were the primary outcomes of this study, and accordingly, Wellworks-2 was powered to detect statistically significant and meaningful changes in smoking and eating behaviours with the worksite as the level of randomisation and analysis.18 Power calculations for OSH programme score changes are presented in the Analysis section below, using the fixed sample size determined on the basis of health behaviour changes.

Policy implications

  • The observation of substantial room for improvement at most sites supports the need for policy and other intervention in the OSH programme area.

  • The observed intervention related improvement in “management commitment and employee participation” suggests likely benefits from policy and other intervention in this area.

  • Further empirical research is needed to improve and validate OSH programme assessment methods and to evaluate the effectiveness of various intervention approaches.

The principal selection criteria for Wellworks-2 study sites were: (1) manufacturing industry worksites employing between 400 and 2000 workers; (2) probable use of hazardous substances; and (3) turnover rate <20% to avoid excessive loss to follow up. We used Dunn’s Direct Access (DDA) to identify 89 manufacturing companies (Standard Industrial Codes 20–39) of appropriate size that were located in eastern Massachusetts. These 89 companies were contacted with recruitment letters and follow up telephone calls; 41 were determined to be eligible for the study. Seventeen of these 41 sites were recruited to the study, including three sites from a single, large company. Median establishment size was 596 (mean = 721) employees. Employee demographics are presented elsewhere.18 The types of manufacturing conducted at the participating worksites included adhesives, food products, high technology, jewellery, motor controls, paper products, newspapers, abrasive products, automobile parts, and metal fabrication.23

Fifteen sites completed the intervention and final assessments (seven HP+ integrated intervention sites, and eight HP control sites). Two sites dropped out after baseline OSH programme assessments, one due to a plant closure and the other due to inadequate participation in baseline employee surveys and intervention planning.

Intervention methods

The 16 month intervention began following completion of baseline assessments. Wellworks-2 encouraged companies to adopt a proactive, upstream preventive approach, going beyond compliance with legal standards set by OSHA.18,23 Management was targeted as representative of the organisation because management has both primary control over—and primary responsibility for—providing a safe and healthy work environment. Though the management of hazardous substance issues was targeted in particular, the upstream approach was emphasised as applicable to all aspects of OSH.

Operationally, the Wellworks-2 management intervention was conducted primarily through contacts with middle and upper management, led by a Wellworks-2 staff industrial hygienist. In non-OSH intervention control worksites, Wellworks-2 health promotion staff provided consultation to management on tobacco control policies, food service, and catering policies over the 16 month intervention period.

Baseline OSH programme assessments, in combination with walk-through assessments of the physical work environment,23 identified site specific OSH needs and were used to tailor the intervention at integrated intervention sites, particularly to guide ongoing management or organisational level intervention. Baseline OSH programme assessment data were reviewed in detail by Wellworks-2 staff and qualitatively summarised in reports presented at the beginning of the intervention period both orally and in written form to OSH staff, management, and unions (if present) at each of the intervention worksites. These baseline reports were attended typically by 5–10 middle and upper managers, usually including OSH mangers, general managers, and department heads. Multiple copies (depending on size and number of departments) of written reports were distributed at the oral presentation. Wellworks staff presenters encouraged the circulation of the reports to middle and upper management.

During the intervention period, management intervention methods included one-on-one consultation and technical assistance with managers, group educational sessions to management, group educational sessions and consultation with OSH committees, and communication through written materials. Quantitative intervention process tracking documented a mean of 18 OSH specific management contacts in the intervention sites (with a mean total of 25 management contacts including health promotion), versus no OSH specific management contacts in control sites (mean total of nine management contacts on health promotion).24 Contacts during the intervention period were most commonly with OSH managers and other middle management (for example, OH nurse, human resources personnel), sometimes upper management (for example, production manager, manufacturing manager, operations manager), but rarely with top management (for example, vice president of manufacturing operations, controller).

Site specific management consultations focused on achieving improvements in OSH programme elements by advocating for: (1) upper management commitment to OSH, integration of OSH as a core business function, and the involvement of upper managers in OSH activities and decision making; (2) employee participation in OSH management structures (such as OSH committees, hazard analysis procedures, and near-miss or accident investigation strategies); (3) improvement in human and material resources for hazard analysis, control, and prevention—based on industrial hygiene (hierarchy of control) principles; and (4) ongoing OSH education and training for salaried as well as hourly employees, and revision and improvement of OSH training and education activities.

Finally, we developed tailored (site specific) Wellworks-2 newsletters towards the end of the intervention period, including one or more OSH improvement success stories for each site. These were distributed to all middle and upper management at each intervention site.

Data collection

All baseline OSH programme assessments were conducted by one Wellworks-2 staff industrial hygienist (ML). Because this industrial hygienist was subsequently involved in intervention delivery, all post-intervention assessments were administered by a second staff industrial hygienist (RY) to avoid bias. In order to make baseline and final assessment methods comparable, both hygienists were involved in the development of the OSH programme assessment instrument and a detailed protocol for its administration. Data collection methods were thus standardised to the extent feasible given the need for different assessors at baseline and final. One assessment was done per site with one informant—the person most responsible for OSH on site. These face-to-face interviews were conducted on-site with OSH managers or other specifically OSH responsible persons (including occupational health nurses at two sites) where available (14 sites at baseline; 13 at final), and with human resource or personnel directors at other sites (one site at baseline; two sites at final). The same site representative was interviewed at baseline and final at nine of the sites.

OSH programme assessment interviews were conducted directly following detailed walk-through assessments of the physical work environment.23 In most cases, the site representative who hosted the walk-through assessments was also the interviewee for the OSH programme assessment. While OSH programme interviews took 1.0–1.5 hours to administer, the total time for the work environment walk-through and OSH programme assessments was 1–2 days per site. The time between (pre-randomisation) baseline assessments and (post-intervention) final assessments averaged 24 months (range 22–26 months).

Measures

OSH programmes were assessed using an instrument adapted from the OSHA’s 1995 Program Evaluation Profile.25 Our adaptation included 91 binary indicator variables (detailed in table 1) grouped under OSHA’s four programme “Essential Elements” of: management commitment and employee participation (43 indicators), workplace analysis (11 indicators), hazard prevention and control (8 indicators), and education and training (14–29 indicators, depending on variable number of potential exposures for which training might be required). Percentages of OSH favourable responses were then multiplied by OSHA weights for each Essential Element,25 and totalled as an overall programme score on a 100 point scale. OSHA Essential Element weights were as follows: management commitment and employee participation (highest possible contribution of 36/100 points), workplace analysis (28/100 points), hazard prevention and control (24/100 points), and education and training (12/100 points).25 These same Essential Elements are included in the 1998 draft OSH programme rule, complemented by a new Element on periodic evaluation.6

Table 1

 Wellworks binary OSH programme evaluation items at baseline, organised by 1995 OSHA programme evaluation plan Essential Elements

Data were also collected on several indicators that were not used for quantitative scoring. These include the numbers of full time employees with OSH as their primary responsibility, the qualifications of such staff (that is, safety professional, industrial hygienist, occupational health nurse, and other), and the percentage of OSH staff work time devoted to handling Workers’ Compensation claims. Other questions included whether the organisation is self insured for Workers’ Compensation, whether any employees are potentially exposed to a range of hazards with detailed OSHA standards (that is, lead, cadmium, asbestos, noise, and other), and whether any employees are subject to medical surveillance for such specific hazards.

Data analysis

Frequency distributions of baseline OSH programme scores are presented as histograms for each Essential Element and for the total score. Because the same baseline scores were assigned to each of the three sites for the single, large multi-site company, baseline assessments are presented for 15 sites.

For each Essential Element and the total, change scores were computed by subtracting baseline from final scores. The difference in mean change scores between intervention and control sites was tested using mixed model analysis of variance with randomisation block included as a random effect and intervention group as the fixed effect. Including randomisation block controls for the within block correlation in the outcome; for the three worksites in the single large company, intra-block correlation was introduced by the use of identical baseline measures. All p values are reported as two sided. Using baseline score distributions, we estimated 80% power to detect a mean improvement of 19 points for the total programme score at a 5% significance level in the intervention group (assuming within worksite correlation of baseline and final scores of 0.5 and no change in the control group).

A second mixed model experimental analysis was conducted using the difference in mean change score between intervention and control sites as the dependent variable, but also controlling for the baseline level of each score.

RESULTS

OSH programme scores at baseline

At baseline, there was considerable variation in Essential Element scores across sites, particularly in “management commitment and employee participation” and “workplace analysis” (fig 1, panels A and B; table 1). Most sites scored fairly well (high percentage of indicators affirmative) on “hazard prevention and control” and “training and education”, leaving relatively little room for improvement as judged by our instrument (fig 1, panels C and D; table 1).

Figure 1

 Baseline frequency distributions of OSH programme scores, by Essential Element and total. Panel A: management commitment and employee participation (OSHA Essential Element weight = maximum 36/100 points). Panel B: workplace analysis (weight = maximum of 28/100 points). Panel C: hazard prevention and control (weight = maximum of 24/100 points). Panel D: health & safety training & education (weight = maximum of 12/100 points). Panel E: total weighted score on 100 point scale (sum of four Essential Element scores).

Review of specific indicators shows numerous opportunities for programme improvement (table 1). For example: review of OSH performance was a formal part of manager and supervisor overall job performance at less than half of the sites; designating an annual OSH budget was rare (2/15 sites); one third of sites had formal procedures for employees to report OSH hazards and receive feedback; only six of 15 sites had line employees participating in incident investigations; and only two of nine sites that employed workers who were not fully fluent in English provided OSH information materials in appropriate languages (table 1). For overall programmes, most sites scored in the 60–80% range on the 100 point scale (n = 10), with one site scoring above at 82% (fig 1, panel E). Four sites fell below 60% (34%, 39%, 49%, and 51%), suggesting weak programmes.

Descriptive data not used for scoring

At baseline, 12 of 15 sites indicated that they had one or more full time employees who have health and safety as their primary responsibility (five sites reported one such employee, two sites with two, three sites with three, and two sites with four). With respect to the qualifications of such employees, eight sites reported the presence of one or more OH nurses, four sites reported the presence of industrial hygienists, four sites reported the presence of safety professionals, and four sites reported the presence of employees with “other” qualifications. Five sites reported the presence of a part time occupational physician on-site.

Nine of 15 sites reported that they were self insured for Workers’ Compensation (WC). Seven sites reported that less than 10% of OSH staff time was spent handling WC claims, five sites reported 10–25%, and one site reported that more than 50% of OSH staff time was spent handling WC claims.

Information on the types of OSH hazards present at each site can be gleaned from the conditional questions on OSH training. For example, 15 of 15 sites acknowledge the presence of hazardous substances of some sort (as implied by all 15 providing hazard communication training) (table 1). The presence of specific hazards with full OSHA standards could trigger specific programmatic requirements in addition to training. In response to queries on whether any employees were potentially exposed to the following hazards and whether any such employees were subject to medical surveillance or monitoring for such hazards, there were: eight sites with employees potentially exposed to lead, with three such sites having employees subject to lead surveillance; one site with employees potentially exposed to cadmium, with that same site having employees subject to cadmium surveillance; six sites with employees with potential asbestos exposure, with two of these sites having employees subject to asbestos surveillance; and 11 sites having employees with potential exposures to excessive noise, with all 11 of these sites having employees subject to noise surveillance. Under “other” particular exposures of concern, sites reported lasers (one site), respirator use (two sites), arsenic (one site), silica (one site), metalworking fluids (one site), blood products (one site), beryllium (one site), furfural (one site), and formaldehyde (one site, with same site indicating that exposures were below the OSHA action level). These sites reported having employees subject to medical surveillance corresponding to these hazards for lasers, respirator use, arsenic, and silica.

Intervention effectiveness evaluation

Table 2 presents crude/unadjusted results by intervention group at baseline and final. Both integrated intervention and control sites showed improvements in most Essential Element scores as well as total OSH programme scores between baseline and final assessments. However, there was a consistent pattern of greater improvements in intervention sites for both crude and adjusted results. For baseline to final changes within each group, the mean change score for the integrated intervention group differs significantly from zero for “workplace analysis” and “total programme” scores. The “total programme” scores increased by 11 points on a 100 point scale for integrated intervention sites. By contrast, though there were slight improvements in four of the five scores for control sites, none of those change scores differed significantly from zero.

Table 2

 Wellworks-2 OSH programme Essential Elements; unadjusted mean scores and standard deviations (SD) at baseline and final, by intervention group

Next, changes from baseline to final were compared between groups using a mixed model experimental analyses after adjustment for intra-block correlation (table 3). Because the control group did increase slightly, though not significantly, the intervention and control groups are significantly different from each other only on the Essential Element of “management commitment and employee participation”. The mixed model experimental analysis above was also repeated controlling for the baseline level of each score. This moderated the intervention related changes slightly, but change scores were not associated with baseline levels for “management commitment and employee participation”, “workplace analysis”, and the “total programme score”. Hence controlling for baseline levels is not called for, and only the first analysis is presented in table 3.

Table 3

 Wellworks-2 OSH programme Essential Elements; mixed model experimental analysis of change scores

To identify those indicator variables that contributed to the greater mean changes in the intervention group, intervention and control groups were compared on an item-by-item basis from baseline to final. For “management commitment and employee participation”, items showing greater mean change in intervention versus controls groups included: “programme updated on a regular basis”, “joint labour-management OSH committees”, “written documentation on OSH committee functions, duties, and authority”, “minutes taken at OSH committee meetings”, “two or more channels used to communicate OSH committee proceedings to employees”, “procedures for employees to report OSH hazards, problems, etc”, “OSH incentive programmes that award participation in OSH activities (and not outcome measures that would discourage reporting)”, and “employee participation in incident/accident investigations”. Most of these items concern “employee participation” and are of a documentary nature (OSH committee and other documentation and communication improvements). Some, however, reflect actions with greater potential to directly affect OSH (for example, joint labour-management OSH committees, movement of incentive programmes towards preventive activity rather than the discouragement of injury/illness reporting, and employee participation in investigations). Most items contributing to greater mean change showed small differences between intervention and control groups. The greatest differential change was for the “OSH incentive programme” item: among controls, there was no change in seven sites and improvement in one; in intervention sites, there were three sites with no changes, and four that improved (by either creating new programmes that award preventive activities, or modifying existing programmes to reward preventive activity rather than decreased incident reporting). For “workplace analysis”, two items showing the greatest mean change in intervention versus controls groups were: “OSH reps have the authority to keep specific processes, machinery, etc out of the plant”, and “conduct of audits, inspections, etc on a monthly (versus less often) basis”.

DISCUSSION

These findings have implications for OSH programmes and for intervention effectiveness research in OSH in general. In addition, the significance of these OSH programme findings, when considered in the broader context of the Wellworks-2 trial, extends across disciplines to include health promotion. These are considered in turn below.

OSH programmes and intervention effectiveness research

Both the cross sectional and experimental analyses presented in this paper provide new insights on OSH programmes. These are presented in turn below, followed by discussion of the strengths and limitations of the methods used.

Baseline cross sectional findings

The Wellworks-2 baseline descriptive data—despite the small sample size—provide some of the first published normative data on OSH programme characteristics. Such information contributes to the evidence base on the need for and feasibility of regulatory or other intervention in this area. Based on our adaptation of OSHA’s 1995 Program Evaluation Plan, most worksites in the Wellworks-2 study had at least basic programmes in place: two thirds of sites were in the 60–80% range (with only one site above that at 82%). Notably, none scored very highly despite the likely positive selection bias. This suggests that there is considerable room for improvement in OSH programmes at most sites, and furthermore—coupled with effectiveness evaluation findings—that improvements are achievable through management focused intervention as conducted in Wellworks-2. The finding of roughly one third of Wellworks-2 sites with programme scores below 60% suggests that programmes are weak in a substantial fraction of large manufacturing worksites in the USA. The distributions of scores by Essential Element and for OSH programmes overall show both good discriminatory power and a reasonable match with the range of prevalent OSH programmes in large US manufacturing worksites.

Effectiveness evaluation findings

The Wellworks-2 intervention resulted in a consistent pattern of greater positive change in OSH programmes in intervention versus control sites. These findings are strengthened by a rigorous experimental design as well as a conservative analysis approach. The significant improvement in “management commitment and employee participation” is particularly noteworthy because this Essential Element is widely acknowledged as the most important of the four, as reflected in its having the highest weighting (36/100 points).3,25 This strengthens the significance of our intervention effectiveness evaluation findings: showing that the most important programme element can be measurably improved through targeted management intervention, and highlighting the need for—as well as the feasibility of—management intervention to improve OSH programmes. By comparison, reviews of international trends in OSH programme initiatives have identified an over-emphasis on changing workers, neglecting the parallel needs to change managers and organisations.2,26,27 The Wellworks-2 results provide further support for refocusing intervention efforts on the management or organisational level, as well as insights on intervention strategies that can be used to achieve this goal.

To complement the quantitative OSH programme scoring findings, further insight on OSH changes over the intervention period can be gleaned from informal interview notes written by the Wellworks Industrial Hygienist after conducting final assessments. These notes are based on examples and anecdotes of OSH changes provided by interviewees at both intervention and control sites. In intervention sites, the following examples were reported. One site expanded its OSH committees and obtained a small grant from the state Workers’ Compensation authority to provide OSH training for committee members; that same site also created a new “near miss” programme which got 73 “hazard identification submittals” in less than two months—with 90% of these reportedly fixed in a timely manner. Another intervention site abolished its safety bingo incentive programme because it was seen as getting in the way of efforts to encourage people to bring issues forward. A third intervention site developed and implemented a comprehensive laboratory chemical hygiene plan and reformulated its OSH committee with “more focus on identifying hazards”. A fourth site reported improvements in hazard prevention and control, particularly in the areas of machine operation, housekeeping, and ergonomic analysis, with “safety presented as cost avoidance”; the same site reported greater involvement of line employees in health and safety decision making, and expanded OSH training efforts.

Anecdotes from control sites included one site where a new health and safety manager was hired, Material Safety Data Sheets became computerised, assembly operations became more automated resulting in less hand soldering, and epoxy use and beryllium compounds were being eliminated. A second control site reported expanded efforts in workplace analysis (including a noise exposure survey), and more and better OSH training, particularly for forklift operators. A third control site was already in OSHA’s Voluntary Protection Program (VPP) at baseline and had progressed from “Merit” to “Star” status (that is, OSHA recognition for excellence at the highest level); this site also reported more effort in hazard prevention and control, a new emphasis on OSH training at off-site locations (for example, warehouses), and a new employee OSH orientation training in development. A fourth control site experienced “significant reductions in employment” due to work being outsourced to China (a whole floor of assembly gone); hence there had been little change in attention to health and safety. A fifth control site was expecting a round of layoffs at baseline that did not occur; this site reported more frequent and regular OSH audits (workplace analysis) and new OSH training and education efforts for engineers (developing on-line materials at appropriate educational level). These anecdotes are consistent with and illustrative of the quantitative scoring patterns observed—with generally positive trends occurring over the intervention period in both intervention and control sites.

The OSH management/organisation intervention was complemented with OSH intervention activities at the level of the worker and the physical work environment, as described in detail elsewhere.18,23 These worker and environmental level interventions overlap conceptually with the organisational. In one such scenario, intervention related worker OSH training could stimulate individual workers to pursue and achieve increased participation in OSH activities, thus in theory increasing organisational level “employee participation”. While it seems most likely that the positive results observed are attributable to the Wellworks-2 management focus, the mutually reinforcing aspects of the three level ecological model and the integration of OSH with health promotion may have also played important roles. Integration in Wellworks-2 has been shown to improve health behaviour outcomes in blue collar workers (see below) and may be an inducement to management to improve organisational OSH efforts (which can benefit organisations with respect to sickness absence, health insurance costs, etc).

Strengths of methods used

There were several strengths of methods and interview instrument used. First, the instrument used attempts to gauge the extent or degree to which a given organisation systematically manages OSH, rather the presence or absence of a management system.1 Further, the presence or absence of specific management systems has been appropriately criticised as being susceptible to representing “paper tigers” with limited implementation in practice.1 The use of OSHA’s 1995 OSH Program Evaluation Profile as the basis for our instrument had the advantage of being a state of the art broad consensus on the key indicators of good OSH programmes and their relative importance. Further, because the assessment approach follows OSHA’s conception of OSH programmes, our findings contribute needed empirical data to inform the development of OSHA’s proposed OSH programme rule. The assessment method is also appropriate for both needs assessment and intervention effectiveness evaluation, and usable by OSH practitioners and others in addition to researchers.

Limitations of methods used

Taken together, the cross sectional and effectiveness evaluation results suggest the need for and likely benefits of intervention in this area; however, our results must also be qualified by the following limitations. With respect to generalisability, we expect that our findings overestimate the quality of OSH programmes in the large manufacturing sector as a whole. Participating companies had to voluntarily agree to participate in the trial; thus we expect our population to be enriched for sites that are receptive to OSH intervention, have relatively good OSH conditions, or both. While this would strengthen the argument that there is a need for improvements in OSH programmes, it would weaken the generalisability of the intervention effectiveness findings (to the extent that companies with poorer programmes are less receptive to intervention). Generalisation to smaller manufacturing worksites, to other industries, and to other states and countries is limited further still. In addition, for analytical and economic feasibility reasons, we relied on the reports of a single management level worksite respondent. This suggests the possibility of social desirability bias artificially inflating scores, which is of particular relevance to the cross sectional baseline score estimates. We believe this possibility is offset by the preceding site-wide walk-through assessment of the physical work environment.23 The walk-through visits (lasting 1–2 days at each site) were guided by the same respondent. The interviewing hygienist had both a sense of the general approach to OSH at the site as well as some rapport with the respondent, which we believe would mitigate social desirability bias. This is further borne out by the observation that there were no sites that scored very highly (82/100 highest). With respect to the effectiveness evaluation, the experimental design rules out any major influence of social desirability bias, in that the same bias would be expected to affect control as well as intervention sites, and thus would cancel out in the analysis. Nevertheless, further research is needed to develop data collection methods for assessing an organisation’s OSH management efforts. The promising findings of this methods development20,21 study should help to justify investment in subsequent studies with larger sample sizes, combined quantitative and qualitative assessment methods, and embedded validation of OSH programme measures against other established metrics (for example, exposures, injury rates).

The instrument used also has several limitations. First, not all important characteristics are amenable to generic scoring. The descriptive data gathered on OSH staffing, self insurance status, and responses to specific health hazards provides insight into the limits of quantitative scoring in general, and to the challenges to strengthening the health relevance of OSH programme assessments in particular (to better balance their safety/injury relevance, which has been the primary driver for OSHA’s approach as well as most others). While OSH staffing is clearly an important part of a systematic approach to managing OSH, using such data for generic scoring is problematic (even after normalising total OSH staffing level to full time equivalent employees) because different hazard profiles, work organisation approaches, etc require different types and quantities of OSH staff. Similarly, while self insurance for Workers’ Compensation is an organisational response to OSH issues, it has not been established whether it reliably translates to stronger OSH management in the form of primary prevention (in addition to its primary purpose of controlling insurance liability). Finally, while determination of organisational responses to specific hazards with full OSHA (or other regulatory) standards is clearly relevant to gauging the quality of an OSH programme, assessment is complex and would require information from numerous sources. For example, a site can only be fairly scored for offering medical surveillance (or not) for a specific hazard if requirements for surveillance are triggered. Triggers include exceeding quantitative exposure assessment action levels and other circumstances, the determination of which could require consultation of exposure monitoring records, and interviews of occupational medicine providers, department supervisors, or others. While this can be accomplished,28,29 it requires substantial additional data collection for each specific exposure as well as requesting more employee release time from the organisation, thus limiting feasibility. Further work is needed to develop more health relevant indicators that can be easily administered and scored, as well as to develop more OSH action oriented rather than OSH documentation oriented indicators. OSH programme assessments would optimally include generic scoring or ratings complemented by some form of qualitative assessment to capture programme characteristics and contextual factors that are not amenable to quantitative assessment.30

With respect to reducing occupational injury and illness, the significance of the positive association of intervention with OSH programme improvements is supported by previous empirical research: from the literature on social and organisational influences on OSH, management commitment and employee participation in OSH are the two constructs that have been most consistently observed as determinants of OSH conditions and injury outcomes.27,31–35 However, directly relating OSH programme scores to occupational injury and illness outcomes was beyond the scope of this methods development study. For the presumed relevance of our OSH programme measures to health outcomes, we are relying on the accumulating evidence of the effectiveness of OSH programmes in reducing injuries, lost worksite rates, disability rates, fatalities, Workers’ Compensation costs, insurance costs, and other outcomes across various industrial sectors.3,8,9 A priority for future research here, reflecting once again the safety dominant emphasis in this area to date, should be validation studies against occupational illness and disease outcomes and their associated exposures.

Integrating health protection and health promotion in Wellworks-2

With respect to health promotion outcomes, the Wellworks-2 intervention resulted in a twofold greater smoking cessation rate among blue collar workers in the integrated intervention condition versus health promotion only.18 Our OSH programme findings suggest that improvements in OSH facilitate improvement in health behaviours among blue collar workers. This reaffirms the soundness and promise of the central principle of Wellworks-2—and of social ecological approaches to health promotion in general—that in addition to intervening on individual behaviour, interventions should also address other modifiable determinants of health and disease.22,36–41 Workplace health promotion and health protection can be mutually reinforcing, promising benefits in both areas by more widespread implementation of integrated approaches.42–46

Conclusions

The OSH programme assessment method developed is broadly applicable to manufacturing work settings. Our findings suggest needs for improvement in OSH programmes in most such worksites. Despite a small sample size, we have shown that sustained management focused intervention can result in improvement in these OSH programme measures. These findings strengthen the accumulating evidence of the need for management focused intervention to improve OSH programmes, and have relevance to the development of policy and other interventions in this area in developed countries. Taken together with the health promotion results, the Wellworks-2 trial shows new synergistic potential for cross disciplinary intervention efforts to improve worker heath by integratively addressing health behaviour and OSH risks in the social context of the workplace.

Acknowledgments

The Wellworks-2 trial was funded by a grant from the US National Cancer Institute (5 R01 CA68087), with OSH programme assessment methods development work funded by a grant from the US Centers for Disease Control/National Institute for Occupational Safety and Health (5 R01 OH04012) and by a Senior Research Fellowship to ADL from the Victorian Health Promotion Foundation (Melbourne VIC, Australia: grant no. 2001-1088). Partial support for integrated intervention materials development was provided by a grant from Liberty Mutual Insurance Company. The Wellworks-2 intervention and evaluation protocol was reviewed and approved by the Dana-Farber Cancer Institute’s Institutional Review Board.

REFERENCES

View Abstract

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.