Objective To explore the difference of cumulative incidence rate of coal workers’ pneumoconiosis (CWP) among four large state-owned coal enterprises in northern China, we created an index system for evaluating the quality of comprehensive measures against CWP and applied the system to evaluate and compare the measures of the four coal enterprises.

Methods A two-round Delphi investigation was conducted to identify the indicators in the index system. The weight values of the indicators were calculated with analytic hierarchy process methods. Measures of CWP, mine annals, records and other information in each coal mine of the four enterprises were collected. The evaluation scores, which ranged from 0 to 100, were calculated and compared with.

Results A three-grade index system with 3 first-grade indicators, 9 second-grade indicators and 44 tertiary-grade indicators was established. The expert authority coefficient (C_r ) was 0.75 and the Kendall’s coefficient of concordance (Kendall’s W) was 0.15 (χ²=193.30, P<0.001). The weight value of ‘Geological conditions’ was 0.43, equal to ‘Dust control engineering technology’, and that of ‘Occupational health management’ was 0.14. The medians and quartiles of the evaluation scores of comprehensive measures against CWP of the four enterprises were 58.38 (54.60~63.02), 64.63 (60.83~67.06), 72.99 (68.92~77.67) and 75.07 (70.73~79.20), respectively.

Conclusions The index system could be effectively used for evaluation and comparison of the comprehensive measures against CWP among different enterprises. The geological conditions and dust control engineering technology played an important role in preventing and controlling CWP.