Article Text
PDF
Abstract
Objectives To investigate inter-reader agreement for the detection of pleural and parenchymal abnormalities using CT in a large cross-sectional study comprising information on individual cumulative exposure to asbestos.
Methods The project was approved by the hospital ethics committee, and all patients received information on the study and gave their written informed consent. In 5511 CT scans performed in a cohort of retired workers previously exposed to asbestos and volunteering to participate in a multiregional survey programme (Asbestos Related Diseases Cohort, ARDCO), double randomised standardised readings, triple in case of disagreement, were performed by seven trained expert radiologists specialised in thoracic imaging and blind to the initial interpretation. Inter-reader agreement was evaluated by calculating the κ-weighted coefficient between pairs of expert readers and results of routine practice and final diagnosis after expert reading.
Results κ-Weighted coefficients between trained experts ranged from 0.28 to 0.52 (fair to good), 0.59 to 0.86 (good to excellent) and 0.11 to 0.66 (poor to good) for the diagnosis of asbestosis, pleural plaques and fibrosis of the visceral pleura, respectively. κ-Weighted coefficients between results of routine practice and final diagnosis after expert reading were 0.13 (poor), 0.53 (moderate) and 0.11 (poor) for the diagnosis of asbestosis, pleural plaques and fibrosis of the visceral pleura, respectively.
Conclusions Interpretation of benign asbestos-related thoracic abnormalities requires standardisation of the reading and trained readers, particularly for participants asking for compensation, and with a view to the longitudinal survey of asbestos-exposed workers.
Statistics from Altmetric.com
What this paper adds
-
Computed tomography (CT) has proved to be much more sensitive and specific than chest radiographs for the detection of pleural changes in their early stages. Although a few studies have addressed inter-reader agreement of CT findings in the context of asbestos exposure, large-scale studies are lacking.
-
In a large study population of asbestos-exposed workers evaluated by CT, inter-reader agreement for the diagnosis of pleural plaques was found to be moderate to excellent between trained expert readers but lower between trained experts and routine practice.
-
When medicolegal issues are involved and for studies attempting to solve the controversial issues dealing with the prognostic value of pleural plaques in assessing risk of respiratory cancers, dedicated training and standardised methods of reading are recommended.
Introduction
Occupational asbestos exposure is associated with the development of non-malignant and malignant respiratory diseases involving the pleura as well as the lung. Chest radiographs are commonly used to screen for benign pleural and pulmonary asbestos-related diseases. However, conventional CT and subsequently high resolution CT (HRCT) have proved to be much more sensitive and specific than chest radiographs for the detection of pleural and parenchymal benign changes in their early stages.1 In view of these better performances, and because workers in France are entitled to financial compensation and early retirement at the age of 50 years in the case of asbestos-related disease, including pleural plaques, a consensus conference held in Paris in January 1999 recommended the use of chest CT scan after a sufficient latency period for the surveillance of workers occupationally exposed to asbestos.2 For medicolegal issues, knowledge about inter-reader agreement for the detection of CT signs of early asbestosis and pleural plaques has therefore become crucial. Although a few studies have dealt with inter-reader agreement of CT findings in the context of asbestos exposure,1 ,3 data from a large-scale study are so far lacking.
The aim of the present study was to investigate inter-reader agreement for the detection of pleural and parenchymal abnormalities using CT in a large cross-sectional study comprising information on individual cumulative exposure to asbestos.
Methods
Study population
The screening programme for asbestos-related diseases, called the Asbestos Post EXposure Survey (APEXS), was organised at the request of the French Ministry of Labour and National Health Insurance between October 2003 and December 2005 in four regions of France: Aquitaine, Rhône-Alpes, Haute-Normandie and Basse-Normandie. Retired or unemployed workers previously occupationally exposed to asbestos and belonging to the General National Health Insurance fund (which covers more than 80% of the French population) were eligible.
They were invited to participate in the programme using different means according to the region (letters targeting age groups below 65 or 67 years and type of previous trade; trade unions, radio, television and newspaper advertisements). Hospital ethics committee approval was obtained, and all patients received information about the study and radiation exposure, and gave their written informed consent. The overall design, exposure assessment,4 ,5 as well as procedures for pulmonary function tests6 ,7 have been previously reported.
Evaluation of exposure to asbestos
Evaluation of individual asbestos exposure was performed by using data from a standardised questionnaire. The cumulative exposure index (CEI) in ‘exposure units×years’6 was the sum of individual exposure indexes calculated for each job, based on the level and duration of exposure to asbestos of each job. CEI was classified into five categories based on the distribution of values (ie, quintiles).
CT procedures and initial reading
Modalities for performing CT scan were set as follows: the entire chest was screened in supine position using a spiral acquisition without injection of contrast material, slice thickness 1.5–5 mm, pitch of 1.5–2.0, 120 kV, 60–150 mA maximum; parenchymal images were reconstructed with sharp filters and visualised with a window width of 1600 HU and window level of −600 HU; and soft tissue images were reconstructed with smooth filters and visualised with window width of 400 HU and window level of 50 HU. A minimum of five HRCT sections of 1 mm thickness performed in prone position between the carina and the inferior pleural recess were added. The following technical variant was also admitted: acquisition of the whole thorax on prone position at full inspiration. Radiologists who participated in the programme were qualified radiologists but not chest radiologists. These radiologists had received specific instructions for modalities for conducting CT scan in the follow-up of asbestos-exposed participants. They reported findings on a non-standardised form. CT reports and either CD-ROM or hard copies were both sent to coordination centres. Participants were classified into those with or without asbestosis, with or without pleural plaques and with or without fibrosis of the visceral pleura.
Trained expert reading
An independent double standardised reading of all the CT scans was performed by seven trained experts, all chest radiologists. Trained experts were involved in two training sessions, which were not part of the main execution of the study, prior to the standardised reading. During these sessions, an interim checking of their reading results was performed between experts. Trained expert readers were blind to the cumulative exposure to asbestos of participants and to the results of the routine practice. Seven trained experts were paired in all unique permutations (21 pairs), and CT films were randomly allocated within these pairings, with films being read independently. When conclusions between experts were discordant for the presence of either asbestosis, pleural plaques, visceral pleura fibrosis or rounded atelectasis, a third interpretation was made by a third independent expert reader (not involved in the first reading of the relevant cases). The final accepted conclusion was the median between the three experts.
Criteria for the diagnosis of benign pleural and parenchymal abnormalities for the expert reading
A CT scan was considered as ‘insufficient’ for either lung and/or pleural analysis when technical defects were likely to influence interpretation, such as: inappropriate slice thickness preventing correct analysis of the parenchyma, failure to produce sections in prone position in the case of suspected interstitial abnormalities observed on images in supine position, or prominent motion artefacts. The presence of parenchymal findings was evaluated in each lung. Septal lines, reticulation, ground-glass opacities, honeycombing and micronodules were those defined according to the Fleischner society glossary of terms.8 According to the extent and location of individual findings, the reader was asked to classify the patient into one of three categories: likely asbestosis, interstitial abnormal findings but unlikely asbestosis, or normal. Extent of parenchymal abnormalities was assessed at three levels: carina, pulmonary veins and pulmonary cul-de-sac.
-
No asbestosis: no interstitial abnormality or few abnormalities or disappearing in supine position.
-
Likely asbestosis: one or several interstitial bilateral abnormalities present at more than one of the three anatomical levels of reference, or present at a single level of reference but on sections distant from each other to more than 1 cm and/or presence of honeycombing.
-
Interstitial abnormal findings but unlikely asbestosis: interstitial abnormalities present on a more limited volume of lung than described above or interstitial abnormalities with pattern suggestive of another infiltrative lung disease (sarcoidosis, emphysema).
Various pleural CT findings were defined according to the Fleischner society glossary of terms.8 Abnormal pleural thickness was classified into four categories by taking into account the most thickened plaque: less than 2 mm, between 2 and 5 mm, between 5 and 10 mm, more than 10 mm. The extent of pleural plaques was classified into four categories according to the visual cumulative extent of pleural plaques detected on each section and virtually reported on the single section at the level of the carina to evaluate the extent as a proportion of the perimeter of a hemithorax. The following thresholds were used: less than 1 cm length, between 1 cm and one-fourth of the perimeter of the hemithorax, between one-fourth and half the perimeter of the hemithorax, more than half the perimeter of the hemithorax. Diffuse pleural thickening, parenchymal bands and crow-feet images were considered to be part of the diagnosis of fibrosis of visceral pleura, as well as rounded atelectasis,9 and all terms used were defined according to the Fleischner society glossary of terms.8 ,10
Statistical analysis
Inter-reader agreement was evaluated by calculating the κ-weighted coefficient between pairs of expert readers for individual CT findings as well as for assessment of asbestosis, pleural plaques, fibrosis of the visceral pleura and round atelectasis, and between results of routine practice and final diagnosis after expert reading. The following interpretation was used for κ coefficient: <0.2=poor agreement; 0.2–0.4=fair agreement; 0.4–0.6=moderate agreement; 0.6–0.8=good agreement; 0.8–1=excellent agreement.11 Statistical analysis was carried out using SAS V.9.2 software.
Results
Study population
The study population consisted of 5511 asbestos-exposed male participants eligible in the screening programme. Age, smoking status and exposure characteristics are reported in table 1. Radiation dose was calculated for each patient and reported on the initial report according to national regulations.
Study population characteristics (n=5511)
CT findings
CT was considered sufficient for evaluating parenchymal and pleural findings by at least one of the readers in 5511 participants. Results from at least two expert readers could be obtained in 4899 participants for individual parenchymal findings and 5446 for pleural findings. In 612 for parenchymal findings and in 65 for pleural findings, missing data from at least one expert precluded analysis of reproducibility. Individual CT findings for the diagnosis of asbestosis, thoracic wall pleural plaques classified according to their thickness and extent, diaphragmatic plaques, findings consistent with visceral pleura fibrosis and rounded atelectasis, identified by each expert reader, and the corresponding κ-weighted values are reported in table 2.
Individual CT findings of trained expert reading and κ-weighted values between expert readers in the detection of asbestosis, pleural plaques, visceral pleura fibrosis and round atelectasis
The final diagnosis for asbestosis was based on the trained expert reading process and could be obtained in 4899 cases and was as follows: asbestosis in 37 (0.8%) and abnormal parenchymal findings not compatible with asbestosis in 362 (7.3%). Based on a similar process, the final diagnosis of pleural plaques was obtained in 1118 participants out of 5446 (20.5%), findings consistent with fibrosis of the visceral pleura in 87 out of 5446 (1.6%), and rounded atelectasis in 14 out of 5446 (0.3%).
Inter-reader agreement between trained experts
For the diagnosis of asbestosis (table 3), κ-weighted values ranged from 0.28 to 0.52 (fair to moderate). For thoracic (see online supplementary table S1) and diaphragmatic (see online supplementary table S2) pleural plaques, κ-weighted values ranged from 0.59 to 0.86 (moderate to excellent) and from 0.57 to 0.88 (moderate to excellent), respectively. For the presence of either thoracic wall or diaphragmatic plaques (table 4), κ-weighted values ranged from 0.59 to 0.86 (moderate to excellent). For evaluating the extent of plaques, κ values were found to be moderate (0.45 (95% CI 0.40 to 0.49); table 2). For the diagnosis of fibrosis of the visceral pleura (online supplementary table S3), κ-weighted values ranged from 0.11 to 0.66 (poor to good).
κ Coefficient values of each pair of trained expert readers for the presence of asbestosis (κ <0.2: poor agreement; 0.2–0.4: fair; 0.4–0.6: moderate; 0.6–0.8: good; 0.8–1: excellent)
κ Coefficient values of each pair of expert readers for the presence of either thoracic or diaphragmatic plaques (κ <0.2: poor agreement; 0.2–0.4: fair; 0.4–0.6: moderate; 0.6–0.8: good; 0.8–1: excellent)
Agreement between trained experts and initial reading
This comparison could be obtained in 5511 participants for the diagnosis of asbestosis, in 5398 participants for the diagnosis of pleural plaques and in 5335 participants for the diagnosis of fibrosis of the visceral pleura. Concordances between results of routine practice and final diagnosis after expert reading for the presence of asbestosis, pleural plaques and fibrosis of the visceral pleura are reported in table 5. κ Values between routine practice and final diagnosis after expert reading process were poor (0.13 (95% CI 0.08 to 0.17)), moderate (0.53 (95% CI 0.50 to 0.56)) and poor (0.11 (95% CI 0.04 to 0.18)) for the detection of abnormal interstitial findings consistent with asbestosis, pleural plaques and fibrosis of the visceral pleura, respectively. While routine practice identified 335 cases of asbestosis, only 26 were confirmed by expert reading, while 11 additional cases not identified by routine practice were added by experts.
Number of concordant diagnoses between initial reading and final diagnoses from expert reading process
Discussion
This study is the largest scale study to date evaluating inter-reader agreement for identification of pleural plaques and asbestosis using CT scan as a diagnostic tool. Inter-reader agreement was found to be moderate to excellent between experts for pleural plaques detection and only moderate between experts and initial reading. For the diagnosis of asbestosis, inter-reader agreement was generally found to be fair to moderate between trained expert readers and poor between trained experts and initial reading.
Interestingly, experts found a very low rate of asbestosis in this series (0.8%). Although the low prevalence of asbestosis in this study population is in agreement with recent data in the asbestosis epidemiology,12 this deserves several comments. Although limited-dose parameters suggested in CT acquisition may induce an increased level of noise and mask early features of asbestosis, low-dose multislice CT has been demonstrated to depict early findings accurately in patients exposed to asbestos.13 The selection bias in participants volunteering to participate in this survey might be an explanation. Nevertheless, the striking difference in the number of asbestosis cases identified by trained experts and initial reading and the resulting poor κ values was likely due to the unknown cut-off between initial features of asbestosis and minor interstitial abnormalities in participants without asbestosis. In such a study population of middle-aged to elderly participants, the presence of discrete lung parenchymal abnormalities has been reported to increase with age.14 ,15 A basal reticular pattern was identified in 60% of healthy individuals older than 75 and in none of those younger than 55.14 This may be in relation to alveolar collapse secondary to elastin degradation with ageing, or to limited interstitial fibrosis.15 Similarly, a thickening of the interlobular septae has been more frequently identified in older individuals.16 The subpleural attenuating curvilinear line defining the subpleural line may be in some but not all healthy individuals a reversible phenomenon, disappearing at imaging in prone position. This finding has been related to focal atelectasis due to labile engorgement of a leash of subpleural lymphatic vessels, which is more prominent in the dorsal part of the lung bases.17 In the CT lung cancer screening study by Vehmas et al18 among asbestos-exposed workers, by adjusting the analysis for smoking status, asbestos exposure and body mass index, increasing age correlated positively with interstitial abnormalities on HRCT findings.18 A study by Gamsu et al19 showed that CT findings of asbestosis are neither perfectly sensitive nor specific for asbestosis. However, it also showed that asbestosis can be diagnosed with confidence when lung changes are bilateral or present at multiple levels.19
A few studies have dealt with inter-reader variability in identifying asbestosis. Copley et al evaluated 74 patients with asbestosis and compared their results to those of 212 patients with idiopathic pulmonary fibrosis.20 They found a κ-weighted coefficient of 0.78 for inter-reader agreement of coarseness of fibrosis and similar findings for usual interstitial pneumonia and asbestosis. However, they explored a majority of patients with late fibrosis (74% of cases). Of note, only 4 patients (5%) had no pleural disease.20 In another study by Copley et al21 including 133 individuals exposed to asbestos, the κ values for CT patterns of asbestosis were as follows: interstitial lines 0.39, coarseness of fibrosis (honeycombing) 0.51.
For pleural plaques, inter-reader agreement was moderate to excellent between trained experts in our series. In a limited number of participants explored by CT, the presence of pleural plaques has been reported with a κ value of 0.87 between two experienced trained readers.21 Regarding CT evaluation of 602 asbestos-exposed employees and 50 controls and using a four-point classification system of pleural lesions, Tiitola et al22 found κ values greater or equal to 0.40 when calcifications, thickness and extent of pleural disease were considered. On the other hand, De Raeve et al3 reported observer variations in identifying pleural lesions in 100 volunteers who had worked in an asbestos-contaminated building. When three radiologists classified CT scans according to a four-point classification system of pleural lesions, κ values ranged from 0.26 to 0.43.3 Thus interobserver variations might be more pronounced in minor pleural lesions, which can occur after low exposure.23
Several reasons may explain interobserver variations in detecting pleural changes on CT. The pleural lesion must be differentiated from normal anatomic structures such as subcostal fat, intercostal muscles or veins that may cause focal abnormalities. Nevertheless, precise evaluation of the presence or absence of pleural plaques is a key issue for studies attempting to solve the controversial issues dealing with the prognostic value of pleural plaques involving the risk of respiratory cancers.24 ,25
Considering the features of visceral pleura fibrosis, we used the state-of-the-art definitions of diffuse pleural thickening, parenchymal bands and crow-feet images. These findings together with rounded atelectasis are presently considered as part of fibrosis of the visceral pleura. Gevenois et al9 used cluster analysis to show that findings can be grouped as one of the three patterns of response to asbestos exposure, namely visceral pleura fibrosis, pleural plaques and asbestosis.9 The prevalence of abnormal findings suggestive of visceral pleura fibrosis was low in our series and κ-weighted values between experts ranged from 0.11 to 0.66, a wide range likely due to the low number of cases with visceral pleura fibrosis.
Our study has limitations. One was the absence of histopathological proof of asbestosis. The current consensus is that there is no justification for open lung biopsy in patients who are suspected of having asbestosis. Diagnosis is currently made on the basis of radiological data and an appropriate history of asbestos exposure. The second limitation is the lack of control group.
Conclusion
In a large study population of asbestos-exposed workers evaluated by CT, inter-reader agreement for the diagnosis of pleural plaques was found to be moderate to excellent between trained expert readers but lower between trained experts and initial reading. This should be taken into consideration when medicolegal issues are involved and for studies attempting to solve the controversial issues dealing with the prognostic value of pleural plaques in assessing the risk of respiratory cancers. Dedicated training and standardised methods of reading are recommended in this context.
Acknowledgments
The authors would like to thank all members of the asbestos postexposure program for their contribution to this survey: participants in program SPP-A/APEXS, ARDCO: E Abboud, B Aubert, J Baron, H Beauvais-March, J Benichou, A Bergeret, A Caillet, P Catilina, S Chamming's, G Christ de Blasi, F Conso, M Coulomb, E Guichard, N Le Stang, B Marchand, MF Marquignon, B Millet, L Mouchot, M Pinet, A Porte, JL Rehel, P Reungoat, R Ribeiro, M Savès, A Sobaszek, A Stoufflet, FX Thomas, L Thorel and the social security practitioners (Aquitaine, Haute et Basse Normandie, Rhône-Alpes).
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
Footnotes
-
Contributors FL, CP and J-CP were involved in design, acquisition, analysis, interpretation, final approval, drafting and integrity of work. GRF, CB, MM, VL, AJ, BC, AG, YB, ML, AL, ES, PB and JA were involved in acquisition, interpretation, final approval and integrity of work.
-
Funding This study was supported by grants from the French National Health Insurance (Occupational Risk Prevention Department), the French Ministry of Labor and Social Relations and the ANSES (07 CRD 51 and EST 2006/1/43).
-
Competing interests None.
-
Patient consent Obtained.
-
Ethics approval Hospital ethics committee approval.
-
Provenance and peer review Not commissioned; externally peer reviewed.