Pulmonary embolism (PE) is a common clinical diagnosis in patients presenting to the emergency department and in hospitalized patients. It is normally treated with long-term anticoagulation therapy to reduce the risks of death and the morbidity associated with chronic pulmonary venous thromboembolism. Because the clinical presentation is often nonspecific and can be mimicked by a range of other conditions, in routine practice, pulmonary CT angiography (CTA) is often used as the imaging method of choice for further investigation [1]. Pulmonary CTA has been shown to be highly sensitive and specific when pretest clinical diagnostic tools are used [2] but surprisingly inaccurate in patients with low pretest probability, with false-positive rates as high as 42% [3]. Unfortunately, adherence to referral guidelines for pulmonary CTA has repeatedly been shown to be low [4, 5]. Nonetheless, many clinicians will initiate anticoagulation therapy on the basis of a positive result, regardless of pretest probability [6], even in isolated subsegmental PE [7].

The risk of hemorrhage related to anticoagulation therapy is potentially significant. A large meta-analysis in 2003 [8] found a 7% annual risk of major bleeding and a 0.4% incidence of bleeding-related fatality in patients treated with oral anticoagulation therapy for venous thromboembolism for longer than 3 months. The practical implications of long-term anticoagulation therapy for the patient are also potentially significant, requiring frequent attendance to their medical practitioners for blood tests, consequent time off from work, potential adverse drug interactions with other medications, adjustments to travel and lifestyle, implications for future dental and medical procedures, and possible negative effects on life insurance status.

With these considerations in mind, it is important to minimize the misdiagnosis of PE. Common artifacts that can lead to a false-positive diagnosis of PE have been well described in the published literature [9–11]. Despite this, however, reported interobserver agreement varies widely, especially in the diagnosis of subsegmental PE [12]. Wide variations in concordance between general and subspecialist radiologists have been reported (89–100%) [13, 14], as well as between residents, fellows, and attending radiologists (87–93%) [15–17]. Although pulmonary CTA examinations are frequently interpreted by general radiologists in most centers, limited data exist on the interobserver agreement between general and subspecialist chest radiologists. A small number of studies [13, 18] have directly compared pulmonary CTA interpretation by general radiologists with that of a single subspecialist chest radiologist. There is very limited analysis of these discrepant cases in terms of PE location within the pulmonary arterial system and potential causes of misdiagnosis. In addition, the absence of a practical reference standard examination makes it difficult to draw conclusions regarding the accuracy of pulmonary CTA in routine clinical practice.

The purpose of this study was to evaluate the rate of overdiagnosis of PE by pulmonary CTA in a tertiary-care university hospital by assessing the degree of discordance between the original reporting radiologists and an expert panel of subspecialty chest radiologists and to attempt to establish patterns of misdiagnosis to try to understand the causes underlying pulmonary CTA misinterpretation.

Materials and Methods Choose Top of page ABSTRACT Materials and Methods << Results Discussion Conclusion References CITING ARTICLES This retrospective study was conducted at University College Hospital Galway, which is a specialist oncology center and a university-affiliated tertiary-care medical center in Galway, Ireland. Approximately 130,000 imaging studies are performed annually in the University College Hospital Galway Radiology Department, which is staffed by 15 attending and nine resident radiologists. The Galway University institutional ethical review board approved this retrospective study and waived the requirement for written informed consent. An electronic search was performed of the approved finalized reports of all consecutive pulmonary CTA examinations performed over a 12-month period between August 1, 2012, and July 31, 2013. Data were collected by both electronic query and manual review of the electronic medical record. All scans were acquired on a 64-MDCT scanner (Somatom Sensation 64, Siemens Healthcare) in the craniocaudal direction with a collimation of 0.6 mm and gantry rotation of 500 milliseconds. Automated dose control software was used with 120 kVp and 200 mA maximum; 80–120 mL of low-osmolar contrast medium (350 Omnipaque, GE Healthcare) was injected through an 18-gauge cannula sited in the antecubital fossa, at a rate of 4–5 mL/s, followed by a 20-mL saline bolus chaser injected at 4 mL/s. Optimal scan acquisition time was determined using a bolus-tracking technique with an ROI placed over the pulmonary trunk. Images were reconstructed with a 512 × 512 matrix and a smooth kernel, with 1-mm axial and 1.5-mm coronal slice thickness and 0.8-mm slice overlap. Images were reviewed using IMPAX (version 6.5, AGFA Healthcare). All studies in which a definite diagnosis of PE was reported were selected for further analysis. Studies reported as nondiagnostic or negative for the presence of PE were excluded (because the purpose of our study was to evaluate the potential rate of over-diagnosed PEs, rather than the overall diagnostic accuracy of pulmonary CTA). All studies were anonymized for independent interpretation on stand-alone workstations by a panel of three subspecialist chest radiologists, each with at least 10 years’ experience in pulmonary CTA interpretation. One radiologist was among the 15 on-site attending radiologists. The other two panel members were reviewers from another tertiary referral center. The final consensus opinion of these three chest radiologists was used as a surrogate reference standard for the diagnosis of PE. Each examiner was blinded to the index report, PE location, clinical history, and other diagnostic test results. An initial interpretation was performed by each of the three chest radiologists independently, in which they recorded the presence or absence of PE, the most proximal level of PE, the lobar location of PE, and the overall quality of the examination (satisfactory or unsatisfactory for diagnosis). After this initial interpretation, a second analysis was then made of those studies in which there was any disagreement among the three chest radiologists (to minimize the risk of overlooking PEs because of interpretation fatigue after reading a large number of pulmonary CTA studies in succession). Where there was any persistent discordance among the three chest radiologists after this second review, the original report was accepted as being correct (i.e., positive for PE). Next, a third and final analysis was performed of those studies for which there was a discrepancy between the consensus opinion of the three chest radiologists and the original report, guided by a partial unblinding of the original report to direct attention to the original reported PE. Where there was unanimous agreement among the three chest radiologists that a pulmonary CTA was negative, a final outcome of negative for PE was recorded. In addition, the following final data were recorded: patient demographics (age and sex); the most proximal PE location according to the modified Boyden classification [19] (pulmonary trunk, main pulmonary artery, lobar pulmonary artery, segmental pulmonary artery, or subsegmental pulmonary artery); number of PEs (solitary vs multiple); quality of contrast enhancement, assessed by calculating the average of the CT number measured in the pulmonary trunk and the right and left main pulmonary arteries with a circular ROI equal to the diameter of the vessel (Fig. 1); and interobserver agreement (modified kappa index) among the three chest radiologists and between the final consensus opinion of the three chest radiologists and the original reporting radiologists.

View larger version (298K)

Finally, the individual discordant cases (those that were considered to be negative for PE) were analyzed separately to attempt to establish a potential underlying cause for the misdiagnosis of PE on the original report, such as movement artifact from breathing or cardiac pulsation; poor contrast opacification of the pulmonary arteries due to Valsalva maneuver, cardiac insufficiency, or other cause of mixing of opacified and unopacified blood; beam-hardening attenuation artifact caused by adjacent high-density structures such as opacified veins, contrast material pooling in the inferior vena cava or right ventricle, or bony structures; and the presence of airspace disease obscuring the underlying pulmonary vasculature. Descriptive numeric values were used for patient and PE demographics (actual values, percentages, mean [± SD], and ranges). Comparisons between groups were performed using the paired t test and for ordinal categories using the chi-square test. A p value of 0.05 or less was considered statistically significant. All statistics were performed using SPSS (version 16, IBM).

Results Choose Top of page ABSTRACT Materials and Methods Results << Discussion Conclusion References CITING ARTICLES There were 937 pulmonary CTA examinations performed over the course of the 12-month study period. Of these, 174 studies (18.6%) were reported as positive for PE by the original radiologist (Table 1), and comprised 84 male and 90 female patients with a mean age of 64 years (range, 17–99 years). PEs were reported as solitary in 67 cases (38.5%) and multiple in 107 cases (61.5%) (Table 2). PEs were more frequently reported in the peripheral segmental and subsegmental arteries (103 cases; 59.2%) than in the more central and lobar arteries (71 cases; 40.8%) (Table 1). Twenty-four patients (13.8%) had a reported solitary subsegmental PE. The average quality of contrast enhancement was 327.0 ± 88 HU (range, 127.1–625.2 HU). View Larger Version TABLE 1: Comparison of Contrast Enhancement, Patient Age, and Pulmonary Embolism (PE) Location in All and Discordant Pulmonary CT Angiography Examinations Reported as Positive for PE by the Original Radiologist View Larger Version TABLE 2: Comparison of Solitary Pulmonary Embolism (PE) in All and Discordant Pulmonary CT Angiography Examinations On final analysis, the panel of three chest radiologists were of the consensus opinion that 45 (25.9%) of the original 174 index cases were negative for the presence of PE. Interobserver agreement between the panel members was almost perfect (weighted κ = 0.835). These discordant cases comprised 25 women (none of whom were pregnant) and 20 men (mean age, 60 years; range, 23–91 years). Overall image quality was considered to be satisfactory for diagnosis in 170 examinations (98%) and inadequate for diagnosis in four examinations (2%). The average quality of contrast enhancement was 291.3 ± 60.9 HU, versus a mean of 338.5 ± 91.8 HU in the group with a concordant diagnosis (p = 0.002). There was discordance between the chest radiologists and the original radiologist in 31 of 67 (46.2%) cases of reported solitary PE, whereas discordance occurred in only 14 of 107 (13.1%) cases where multiple PEs were originally reported. Discordance was highest for cases of reported peripheral PEs (38/103 [36.9%] cases of segmental or subsegmental PEs), with the highest rate for reported solitary subsegmental PEs (16/24 [66.7%] of such cases). Discordance occurred most commonly in the lower lobes, with the most commonly involved vessels being the lateral basal and posterior basal segmental arteries of the left lower lobe; these vessels accounted for 35.5% of all discordant cases diagnosed at the segmental level. Interestingly, no discordant diagnoses were seen in the right middle lobe (where diagnostic difficulty might have been expected because of the more horizontal course of the pulmonary arteries and their greater susceptibility to partial volume averaging effects). The distribution of instances of discordance between the chest radiologists and the original reporting radiologists was relatively even, varying from zero to six cases per radiologist (median, three cases), indicating that this was a generalized rather than an individual phenomenon (Fig. 2).

View larger version (30K)

Causes for the 45 cases of discordance included the following: 24 (53%) cases were due to motion artifact from breathing (19 [42.2%]) or cardiac pulsation (5 [11.1%]), eight (18%) cases were due to poor contrast opacification from Valsalva maneuver (3 [6.7%]) or contrast material mixing (5 [11.1%]), 10 cases (22.2%) were due to attenuation artifact secondary to beam hardening from adjacent high-density structures, and three cases (6.7%) were due to effects from adjacent airspace disease (Fig. 3).

View larger version (45K)



Discussion Choose Top of page ABSTRACT Materials and Methods Results Discussion << Conclusion References CITING ARTICLES This study shows an unexpectedly high rate of overdiagnosis of PE by pulmonary CTA in a tertiary-care university hospital, with an overall rate of 25.9% of all positive pulmonary CTA examinations, increasing to as high as 66.7% of cases where a solitary subsegmental PE was originally reported. Discordance was greatest for solitary PEs, PEs located in segmental and subsegmental pulmonary arteries, and in the lower zones of the lungs. The positive predictive value of pulmonary CTA for the diagnosis of PE was only 74.1% in this study. The published overall rate of positive diagnosis of PE on pulmonary CTA varies from study to study (e.g., 15.4% [14], 16.4% [13], and 17.8% [6]) but usually ranges between 14% [20] and 22% [7]. Differing levels of adherence to referral guidelines, which has been shown to significantly affect positivity rates, may explain some of this variation; for example, 30% of all pulmonary CTA examinations were positive for PE in the multicenter Christopher Study [2], which used strict adherence to a basic pretest risk stratification tool. The rate of index positive cases in our center was 18.6% for the 12-month period studied. After review by the panel of chest radiologists, this was revised downward to 13.8%. In our institution, there is no systematic use of pretest probability scoring (e.g., Well or Revised Geneva scores [21, 22]) and inconsistent use of d-dimer assays. Furthermore, many pulmonary CTA examinations in our institution are ordered by the emergency department before assessment by the admitting medical team. The combination of a lack of pretest probability assessment and either inconclusive or possibly erroneous pulmonary CTA results can cause difficulties in patient diagnosis and management, often leading to repeat imaging and unnecessary anticoagulation therapy. The risks and disadvantages of anticoagulation therapy include hemorrhage (occasionally devastating or fatal) [8], interactions with other medications, inconvenience in terms of attendance for repeated blood tests (which may require time off work), and cost (to both the patient and society) [23]. Furthermore, a diagnosis of PE carries with it implications for life insurance coverage, travel plans, and preparation for other medical or surgical procedures. The diagnosis of PE also places the patient in a higher risk category for future events, which can influence investigations and management if the patient again seeks medical attention for similar symptoms. The significance of a false-positive pulmonary CTA examination should be considered in this context. Previously published studies have shown differences between chest- and non-chest-trained radiologists in the diagnostic accuracy of pulmonary CTA interpretation. In a 2011 study of 70 isolated subsegmental PEs by Pena et al. [13], a reviewing thoracic radiologist reinterpreted 11% of these examinations as negative. In a separate abstract published by Miller et al. [18], a single thoracic radiologist found a false-positive or probable false-positive rate of 11% at all pulmonary artery levels in 508 cases. Compared with these previous studies, the current study is larger and uses a panel of three subspecialty chest radiologists as a more robust surrogate reference standard, rather than relying on one single radiologist's opinion. Although this was a single-center study, our department does not differ in any significant way from any other university hospital imaging center, with the same mix of inpatients, out-patients, emergency department patients, and pregnant patients as might be found in any equivalent tertiary referral center, and with a modern radiology department using conventional MDCT technology and a PACS for the performance and interpretation of pulmonary CTA examinations, staffed by a general mix of experienced subspecialty fellowship-trained radiologists. The high rates of discrepant pulmonary CTA interpretations found in this study raise concerns about the diagnostic accuracy of radiologists in the wider community. However, the generalizability of our results should be confirmed with a larger multicenter study. Causes of diagnostic difficulty in the interpretation of pulmonary CTA examinations are well recognized [9–11]. A full description of such interpretative pitfalls is beyond the scope of this discussion, but potential false-positive findings are known to occur because of partial volume averaging effects secondary to motion (breathing and cardiogenic), poor contrast opacification from mixing of opacified and unopacified blood, beam-hardening attenuation artifact from high-density structures (e.g., contrast agent in the superior vena cava [SVC] and right atrium), and confusion with venous structures and mucus-filled bronchi [9, 10, 24]. In our study, the most common cause identified for the misdiagnosis of PE was motion artifact due to breathing, which accounted for 42.2% of cases. Breathing artifact has previously been shown to be the most common mimic of PE [18] as well as the most common cause of equivocal pulmonary CTA findings in up to 74% of cases [9]. Breathing artifact can most easily be identified on a lung window by the presence of the seagull artifact, the stair-step artifact, and rapid changes in position of vessels on contiguous image slices [10] (Fig. 4). Ways to reduce the level of breathing artifact include administering supplemental oxygen and scanning in the caudocranial direction [24].

View larger version (288K)



View larger version (353K)

The second-most-common confounding artifact was beam-hardening attenuation artifact (Fig. 5) from high-density structures, including pooled contrast agent in the SVC or other adjacent vessels, metallic structures such as pacemakers, or the patient's arms if they cannot be elevated above the chest. The use of a saline chaser helps clear pooled contrast agent from the SVC [10]. Apart from their proximity to a high-density structure, regions of low attenuation related to streak artifact have much higher densities (> 78 HU) than real thrombus and form indistinct borders with contrast agent in the vessels [10]. We also observed that beam-hardening attenuation artifact could often be tracked in a radial pattern from the source of the artifact and could also be identified in other nearby structures.

View larger version (216K)



View larger version (178K)

Other artifacts responsible for misinterpretation included cardiac pulsatility (Figs. 6 and 7), which is most often seen in regions of the lung adjacent to the heart, such as the lingula and the paracardiac segments of the lower lobes; reduced mixing of contrast agent with unopacified blood, which can be due to excessive inflow of unopacified blood from the inferior vena cava or other veins, excessive breath-holding resulting in a Valsalva maneuver, or poor cardiac function and poor mixing of contrast agent; and obscuration of the pulmonary arteries by adjacent parenchymal disease. The latter is attributed to increased local vascular resistance, which leads to reduced flow and flow artifacts [10]. Interestingly, in our study, there were no discrepancies due to confusion between PEs and pulmonary veins or mucus-filled bronchi. This might suggest that discrepancies in the diagnosis of PE arose not because of unfamiliarity with anatomy or to a lack of attention to detail when reading the scan but rather because of perceptual errors resulting from an under-recognition of the other causes of false-positive examinations, as summarized already.

View larger version (339K)



View larger version (290K)



View larger version (212K)

Our study also highlights the difficulty of performing audits of the accuracy of pulmonary CTA interpretation. The original decision by the treating physician to initiate anticoagulation therapy would be taken within the clinical context of the patient's presentation, history of thromboembolic disease, cardiac workup, d-dimer levels, and so forth, and is not based purely on the result of the pulmonary CTA scan. In our study, this clinical information was not included. In this regard, as a specific outcome from our study, it was considered to be neither clinically appropriate nor ethical to revisit the original clinical diagnosis several years later on the basis of the results of an academic study that had not been designed to reexamine all of the clinical information that was originally available. Rather, the purpose of our study was to examine the diagnostic difficulties in the use of pulmonary CTA as a diagnostic study in isolation. In the absence of a practical true reference standard, we opted to rely on the consensus opinion of three experienced chest radiologists, which would be difficult to reproduce on a routine basis because of the time involved in collating and reviewing the necessary examinations. Correlation of pulmonary CTA findings with clinical outcomes (e.g., recurrent thromboembolism or death) is a crude measure of accuracy and would also be difficult to achieve in routine clinical practice. The difficulty in performing regular audits of this very common imaging test highlights the risk of unrecognized diagnostic drift, where an established diagnostic test performs less well over time because of changes in practice and personnel and because of an absence of feedback or correlative reference standard test. Practical measures to reduce the risk of PE misdiagnosis could and should include any of the following: systematic use of pretest probability assessment (which would require buy-in from clinicians and incorporation into imaging protocols); radiology technologists being educated to optimize image quality, focusing on proper patient breathing technique and repeating examinations where appropriate; increased familiarization by radiologists with the range of potential diagnostic pitfalls; encouragement of the use of second opinions by interpreting radiologists, particularly for solitary subsegmental PEs; and regular review of positive pulmonary CTA cases (e.g., at monthly discrepancy or audit meetings). Some of these measures are easier to implement than others, but their importance is underscored by the implications of a false-positive diagnosis of PE.