Purpose: Pathologic fractures could be prevented if reliable methods of fracture risk assessment were available. A multicenter prospective study was conducted to identify significant predictors of physicians' treatment plan for skeletal metastasis based on clinical fracture risk assessments and the proposed CT-based Rigidity Analysis (CTRA).
Experimental Design: Orthopedic oncologists selected a treatment plan for 124 patients with 149 metastatic lesions based on the Mirels method. Then, CTRA was performed, and the results were provided to the physicians, who were asked to reassess their treatment plan. The pre- and post-CTRA treatment plans were compared to identify cases in which the treatment plan was changed based on the CTRA report. Patients were followed for a 4-month period to establish the incidence of pathologic fractures.
Results: Pain, lesion type, and lesion size were significant predictors of the pre-CTRA plan. After providing the CTRA results, physicians changed their plan for 36 patients. CTRA results, pain, and primary source of metastasis were significant predictors of the post-CTRA plan. Follow-up of patients who did not undergo fixation resulted in 7 fractures; CTRA predicted these fractures with 100% sensitivity and 90% specificity, whereas the Mirels method was 71% sensitive and 50% specific.
Conclusions: Lesion type and size and pain level influenced the physicians' plans for the management of metastatic lesions. Physicians' treatment plans and fracture risk predictions were significantly influenced by the availability of CTRA results. Due to its high sensitivity and specificity, CTRA could potentially be used as a screening method for pathologic fractures. Clin Cancer Res; 21(11); 2514–9. ©2015 AACR.
This is the first prospective multicenter study that evaluates the impact of CT-based Structural Rigidity Analysis (CTRA) results on physicians' treatment plans for patients with appendicular metastatic lesions. The results of this study suggest that the existing gap, between clinical guidelines and physicians' recommendations, in the decision-making process for the selection of surgical or nonsurgical treatment must be narrowed by more advanced prognostic tools, such as CTRA. We are presenting a unique method based on the principles of composite beam theory—an analytical framework that accounts for both the material properties of the individual elements that make up a structure and the overall geometry of the structure itself. These mechanical and engineering principles have been translated into the clinical setting with the use of readily available imaging techniques that are capable of noninvasive measurements of bone density and cross-sectional geometry.
The skeleton is the third most common site of metastatic cancer, and one third to half of all cancers metastasize to bone (1). Long bone skeletal metastases are common in the United States, with more than 280,000 new cases every year (2). As a result of new and aggressive treatments, patients with cancer are living longer, but at sites of skeletal metastasis, patients may experience intractable pain and pathologic fractures (3, 4). The dilemma is to decide whether the metastatic tumor has weakened the bone sufficiently such that a pathologic fracture is imminent. Although guidelines have been previously put into effect, most clinicians make subjective assessments regarding fracture risk and treatment selection based on plain radiographs, using empirical methods now recognized to be inaccurate (5).
Retrospective studies have identified pain, activity level, lesion geometry, lesion anatomic site, and lesion type as fracture predictor candidates for metastatic tumors (6–12). Given that skeletal metastasis is initially diagnosed from the evaluation of plain radiographs, several investigators have attempted to estimate fracture risk by measuring the geometry of the lesion using radiographs. Two frequently cited criteria are considered indications for prophylactic stabilization: a metastatic defect greater than 2.5 cm in diameter and/or cortical destruction that is more than 50% of the bone's diameter (6, 9, 13–15). However, they have not been confirmed in experimental or prospective in vivo studies (16).
Mirels (17) developed a scoring system to quantify the risk of sustaining a pathologic fracture in a long bone by combining four risk factors: site (upper extremity, lower extremity, peritrochanteric); pain (mild, moderate, severe); lesion type (blastic, mixed, lytic); and lesion size (<1/3, 1/3–2/3, >2/3 of diameter of the bone). Summation of these factors into a single score provided greater accuracy than any single factor for determining fracture risk. Based on Mirels criteria, lesions with overall scores less than 7 could be irradiated, whereas prophylactic stabilization was recommended for scores greater than 9.
Although Mirels score is currently the only available tool for screening metastatic appendicular lesions, it has several limitations: It is based on the two-dimensional representation of a three-dimensional structure in a plain radiograph, often with inadequate resolution to assess the size and nature of the lesion. The specificity is less than 35% (17), and the strict application of the score will result in unnecessary surgeries in two thirds of surgical cases, while exposing patients to operative risks and complications (18). There are also conflicting reports on the sensitivity and specificity of Mirels criteria in different anatomical sites (19) and among different medical specialties (18, 20, 21), further emphasizing the need for a more objective and precise tool to assess fracture risk in metastatic lesions.
We have developed and validated a technique called Computed Tomography-based Structural Rigidity Analysis (CTRA) to accurately predict and monitor fracture risk associated with metastatic lesions based on quantification of changes in bone geometry and density (22–25). We hypothesize that CTRA significantly guides physicians in the appropriate selection of therapeutic plans for patients with skeletal metastasis and improves metastatic fracture risk prediction when compared with current clinical guidelines. To that end, we designed a multicenter prospective study to identify those factors that determine the treatment plan recommended by orthopedic oncologists for patients with appendicular skeletal metastasis; to evaluate whether inclusion of CTRA results can alter the treatment plan outlined by the physician; and to evaluate “prospectively” whether CTRA is more accurate at predicting pathologic fractures than current clinical and radiographic fracture risk assessments.
Materials and Methods
Institutional Review Board (IRB) approvals were obtained from participating institutions (Upstate Medical University, Rhode Island Hospital, University of Minnesota Medical Center, Sinai Hospital, MD Anderson Cancer Center, McGill University Health Centre, Marshall University, and Beth Israel Deaconess Medical Center). Enrollment took place at the time of first presentation to orthopedic oncology care. One hundred twenty-four patients with 149 metastatic lesions, who met the inclusion criteria of having at least one appendicular skeletal metastasis and no previous history of metastatic disease, were enrolled into the study between 2009 and 2012. The patients' age, sex, height, weight, type of primary cancer, characteristics of the metastatic lesion (size, type, and location) and pain level (mild, moderate, and severe/functional) were obtained upon enrollment. General health status was assessed using the SF-36 physical component summary (PCS; refs. 26, 27). The study has been registered at clinicaltrials.gov under the number NCT02109952.
The enrolling physicians were asked to complete a pre-CTRA survey and select a treatment plan (observation, chemotherapy ± radiation, surgical stabilization) based on their fracture risk assessment using standard clinical and radiographic guidelines. Then, CT scans of the involved bones (including the lesion and adjacent intact bone) were obtained with a hydroxyapatite (HA) phantom (CIRS Tissue Simulation and Phantom Technology) to convert the X-ray attenuation for each pixel to bone mineral density and to enable comparison of cases from different imaging sites. All institutions followed a standard CT imaging protocol (axial slices of 1–2 mm in thickness; inclusion of 1–2 cm imaging of the bone beyond the distal and proximal ends of the lesion). CTRA was performed for research purposes only and served two purposes: (i) evaluate how the availability of the results would change treatment recommendations (post-CTRA plan); and (ii) evaluate prospectively the diagnostic performance of CTRA and Mirels score in the subgroup of patients who did not undergo prophylactic stabilization. The CTRA report was sent to the physicians, who were asked to submit a post-CTRA survey, stating how the CTRA results would have changed their treatment plan. The pre- and post-CTRA treatment plans were compared to identify cases where the treatment plan changed as a result of the CTRA results. The patients who did not undergo prophylactic stabilization (there were three reasons why patients did not undergo prophylactic fixation in spite of a high Mirels score: (i) Physicians' decision: Some physicians considered that the patient did not have an increased risk of fracture in spite of a high Mirels score; (ii) the patient was unfit to undergo a major surgical procedure; and (iii) the patient decided against the procedure, were followed over a 4-month period, and pathologic fractures at the lesions sites were recorded at incidence or at subsequent visits.
Calculation of fracture risk using structural rigidity analysis
For each trans-axial CT image, axial (EA), bending (EI), and torsional (GJ) rigidities for the affected bone and the contralateral (unaffected) bone were calculated by summing the modulus-weighted area of each pixel within the bone contour by the position of the pixel relative to the centroid of the bone cross-section (Fig. 1). EA provides a measure of the bone's resistance to uniaxial loads; EI provides a measure of the bone's resistance to bending moments; and GJ provides a measure of the bone's resistance to torsional moments. A detailed account of rigidity calculations can be found in Supplementary Appendix A.
The primary outcome measure of the study was treatment recommendation (surgery vs. no surgery) as a function of the Mirels and CTRA methods. For the Mirels method, lesions with scores of 9 or higher were considered at high risk for fracture, whereas for the CTRA method, lesions with a reduction in EA, EI, or GJ greater than 33%, when compared with the contralateral control bones, were deemed at high risk for fracture. This threshold was determined from receiver operating characteristic (ROC) analyses conducted in a previous study of breast cancer patients with skeletal metastasis (24).
The McNemar test for a 2 × 2 contingency table was used to assess whether the CTRA and Mirels methods assigned a given patient to different treatment groups. To identify the predictors of the physicians' pre- and post-CTRA plans and to quantify the influence of the CTRA results on the physician's post-CTRA plan, logistic regression with a generalized estimating equations strategy (GEE) was used to establish the probability of assigning a patient to surgery with significance assessed by the Wald χ2 test (28). The covariates were patient age, source of metastasis, the four Mirels subcategories (lesion size, type, location, and patient pain level), and the SF-36 PCS. Odds ratios (OR) and 95% confidence intervals (CI) were calculated for significant multivariable predictors, and the c-index was used to assess the overall predictive accuracy of the multivariable models. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and positive and negative likelihood ratios (LLR+ and LLR−) of the Mirels and CTRA methods to predict pathologic fractures were calculated using 2 × 2 tables. ROC curve methodology was applied to determine the area under the curve (AUC) for both pre- and post-CTRA treatment plans and for predicting fracture based on the Mirels and CTRA methods (26). This approach has the advantage of incorporating covariates into the analysis and provides a more precise estimation of the AUC (29). The AUC's of the Mirels and CTRA methods were compared using the trapezoidal rule of Hanley and McNeil (30, 31).
Two independent readers (N. Calderon and J.A. Hipp) performed CTRA on a total of 10 lesions to determine the interobserver agreement of CTRA interpretations, which were recorded as a binary variable: at risk for fracture or not at risk for fracture. To determine intraobserver variability, CTRA of 10 lesions were performed two times by reader 1 (N. Calderon), with an interval of 1 year between readings, who was blinded to the previous interpretation. Interobserver and intraobserver agreements in the interpretation of CTRA results were determined by using Cohen κ statistics (32). A κ value of 0.20 or less indicated slight agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement; 0.61–0.80, substantial agreement; and 0.81–1.00, excellent agreement (33).
Statistical analysis was performed using the IBM SPSS Statistics software package (version 22.0, IBM). Two-tailed values of P < 0.05 were considered statistically significant. Power analysis indicated that the number of patients and number of events provided at least 80% power to capture differences of 20% or more in predictive accuracy (AUC or c-index) based on ROC curve analysis between CTRA and Mirels methods (version 7.0, nQuery Advisor, Statistical Solutions).
The Mirels criteria assigned 96 lesions (96/149; 64%) to the high-risk group (Mirels score > 9), whereas the physicians recommended surgery for 64 lesions (64/149; 43%), all part of the 96 lesions selected by the Mirels score (P < 0.001). Eighty-five patients (57%) did not undergo prophylactic stabilization, and 65 of those 85 patients were followed over the following 4-month period. Seven new fractures, all at the lesion sites, were reported during follow-up in 7 different patients. All 7 new fractures were correctly predicted to fracture using the CTRA method (100% sensitivity). Of the 58 lesions that did not fracture, CTRA predicted 52 not to fracture (90% specificity). However, only 5 of the 7 new fractures were correctly predicted to fracture using the Mirels method (71% sensitive; Table 3); and of the 58 lesions that did not fracture, the Mirels method predicted only 29 of them to not fracture (50% specific). Sensitivity was higher using CTRA (not significant due to the small number fractures, n = 7), and specificity was significantly higher using CTRA compared with the Mirels method (P = 0.002). The overall accuracy was 91% using the CTRA method and 52% with the Mirels method (Table 3 and Fig. 2).
Multivariable logistic regression modeling of the pre-CTRA plan confirmed that pain level (P < 0.001), lesion type (P < 0.001), and lesion size (P = 0.04) were significant predictors of the physician's initial plan (Table 4), with pain level as the strongest independent predictor of the physician's initial plan (OR, 9.2; 95% CI, 3.8–22.3 per 1-point increase). Lesion type was a significant predictor of the pre-CTRA plan as well, with lytic lesions having the highest and blastic lesions the lowest probability of being assigned to surgery (OR, 8.8; 95% CI, 2.7–28.2 per 1-point increase). Modeling the physician's post-CTRA plan revealed that CTRA and pain level were the only significant predictors (Table 4). Based on this model, CTRA was the most significant predictor of the physician's plan (OR, 118.1; 95% CI, 25.0–557.2), meaning that after controlling for the rest of the predictors, a positive CTRA report increased the probability of assigning a patient to surgery, by at least 25 times. Pain level (OR, 4.2; 95% CI, 2.0–8.9 per each 1-point increase) was also a significant predictor of the physician's post-CTRA plan.
Intraobserver agreement for CTRA analysis at our laboratory was excellent, with κ = 1 ± 0 (SEM; P < 0.01). Interobserver agreement between the two readers also showed excellent agreement, with κ = 1 ± 0 (SEM; P < 0.01) for fracture risk prediction with CTRA alone.
Prophylactic surgery can mitigate the pain and loss of function that occur after pathologic fractures. However, the morbidity and cost of surgical treatment would decrease with a more precise determination of fracture risk. Our goal was to compare the relative predictive values of the CTRA and the Mirels methods in fracture risk assessment. Furthermore, we sought to study the current clinical evaluation methodology among orthopedic oncologists that led to an initial treatment plan, and whether the CTRA data might contribute to this decision-making process.
Our results show that there is a discrepancy between the Mirels criteria and the physicians' pre-CTRA treatment plan in 1 of every 3 patients. If the Mirels method was the only factor to be considered in making a surgical decision, then 64% of cases (96 patients) would have been assigned to surgery. However, the enrolling physicians selected only 43% of cases (64 patients) for surgery as their initial plan, before receiving the CTRA data. Upon reviewing their clinical documentation for those initial plans, we found that both pain level and lesion type were significant predictors of their pre-CTRA plans.
Modeling the physician's post-CTRA plan revealed that only CTRA and pain level were significant predictors of the post-CTRA plan: when CTRA results indicated a reduction in EA, EI, or GJ of less than 33% when compared with the contralateral limb, the patients had a low probability of being assigned to surgery regardless of the pain level. However, if the CTRA indicated a reduction greater than 33% in EA, EI, or GJ, the probability of being assigned to surgery increases with the increasing level of pain from 1.9% (mild pain), to 14.5% (moderate pain), to 59.4% (severe pain).
The association of pain and fracture risk has been studied extensively in the literature. Keene and colleagues (16), in their retrospective study of proximal femoral metastases from breast cancer, indicated that pain was a nonreliable indicator of an impending fracture. In Mirels original series, 73% of the patients reported mild and moderate pain, whereas only 10% (6 of 57) developed a fracture, and all patients with functional pain (caused by mechanical weakness of the bone that can no longer support the normal stresses of daily activities) eventually fractured (17). This may be explained by the strong association between functional pain and lesion size in the Mirels study, as 90% of his patients with functional pain had a lesion size of larger than two thirds of the diameter of the bone. We did not observe the same association between pain level and the size of the lesion in our study (P = 0.14), and patients with different levels of pain had similar distributions of lesion sizes. Our results also show that, regardless of the evidence supporting the association of pain with fracture risk, pain is the most significant predictor of the physicians' pre-CTRA plan, and it remains the most important predictor after CTRA when the CTRA data are presented to the enrolling physicians. This may reflect concerns, apart from risk of fracture, by the enrolling physicians in assigning their patients to surgery or no surgery that warrant additional research.
The significance of the size and location of the lesions in modeling a physician's decision-making process may have been affected by the fact that almost half of the lesions (n = 73, 49%) were large, involving more than one third of the diameter of the bone, and the majority of them (n = 142, 95%) were located in the lower extremity. This reflects the characteristics of the patient population seen by orthopedic oncologists: These medical specialists may encounter patients who have metastatic bone lesions at a later stage in their disease than those patients seen by medical oncologists.
CTRA had a significantly better diagnostic performance than the Mirels score. However, we were unable to study the natural history of the metastatic lesions because of ethical considerations; surgeons felt obligated to treat a bone lesion if they suspected that the affected bone was at increased risk for fracture. Therefore, we could not evaluate the diagnostic performance of CTRA risk predictions in the whole cohort. Instead, we prospectively evaluated the diagnostic performance of both fracture risk assessment methods in the subgroup of patients that did not undergo prophylactic fixation. We acknowledge that this subgroup possessed a smaller risk of fracture than that of the general population, limiting the generalizability of the results. That said, it is safe to say that CTRA has a better sensitivity and specificity than the Mirels method in patients with a low pretest risk of fracture. This is particularly important, as it underlines the risk of performing unnecessary procedures in a high percentage of patients if the Mirels method is followed strictly. Furthermore, it means that CTRA could potentially be used as a screening method in patients who present early in the process of disseminated disease.
There are some limitations associated with the present study. First, the study population was enrolled at the time of consultation to an orthopedic oncologist. These patients, on average, are at an advanced stage in the disease process and as such limit the generalizability of the results. In addition, patient follow-up lasted only 4 months, which can be a limited period of time to identify all the possible pathologic fractures that could potentially present in the study cohort. However, the duration of follow-up was based on the consensus opinion of orthopedic oncologists who considered that tumor–host bone interactions change significantly after 4 months, providing an adequate time frame to identify a significant number of new fractures. This 4-month follow-up period has previously been used in pathologic fracture risk prediction studies (24). Nevertheless, further studies are necessary to establish the diagnostic performance of CTRA for delayed pathologic fractures. Finally, the orthopedic surgeons made the decision to operate by their own determinations, which means that not one single set of strict criteria was used to assign patients for surgery. We believe that the conditions in this study more accurately reflect the clinical scenario, where factors other than fracture risk scores (including the patients' preferences and autonomy and the clinician's personal expertise) were taken into account in the decision-making process.
This is the first prospective multicenter study that evaluates the impact of CTRA results on physicians' treatment plans for patients with appendicular metastatic lesions. The results of this study suggest that the existing gap, between clinical guidelines and physicians' recommendations, in the decision-making process for the selection of surgical or nonsurgical treatment must be narrowed by more advanced prognostic tools such as CTRA. Our ultimate goal is to expand this study to include additional institutes and subspecialties to evaluate the value of CTRA in a larger patient population who are at earlier stages of disease and therefore may derive greater benefit from CTRA as a prognostic tool.
Disclosure of Potential Conflicts of Interest
T.A. Damron is co-principal investigator on a Musculoskeletal Tumor Society study. No potential conflicts of interest were disclosed by the other authors.
Conception and design: A. Nazarian, V. Entezari, D. Zurakowski, A.J. Aboulafia, M.E. Anderson, M.C. Gebhardt, E.Y. Cheng, M. Yaszemski, T.A. Damron, B.D. Snyder
Development of methodology: A. Nazarian, V. Entezari, D. Zurakowski, J.A. Hipp, A.J. Aboulafia, M.C. Gebhardt, E.Y. Cheng, M. Yaszemski, B.D. Snyder
Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): A. Nazarian, V. Entezari, J.C. Villa-Camacho, P.P. Lin, F.H. Cheung, A.J. Aboulafia, R. Turcotte, M.E. Anderson, M.C. Gebhardt, E.Y. Cheng, R.M. Terek, T.A. Damron, B.D. Snyder
Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): A. Nazarian, V. Entezari, D. Zurakowski, N. Calderon, J.C. Villa-Camacho, P.P. Lin, A.J. Aboulafia, M.C. Gebhardt, E.Y. Cheng, R.M. Terek, M. Yaszemski, B.D. Snyder
Writing, review, and/or revision of the manuscript: A. Nazarian, V. Entezari, D. Zurakowski, J.A. Hipp, J.C. Villa-Camacho, A.J. Aboulafia, R. Turcotte, M.E. Anderson, M.C. Gebhardt, E.Y. Cheng, R.M. Terek, M. Yaszemski, T.A. Damron, B.D. Snyder
Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): A. Nazarian, V. Entezari, N. Calderon, P.P. Lin, A.J. Aboulafia, M.C. Gebhardt, B.D. Snyder
Study supervision: A. Nazarian, R. Turcotte, M.C. Gebhardt, M. Yaszemski, B.D. Snyder
Other (processed through local IRB): R. Turcotte
Other (PI for grant and developed the CTRA methodology): B.D. Snyder
The Musculoskeletal Tumor Society, Boston Children's Hospital Orthopaedic Surgery Foundation, and the National Institutes of Health T32 COMET Program (AR055885, to A. Nazarian), LRP (L30 AR056606, to A. Nazarian), and R01 (AR056212, to M. Yaszemski) provided financial support for this project.
The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
The authors thank the patients who volunteered to participate in this study and the staff of the enrolling physicians, particularly Ms. Tina Craig, for collecting and submitting all data for analysis. The authors are grateful to Dr. Russell Phillips, Chief of the Division of General Medicine and Primary Care at BIDMC, Drs. Jeffrey Katz and Elena Losina from the Department of Orthopedic Surgery and the Arthritis Center for Outcomes Research at BWH, and Dr. Ryan Porter from the Center for Advanced Orthopaedic Studies at BIDMC for their thoughtful comments and suggestions to further improve the article.
Note: Supplementary data for this article are available at Clinical Cancer Research Online (http://clincancerres.aacrjournals.org/).
- Received October 14, 2014.
- Revision received January 24, 2015.
- Accepted February 9, 2015.
- ©2015 American Association for Cancer Research.