Research Paper Volume 13, Issue 9 pp 13195—13210

Morphology-based radiomics signature: a novel determinant to identify multiple intracranial aneurysms rupture

Xin Tong1,2, *, , Xin Feng3, *, , Fei Peng1,2, , Hao Niu1,2, , Baorui Zhang1,2, , Fei Yuan1,2, , Weitao Jin4, , Zhongxue Wu1,2, , Yuanli Zhao4, , Aihua Liu1,2, , Daming Wang3, ,

  • 1 Beijing Neurosurgical Institute, Capital Medical University, Beijing, China
  • 2 Department of Interventional Neuroradiology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
  • 3 Department of Neurosurgery, Beijing Hospital, National Center of Gerontology, Graduate School of Peking Union Medical College, Beijing, China
  • 4 Department of Neurosurgery, Peking University International Hospital, Beijing, China
* Equal contribution

Received: August 3, 2020       Accepted: November 27, 2020       Published: May 10, 2021      

https://doi.org/10.18632/aging.203001
How to Cite

Copyright: © 2021 Tong et al. This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Abstract

We aimed to develop and validate a morphology-based radiomics signature nomogram for assessing the risk of intracranial aneurysm (IA) rupture. A total of 254 aneurysms in 105 patients with subarachnoid hemorrhage and multiple intracranial aneurysms from three centers were retrospectively reviewed and randomly divided into the derivation and validation cohorts. Radiomics morphological features were automatically extracted from digital subtraction angiography and selected by the least absolute shrinkage and selection operator algorithm to develop a radiomics signature. A radiomics signature-based nomogram was developed by incorporating the signature and traditional morphological features. The performance of calibration, discrimination, and clinical usefulness of the nomogram was assessed. Ten radiomics morphological features were selected to build the radiomics signature model, which showed better discrimination with an area under the curve (AUC) equal to 0.814 and 0.835 in the derivation and validation cohorts compared with 0.747 and 0.666 in the traditional model, which only include traditional morphological features. When radiomics signature and traditional morphological features were combined, the AUC increased to 0.842 and 0.849 in the derivation and validation cohorts, thus showing better performance in assessing aneurysm rupture risk. This novel model could be useful for decision-making and risk stratification for patients with IAs.

Introduction

Multiple intracranial aneurysms (MIAs) are encountered in approximately 20.1% of patients harboring unruptured IAs [1]. When a patient with unruptured MIAs is detected, determining which of the aneurysm is most likely to rupture is critical for determining the treatment strategy. For patients with both aneurysmal subarachnoid hemorrhage (aSAH) and MIAs, correct identification of the ruptured IA is also critical for planning treatment. After the IAST (International Subarachnoid Aneurysm Trial) research outcomes were reported, endovascular treatment has become the treatment choice for most ruptured IAs [2]. Therefore, rupture discrimination of MIAs is increasingly important because unlike in microsurgery, one cannot visually confirm the rupture in endovascular treatment.

Morphological parameters such as size [35], aspect ratio (AR) [69], size ratio (SR) [1012], irregular shape [68], and flow angles [13, 14] have been suggested to be significant risk factors for IA rupture. However, studies on these morphological characteristics have reported conflicting results. This can be explained by the fact that these parameters are usually manually measured using different imaging techniques and measurement methodologies, and could differ among evaluators. In this study, we introduced objective and quantitative morphological features extracted from radiomics.

As an emerging branch of artificial intelligence, radiomics has been used to automatically extract a large number of objective and quantitative imaging features to select those most significantly associated clinical, pathological, molecular, and genetic characteristics so as to improve the diagnostic and prognostic accuracy and the evaluation of therapeutic efficacy [1517]. There are only a few reports related to the application of radiomics-extracted morphological features in the detection of IAs and the stability of assessing small IAs [18, 19]. To the best of our knowledge, there are no reports illustrating whether a morphology-based radiomics signature will enable the superior assessment of the risks of IA rupture.

In this study, a subgroup of the patients from 3 centers with MIAs who developed aSAH were analyzed to characterize radiomics-extracted and traditional morphological features of the ruptured IA relative to other concomitant unruptured IAs. This was done to avoid confounding patient-specific characteristics [6, 7, 20]. We then developed and validated a combined nomogram that incorporates both the radiomics signature and traditional morphological features for the assessment of the individual risk of IA rupture in patients with MIAs.

Materials and Methods

Ethical statement

This study was approved by the institutional research ethics boards of each center and conducted in accordance with the ethical standards and according to the 1964 Declaration of Helsinki.

Study population

We retrospectively collected images and medical records from a consecutive series of patients with both aSAH and MIA treated at 3 hospitals in China (Beijing Tiantan Hospital, Beijing Hospital, and Peking University International Hospital) from January 2016 to December 2018. The inclusion criteria were as follows: patients with the diagnosis of aSAH and having at least two saccular IAs; patients with 3D-DSA diagnosed by Siemens Artis Zee System (Siemens Healthcare, Erlangen, Germany); sufficient image quality for segmentation; and patients with available clinical information. The ruptured IA was confirmed through microscopic visual assessment for patients with craniotomy treatment or through a definitive hemorrhage pattern on CT for patients who underwent endovascular or no treatment [21] (Figure 1). Patients in whom the ruptured IA could not be confirmed and those with other cerebrovascular diseases were excluded. Patients with aneurysm that has un-clear neck from the parent vessel were also excluded. All aneurysms included were randomly divided into 2 cohorts: the derivation cohort (70% of the IAs) and the validation cohort (30% of the IAs).

Definitive hemorrhage pattern to confirm the ruptured aneurysm for patients who underwent endovascular or no treatment. A 67-year-old woman presented with subarachnoid hemorrhage (A) was found to have left and right internal carotid aneurysms (B–D). The ruptured aneurysm is the right internal carotid aneurysm.

Figure 1. Definitive hemorrhage pattern to confirm the ruptured aneurysm for patients who underwent endovascular or no treatment. A 67-year-old woman presented with subarachnoid hemorrhage (A) was found to have left and right internal carotid aneurysms (BD). The ruptured aneurysm is the right internal carotid aneurysm.

Aneurysm segmentation and radiomics morphological features extraction

The DSA image of each IA was reconstructed using an open-source software 3D Slicer [22] (version 4.9.0; http://www.slicer.org). In this study, the IA was defined as the region of interest (ROI). After reconstruction, IAs were manually segmented from the parent vessel at the neck of the IA [18, 19]. Two researchers (X.T. and X.F.) performed aneurysm segmentation. Disparities regarding the aneurysm reconstruction and segmentation between the researchers were further evaluated by 2 senior neurointerventionists each with 15 years of experience (AH.L. and DM.W.). The extraction of radiomics morphological features for each IA was completed with PyRadiomics in Python [16, 17, 19, 2224], an open-source platform capable of extracting a large panel of morphological features from medical images. This radiomic quantification platform sets the reference standardization for feature definition and image processing [22]. In this study, 17 radiomics morphological features were extracted (Table 1). Detailed information on these features can be found at https://pyradiomics.readthedocs.io. A flow chart detailing the image processing operations performed in the present study is illustrated in Figure 2.

Table 1. Characteristics of the unruptured and ruptured IAs.

Characteristic,
mean±SD or Num(%)
Derivation cohort (n=176)Validation cohort (n=78)
Unruptured IA
(n=113)
Ruptured IA
(n=63)
PUnruptured IA
(n=35)
Ruptured IA
(n=43)
P
Traditional morphological features
Size (mm)3.8±1.45.6±2.6<0.0014.2±1.85.5±2.10.007
Neck (mm)3.4±1.03.8±1.30.0203.6±1.73.9±1.20.357
Height (mm)3.3±1.24.9±2.3<0.0013.4±1.34.8±1.9<0.001
Width (mm)3.4±1.14.4±1.8<0.0013.7±1.84.6±1.70.034
Diameter of parent artery (mm)3.3±1.03.2±0.90.3833.0±1.03.2±0.80.462
Branching to parent ratio1.2±0.31.2±0.40.3871.2±0.21.4±0.40.010
Neck to parent ratio1.1±0.51.3±0.60.0351.2±0.71.3±0.60.599
Aspect ratio1.0±0.41.3±0.6<0.0011.1±0.51.3±0.60.132
Size ratio1.0±0.51.5±0.8<0.0011.3±0.81.6±0.70.064
Width-Height ratio1.1±0.41.0±0.40.1551.1±0.31.0±0.30.353
Posterior circulation location16(14.2)12(19.0)0.3956(17.1)9(20.9)0.673
Irregular shape33(29.2)34(54.0)0.00110(28.6)23(53.5)0.027
Bifurcation aneurysm35(31.0)27(42.9)0.11411(31.4)24(55.8)0.031
Inflow angle (°)105.5±24.7110.8±27.60.190110.3±26.9109.7±29.10.927
Outflow angle (°)97.7±26.992.9±27.60.266102.6±29.296.5±25.80.329
Branching angle (°)134.6±23.7130.0±24.20.217124.9±21.2127.5±26.80.635
Radiomics morphological features
Elongation0.802±0.1260.671±0.133<0.0010.811±0.1120.717±0.1180.001
Flatness0.676±0.1160.560±0.121<0.0010.690±0.1130.589±0.107<0.001
Compactness 10.039±0.0030.036±0.004<0.0010.040±0.0030.038±0.003<0.001
Compactness 20.544±0.0750.470±0.104<0.0010.571±0.0760.506±0.078<0.001
Sphericity0.814±0.0390.773±0.062<0.0010.828±0.0390.794±0.0430.001
Surface volume Ratio2.069±0.5801.867±0.6280.0332.009±0.6331.743±0.5920.060
Spherical disproportion1.231±0.0641.303±0.117<0.0011.210±0.0601.263±0.0720.001
Minor axis length3.502±0.9094.130±1.5230.0013.575±1.0144.292±1.3430.011
Least axis length2.953±0.8063.467±1.3420.0023.055±0.9543.572±1.3160.056
Major axis length4.478±1.4186.390±2.586<0.0014.539±1.7836.145±2.1830.001
Surface area61.371±37.292103.555±82.469<0.00165.908±48.541104.050±71.9510.009
Voxel volume37.540±36.00980.367±100.824<0.00144.097±51.71882.383±83.1450.020
Mesh volume36.675±35.81579.223±100.414<0.00143.417±51.35381.385±82.9540.021
Maximum 2D diameter column4.792±1.4296.233±2.292<0.0014.806±1.6236.257±2.1570.002
Maximum 2D diameter row4.759±1.3586.245±2.726<0.0014.976±1.5906.194±2.3690.011
Maximum 2D diameter slice4.920±1.4826.301±2.225<0.0014.894±1.5326.365±2.3910.002
Maximum 3D diameter5.394±1.6317.843±4.553<0.0015.458±1.9587.371±2.7000.001
Radiomics signature
Rad-score-1.020±1.1180.730±2.004<0.001-1.123±1.1490.327±1.245<0.001
Flow chart of the study. The aneurysm was reconstructed from DSA images and using 3D slicer. The segmentation was performed by threshold and checked layer by layer. Then, the segmented label map and volume files were entered in the Pyradiomics package in the Python platform, and 17 radiomics morphological features were extracted for each aneurysm. The least absolute shrinkage and selection operator binary logistic were used to select the potential assessment factors and develop a radiomics signature. Along the radiomics morphological features, 16 traditional morphological features were combined and entered in the model construction analysis. Finally, the optimal model was performed in the nomogram.

Figure 2. Flow chart of the study. The aneurysm was reconstructed from DSA images and using 3D slicer. The segmentation was performed by threshold and checked layer by layer. Then, the segmented label map and volume files were entered in the Pyradiomics package in the Python platform, and 17 radiomics morphological features were extracted for each aneurysm. The least absolute shrinkage and selection operator binary logistic were used to select the potential assessment factors and develop a radiomics signature. Along the radiomics morphological features, 16 traditional morphological features were combined and entered in the model construction analysis. Finally, the optimal model was performed in the nomogram.

Traditional morphological data acquisition

Traditional morphological features recorded from the angiograms included the IA number, location, size, neck, weight, inflow angle, outflow angle, branching angle, and diameters of the parent and branch vessels (Figure 3). Aneurysms with daughter sacs, multiple lobes, or other types of wall protrusions were defined as irregular. NPR, BPR, AR, SR, and WH ratio were calculated (Figure 3). All angiograms of IAs were re-evaluated and measured by two authors (X.F. and X.T.) and confirmed by two senior neurointerventionists (AH.L. and DM.W.).

Measurements of the traditional morphological features. (II-A) schematically shows the aneurysm size and diameters of parent and branch vessels measurements, with the height (a), width (b), neck width (c), the diameter of the parent vessel (d), diameters of the branching vessel (e and f). (II-B) shows the angle measurements. The outflow angle (A) was defined as the angle at which the aneurysm flows outward to the distal parent artery in the sidewall aneurysm or to the daughter branch most approaching 180° in the bifurcation aneurysm. The inflow angle (B) was defined as the angle from the parent artery into the aneurysm. The main branching angle (C) was defined as the angle of the parent artery in the sidewall aneurysm or the angle between the parent artery and the daughter branch most approaching 180° in bifurcation aneurysm. In addition, several indicators were calculated: aspect ratio (AR) was defined as the ratio of aneurysm height (a) to the neck width (c); size ratio (SR) was defined as the ratio of aneurysm height (a) to the parent vessel diameter (d); WH ratio was defined as the ratio of aneurysm width (b) to the height (a); branching to parent ratio (BPR) was defined as the ratio of the sum of the diameters of the branch vessels (e + f) to the diameter of the parent artery (d) (in case of a sidewall aneurysm, the BPR was set to 1); neck to parent ratio (NPR) was defined as the ratio of the aneurysm neck width (c) to the parent artery diameter (d).

Figure 3. Measurements of the traditional morphological features. (II-A) schematically shows the aneurysm size and diameters of parent and branch vessels measurements, with the height (a), width (b), neck width (c), the diameter of the parent vessel (d), diameters of the branching vessel (e and f). (II-B) shows the angle measurements. The outflow angle (A) was defined as the angle at which the aneurysm flows outward to the distal parent artery in the sidewall aneurysm or to the daughter branch most approaching 180° in the bifurcation aneurysm. The inflow angle (B) was defined as the angle from the parent artery into the aneurysm. The main branching angle (C) was defined as the angle of the parent artery in the sidewall aneurysm or the angle between the parent artery and the daughter branch most approaching 180° in bifurcation aneurysm. In addition, several indicators were calculated: aspect ratio (AR) was defined as the ratio of aneurysm height (a) to the neck width (c); size ratio (SR) was defined as the ratio of aneurysm height (a) to the parent vessel diameter (d); WH ratio was defined as the ratio of aneurysm width (b) to the height (a); branching to parent ratio (BPR) was defined as the ratio of the sum of the diameters of the branch vessels (e + f) to the diameter of the parent artery (d) (in case of a sidewall aneurysm, the BPR was set to 1); neck to parent ratio (NPR) was defined as the ratio of the aneurysm neck width (c) to the parent artery diameter (d).

Statistical analysis

The continuous variables of patients’ baseline characteristics are presented as mean ± SD, and categorical variables are presented as percentages. All analyses were performed with IBM SPSS Statistics for Windows, version 25.0 (IBM Corp., Armonk, N.Y., USA) and R software (R Foundation for Statistical Computing, Vienna, Austria.). The major R software packages used in this study are listed in Supplementary Table 1. Statistical significance was set at p < 0.05.

Feature selection and morphology-based radiomics signature construction

The assessment analysis included 16 traditional morphological features and 17 radiomics morphological features. Each traditional morphological feature with a probability value < 0.20 in the univariate analysis was included in the multivariate logistic regression analysis.

We used the LASSO algorithm to select the most discriminative radiomics morphological features from the derivation cohort. A rad-score was constructed for each IA from a linear combination of selected features that were weighted based on their respective LASSO coefficients (including both coefficient <0 and >0). The potential association of this morphology-based radiomics signature with IA rupture was assessed in the derivation and validation cohort by using a Mann-Whitney U test. Diagnostic validation was also performed in the derivation and the validation cohort.

Development of aneurysm rupture assessment models

In this study, we constructed 3 models by the multivariate logistic regression analysis: 1) the MRS model, which included the radiomics signature and traditional morphological features, 2) the MRF model, which included the radiomics and traditional morphological features, and 3) the MTF model, which included traditional morphological features only. VIFs (< 2 were considered insignificant) were determined to evaluate the collinearity of combinations of final variables in models.

Nomogram model selection and performance assessment

The performance of the models was tested in the derivation and validation cohorts. We used AUCs to compare the discriminatory efficacy of the different models. Based on these analyses, the optimal model was developed into the nomogram. The discrimination accuracy and the calibration of the nomogram were assessed using AUCs, calibration curves, and the Hosmer-Lemeshow test. The Brier score, ranging from 0 (excellent discriminate ability) to 1 (worst discriminate ability), was used to determine the overall performance of each model.

Clinical usefulness

DCA evaluates assessment models and visualizes the net benefit derived from the use of a specific assessment model. Thus, we conducted DCA to determine the clinical usefulness of the nomogram in this study.

Results

Study population

A total of 254 IAs in 105 patients were included, including 148 unruptured IAs and 106 ruptured IAs (one patient had two ruptured IAs 12 months apart). Among the 105 patients, 75 (71.4%) were women. The median age of the patients was 58.5 years (range, 31 - 85 years). Current smoking and hypertension were present in 26 (24.8%) and 64 (61.9%) patients, respectively. (Supplementary Table 2) Ruptured IA was not the largest IA was found in 32 (30.5%) patients. In this study, 176 IAs were randomized into the derivation subset, and 78 IAs were randomized into the validation subset. Details of IAs in the derivation and validation subsets are shown in Table 1.

Features selection and morphology-based radiomics signature construction

The comparison of traditional morphological and radiomics features between ruptured and unruptured aneurysms is summarized in Table 1. The eleven traditional morphological features in the derivation cohort with p < 0.20 that were analyzed using multivariable logistic regression included: size, neck, height, width, AR, SR, irregular shape, neck, branching to parent ratio (BPR), neck to parent ratio (NPR), and bifurcation.

Ten radiomics morphological features were identified as potential indicators, including surface volume ratio, spherical disproportion, maximum 3D diameter, maximum 2D diameter slice, major axis length, least axis length, maximum 2D diameter column, maximum 2D diameter row, elongation, and flatness. These ten features had coefficients > 0 in the least absolute shrinkage and selection operator (LASSO) logistic regression model and were included in the radiomics signature score (rad-score) calculation formula (Figure 4).

Radiomics signature score (rad-score) calculation. (A) Radiomic features ranked by coefficients of the least absolute shrinkage and selection operator (LASSO) binary logistic regression model. The flatness was the most correlated indicator with IA rupture. (B) Radiomics signature (rad-score) was constructed from a linear combination of selected features that were weighted based on their respective LASSO coefficients.

Figure 4. Radiomics signature score (rad-score) calculation. (A) Radiomic features ranked by coefficients of the least absolute shrinkage and selection operator (LASSO) binary logistic regression model. The flatness was the most correlated indicator with IA rupture. (B) Radiomics signature (rad-score) was constructed from a linear combination of selected features that were weighted based on their respective LASSO coefficients.

Performance of the morphology-based radiomics signature

The rad-scores of the ruptured group were significantly higher than that of the unruptured group in both the derivation and validation cohorts (p < 0.001, Table 1, Figure 5). A significant association between the rad-score and IA rupture was also found in the IAs at a different location (Supplementary Table 3). The radiomics signature yielded areas under the curve (AUCs) of 0.814 (CI 95%, 0.746-0.881) in the derivation cohort and 0.835 (CI 95%, 0.739-0.930) in the validation cohort (Figure 6).

Violin plots of the radiomics signature score (rad-score). There was a significant difference in the rad-score between unruptured IA and ruptured IA in the derivation cohort (p A), which was then confirmed in the validation cohort (p B).

Figure 5. Violin plots of the radiomics signature score (rad-score). There was a significant difference in the rad-score between unruptured IA and ruptured IA in the derivation cohort (p < 0.001, III-A), which was then confirmed in the validation cohort (p < 0.001, III-B).

Rad-score for every aneurysm in each in the derivation (A) and validation cohort (B).

Figure 6. Rad-score for every aneurysm in each in the derivation (A) and validation cohort (B).

Individualized nomogram model development and selection

Multivariate logistic regression analysis yielded 3 rupture discriminations models:

the morphology-based radiomics signature (MRS) model, which include radiomics signature and traditional morphological features,

odds MRS = [0.90(Radiomics Signature) + 0.79(SR) + 1.23(BPR) – 2.84];

the morphology-based radiomics features (MRF) model, which include radiomics features and traditional morphological features,

odds MRF = [0.26(Maximum 3D Diameter) +0.88 (SR) - 5.89(Flatness) + 0.31]; and

the morphology-based traditional features (MTF) model, which include only traditional morphological features,

odds MTF = [1.66 (AR) + 0.96 (NPR) + 0.73 (Irregular shape) – 3.90];

where odds are the ratios of the probability of ruptured status to the probability of unruptured status of an IA.

Independently significant discriminants of the MRS model included radiomics signature, SR, and BPR. Discriminants of the MRF model included flatness, maximum 3D diameter, and SR. Discriminants of the MTF model include AR, NPR, and irregular shape (Table 2). We found that only SR was incorporated in both the MRS model (OR, 3.430; CI 95%, 1.275-4.521) and the MRF model (OR, 2.400; CI 95%, 1.226-4.900).

Table 2. Multivariate analysis for patient with aSAH and MIAs.

ModelRisk factorsOdds ratio (95% CI)PVIF
MRS
Model
Radiomics Signature2.466(1.756-3.618)0.0031.116
SR3.430(1.275-4.521)0.0231.073
BPR3.430(1.275-9.890)0.0181.056
MRF
Model
Flatness0.003(<0.001-0.0600)<0.0011.044
Maximum 3D Diameter1.359(0.109-16.488)0.0101.307
SR2.400(1.226-4.900)0.0131.295
MTF
Model
AR5.240(2.451-12.099)<0.0011.149
NPR2.621(1.373-5.257)0.0041.144
Irregular shape2.078(1.036-4.173)0.0391.022
MRS model, morphology-based radiomics signature model; MRF model, morphology-based radiomics features model; MTF model, morphology-based traditional features model; SR, size ratio; AR, aspect ratio; NPR, neck to parent diameter ratio; VIF, variation inflation factors.

For all three models, the variance inflation factors (VIFs) of all the candidate indicators ranged from 1.050–1.307, demonstrating that there was no collinearity in any of the three models. The MRS model (AUC, 0.842; 95% CIs, 0.786-0.899) assessed the risk of aneurysm rupture in the derivation cohort more accurately than the MTF model (AUC, 0.747; 95% CIs, 0.673-0.822) or the MRF model (AUC, 0.820; 95% CIs, 0.755-0.885) (Figure 7). In the validation cohort, the MRS model (AUC, 0.849, 95% CIs, 0.752-0.946) also had the largest AUC compared with the MTF (AUC, 0.666; 95% CIs, 0.539-0.793) and MRF models (AUC, 0.799; 95% CIs, 0.697-0.902). As the optimal model, the MRS model was developed as the nomogram (Figure 8A).

The area under the curves (AUCs) shows that the morphology-based radiomics signature model (A) has better discrimination compared with the morphology-based radiomics features model (B) and morphology-based radiomics features model (C). Radiomics morphological feature selection used the LASSO binary logistic regression model (D).

Figure 7. The area under the curves (AUCs) shows that the morphology-based radiomics signature model (A) has better discrimination compared with the morphology-based radiomics features model (B) and morphology-based radiomics features model (C). Radiomics morphological feature selection used the LASSO binary logistic regression model (D).

The morphology-based radiomics signature model was developed into nomogram (A). Calibration curves suggest that our nomogram performed well in both the derivation (B) and validation (C) cohorts.

Figure 8. The morphology-based radiomics signature model was developed into nomogram (A). Calibration curves suggest that our nomogram performed well in both the derivation (B) and validation (C) cohorts.

Nomogram model validation and performance assessment

The AUC of 0.842 in the derivation cohort and 0.849 in the validation cohort showed good discrimination by the nomogram. Brier scores of 0.164 and 0.168 showed good overall performance of the MRS model in the derivation and validation cohorts, respectively (Table 3). The calibration curve of the nomogram demonstrated good agreement between estimating and observation results (Figure 8). The Hosmer-Lemeshow test of the nomogram in the derivation cohort (p = 0.782) showed no departure from perfect fit, which was confirmed in the validation cohort (p = 1.000).

Table 3. Evaluation of discrimination and calibration abilities of the models.

MRS modelMRF modelMTF model
Derivation cohortValidation cohortTotal cohortDerivation cohortValidation cohortTotal cohortDerivation cohortValidation cohortTotal cohort
Brier score0.1600.1370.1590.1620.1740.1730.1900.2270.208
AUC (95% CI)0.842
(0.786-0.899)
0.849
(0.752-0.946)
0.840
(0.790-0.890)
0.820
(0.755-0.885)
0.799
(0.697-0.902)
0.816
(0.764-0.869)
0.747
(0.673-0.822)
0.666
(0.539-0.793)
0.729
(0.667-0.791)
Hosmer-Lemeshow test (p value)0.3661.0001.0000.0980.4940.9500.1390.1710.954
AUC, area under the receiver operating characteristic curve; MRS model, morphology-based radiomics signature model; MRF model, morphology-based radiomics features model; MTF model, morphology-based traditional features model.

Clinical usefulness

Compared to the MRF and MTF models, the MRS model showed larger net benefits in the decision curve analysis (DCA) (Figure 9). The DCA showed that if the probability of IA rupture generated by the MRS nomogram model is between 0.10–0.80, the use of nomogram to assess the IA rupture risk adds more benefit than either of the treat-all or the treat-none strategies (Figure 9).

The decision curve analysis demonstrates the morphology-based radiomics signature model (MRS model) has a larger net benefit compared with the morphology-based radiomics features model (MRF model) and morphology-based traditional features model (MTF model) for the assessment of aneurysm rupture risk.

Figure 9. The decision curve analysis demonstrates the morphology-based radiomics signature model (MRS model) has a larger net benefit compared with the morphology-based radiomics features model (MRF model) and morphology-based traditional features model (MTF model) for the assessment of aneurysm rupture risk.

Discussion

Our study has important implications for clinical practice. To our knowledge, this study was the first to attempt to establish a reliable morphology-based radiomics signature nomogram to assess the risk of IA rupture in MIA patients. This nomogram incorporated both radiomics signature and traditional morphological features and showed larger benefit gains in assessing the risk of IA rupture compared with models based on radiomics or traditional morphological features.

Radiomics signature was significantly associated with MIA rupture

Previous studies have shown that IA size is the most important morphological risk factor for ruptures [35, 25]. However, other studies showed that in 12.5%–33% of patients with MIA, the ruptured IA was not the largest one [6, 7]. In our study, similarly, this was 30.5%. An irregular shape has also been reported as one of the most important factors for predicting IA rupture [68]. Some studies defined IA with multiple lobes, daughter sacs, or other types of wall protrusions as irregular [57, 26]. Such descriptions are qualitative and rely on the evaluator's experience with an ill-defined threshold. A few other studies used morphology indexes such as AR [69], SR [1012], and flow angles [13, 14] for IA rupture risk assessment. However, these studies reported conflicting results. This may be because traditional morphological features were manually measured using different imaging techniques and measurement methodologies, which could differ among evaluators. Furthermore, compared with the radiomic morphological features automatically obtained based on artificial intelligence, traditional morphological features that are mainly measured in a 2-dimensional projection are not sufficient to judge the overall morphology of the IA. All these contradictions emphasize the need for a new tool to assess aneurysm rupture risk.

In this study, we introduced radiomics morphological features and developed a morphology-based radiomics signature nomogram model in patients with aSAH and MIA. We used LASSO regression to select potential indicators and combined them via a linear equation weighted by each indicator's respective coefficients to construct the radiomics signature. This method allowed us to incorporate individual radiomics morphological features into a feature panel in order to do multi-feature analyses and it has been used in several recent studies [17, 27, 28]. We found that the rad-scores of the ruptured group were significantly higher than that of the unruptured group in both derivation and validation sets (Figures 5, 6). More importantly, our result shows that the morphology-based radiomics signature performs well in assessing ruptured aneurysms in patients with SAH and MIAs, with an AUC of 0.814 in the derivation cohort and 0.835 in the validation cohort (Figure 6). After grouping by location, the radiomics signature still showed a better stratified ability for detecting IAs at different locations (Supplementary Table 3). Furthermore, the radiomics signature had better assessment accuracy than the MTF model based on traditional morphological features only (AUC, 0.814 vs. 0.747). These results confirm that the morphology-based radiomics signature, as a new indicator established by radiomics and machine learning, has great values in IA rupture risk assessment.

Development and validation of morphology-based radiomics signature nomogram

Several methods, such as clinical risk factor assessments and scoring systems, have been used in IA stratification in clinical practice [4, 5, 29]. However, the usage of these clinical tools in clinical practice remains controversial [3032], especially for patients with MIAs. This study intended to establish a relatively accurate, convenient, and noninvasive method for assessing IA rupture based on radiomics-extracted morphological features. Therefore, we established three different models and selected the optimal model to develop a nomogram. As a result, we found that the MRS model (AUC, 0.842), which incorporated radiomics signature and traditional morphological features, was more accurate in IA risk assessment than the MRF model (AUC, 0.820) and the MTF model (AUC, 0.747). The DCA also showed that the MRS model was associated with larger net benefits than the MRF and MTF models. Our study results with those mentioned above indicate that the combination of radiomics-extracted and clinical traditional morphological variables has a complementary and synergistic effect in assessing IA rupture in patients with MIA.

In this study, we found that all three models had some shared and some distinct characters. All models included features that reflect the shape of the IA, such as radiomics signature in the MRS model, flatness in the MRF model, as well as AR and irregular shape in the MTF model. An irregular shape has been reported as one of the most important factors for predicting IA rupture [68]. These geometries tend to harbor vortices, which may promote the infiltration of inflammatory cells [33]. Moreover, the local disturbed flow pattern coursed by the irregular shape of IAs may be more vulnerable to form thrombus, which could aggravate inflammatory cell infiltration and the proteolytic degradation of the IA wall [3436]. In the MTF model, AR and irregular shape, as typical morphological features, have been widely studied and correlated with IA rupture [69]. In the MRF model, the flatness was a novel morphological radiomics index, which shows the relationship between the largest and smallest principal components in shape [1719]. Liu et al. identified flatness as the most important determinant to predict small IA stability [19]. In our study, LASSO binary logistic regression analysis showed that flatness was also the most correlated indicator for IA rupture (Figure 4). Compared to AR, NPR, and irregular shape, flatness may be a more objective and reliable way to describe the IA morphology. In the MRS model, the radiomics signature reflected the overall morphology of the aneurysm more comprehensively than a single traditional or radiomics morphological feature. Interestingly, NPR was included in the MTF model, and SR was included in the MRS and MRF model. NPR is defined as the ratio of the neck width to the parent artery diameter, which is affected by the IA location and parent vessel. SR was found to be a more robust morphometric feature than AR or size, which can also be affected by the location and parent vessel of the IA [1012]. In addition, maximum 3D diameter, which was included in the MRF model, represented the largest diameter of the IA in 3 dimensions and could be more accurate than traditional size measurements relying on manual measurement by the naked eye [1719].

Previous studies have linked IA rupture and radiomics features. Liu et al. attempted to predict aneurysm stability with radiomics morphological features. However, 13.7% in their case group were growing or symptomatic unruptured aneurysm, so it was not suitable for identifying IA ruptured in MIA patients [19]. In addition to morphological features, more radiomic features could be extracted from CT or MR imaging. However, this is a technical challenge for IA because IA is a cystic vascular disease, unlike other substantial lesions such as tumors. In fact, as the gold standard for diagnosis and evaluation of IA [37]. 3D digital subtraction angiography (3D-DSA) reconstruction mainly reflects the morphological characteristics of the aneurysm and parent artery, which are very important factors affecting the risk of aneurysm rupture. Therefore, we only extracted morphological-based radiomics features from 3D-DSA. Further research will be conducted using on high-resolution magnetic resonance images to extract more imaging features (such as grayscale) other than the morphological factors.

Strengths and limitations

Our study was specifically designed to yield accurate and reliable results. Radiomics is the mining of quantitative and objective image parameters and the evaluation of IA morphological characteristics in multiple dimensions. We included patients who had both ruptured and unruptured IAs. Similarly, three previous articles have also studied this subgroup of patients [7, 9, 38]. All IAs were exposed to the same internal milieu within this study population as patient-specific parameters and standardized by their own internal controls. This makes our model more accurate [7, 9, 38]. Furthermore, we used a nomogram to develop our IA rupture risk assessment tool. Nomograms have been widely accepted as reliable tools to determine quantitative risk factors for clinical events.

This study has some limitations. First, our study was based on post IA rupture imaging, which may not accurately reflect the original morphology of existing IAs. This shortcoming may be further magnified by the accuracy of the radiomic morphological feature extraction. Second, the retrospective design of our study may have introduced some bias, and we did not have an external validation. The number of cases in the verification group was relatively small (76 IAs). However, because only 2 or 3 variables were used in each model, this sample size was statistically satisfactory. Third, this research is based on semi-automatic segmentation procession, and manual segmentation of IA neck may introduce discrepancy. Fourth, incorporated both radiomics signature and traditional morphological features, our model does not fully cover the shortcomings of manual extraction of parameters, so further research needs to consider fully automatic extraction of all aneurysm morphological and radiomic parameters. Lastly, models in this study did not include factors such as hemodynamics [20], genetics [39], and wall enhancement [40]. The inclusion of these parameters may improve the assessment accuracy of the model but may also increase the complexity of the model and limit its clinical application [25].

Conclusions

To the best of our knowledge, this study was the first attempt to establish a reliable morphology-based radiomics signature nomogram to assess the risk of IA rupture in MIA patients. This nomogram may provide a relatively accurate, convenient, and noninvasive method for the decision-making and risk stratification for patients with MIAs.

Availability of data and material

Some or all data, models, or code generated or used during the study are available from the corresponding author by request.

Consent for participation and publication

Informed consents for participation and publication were obtained from all participants.

Supplementary Materials

Supplementary Tables

Acknowledgments

We thank Yuesong Pan for the analysis and interpretation of data; Yunyun Xiong for advice and critical discussion.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Funding

This work was supported by the Beijing Municipal Administration of Hospitals' Ascent Plan (DFL20190501), the Non-profit Central Research Institute Fund of Chinese Academy of Medical Sciences (No. 2019TX320002), and the Natural Science Foundation of China (No. 81771233).

References

  • 1. Jabbarli R, Dinger TF, Darkwah Oppong M, Pierscianek D, Dammann P, Wrede KH, Kaier K, Köhrmann M, Forsting M, Kleinschnitz C, Sure U. Risk Factors for and Clinical Consequences of Multiple Intracranial Aneurysms: A Systematic Review and Meta-Analysis. Stroke. 2018; 49:848–55. https://doi.org/10.1161/STROKEAHA.117.020342 [PubMed]
  • 2. Molyneux AJ, Kerr RS, Yu LM, Clarke M, Sneade M, Yarnold JA, Sandercock P, and International Subarachnoid Aneurysm Trial (ISAT) Collaborative Group. International subarachnoid aneurysm trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomised comparison of effects on survival, dependency, seizures, rebleeding, subgroups, and aneurysm occlusion. Lancet. 2005; 366:809–17. https://doi.org/10.1016/S0140-6736(05)67214-5 [PubMed]
  • 3. Wiebers DO, Whisnant JP, Huston J 3rd, Meissner I, Brown RD Jr, Piepgras DG, Forbes GS, Thielen K, Nichols D, O’Fallon WM, Peacock J, Jaeger L, Kassell NF, et al, and International Study of Unruptured Intracranial Aneurysms Investigators. Unruptured intracranial aneurysms: natural history, clinical outcome, and risks of surgical and endovascular treatment. Lancet. 2003; 362:103–10. https://doi.org/10.1016/s0140-6736(03)13860-3 [PubMed]
  • 4. Greving JP, Wermer MJ, Brown RD Jr, Morita A, Juvela S, Yonekura M, Ishibashi T, Torner JC, Nakayama T, Rinkel GJ, Algra A. Development of the PHASES score for prediction of risk of rupture of intracranial aneurysms: a pooled analysis of six prospective cohort studies. Lancet Neurol. 2014; 13:59–66. https://doi.org/10.1016/S1474-4422(13)70263-1 [PubMed]
  • 5. Morita A, Kirino T, Hashi K, Aoki N, Fukuhara S, Hashimoto N, Nakayama T, Sakai M, Teramoto A, Tominari S, Yoshimoto T, and UCAS Japan Investigators. The natural course of unruptured cerebral aneurysms in a Japanese cohort. N Engl J Med. 2012; 366:2474–82. https://doi.org/10.1056/NEJMoa1113260 [PubMed]
  • 6. Björkman J, Frösen J, Tähtinen O, Backes D, Huttunen T, Harju J, Huttunen J, Kurki MI, von Und Zu Fraunberg M, Koivisto T, Manninen H, Jääskeläinen JE, Lindgren AE. Irregular Shape Identifies Ruptured Intracranial Aneurysm in Subarachnoid Hemorrhage Patients With Multiple Aneurysms. Stroke. 2017; 48:1986–89. https://doi.org/10.1161/STROKEAHA.117.017147 [PubMed]
  • 7. Backes D, Vergouwen MD, Velthuis BK, van der Schaaf IC, Bor AS, Algra A, Rinkel GJ. Difference in aneurysm characteristics between ruptured and unruptured aneurysms in patients with multiple intracranial aneurysms. Stroke. 2014; 45:1299–303. https://doi.org/10.1161/STROKEAHA.113.004421 [PubMed]
  • 8. Beck J, Rohde S, el Beltagy M, Zimmermann M, Berkefeld J, Seifert V, Raabe A. Difference in configuration of ruptured and unruptured intracranial aneurysms determined by biplanar digital subtraction angiography. Acta Neurochir (Wien). 2003; 145:861–65. https://doi.org/10.1007/s00701-003-0124-0 [PubMed]
  • 9. Nader-Sepahi A, Casimiro M, Sen J, Kitchen ND. Is aspect ratio a reliable predictor of intracranial aneurysm rupture? Neurosurgery. 2004; 54:1343–47. https://doi.org/10.1227/01.neu.0000124482.03676.8b [PubMed]
  • 10. Mocco J, Brown RD Jr, Torner JC, Capuano AW, Fargen KM, Raghavan ML, Piepgras DG, Meissner I, Huston J III, and International Study of Unruptured Intracranial Aneurysms Investigators. Aneurysm Morphology and Prediction of Rupture: An International Study of Unruptured Intracranial Aneurysms Analysis. Neurosurgery. 2018; 82:491–96. https://doi.org/10.1093/neuros/nyx226 [PubMed]
  • 11. Ma D, Tremmel M, Paluch RA, Levy EI, Meng H, Mocco J. Size ratio for clinical assessment of intracranial aneurysm rupture risk. Neurol Res. 2010; 32:482–86. https://doi.org/10.1179/016164109X12581096796558 [PubMed]
  • 12. Rahman M, Smietana J, Hauck E, Hoh B, Hopkins N, Siddiqui A, Levy EI, Meng H, Mocco J. Size ratio correlates with intracranial aneurysm rupture status: a prospective study. Stroke. 2010; 41:916–20. https://doi.org/10.1161/STROKEAHA.109.574244 [PubMed]
  • 13. de Rooij NK, Velthuis BK, Algra A, Rinkel GJ. Configuration of the circle of Willis, direction of flow, and shape of the aneurysm as risk factors for rupture of intracranial aneurysms. J Neurol. 2009; 256:45–50. https://doi.org/10.1007/s00415-009-0028-x [PubMed]
  • 14. Baharoglu MI, Lauric A, Gao BL, Malek AM. Identification of a dichotomy in morphological predictors of rupture status between sidewall- and bifurcation-type intracranial aneurysms. J Neurosurg. 2012; 116:871–81. https://doi.org/10.3171/2011.11.JNS11311 [PubMed]
  • 15. Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, van Stiphout RG, Granton P, Zegers CM, Gillies R, Boellard R, Dekker A, Aerts HJ. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer. 2012; 48:441–46. https://doi.org/10.1016/j.ejca.2011.11.036 [PubMed]
  • 16. Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images Are More than Pictures, They Are Data. Radiology. 2016; 278:563–77. https://doi.org/10.1148/radiol.2015151169 [PubMed]
  • 17. Huang YQ, Liang CH, He L, Tian J, Liang CS, Chen X, Ma ZL, Liu ZY. Development and Validation of a Radiomics Nomogram for Preoperative Prediction of Lymph Node Metastasis in Colorectal Cancer. J Clin Oncol. 2016; 34:2157–64. https://doi.org/10.1200/JCO.2015.65.9128 [PubMed]
  • 18. Liu Q, Jiang P, Jiang Y, Ge H, Li S, Jin H, Li Y. Prediction of Aneurysm Stability Using a Machine Learning Model Based on PyRadiomics-Derived Morphological Features. Stroke. 2019; 50:2314–21. https://doi.org/10.1161/STROKEAHA.119.025777 [PubMed]
  • 19. Zhang Y, Ma C, Liang S, Yan P, Liang F, Guo F, Jiang C. Morphologic Feature Elongation Can Predict Occlusion Status Following Pipeline Embolization of Intracranial Aneurysms. World Neurosurg. 2018; 119:e934–40. https://doi.org/10.1016/j.wneu.2018.08.007 [PubMed]
  • 20. Rajabzadeh-Oghaz H, Wang J, Varble N, Sugiyama SI, Shimizu A, Jing L, Liu J, Yang X, Siddiqui AH, Davies JM, Meng H. Novel Models for Identification of the Ruptured Aneurysm in Patients with Subarachnoid Hemorrhage with Multiple Aneurysms. AJNR Am J Neuroradiol. 2019; 40:1939–46. https://doi.org/10.3174/ajnr.A6259 [PubMed]
  • 21. Orning JL, Shakur SF, Alaraj A, Behbahani M, Charbel FT, Aletich VA, Amin-Hanjani S. Accuracy in Identifying the Source of Subarachnoid Hemorrhage in the Setting of Multiple Intracranial Aneurysms. Neurosurgery. 2018; 83:62–68. https://doi.org/10.1093/neuros/nyx339 [PubMed]
  • 22. van Griethuysen JJ, Fedorov A, Parmar C, Hosny A, Aucoin N, Narayan V, Beets-Tan RG, Fillion-Robin JC, Pieper S, Aerts HJ. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Res. 2017; 77:e104–07. https://doi.org/10.1158/0008-5472.CAN-17-0339 [PubMed]
  • 23. Aerts HJ, Velazquez ER, Leijenaar RT, Parmar C, Grossmann P, Carvalho S, Bussink J, Monshouwer R, Haibe-Kains B, Rietveld D, Hoebers F, Rietbergen MM, Leemans CR, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun. 2014; 5:4006. https://doi.org/10.1038/ncomms5006 [PubMed]
  • 24. Haubold J, Demircioglu A, Gratz M, Glas M, Wrede K, Sure U, Antoch G, Keyvani K, Nittka M, Kannengiesser S, Gulani V, Griswold M, Herrmann K, et al. Non-invasive tumor decoding and phenotyping of cerebral gliomas utilizing multiparametric 18F-FET PET-MRI and MR Fingerprinting. Eur J Nucl Med Mol Imaging. 2020; 47:1435–45. https://doi.org/10.1007/s00259-019-04602-2 [PubMed]
  • 25. Etminan N, Rinkel GJ. Unruptured intracranial aneurysms: development, rupture and preventive management. Nat Rev Neurol. 2016; 12:699–713. https://doi.org/10.1038/nrneurol.2016.150 [PubMed]
  • 26. Tominari S, Morita A, Ishibashi T, Yamazaki T, Takao H, Murayama Y, Sonobe M, Yonekura M, Saito N, Shiokawa Y, Date I, Tominaga T, Nozaki K, et al, and Unruptured Cerebral Aneurysm Study Japan Investigators. Prediction model for 3-year rupture risk of unruptured cerebral aneurysms in Japanese patients. Ann Neurol. 2015; 77:1050–59. https://doi.org/10.1002/ana.24400 [PubMed]
  • 27. Sparano JA, Gray RJ, Makower DF, Pritchard KI, Albain KS, Hayes DF, Geyer CE Jr, Dees EC, Perez EA, Olson JA Jr, Zujewski J, Lively T, Badve SS, et al. Prospective Validation of a 21-Gene Expression Assay in Breast Cancer. N Engl J Med. 2015; 373:2005–14. https://doi.org/10.1056/NEJMoa1510764 [PubMed]
  • 28. Wang K, Qiao Z, Zhao X, Li X, Wang X, Wu T, Chen Z, Fan D, Chen Q, Ai L. Individualized discrimination of tumor recurrence from radiation necrosis in glioma patients using an integrated radiomics-based model. Eur J Nucl Med Mol Imaging. 2020; 47:1400–11. https://doi.org/10.1007/s00259-019-04604-0 [PubMed]
  • 29. Etminan N, Brown RD Jr, Beseoglu K, Juvela S, Raymond J, Morita A, Torner JC, Derdeyn CP, Raabe A, Mocco J, Korja M, Abdulazim A, Amin-Hanjani S, et al. The unruptured intracranial aneurysm treatment score: a multidisciplinary consensus. Neurology. 2015; 85:881–89. https://doi.org/10.1212/WNL.0000000000001891 [PubMed]
  • 30. Pagiola I, Mihalea C, Caroff J, Ikka L, Chalumeau V, Iacobucci M, Ozanne A, Gallas S, Marques M, Nalli D, Carrete H, Caldas JG, Frudit ME, et al. The PHASES score: To treat or not to treat? Retrospective evaluation of the risk of rupture of intracranial aneurysms in patients with aneurysmal subarachnoid hemorrhage. J Neuroradiol. 2020; 47:349–52. https://doi.org/10.1016/j.neurad.2019.06.003 [PubMed]
  • 31. Rutledge C, Jonzzon S, Winkler EA, Raper D, Lawton MT, Abla AA. Small Aneurysms with Low PHASES Scores Account for Most Subarachnoid Hemorrhage Cases. World Neurosurg. 2020; 139:e580–84. https://doi.org/10.1016/j.wneu.2020.04.074 [PubMed]
  • 32. Sturiale CL, Stumpo V, Ricciardi L, Trevisi G, Valente I, D’Arrigo S, Latour K, Barbone P, Albanese A. Retrospective application of risk scores to ruptured intracranial aneurysms: would they have predicted the risk of bleeding? Neurosurg Rev. 2020. [Epub ahead of print]. https://doi.org/10.1007/s10143-020-01352-w [PubMed]
  • 33. Xiang J, Natarajan SK, Tremmel M, Ma D, Mocco J, Hopkins LN, Siddiqui AH, Levy EI, Meng H. Hemodynamic-morphologic discriminants for intracranial aneurysm rupture. Stroke. 2011; 42:144–52. https://doi.org/10.1161/STROKEAHA.110.592923 [PubMed]
  • 34. Kataoka K, Taneda M, Asai T, Kinoshita A, Ito M, Kuroda R. Structural fragility and inflammatory response of ruptured cerebral aneurysms. A comparative study between ruptured and unruptured cerebral aneurysms. Stroke. 1999; 30:1396–401. https://doi.org/10.1161/01.str.30.7.1396 [PubMed]
  • 35. Meng H, Tutino VM, Xiang J, Siddiqui A. High WSS or low WSS? Complex interactions of hemodynamics with intracranial aneurysm initiation, growth, and rupture: toward a unifying hypothesis. AJNR Am J Neuroradiol. 2014; 35:1254–62. https://doi.org/10.3174/ajnr.A3558 [PubMed]
  • 36. Frösen J, Tulamo R, Paetau A, Laaksamo E, Korja M, Laakso A, Niemelä M, Hernesniemi J. Saccular intracranial aneurysm: pathology and mechanisms. Acta Neuropathol. 2012; 123:773–86. https://doi.org/10.1007/s00401-011-0939-3 [PubMed]
  • 37. Brown RD Jr, Broderick JP. Unruptured intracranial aneurysms: epidemiology, natural history, management options, and familial screening. Lancet Neurol. 2014; 13:393–404. https://doi.org/10.1016/S1474-4422(14)70015-8 [PubMed]
  • 38. Sadatomo T, Yuki K, Migita K, Taniguchi E, Kodama Y, Kurisu K. Morphological differences between ruptured and unruptured cases in middle cerebral artery aneurysms. Neurosurgery. 2008; 62:602–09. https://doi.org/10.1227/01.NEU.0000311347.35583.0C [PubMed]
  • 39. Alg VS, Sofat R, Houlden H, Werring DJ. Genetic risk factors for intracranial aneurysms: a meta-analysis in more than 116,000 individuals. Neurology. 2013; 80:2154–65. https://doi.org/10.1212/WNL.0b013e318295d751 [PubMed]
  • 40. Lv N, Karmonik C, Chen S, Wang X, Fang Y, Huang Q, Liu J. Wall Enhancement, Hemodynamics, and Morphology in Unruptured Intracranial Aneurysms with High Rupture Risk. Transl Stroke Res. 2020; 11:882–89. https://doi.org/10.1007/s12975-020-00782-4 [PubMed]