Online ISSN: 1945-4589
Learn about our FREE Post-Publication Promotion Services
Research Paper Volume 11, Issue 8 pp 2185—2201

Predicting progression from mild cognitive impairment to Alzheimer’s disease on an individual subject basis by applying the CARE index across different independent cohorts

Jiu Chen1,2,3, , Gang Chen3, , Hao Shu1,3, , Guangyu Chen3, , B. Douglas Ward3, , Zan Wang1, , Duan Liu1, , Piero G. Antuono4, , Shi-Jiang Li3, , Zhijun Zhang1,5, , Alzheimer’s Disease Neuroimaging Initiative6, ,

  • 1 Department of Neurology, Affiliated ZhongDa Hospital, School of Medicine, Southeast University, Nanjing, China
  • 2 Institute of Neuropsychiatry, the Affiliated Brain Hospital of Nanjing Medical University, Fourth Clinical College of Nanjing Medical University, Nanjing, China
  • 3 Department of Biophysics, Medical College of Wisconsin, Milwaukee, WI 53226, USA
  • 4 Department of Neurology, Medical College of Wisconsin, Milwaukee, WI 53226, USA
  • 5 Department of Psychology, Xinxiang Medical University, Xinxiang, China
  • 6 A complete listing of ADNI investigators can be found at http://adni.loni.usc.edu/wp-content/uploads/ how_to_apply/ADNI_Acknowledgement_List.pdf

Received: December 1, 2018       Accepted: March 19, 2019       Published: April 29, 2019      

https://doi.org/10.18632/aging.101883
How to Cite

Copyright: Chen et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY 3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Abstract

The purposes of this study are to investigate whether the Characterizing Alzheimer’s disease Risk Events (CARE) index can accurately predict progression from mild cognitive impairment (MCI) to Alzheimer’s disease (AD) on an individual subject basis, and to investigate whether this model can be generalized to an independent cohort. Using an event-based probabilistic model approach to integrate widely available biomarkers from behavioral data and brain structural and functional imaging, we calculated the CARE index. We then applied the CARE index to identify which MCI individuals from the ADNI dataset progressed to AD during a three-year follow-up period. Subsequently, the CARE index was generalized to the prediction of MCI individuals from an independent Nanjing Aging and Dementia Study (NADS) dataset during the same time period. The CARE index achieved high prediction performance with 80.4% accuracy, 75% sensitivity, 82% specificity, and 0.809 area under the receiver operating characteristic (ROC) curve (AUC) on MCI subjects from the ADNI dataset over three years, and a highly validated prediction performance with 87.5% accuracy, 81% sensitivity, 90% specificity, and 0.861 AUC on MCI subjects from the NADS dataset. In conclusion, the CARE index is highly accurate, sufficiently robust, and generalized for predicting which MCI individuals will develop AD over a three-year period. This suggests that the CARE index can be usefully applied to select individuals with MCI for clinical trials and to identify which individuals will convert from MCI to AD for administration of early disease-modifying treatment.

Introduction

Mild cognitive impairment (MCI) has been conceptualized as a transitional clinical state between normal and Alzheimer’s disease (AD)-type dementia [1]. MCI has been considered as a key prognostic and therapeutic target in the management of AD. However, not all MCI subjects convert to AD, and many individuals remain cognitively stable or revert to normal status [2]. Early detection of those progressive MCI individuals is of increasing clinical importance in the enrichment of clinical trials of disease-modifying therapies [3].

Building an effective and accurate prognostic model that predicts the progression from MCI to AD is clinically important. To this end, a large number of studies have combined magnetic resonance imaging (MRI)-based features with positron emission tomography (PET) [4]; cerebrospinal fluid (CSF) [5]; PET and CSF [4, 6]; assessment of cognitive function (CF) and CSF [7]; and CF, PET, and CSF measures [8, 9].

Significant progress has been made using machine learning tools. Most of the above-mentioned studies have integrated CSF and PET measures. This approach using multiple technologies (CF, CSF, PET, and MRI) is often not available in clinical practice. In particular, while the prediction performance of these models has been validated extensively in a nested cross-validation loop [10] and the traditional leave-one-out approach [11], they have not been validated in an independent population. This will most likely lead to biased results and over-optimistic accuracies [12]. Although there exist some methods using the outer cross-validation loop to evaluate generalizability [13], these have limited predictability in differentiating individuals with MCI who convert to AD from non-converters. However, an effective, accurate, and clinically useful prognostic model should be robust and generalized for both the routine clinical setting and drug trials.

Recent work has begun to address this limitation. Using cortical thickness regions and subcortical volumes, Westman and colleagues combine Alzheimer’s Disease Neuroimaging Initiative (ADNI) data with the AddNeuroMed dataset [14] to predict progression of MCI to AD [15]. However, they report a low robustness with accuracies of 58% in ADNI data and 70% in AddNeuroMed data. Using MRI, PET, and CSF biomarkers, Prestia and colleagues combined ADNI data with the Translational Outpatient Memory Clinic (TOMC) dataset to predict MCI progression to AD [6]. They also obtained low accuracies with ranges of 49–63% and 51–75% in ADNI and TOMC, respectively. Thus, for the early detection of AD progression, the accuracy of predicting progression from MCI to AD needs to be improved using sensitive, generalizable, widely available, cost-effective, and minimally invasive tools.

Here, we will address this challenge with our newly developed framework, termed the characterizing AD risk events (CARE) index [16]. The CARE index estimates the probabilities of occurrence and nonoccurrence of a series of biomarkers by using an event-based probabilistic model [16]. In our previously published study, the CARE index was proven to accurately stage each individual across the whole AD spectrum [16]. It is of crucial importance to note that predicting the conversion over a short future time is an easier problem than over a longer one and less clinically useful (Eskildsen et al., 2013; Young et al., 2013), and means a smaller proportion of MCI subjects will likely convert to AD at a later time, which reduces the positive predictive value of the classification result (Young et al., 2013). Therefore, we have chosen a three-year follow-up period. In this study, we applied the CARE index to the ADNI dataset to distinguish those individuals with MCI who progressed to AD from those who did not, during a three-year follow-up period. Subsequently, we generalized the CARE index to predict which MCI from the independent Nanjing Aging and Dementia Study (NADS) dataset would convert to AD during the same time period (Figure 1). Based on the previous literature [16], We hypothesized that the CARE index score accurately predicted MCI-to-AD progression with high sensitivity and specificity at the individual patient level over 3 years in the ADNI dataset. We further hypothesized that the excellent prediction performance of the CARE index could be validated in the independent NADS dataset and achieve good between-cohort generalization during the same time-period.

Schematic diagram of the proposed classification, independent validation, and prediction framework. First, we calculated each individual CARE index score for MCI subjects in the ADNI and NADS datasets. Second, we utilized a CARE index score (A score on the CARE index is equivalent to the subject’s disease stage.) to classify N-MCI and P-MCI subjects in the ADNI dataset. Third, we applied the ‘CARE index stage “classifier” determined from the ADNI dataset to predict the conversion of MCI subjects in the NADS dataset. The ROC curve was used to assess the performance of the CARE index stage classifier and the CARE index stage prediction classifier, respectively. In addition, we assessed the performance differences of CARE index stage classification and prediction and original indices (AVLT, MMSE, GM, and FC indices) by comparing these ROCs across datasets. Abbreviations: MCI, mild cognitive impairment; ADNI, Alzheimer’s Disease Neuroimaging Initiative; NADS, Nanjing Aging and Dementia Study; AD, Alzheimer’s disease; MMSE, Mini-Mental State Examination; AVLT, Rey Auditory Verbal Learning Test; MRI, magnetic resonance imaging; AAL, automated anatomical labeling; GM, grey matter; BOLD, blood oxygenation level dependent; FC, functional connectivity; CARE, characterizing AD risk event; ROC, receiver operating characteristic; P-MCI, progressive MCI, including MCI subjects who progressed to AD-type dementia at the three-year follow-up; N-MCI, no-progressive MCI, including MCI subjects who had not progressed to dementia at the three-year follow-up.

Figure 1. Schematic diagram of the proposed classification, independent validation, and prediction framework. First, we calculated each individual CARE index score for MCI subjects in the ADNI and NADS datasets. Second, we utilized a CARE index score (A score on the CARE index is equivalent to the subject’s disease stage.) to classify N-MCI and P-MCI subjects in the ADNI dataset. Third, we applied the ‘CARE index stage “classifier” determined from the ADNI dataset to predict the conversion of MCI subjects in the NADS dataset. The ROC curve was used to assess the performance of the CARE index stage classifier and the CARE index stage prediction classifier, respectively. In addition, we assessed the performance differences of CARE index stage classification and prediction and original indices (AVLT, MMSE, GM, and FC indices) by comparing these ROCs across datasets. Abbreviations: MCI, mild cognitive impairment; ADNI, Alzheimer’s Disease Neuroimaging Initiative; NADS, Nanjing Aging and Dementia Study; AD, Alzheimer’s disease; MMSE, Mini-Mental State Examination; AVLT, Rey Auditory Verbal Learning Test; MRI, magnetic resonance imaging; AAL, automated anatomical labeling; GM, grey matter; BOLD, blood oxygenation level dependent; FC, functional connectivity; CARE, characterizing AD risk event; ROC, receiver operating characteristic; P-MCI, progressive MCI, including MCI subjects who progressed to AD-type dementia at the three-year follow-up; N-MCI, no-progressive MCI, including MCI subjects who had not progressed to dementia at the three-year follow-up.

Results

Baseline demographic and neuropsychological characteristics

The demographic and neuropsychological characteristics of the included groups are listed in Table 1. As expected, in the ADNI dataset, N-MCI and P-MCI showed no significant differences in age, gender, education, and MMSE scores (p > 0.05). In the NADS dataset, P-MCI showed no significant differences in gender and education (p > 0.05), but higher age and lower MMSE scores than N-MCI (p < 0.05). Compared with N-MCI, P-MCI showed significant deficits in performance in multiple domains of cognitive functions, including episodic memory, information processing speed, and executive function (all p < 0.05). Furthermore, there was no difference in the conversion rates of different MCI state transitions between the ADNI and NADS datasets (χ2 = 0.078, p = 0.78, see Supplementary Table 1).

Table 1. Demographics and clinical measures of N-MCI and P-MCI subjects at baseline.

ItemN-MCIP-MCIt value (χ2)p value
n=34n=12
ADNI data
   Age (years)70.24 (7.20)73.90(5.41)-1.6050.116
   Gender (male/female)17/176/60.0001.000
   Education level (years)15.76(2.73)15.75(2.45)0.0160.987
   MMSE27.71(1.77)27.42(1.38)0.5130.610
   ADAS-Cog8.62(3.23)13.08(4.21)-3.8000.000*
   AVLT37.47(10.20)29.25(5.85)3.3820.002*
ItemN-MCIP-MCIt value (χ2)p value
n=40n=16
NADS data
   Age (years)66.73(7.05)72.93(5.79)-3.1230.003*
   Gender (male/female)23/179/70.0070.932
   Education level (years)11.93(3.48)11.38(2.94)0.5570.580
   MMSE27.53(1.60)24.56(2.63)5.1540.000*
Composite Z scores of each cognitive domain
   Episodic memory0.33(0.57)-0.82(0.73)6.3380.000*
   Information processing speed0.17(0.73)-0.42(0.71)2.7800.007*
   Executive function0.22(0.52)-0.56(0.49)5.1320.000*
   Visuospatial function0.05(0.52)-0.13(0.52)1.1480.255
Notes: Values are expressed as the mean (standard deviation, SD). * Significant differences are found between P-MCI and N-MCI subjects. P values are obtained by t test except for gender (chi-square test). The performances of MMSE are presented as raw scores. The level of each cognitive domain is denoted by the composite Z scores.
Abbreviations: MCI, mild cognitive impairment; MMSE, Mini-Mental State Examination; ADNI, Alzheimer’s Disease Neuroimaging Initiative; NADS, Nanjing Aging and Dementia Study; P-MCI, progressive MCI, including MCI subjects who have progressed to AD-type dementia at the three-year follow-up; N-MCI, nonprogressive MCI, including MCI subjects who have not progressed to dementia at the three-year follow-up. ADAS-Cog, Alzheimer’s Disease Assessment Scale-Cognitive Subscale; AVLT, Rey Auditory Verbal Learning Test

Discriminating N-MCI/P-MCI

Both ADNI and NADS datasets showed that the CARE index differentiated P-MCI subjects from N-MCI subjects at baseline (see Figure 2). The comparative results on individual biomarker indices between N-MCI and P-MCI subjects are provided in Supplementary Figure 1.

Classification of N-MCI and P-MCI in the ADNI dataset by the CARE index. (A) Number of patients in each diagnostic category at each CARE index stage at baseline; data from the ADNI dataset. N-MCI subjects are represented in blue and P-MCI subjects in red. Each CARE index stage on the x-axis corresponds to the occurrence of a new biomarker transition event. Stage 0 corresponds to no events having occurred and stage 10 to all events having occurred. The optimal temporal sequence, Soptimal, of the 10 AD Biomarkers was used to calculate the CARE index. The Soptimal was estimated by the event-based probabilistic model. The Soptimal biomarker sequence is 1) increased HIP FCI, 2) decreased PCC FCI, 3) decreased Aβ concentration, 4) increased p-tau concentration, 5) decreased MMSE score, 6) increased ADAS score, 7) decreased HIP GMI, 8) decreased AVLT score, 9) decreased FG GMI, and 10) increased FG FCI. The details of the calculation of the Soptimal biomarker sequence and CARE index score can be found in our previously published studies [16] and are also provided in SI Methods. (B) Boxplot representing the distribution comparison of N-MCI and P-MCI subjects. For each boxplot, the band represents the median value, the box represents the interquartile range, and whiskers show the range of data without outliers (an outlier being defined as any value that lies more than one-and-a-half times the interquartile range from either end of the box). Differences were assessed between the two groups using Mann-Whitney tests; ***p C) The power of receiver operating characteristic (ROC) curve of the CARE index “classifier” in classifying the diagnosis of P-MCI versus N-MCI at baseline in the ADNI dataset. Note: Numbers next to the ROC curve indicate the CARE index threshold. The values of sensitivity, specificity, and odds ratio in lower right of the figure present the optimum values under the optimum CARE index threshold (red piont). Abbreviations: ADNI, Alzheimer’s Disease Neuroimaging Initiative; AD, Alzheimer’s disease; P-MCI, progressive MCI, including MCI subjects who progressed to AD-type dementia at the three-year follow-up; N-MCI, nonprogressive MCI, including MCI subjects who had not progressed to dementia at the three-year follow-up; MCI, mild cognitive impairment; CARE, characterizing AD risk event; ROC, receiver operating characteristic; AUC, area under curve; Opt, optimum; HIP, hippocampus; PCC, posterior cingulate cortex; FG, fusiform gyrus; FCI, functional connectivity indices; GMI, gray matter indices; Aβ, β-amyloid; p-tau, phosphorylated tau; MMSE, Mini-Mental State Examination; ADAS-Cog, Alzheimer’s Disease Assessment Scale-Cognitive Subscale; AVLT, Rey Auditory Verbal Learning Test.

Figure 2. Classification of N-MCI and P-MCI in the ADNI dataset by the CARE index. (A) Number of patients in each diagnostic category at each CARE index stage at baseline; data from the ADNI dataset. N-MCI subjects are represented in blue and P-MCI subjects in red. Each CARE index stage on the x-axis corresponds to the occurrence of a new biomarker transition event. Stage 0 corresponds to no events having occurred and stage 10 to all events having occurred. The optimal temporal sequence, Soptimal, of the 10 AD Biomarkers was used to calculate the CARE index. The Soptimal was estimated by the event-based probabilistic model. The Soptimal biomarker sequence is 1) increased HIP FCI, 2) decreased PCC FCI, 3) decreased Aβ concentration, 4) increased p-tau concentration, 5) decreased MMSE score, 6) increased ADAS score, 7) decreased HIP GMI, 8) decreased AVLT score, 9) decreased FG GMI, and 10) increased FG FCI. The details of the calculation of the Soptimal biomarker sequence and CARE index score can be found in our previously published studies [16] and are also provided in SI Methods. (B) Boxplot representing the distribution comparison of N-MCI and P-MCI subjects. For each boxplot, the band represents the median value, the box represents the interquartile range, and whiskers show the range of data without outliers (an outlier being defined as any value that lies more than one-and-a-half times the interquartile range from either end of the box). Differences were assessed between the two groups using Mann-Whitney tests; ***p < 0.001. (C) The power of receiver operating characteristic (ROC) curve of the CARE index “classifier” in classifying the diagnosis of P-MCI versus N-MCI at baseline in the ADNI dataset. Note: Numbers next to the ROC curve indicate the CARE index threshold. The values of sensitivity, specificity, and odds ratio in lower right of the figure present the optimum values under the optimum CARE index threshold (red piont). Abbreviations: ADNI, Alzheimer’s Disease Neuroimaging Initiative; AD, Alzheimer’s disease; P-MCI, progressive MCI, including MCI subjects who progressed to AD-type dementia at the three-year follow-up; N-MCI, nonprogressive MCI, including MCI subjects who had not progressed to dementia at the three-year follow-up; MCI, mild cognitive impairment; CARE, characterizing AD risk event; ROC, receiver operating characteristic; AUC, area under curve; Opt, optimum; HIP, hippocampus; PCC, posterior cingulate cortex; FG, fusiform gyrus; FCI, functional connectivity indices; GMI, gray matter indices; Aβ, β-amyloid; p-tau, phosphorylated tau; MMSE, Mini-Mental State Examination; ADAS-Cog, Alzheimer’s Disease Assessment Scale-Cognitive Subscale; AVLT, Rey Auditory Verbal Learning Test.

As shown in Figure 2, the CARE index discriminated MCI progression to AD with an area under curve (AUC) of 0.81. The optimal CARE index threshold for discrimination between P-MCI and N-MCI subjects was found at a CARE index score of 6.54, with a sensitivity of 75.0%, specificity of 82.4%, odds ratio (OR) of 14.0, and relative risk (RR) of 6.20.

Independent validation in different cohorts

As shown in Figure 3, the CARE index had significant power to discriminate P-MCI subjects from N-MCI subjects, with an AUC of 0.86. At the CARE index of 6.87, the prediction has high sensitivity (81.3%), specificity (90.0%), and OR (39.0). When combining both the ADNI and NADS datasets on MCI subjects as a single cohort, supplemental analysis also showed that the CARE index discriminated P-MCI subjects from N-MCI subjects on an individual subject basis, with 83.3% accuracy, 82.0% balanced accuracy, 79% sensitivity, 85% specificity, AUC of 0.84 in receiver operating characteristic (ROC) curves on MCI subjects. (See Supplementary Figure 2, Supplementary Table 2.)

Independent validation with NADS dataset for prediction of N-MCI and P-MCI subjects using the CARE index. (A) Number of patients in each diagnostic category at each individual CARE index stage at baseline from the NADS dataset. N-MCI subjects are represented in blue and P-MCI subjects in red. (B) Boxplot representing the distribution comparison of N-MCI and P-MCI subjects. For each boxplot, the band represents the median value, the box represents the interquartile range, and whiskers show the range of data without outliers (an outlier being defined as any value that lies more than one and a half times the interquartile range from either end of the box). Differences were assessed between the two groups using Mann-Whitney tests; ***p C) The power of the ROC curve of the CARE index “classifier” in predicting P-MCI versus N-MCI at baseline in the NADS dataset. Note: Numbers next to ROC curve indicate CARE index threshold. The values of sensitivity, specificity, and odds ratio in lower right of the figure present the optimum values under the optimum CARE index threshold (red piont). Abbreviations: NADS, Nanjing Aging and Dementia Study; AD, Alzheimer’s disease; P-MCI, progressive MCI, including MCI subjects who progressed to AD-type dementia at the three-year follow up; N-MCI, nonprogressive MCI, including MCI subjects who had not progressed to dementia at the three-year follow up; MCI, mild cognitive impairment; CARE, characterizing AD risk event; ROC, receiver operating characteristic; AUC, area under curve.

Figure 3. Independent validation with NADS dataset for prediction of N-MCI and P-MCI subjects using the CARE index. (A) Number of patients in each diagnostic category at each individual CARE index stage at baseline from the NADS dataset. N-MCI subjects are represented in blue and P-MCI subjects in red. (B) Boxplot representing the distribution comparison of N-MCI and P-MCI subjects. For each boxplot, the band represents the median value, the box represents the interquartile range, and whiskers show the range of data without outliers (an outlier being defined as any value that lies more than one and a half times the interquartile range from either end of the box). Differences were assessed between the two groups using Mann-Whitney tests; ***p < 0.001. (C) The power of the ROC curve of the CARE index “classifier” in predicting P-MCI versus N-MCI at baseline in the NADS dataset. Note: Numbers next to ROC curve indicate CARE index threshold. The values of sensitivity, specificity, and odds ratio in lower right of the figure present the optimum values under the optimum CARE index threshold (red piont). Abbreviations: NADS, Nanjing Aging and Dementia Study; AD, Alzheimer’s disease; P-MCI, progressive MCI, including MCI subjects who progressed to AD-type dementia at the three-year follow up; N-MCI, nonprogressive MCI, including MCI subjects who had not progressed to dementia at the three-year follow up; MCI, mild cognitive impairment; CARE, characterizing AD risk event; ROC, receiver operating characteristic; AUC, area under curve.

Generalization in different cohorts

To validate the generalizability of the CARE index to discriminate P-MCI subjects from N-MCI subjects, we applied the optimal the CARE index threshold (6.54) to discriminate between the P-MCI and N-MCI subjects in the ADNI dataset and the MCI subjects in the NADS dataset. We found that, of the 56 MCI subjects, 48 (85.7%) were predicted correctly. MCI subjects with a CARE index score above the threshold have a high diagnostic odds ratio (OR = 33.33, 95% CI = 6.33–145.30) and relative risk (RR = 9.15, 95% CI = 2.98–28.13) in MCI progression relative to non-progression to AD. (See Table 2.)

Table 2. Estimated risk of the 3-year conversion to probable AD in MCI patients in NADS dataset using the optimal cutoff value of CARE index from ADNI dataset.

DatasetCARE index cutoffTotalMCI
P-MCIN-MCIOR95% CI for ORRR95% CI for RRACCSensitivitySpecificity
ADNI
Converters
(> = 6.54)		1596
Nonconverters
(< 6.54)		3132814.002.89–67.726.201.96–19.6280.43%75.0%82.4%
NADS
Converters
(> = 6.54)		18135
Nonconverters
(< 6.54)		3833530.336.33–145.309.152.98–28.1385.71%81.3%87.5%
Note: 6.54 is the optimal cutoff value found using the ROC analysis to predict with optimal sensitivity and specificity those MCI patients from the ADNI dataset who did or do not convert to AD within three years.
Abbreviations: ADNI, Alzheimer’s Disease Neuroimaging Initiative; NADS, Nanjing Aging and Dementia Study; CARE, characterizing AD risk event; P-MCI, progressive MCI, including MCI subjects who progressed to AD-type dementia at the three-year follow-up; N-MCI, nonprogressive MCI, including MCI subjects who had not progressed to dementia at the three-year follow-up; MCI, mild cognitive impairment; AD, Alzheimer’s Disease; OR, odds ratio; RR, relative risk; CI, confidence interval; ACC, accuracy.

Robustness and powerfulness of the CARE index compared to a single biomarker

As shown in Figure 4 and Table 3, in the ADNI dataset, the CARE index performed better than each of the seven selected biomarker indices in discriminating P-MCI subjects from N-MCI subjects. (See Figure 4 and Table 3). In the NADS dataset, although the best single biomarker index was an AVLT score with an AUC of 0.876, the CARE index power in discriminating P-MCI subjects from N-MCI subjects showed high generalization and stability across datasets, whereas other individual biomarkers did not. (See Table 3.)

Comparisons of the power of ROC curve of the CARE index with behavioral, gray matter, and functional indices in predicting the P-MCI versus N-MCI subjects in the ADNI and NADS datasets. (A–C) represent comparisons of the power of ROC curve of the CARE index and individual behavioral, gray matter, and functional indices in the ADNI dataset, respectively. (D–F) represent comparisons of the power of ROC curve of the CARE index and individual behavioral, gray matter, and functional indices in the NADS dataset, respectively. Abbreviations: ADNI, Alzheimer’s Disease Neuroimaging Initiative; AD, Alzheimer’s disease; P-MCI, progressive MCI, including MCI subjects who progressed to AD-type dementia at the three-year follow up; N-MCI, nonprogressive MCI, including MCI subjects who had not progressed to dementia at the three-year follow up; MCI, mild cognitive impairment; CARE, characterizing AD risk event; ROC, receiver operating characteristic; AUC, area under curve; HIP, hippocampus; PCC, posterior cingulate cortex; FG, fusiform gyrus; FCI, functional connectivity indices; GMI, gray matter indices; Aβ, β-amyloid; p-tau, phosphorylated tau; MMSE, Mini-Mental State Examination; ADAS-Cog, Alzheimer’s Disease Assessment Scale-Cognitive Subscale; AVLT, Rey Auditory Verbal Learning Test.

Figure 4. Comparisons of the power of ROC curve of the CARE index with behavioral, gray matter, and functional indices in predicting the P-MCI versus N-MCI subjects in the ADNI and NADS datasets. (A–C) represent comparisons of the power of ROC curve of the CARE index and individual behavioral, gray matter, and functional indices in the ADNI dataset, respectively. (D–F) represent comparisons of the power of ROC curve of the CARE index and individual behavioral, gray matter, and functional indices in the NADS dataset, respectively. Abbreviations: ADNI, Alzheimer’s Disease Neuroimaging Initiative; AD, Alzheimer’s disease; P-MCI, progressive MCI, including MCI subjects who progressed to AD-type dementia at the three-year follow up; N-MCI, nonprogressive MCI, including MCI subjects who had not progressed to dementia at the three-year follow up; MCI, mild cognitive impairment; CARE, characterizing AD risk event; ROC, receiver operating characteristic; AUC, area under curve; HIP, hippocampus; PCC, posterior cingulate cortex; FG, fusiform gyrus; FCI, functional connectivity indices; GMI, gray matter indices; Aβ, β-amyloid; p-tau, phosphorylated tau; MMSE, Mini-Mental State Examination; ADAS-Cog, Alzheimer’s Disease Assessment Scale-Cognitive Subscale; AVLT, Rey Auditory Verbal Learning Test.

Table 3. Sensitivity for P-MCI, specificity for N-MCI, AUC, and accuracy of CARE index and each of seven selected biomarker indices when applying the optimal threshold from the ADNI dataset to the NADS dataset.

ADNIPredictorsAUCpSensitivitySpecificityAccuracyBalanced AccuracyOpt. Threshold
RankP-MCI + N-MCI95% CIRankP-MCIRankN-MCIRank%Rank%
CARE index10.8090.68-0.940.0220.7510.82180.4178.76.54
MMSE60.5780.42-0720.4220.7580.44276.1859.628.5
AVLT30.7610.61-0.870.0860.6920.79852.2373.028.5
HIPFCI80.5150.36-0.670.8880.5040.71565.2760.30.66
PCCFCI50.6350.48-0.770.1710.8360.50658.7566.72.28
FGFCI40.6590.51-0.790.1070.6740.71469.6468.610.57
HIPGMI20.7620.61-0.880.0120.7530.74373.9274.30.40
FGGMI60.5780.42-0.720.4220.7560.50756.5662.50.55
NADSAUCpSensitivitySpecificityAccuracyBalanced AccuracyOpt. Threshold
RankP-MCI + N-MCI95% CIRankP-MCIRankN-MCIRank%Rank%
CARE index20.8610.74-0.940.0020.8130.88185.7184.46.54
MMSE30.8370.71-0.920.0010.9480.32850.0463.128.5
AVLT10.8760.74-0.950.0030.7520.90185.7282.528.5
HIPFCI60.5880.45-0.720.3140.6970.53757.1560.60.66
PCCFCI70.5670.43-0.700.4480.3150.70658.9850.62.28
FGFCI80.5630.42-0.700.4760.3850.70560.7753.810.57
HIPGMI40.8110.68-0.900.0060.3810.95378.6366.30.40
FGGMI50.6340.50-0.760.1250.4440.78467.9560.60.55
Abbreviations: ADNI, Alzheimer’s Disease Neuroimaging Initiative; NADS, Nanjing Aging and Dementia Study; CARE, characterizing AD risk event; P-MCI, progressive MCI, including MCI subjects who progressed to AD-type dementia at the three-year follow-up; N-MCI, nonprogressive MCI, including MCI subjects who had not progressed to dementia at the three-year follow-up; MCI, mild cognitive impairment; AD, Alzheimer’s Disease; AVLT, Rey Auditory Verbal Learning Test; MMSE, Mini-Mental State Examination; HIP, hippocampus; PCC, posterior cingulate cortex; FUS, fusiform gyrus; GM, gray matter; GMI, gray matter indices; FCI, functional connectivity indices; Opt, optima.

Behavioral significance of the changes in CARE index measured at baseline and the three-year follow-up

The changes in CARE index score measured at baseline and the three-year follow-up were negatively correlated with changes in mini-mental state examination (MMSE) scores, and the composite Z scores of episodic memory, information processing speed, and visuospatial function in MCI (p < 0.05). (See Figure 5 and Supplementary Table 3.)

The changes in the CARE index correlated with the changes in cognitive performance and other clinical variables measured between baseline and the three-year follow up in MCI subjects from the NADS dataset. Notes: 1) In each of these scatter diagrams, there is a significant correlation between changes in the CARE index and changes in the cognitive performance or clinical measures (p p

Figure 5. The changes in the CARE index correlated with the changes in cognitive performance and other clinical variables measured between baseline and the three-year follow up in MCI subjects from the NADS dataset. Notes: 1) In each of these scatter diagrams, there is a significant correlation between changes in the CARE index and changes in the cognitive performance or clinical measures (p < 0.05, FDR corrected). 2) In Figure 4, the total number of subjects selected at baseline is equivalent of those at the three-year follow-up from the NADS dataset (MCI, 44 subjects). All subjects at the three-year follow-up had at least one resting-state functional connectivity MRI scan with a corresponding anatomical scan and had scores on the MMSE and AVLT. Abbreviations: NADS, Nanjing Aging and Dementia Study; MCI, mild cognitive impairment; CARE, characterizing AD risk event; EM, episodic memory; IPS, information processing speed; VF, visuospatial function; MMSE, Mini-Mental State Examination. A statistical threshold was set at a p < 0.05 (false discovery rate [FDR]-corrected]. Δ represents the changes.

Discussion

The most novel result of this study is that the CARE index allows us to accurately predict MCI-to-AD progression on an individual subject basis across different cohorts.

This study showed no differences in the conversion rates in individuals with MCI after a 3-year follow-up between ADNI (26.1%) and NADS (28.6%), which agrees with these findings based on ADNI that showed the conversion rates of 26 to 48% after a 3-year follow-up (Devanand DP et al. 2008; Okello A et al. 2009). This suggests our results are unbiased and convincing.

Recently, an excellent study by Young and colleagues used an EBP model to develop a biomarker prognostic model that predicts the MCI-to-AD conversion over three years [17]. They obtained the relatively high balanced accuracy of 77%, sensitivity of 86%, and specificity of 69%. On the basis of their study, we have made significant improvements from a practical perspective, as follow: First, the CARE index only integrates widely available, cost-effective, and noninvasive markers (i.e., behavioral, structural, and functional MRI markers). In particular, we added brain functional connectivity, which improved the prediction performance with regard to MCI-to-AD progression. It is well established that the measures of brain connectivity at rest are sensitive to AD progression [18] and the incorporation of brain connectivity measures with other biomarkers can add further predictive information to the prognostic model [19, 20]. Second, the CARE index can be applied to predict MCI-to-AD progression in an independent population with a very high prediction performance.

A strength of this work is that the overall predictive performance of the CARE index was not only high but also has a fairly balanced sensitivity/specificity, which has significantly outperformed other existing work thus far. Several studies have developed models based on various combinations of CF, MRI, PET, and CSF biomarkers. They have applied these models to distinguish MCI converters from MCI non-converters over a three-year follow-up period, only attaining accuracies in the range of 0.49–0.75 and AUCs in the range of 0.64–0.76 [4–6, 10, 21]. The high prediction performance for the CARE index may be the reason why all selected markers are well-studied and well-established AD markers, These markers are significantly associated with conversion from MCI to AD [22, 23]. The structural and functional impairment of the hippocampus, hallmarks of AD, is consistently considered as a valuable predictor of progression from MCI to AD [24]. The default mode network (PCC as a core hub) abnormalities are associated with episodic memory impairment early in AD [25, 26], representing AD disease progression [27]. Furthermore, the CARE index is a staging system for disease monitoring, which can reveal a detailed evaluation of patient state. Using a baseline MRI marker, Wee et al. can predict the MCI-to-AD conversion with relatively high accuracy (AUC = 0.84) [13], although they report limited sensitivity (75.1%) and low specificity (63.5%). Furthermore, there are two studies that report results comparable to those in our study. Using CSF and plasma markers, Lehallier et al. report a good ability to predict the MCI-to-AD conversion within three years with relatively high accuracy (80%), sensitivity (88%), specificity (70%), and balanced accuracy (79%) [28]. However, they evaluate the predictive performance using a form of internal validation in the same dataset. This method may lead to overfitting in the predictive model [12, 29]. Using CF and MRI biomarkers, a recent study also achieved a relatively high accuracy (79.9%), sensitivity (83.4%), specificity (76.4%), AUC of 0.87, and balanced accuracy (79.9%) [30]. However, they use a nested stratified cross-validation procedure to evaluate the predictive performance in the same dataset [12, 29]. Using MRI and plasma biomarkers, Liu and colleagues achieved a high accuracy (96%), AUC of 0.82, and sensitivity (95%), but at the cost of a very low specificity (65%) [31]. Using MRI and functional MRI (functional connectivity) biomarkers, Serra and colleagues can predict MCI-to-AD progression with high accuracy (89.7%), sensitivity (84.6%), and specificity (93.8%) [19]. However, they performed a simple discriminant analysis with no form of internal or external validation. This may especially lead to overfitting in the predictive model. Furthermore, they used a relatively small sample size (a total of 31 MCI subjects) and a relatively short-term, two-year follow-up with a high conversion rate of 45.2% (14 converters from a total of 31 MCI subjects). This conversion rate after a two-year follow-up is considerably higher than the 35% and 37.5% reported by other studies [32, 33], respectively. A previous study indicated disease progression may present more dynamic features within two years and demonstrated relatively stable performance over two years [34]. It is of crucial importance to note that predicting the conversion over a short timespan means a proportion of the MCI subjects will likely convert to AD at a later time, which reduces the positive predictive value of the classification result [35]. Most importantly, the prediction performance of the models in all of the above-mentioned studies has not been validated in an independent cohort.

Another principal novelty of this study is that we were able to replicate our results in an independent cohort, which suggests that the CARE index is a robust and accurate model with good generalizability. Recently, a few studies have begun to address the generalization bottleneck; however, their prediction performance could not be transferred from one cohort to other cohorts [6, 15]. And they obtained a low robustness across cohorts, with low accuracies. The methodological differences may account for the CARE index’s superior prediction performance, robustness, and generalizability compared to those of their models. First, unlike above-mentioned models, the CARE index estimates the probabilities of occurrence and nonoccurrence of each biomarker on an individual basis by using the EBP model, rather than by dichotomizing biomarker status based on the cutoff point threshold or clinical diagnosis information [16]. The CARE index emphasizes the temporally dependent process of diverse pathophysiological events underlying AD development and can accurately stage each individual across the whole AD spectrum [16]. The probabilities of occurrence and nonoccurrence of each pathophysiological event are affected only by the disease progression of the individual and not by the whole nature of the different independent cohorts, such as multicenter or single-center, or cohort heterogeneity. Second, Prestia and colleagues’ models use high-dimensional input in prediction, which are often related to the so-called “curse of dimensionality” [36]. which significantly hampers modeling generalization. The explanation may be that many machine learning methods may lose their ability to generalize to unseen examples with the increase in the number of available input features, due to the discordance between the sample size and the increased dimensionality [4]. Therefore, the CARE index exhibits a remarkable advantage in predicting the MCI-to-AD conversion across different cohorts.

In the current study, our findings support the fact that combining different biomarkers can predict the conversion status and cognitive decline. The CARE index associated with conversion likely reflects disease severity, i.e., how close an individual is to a significant clinical transition. The CARE index integrates biomarkers derived from behavior, structural, and resting-sate functional MRI modalities. It has important clinical advantages such as cost-efficiency, noninvasiveness, very high test–retest reliability, strong validity for AD pathological process, and clinical popularity. Most importantly, the excellent predictive performance of CARE index in a multicenter study (ADNI) can be applied in a clinical single-center study (NADS). Therefore, the CARE index can be generalized for use in precisely selecting individuals with MCI for a clinical trial. It can aid the development of new disease-modifying drugs by assigning possible surrogate markers of disease progression, and it can reduce the number of subjects needed to detect a significant drug effect [37]. Ultimately, it is reasonable to speculate that it would be desirable for the CARE index to be available to clinicians.

Some limitations of this study deserve comment. First, sample sizes of P-MCI and N-MCI subjects in the separate ADNI and NADS cohorts, especially the P-MCI subjects, are relatively small. However, it is important to note that our supplementary analysis combining ADNI and NADS cohorts still showed a very high power in predicting conversion from MCI to AD. Therefore, our results remain convincing. Second, we only used parts of MCI subjects from ADNI cohorts due to our entry criteria, which may affect our results. Finally, this study assessed only the predictive power of the CARE index during a three-year follow-up period. Future studies with longer and multiple follow-up times (at four years, five years, and so on) are needed to refine and generalize our CARE index estimates. Furthermore, the diffusion tensor imaging (DTI) has been considered as an advanced technology on studying the MCI/AD patients. Therefore, the further studies will corroborate and extend high prediction performance of the CARE index and explore whether combining DTI, brain structural and functional MRI can improve this high prediction performance. Finally, other cognitive measures and non-invasive biomarkers may play crucial roles in improving prediction performance that were not addressed here (such as the CDR–sum of boxes and diffusion tensor imaging measures)

In conclusions, the CARE index is sufficiently robust and generalizable for predicting which MCI individuals, across datasets, will develop AD over three years. It can be usefully applied to select individual subjects with MCI for clinical trials and to predict with high sensitivity and specificity for early treatment which individual subjects with MCI will convert to AD in the future.

Methods

Subjects

The data for study subjects were obtained from two independent datasets: ADNI and NADS.

ADNI

Baseline data used in this study were consistent with data in our previously published study [16]. The details of the ADNI information are provided in SI Methods S.1.

We selected a total of 74 MCI subjects with a baseline diagnosis of amnestic MCI, based on the requirements in our previously published study [16]. Finally, 46 subjects had a three-year follow-up clinical diagnosis of MCI. According to the follow-up clinical diagnosis by the National Institute of Neurological and Communicative Disorders and Stroke or the Alzheimer’s Disease and Related Disorders Association (NINCDS-ADRDA) criteria for the diagnosis of probable AD [38], those MCI subjects who progressed to AD within 36 months of entering the study were labeled as progressive MCI (P-MCI), and those who did not progress were labeled as non-progressive MCI (N-MCI) subjects. The clinical statuses for P-MCI and N-MCI subjects are employed as the “ground truth” in our classification experiments as described below. The characteristics of the MCI subjects are provided in Table 1.

NADS

The NADS study recruited 87 subjects with a baseline diagnosis of amnestic MCI status. Written informed consent was obtained from all of the participants, and the study was approved by the responsible Human Participants Ethics Committee of the Affiliated ZhongDa Hospital, Southeast University, Nanjing, China.

All amnestic MCI subjects met the diagnostic criteria proposed by Petersen and colleagues [39] and the current revised consensus criteria of the International Working Group on amnestic MCI [40]. The inclusion and exclusion criteria (see details in SI Methods S.2) used to choose subjects can be found in our previously published studies [41]. Finally, 56 subjects had a three-year follow-up clinical diagnosis of amnestic MCI. The clinical status of each MCI subject was reevaluated at 36 months and classified into the N-MCI and P-MCI groups, as described above. The characteristics of the MCI subjects are provided in Table 1.

Neuropsychological assessment

In the NADS dataset, all subjects underwent a standardized clinical interview and comprehensive neuropsychological assessments that were performed by neuropsychologists (Dr. Gu, Gao, and Yan). These assessments included mini-mental state examination (MMSE), Mattis Dementia Rating Scale (MDRS); Auditory Verbal Learning Test–immediate recall (AVLT-IR); Auditory Verbal Learning Test–5-min delayed recall (AVLT-5-min-DR); Auditory Verbal Learning Test–20-min delayed recall (AVLT-20-min-DR); Logical Memory Test–immediate recall (LMT-IR); Logical Memory Test–20-min delayed recall (LMT-20-min-DR); Rey-Osterrieth Complex Figure Test (ROCFT); Rey-Osterrieth Complex Figure Test –20-min delayed recall (ROCFT-20min-DR); Trail-Making Tests A and B (TMT–A and B); Digital Symbol Substitution Test (DSST); Digit Span Test (DST); Stroop Color and Word Test A, B, and C; Verbal Fluency Test (VFT); Semantic Similarity (Similarity) test; and Clock Drawing Test (CDT). These tests were used to evaluate multi-domains of cognitive function, including general cognitive function, episodic memory, information processing speed, executive function, and visuo-spatial function, respectively. Note that all subjects in the NADS dataset had both scores of comprehensive neuropsychological assessment and at least one R-fMRI scan with corresponding anatomical scans at 3-year follow-up to investigate the links between the changes of characterizing AD risk event and the changes of neuropsychological performance. Furthermore, we have standardized the tests in the NADS dataset into the ADNI dataset when the cognitive measure scores obtained across different study datasets may have different testing composition.

MRI data acquisition

ADNI dataset

The ADNI data acquisition process is described at http://adni.loni.ucla.edu/. The details regarding image acquisition are provided in SI Methods S.4.

NADS dataset

The details regarding image acquisition parameters are provided in SI Methods S.4 and in our previously published studies [42].

Image preprocessing

Conventional preprocessing steps were conducted using Analysis of Functional NeuroImages (AFNI) software, SPM8, and MATLAB (Chen et al., 2016). The preprocessing allows for T1-equilibration (removing the first 15 s of R-fMRI data); slice-acquisition-dependent time shift correction (3dTshift); motion correction (3dvolreg); detrending (3dDetrend); despiking (3dDespike); spatial normalization (original space to the Montreal Neurological Institute [MNI] space, SPM8); averaging white matter and CSF signal retrieval (3dROIstats) using standard SPM white matter and CSF mask in the MNI space; white matter, CSF signal, and motion effect removal (3dDeconvolve); global signal removal necessity check (the global signal will be removed if necessary) [5]; and low-frequency band-pass filtering (3dFourier, 0.015–0.1Hz).

Biomarker events, expected stage, and missing biomarker

According to our previously published studies [16], 10 well-studied AD biomarkers were selected, each representing an event that dynamically occurs along with AD progression. These biomarkers include three functional connectivity indices (FCI) from the hippocampus (HIPFCI), the posterior cingulate cortex (PCCFCI), and the fusiform gyrus (FUSFCI); two gray matter concentration indices (GMI) from the hippocampus (HIPGMI) and fusiform gyrus (FUSGMI); two CSF biomarkers (Aβ1-42 and p-tau levels); and three cognitive biomarkers (MMSE, ADAS-Cog, and AVLT scores). Detailed methods to extract FCI and GMI are provided in SI Methods S.6 and S.7.

The optimal temporal sequence, Soptimal, is determined by the event-based probabilistic (EBP) model. The Soptimal sequence, in which these biomarker events occur, was well studied in our previously published work [16]. The mathematical detail of the EBP model is described previously [16, 17] and in SI Methods S.8, S.9, and S.10.

In the case of missing biomarkers, the mathematical detail of the Soptimal sequence of biomarker events with missing data is described in SI Methods S.8–S.12.

Individual CARE index

We numbered each of the 10 biomarker events by their order of occurrence in Soptimal; collectively, these events comprise the index for CARE index. Each subject’s CARE index score is defined as that at which the order number had the highest likelihood value in Soptimal (SI Methods S.11). Note that in this study, three biomarkers (two CSF biomarkers of Aβ1-42 and p-tau levels and ADAS-Cog scores) were missing in the NADS dataset. Therefore, the same biomarkers in the ADNI dataset were selected to be consistent with those in the NADS dataset to compute the CARE index.

Statistical analysis

Demographic and neuropsychological data

The statistical analyses were conducted with SPSS 22.0 software. The two-sample t-test, chi-square (χ2) test, and Mann-Whitney U tests were used to compare the differences in demographic data, neuropsychological performance, each individual biomarker feature, and the CARE index between N-MCI and P-MCI subjects. A statistical threshold was set at a p < 0.05.

MCI conversion prediction

We evaluated the power of the CARE index and of individual biomarkers to discriminate P-MCI subjects from N-MCI subjects with the use of receiver operating characteristic (ROC) curves [43]. To demonstrate the CARE index’s power to predict clinical progression from MCI to AD, the area under the ROC (AUC) values were employed to compare the CARE index and individual biomarkers using a nonparametric method for correlated samples [44]. The optimal cutoff value of the CARE index for discriminating P-MCI subjects from N-MCI subjects was extracted from the ADNI dataset, generating optimal sensitivity and specificity values, accuracy, odds ratio (OR), and relative risk (RR). Furthermore, we applied the CARE index classifier obtained from the ADNI dataset to the NADS dataset to validate the generalizability of the classifier in discriminating between P-MCI and N-MCI subjects.

To compare the stability and generalizability of the CARE index to discriminate between P-MCI and N-MCI subjects with those of individual biomarkers, we also applied the optimal classifier of each biomarker obtained from the ADNI dataset to the NADS dataset. To avoid unbalanced class frequency to lead to discrepancies between sensitivity and specificity [7, 35], we also reported the balanced accuracy: defined as (sensitivity + specificity)/2 [45]. To directly observe the power of each biomarker in distinguishing P-MCI subjects from N-MCI subjects across datasets, we ranked the AUC, optimal sensitivity and specificity, accuracy, and balanced accuracy of the CARE index and individual biomarkers.

In addition, to avoid limitations due to the relatively small sample size and the differences in the predictive power of the CARE index between the ADNI and NADS cohorts due to the differences of MCI heterogeneity in general, we performed a supplementary analysis with the combined cohort of ADNI and NADS datasets.

Behavioral significance of the changes in CARE index measured at baseline and the three-year follow-up

We performed a multiple linear regression model analysis to examine the relationships between the changes in CARE index and the changes in cognitive performance or clinical variables at baseline and the three-year follow-up in MCI subjects (See details in SI Methods S.13). The statistical threshold was set at a p < 0.05 (FDR-corrected).

Supplementary Materials

Supplementary Information

Abbreviations

CARE: Characterizing Alzheimer’s disease Risk Events; MCI: mild cognitive impairment; AD: Alzheimer’s disease; ADNI: Alzheimer’s Disease Neuroimaging Initiative; NADS: Nanjing Aging and Dementia Study; ROC: receiver operating characteristic; AUC: receiver operating characteristic curve; PET: positron emission tomography; CSF: cerebrospinal fluid; CF: cognitive function; TOMC: Translational Outpatient Memory Clinic; P-MCI: progressive MCI; N-MCI: non-progressive MCI; FCI: functional connectivity indices; HIPFCI: functional connectivity indices from the hippocampus; PCCFCI: functional connectivity indices from posterior cingulate cortex; FUSFCI: functional connectivity indices from fusiform gyrus; GMI: gray matter concentration indices; HIPGMI: gray matter concentration indice from the hippocampus; FUSGMI: gray matter concentration indice from fusiform gyrus; OR: odds ratio; RR: relative risk; EBP: event-based probabilistic.

Author Contributions

S-JL and Z-JZ contributed study concept and design. JC, GC, HS, GC, DW, ZW, DL, and PGA acquired, analyzed or interpreted data. JC, S-JL and Z-JZ drafted and revised the manuscript. All the authors approve the final manuscript.

Acknowledgments

We sincerely thank Ms. Lydia Washechek, BA, for editorial assistance.

Conflicts of Interest

Dr. Shi-Jiang Li reports personal fees for consulting outside the submitted work from Brain Symphonics, LLC, and Bristol-Myers Squibb Company. Brain Symphonics, LLC, has licensed related patents from the Medical College of Wisconsin. The other authors declare no conflicts of interest.

Funding

This study was supported by the National Natural Science Foundation of China (Nos. 81420108012, 91432000, 81500919, 81671046, and 81701675); the Key Program for Clinical Medicine and Science and Technology: Jiangsu Province Clinical Medical Research Center (Nos. BL2014077); the Nanjing Medical Science and Technology Development Foundation-Nanjing Outstanding Youth Fund Projects (No. JQX18005), the Cooperative Research Project of Southeast University-Nanjing Medical University (No. 2018DN0031), and the Key Research and Development Plan (Social Development) Project of Jiangsu Province (No. BE2018608), the Disciplinary group of Psychology and Neuroscience, Xinxiang Medical University (No. 2016PN-KFKT-01); and National Institutes of Health grants UF1 AG051216 and R44 AG035405.

References

  • 1. Petersen RC. Mild cognitive impairment as a diagnostic entity. J Intern Med. 2004; 256:183–94. https://doi.org/10.1111/j.1365-2796.2004.01388.x [PubMed]
  • 2. Ganguli M, Dodge HH, Shen C, DeKosky ST. Mild cognitive impairment, amnestic type: an epidemiologic study. Neurology. 2004; 63:115–21. https://doi.org/10.1212/01.WNL.0000132523.27540.81 [PubMed]
  • 3. Petersen RC, Roberts RO, Knopman DS, Boeve BF, Geda YE, Ivnik RJ, Smith GE, Jack CRJr. Mild cognitive impairment: ten years later. Arch Neurol. 2009; 66:1447–55. https://doi.org/10.1001/archneurol.2009.266 [PubMed]
  • 4. Hinrichs C, Singh V, Xu G, Johnson SC, and Alzheimers Disease Neuroimaging Initiative. Predictive markers for AD in a multi-modality framework: an analysis of MCI progression in the ADNI population. Neuroimage. 2011; 55:574–89. https://doi.org/10.1016/j.neuroimage.2010.10.081 [PubMed]
  • 5. Westman E, Muehlboeck JS, Simmons A. Combining MRI and CSF measures for classification of Alzheimer’s disease and prediction of mild cognitive impairment conversion. Neuroimage. 2012; 62:229–38. https://doi.org/10.1016/j.neuroimage.2012.04.056 [PubMed]
  • 6. Prestia A, Caroli A, Herholz K, Reiman E, Chen K, Jagust WJ, Frisoni GB, and Translational Outpatient Memory Clinic Working Group, and Alzheimer’s Disease Neuroimaging Initiative. Diagnostic accuracy of markers for prodromal Alzheimer’s disease in independent clinical series. Alzheimers Dement. 2013; 9:677–86. https://doi.org/10.1016/j.jalz.2012.09.016 [PubMed]
  • 7. Cui Y, Liu B, Luo S, Zhen X, Fan M, Liu T, Zhu W, Park M, Jiang T, Jin JS, and Alzheimer’s Disease Neuroimaging Initiative. Identification of conversion from mild cognitive impairment to Alzheimer’s disease using multivariate predictors. PLoS One. 2011; 6:e21896. https://doi.org/10.1371/journal.pone.0021896 [PubMed]
  • 8. Landau SM, Harvey D, Madison CM, Reiman EM, Foster NL, Aisen PS, Petersen RC, Shaw LM, Trojanowski JQ, Jack CRJr, Weiner MW, Jagust WJ, and Alzheimer’s Disease Neuroimaging Initiative. Comparing predictors of conversion and decline in mild cognitive impairment. Neurology. 2010; 75:230–38. https://doi.org/10.1212/WNL.0b013e3181e8e8b8 [PubMed]
  • 9. Chen K, Ayutyanont N, Langbaum JB, Fleisher AS, Reschke C, Lee W, Liu X, Bandy D, Alexander GE, Thompson PM, Shaw L, Trojanowski JQ, Jack CRJr, et al, and Alzheimer’s Disease Neuroimaging Initiative. Characterizing Alzheimer’s disease using a hypometabolic convergence index. Neuroimage. 2011; 56:52–60. https://doi.org/10.1016/j.neuroimage.2011.01.049 [PubMed]
  • 10. Ritter K, Schumacher J, Weygandt M, Buchert R, Allefeld C, Haynes JD. Multimodal prediction of conversion to Alzheimer’s disease based on incomplete biomarkers. Alzheimers Dement (Amst). 2015; 1:206–15. https://doi.org/10.1016/j.dadm.2015.01.006 [PubMed]
  • 11. Douaud G, Menke RA, Gass A, Monsch AU, Rao A, Whitcher B, Zamboni G, Matthews PM, Sollberger M, Smith S. Brain microstructure reveals early abnormalities more than two years prior to clinical progression from mild cognitive impairment to Alzheimer’s disease. J Neurosci. 2013; 33:2147–55. https://doi.org/10.1523/JNEUROSCI.4437-12.2013 [PubMed]
  • 12. Wolz R, Julkunen V, Koikkalainen J, Niskanen E, Zhang DP, Rueckert D, Soininen H, Lötjönen J, and Alzheimer’s Disease Neuroimaging Initiative. Multi-method analysis of MRI images in early diagnostics of Alzheimer’s disease. PLoS One. 2011; 6:e25446. https://doi.org/10.1371/journal.pone.0025446 [PubMed]
  • 13. Wee CY, Yap PT, Shen D, and Alzheimer’s Disease Neuroimaging Initiative. Prediction of Alzheimer’s disease and mild cognitive impairment using cortical morphological patterns. Hum Brain Mapp. 2013; 34:3411–25. https://doi.org/10.1002/hbm.22156 [PubMed]
  • 14. Lovestone S, Francis P, Kloszewska I, Mecocci P, Simmons A, Soininen H, Spenger C, Tsolaki M, Vellas B, Wahlund LO, Ward M, and AddNeuroMed Consortium. AddNeuroMed—the European collaboration for the discovery of novel biomarkers for Alzheimer’s disease. Ann N Y Acad Sci. 2009; 1180:36–46. https://doi.org/10.1111/j.1749-6632.2009.05064.x [PubMed]
  • 15. Westman E, Simmons A, Muehlboeck JS, Mecocci P, Vellas B, Tsolaki M, Kłoszewska I, Soininen H, Weiner MW, Lovestone S, Spenger C, Wahlund LO, and AddNeuroMed consortium, and Alzheimer’s Disease Neuroimaging Initiative. AddNeuroMed and ADNI: similar patterns of Alzheimer’s atrophy and automated MRI classification accuracy in Europe and North America. Neuroimage. 2011; 58:818–28. https://doi.org/10.1016/j.neuroimage.2011.06.065 [PubMed]
  • 16. Chen G, Shu H, Chen G, Ward BD, Antuono PG, Zhang Z, Li SJ, Alzheimer’s Disease Neuroimaging Initiative. Staging Alzheimer’s Disease Risk by Sequencing Brain Function and Structure, Cerebrospinal Fluid, and Cognition Biomarkers. J Alzheimers Dis. 2016; 54:983–993. https://doi.org/10.3233/JAD-160537 [PubMed]
  • 17. Young AL, Oxtoby NP, Daga P, Cash DM, Fox NC, Ourselin S, Schott JM, Alexander DC, and Alzheimer’s Disease Neuroimaging Initiative. A data-driven model of biomarker changes in sporadic Alzheimer’s disease. Brain. 2014; 137:2564–77. https://doi.org/10.1093/brain/awu176 [PubMed]
  • 18. Greicius MD, Menon V. Default-mode activity during a passive sensory task: uncoupled from deactivation but impacting activation. J Cogn Neurosci. 2004; 16:1484–92. https://doi.org/10.1162/0898929042568532 [PubMed]
  • 19. Serra L, Cercignani M, Mastropasqua C, Torso M, Spanò B, Makovac E, Viola V, Giulietti G, Marra C, Caltagirone C, Bozzali M. Longitudinal Changes in Functional Brain Connectivity Predicts Conversion to Alzheimer’s Disease. J Alzheimers Dis. 2016; 51:377–89. https://doi.org/10.3233/JAD-150961 [PubMed]
  • 20. Challis E, Hurley P, Serra L, Bozzali M, Oliver S, Cercignani M. Gaussian process classification of Alzheimer’s disease and mild cognitive impairment from resting-state fMRI. Neuroimage. 2015; 112:232–43. https://doi.org/10.1016/j.neuroimage.2015.02.037 [PubMed]
  • 21. Eskildsen SF, Coupé P, García-Lorenzo D, Fonov V, Pruessner JC, Collins DL, and Alzheimer’s Disease Neuroimaging Initiative. Prediction of Alzheimer’s disease in subjects with mild cognitive impairment from the ADNI cohort using patterns of cortical thinning. Neuroimage. 2013; 65:511–21. https://doi.org/10.1016/j.neuroimage.2012.09.058 [PubMed]
  • 22. Schroeter ML, Stein T, Maslowski N, Neumann J. Neural correlates of Alzheimer’s disease and mild cognitive impairment: a systematic and quantitative meta-analysis involving 1351 patients. Neuroimage. 2009; 47:1196–206. https://doi.org/10.1016/j.neuroimage.2009.05.037 [PubMed]
  • 23. deToledo-Morrell L, Stoub TR, Bulgakova M, Wilson RS, Bennett DA, Leurgans S, Wuu J, Turner DA. MRI-derived entorhinal volume is a good predictor of conversion from MCI to AD. Neurobiol Aging. 2004; 25:1197–203. https://doi.org/10.1016/j.neurobiolaging.2003.12.007 [PubMed]
  • 24. Chen J, Zhang Z, Li S. Can multi-modal neuroimaging evidence from hippocampus provide biomarkers for the progression of amnestic mild cognitive impairment? Neurosci Bull. 2015; 31:128–40. https://doi.org/10.1007/s12264-014-1490-8 [PubMed]
  • 25. Buckner RL. Memory and executive function in aging and AD: multiple factors that cause decline and reserve factors that compensate. Neuron. 2004; 44:195–208. https://doi.org/10.1016/j.neuron.2004.09.006 [PubMed]
  • 26. Greicius MD, Srivastava G, Reiss AL, Menon V. Default-mode network activity distinguishes Alzheimer’s disease from healthy aging: evidence from functional MRI. Proc Natl Acad Sci USA. 2004; 101:4637–42. https://doi.org/10.1073/pnas.0308627101 [PubMed]
  • 27. Petrella JR, Sheldon FC, Prince SE, Calhoun VD, Doraiswamy PM. Default mode network connectivity in stable vs progressive mild cognitive impairment. Neurology. 2011; 76:511–17. https://doi.org/10.1212/WNL.0b013e31820af94e [PubMed]
  • 28. Lehallier B, Essioux L, Gayan J, Alexandridis R, Nikolcheva T, Wyss-Coray T, Britschgi M, Alzheimer’s Disease Neuroimaging Initiative. Combined Plasma and Cerebrospinal Fluid Signature for the Prediction of Midterm Progression From Mild Cognitive Impairment to Alzheimer Disease. JAMA Neurol. 2016; 73:203–212. Epub 2015 Dec 14. https://doi.org/10.1001/jamaneurol.2015.3135 [PubMed]
  • 29. Querbes O, Aubry F, Pariente J, Lotterie JA, Démonet JF, Duret V, Puel M, Berry I, Fort JC, Celsis P, and Alzheimer’s Disease Neuroimaging Initiative. Early diagnosis of Alzheimer’s disease using cortical thickness: impact of cognitive reserve. Brain. 2009; 132:2036–47. https://doi.org/10.1093/brain/awp105 [PubMed]
  • 30. Korolev IO, Symonds LL, Bozoki AC, and Alzheimer’s Disease Neuroimaging Initiative. Predicting Progression from Mild Cognitive Impairment to Alzheimer’s Dementia Using Clinical, MRI, and Plasma Biomarkers via Probabilistic Pattern Classification. PLoS One. 2016; 11:e0138866. https://doi.org/10.1371/journal.pone.0138866 [PubMed]
  • 31. Liu H, Zhou X, Jiang H, He H, Liu X, Weiner MW, Aisen P, Petersen R, Jack CR, Jagust W, Trojanowki JQ, Toga AW, Beckett L, et al, and Alzheimer’s Disease Neuroimaging Initiative. A semi-mechanism approach based on MRI and proteomics for prediction of conversion from mild cognitive impairment to Alzheimer’s disease. Sci Rep. 2016; 6:26712. https://doi.org/10.1038/srep26712 [PubMed]
  • 32. Gomar JJ, Bobes-Bascaran MT, Conejero-Goldberg C, Davies P, Goldberg TE, and Alzheimer’s Disease Neuroimaging Initiative. Utility of combinations of biomarkers, cognitive markers, and risk factors to predict conversion from mild cognitive impairment to Alzheimer disease in patients in the Alzheimer’s disease neuroimaging initiative. Arch Gen Psychiatry. 2011; 68:961–69. https://doi.org/10.1001/archgenpsychiatry.2011.96 [PubMed]
  • 33. Sohn BK, Yi D, Seo EH, Choe YM, Kim JW, Kim SG, Choi HJ, Byun MS, Jhoo JH, Woo JI, Lee DY. Comparison of regional gray matter atrophy, white matter alteration, and glucose metabolism as a predictor of the conversion to Alzheimer’s disease in mild cognitive impairment. J Korean Med Sci. 2015; 30:779–87. https://doi.org/10.3346/jkms.2015.30.6.779 [PubMed]
  • 34. Lebedev AV, Westman E, Van Westen GJ, Kramberger MG, Lundervold A, Aarsland D, Soininen H, Kłoszewska I, Mecocci P, Tsolaki M, Vellas B, Lovestone S, Simmons A, and Alzheimer’s Disease Neuroimaging Initiative and the AddNeuroMed consortium. Random Forest ensembles for detection and prediction of Alzheimer’s disease with a good between-cohort robustness. Neuroimage Clin. 2014; 6:115–25. https://doi.org/10.1016/j.nicl.2014.08.023 [PubMed]
  • 35. Young J, Modat M, Cardoso MJ, Mendelson A, Cash D, Ourselin S, and Alzheimer’s Disease Neuroimaging Initiative. Accurate multimodal probabilistic prediction of conversion to Alzheimer’s disease in patients with mild cognitive impairment. Neuroimage Clin. 2013; 2:735–45. https://doi.org/10.1016/j.nicl.2013.05.004 [PubMed]
  • 36. Bellman R. Adaptive Control Processes: A Guided Tour. Princeton University Books, 1961.
  • 37. Hampel H, Frank R, Broich K, Teipel SJ, Katz RG, Hardy J, Herholz K, Bokde AL, Jessen F, Hoessler YC, Sanhai WR, Zetterberg H, Woodcock J, Blennow K. Biomarkers for Alzheimer’s disease: academic, industry and regulatory perspectives. Nat Rev Drug Discov. 2010; 9:560–74. https://doi.org/10.1038/nrd3115 [PubMed]
  • 38. McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer’s Disease. Neurology. 1984; 34:939–44. https://doi.org/10.1212/WNL.34.7.939 [PubMed]
  • 39. Petersen RC, Smith GE, Waring SC, Ivnik RJ, Tangalos EG, Kokmen E. Mild cognitive impairment: clinical characterization and outcome. Arch Neurol. 1999; 56:303–08. https://doi.org/10.1001/archneur.56.3.303 [PubMed]
  • 40. Winblad B, Palmer K, Kivipelto M, Jelic V, Fratiglioni L, Wahlund LO, Nordberg A, Bäckman L, Albert M, Almkvist O, Arai H, Basun H, Blennow K, et al. Mild cognitive impairment—beyond controversies, towards a consensus: report of the International Working Group on Mild Cognitive Impairment. J Intern Med. 2004; 256:240–46. https://doi.org/10.1111/j.1365-2796.2004.01380.x [PubMed]
  • 41. Chen J, Shu H, Wang Z, Zhan Y, Liu D, Liao W, Xu L, Liu Y, Zhang Z. Convergent and divergent intranetwork and internetwork connectivity patterns in patients with remitted late-life depression and amnestic mild cognitive impairment. Cortex. 2016; 83:194–211. https://doi.org/10.1016/j.cortex.2016.08.001 [PubMed]
  • 42. Chen J, Shu H, Wang Z, Liu D, Shi Y, Zhang X, Zhang Z. The interaction of APOE genotype by age in amnestic mild cognitive impairment: a voxel-based morphometric study. J Alzheimers Dis. 2015; 43:657–68. https://doi.org/10.3233/JAD-141677 [PubMed]
  • 43. McNeil BJ, Hanley JA. Statistical approaches to the analysis of receiver operating characteristic (ROC) curves. Med Decis Making. 1984; 4:137–50. https://doi.org/10.1177/0272989X8400400203 [PubMed]
  • 44. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988; 44:837–45. https://doi.org/10.2307/2531595 [PubMed]
  • 45. Brodersen KH, Ong CS, Stephan KE, Buhmann JM. The balanced accuracy and its posterior distribution. 2010 20th international conference, IEEE. 2010; 3121–3124.
Download PDF
Corresponding Authors

Zhijun Zhang
janemengzhang@vip.163.com

Shi-Jiang Li
sjli@mcw.edu

Keywords
Alzheimer’s disease mild cognitive impairment Characterizing Alzheimer’s disease Risk Events (CARE) index Alzheimer’s Disease Neuroimaging Initiative (ADNI)
Paper Sections