Journal of Alzheimers Disease & Parkinsonism
Open Access

AD: Alzheimer’s Disease; ADLS: Alzheimer’s Disease Likelihood Score; ADNI: Alzheimer’s Disease Neuroimaging Initiative; AI: Artificial Intelligence; AIBL: Australian Imaging, Biomarker and Lifestyle; ApoE: Apolipoprotein E; Aβ: Amyloid Beta; BAAD: Brain Anatomical Analysis Using Diffeomorphic Deformation; CNN: Convolutional Neural Network; CSF: Cerebrospinal Fluid; DARTEL: Diffeomorphic Anatomical Registration Through Exponentiated Lie Algebra; DLB: Dementia with Lewy Bodies; DMTs: Disease Modifying Therapies; FDG: 18 F-labeled Fluoro-2-Deoxyglucose; IXI: Information EXtraction from Images; JADNI: Japanese ADNI; LONI: Laboratory of Neuroimaging; LPBA40: LONI Probabilistic Brain Atlas; MCC: Matthews Correlation Coefficient; MCI: Mild Cognitive Impairment; MIRIAD: Minimal Interval Resonance Imaging in Alzheimer’s Disease; ML: Machine Learning; MMSE: Mental State Examination; MNI: Montreal Neurological Institute; MR: Magnetic Resonance; MRI: Magnetic Resonance Imaging; NIA-AA: National Institute on Aging and the Alzheimer’s Association; NPV: Negative Predictive Value; PET: Positron Emission Tomography; PPV: Positive Predictive Value; ROC: Receiver Operating Characteristic; ROI: Region of Interest; SUVR: Standardized Uptake Value Ratio; SVM: Support Vector Machine; SVMst: SVM that was trained by MRI only; CVMcog: SVM that was trained by MRI and MMSE; TIV: Total Intracranial Volume; VBM: Voxel-Based Morphometry; VSRAD: Voxel-Based Specific Regional Analysis system for Alzheimer’s Disease; p-tau: Phosphorylated tau; pMCI: progressive MCI; sMCI: stable MCI; t-tau: Total Tau

The proportion of dementia patients in society has been increasing as life expectancy improves, and preventive measures are an important social concern. Alzheimer’s disease International estimated that over 50 million people worldwide were living with dementia in 2019. The current annual cost for dementia care is estimated at 1.0 trillion US dollars, a figure set to double by 2030. According to the Ministry of Health, Labor and Welfare in Japan, the social cost of dementia in Japan in 2014 was about 14.5 trillion yen and is estimated to increase to 24.3 trillion yen in 2060.

The target of ongoing disease modifying therapies (DMTs) for Alzheimer’s disease (AD) has shifted from symptomatology-based mild cognitive impairment (MCI) to subjects with biologically high risk derived from biomarkers that explain the pathogenesis of dementia [1]. MCI is considered to be at the cognitive boundary between normal aging and dementia. According to the National Institute on Aging and the Alzheimer’s Association (NIA-AA), based on culturally appropriate normative data, cognitive test scores for individuals with MCI typically range from 1.0 to 1.5 standard deviations below the mean for their age and education matched peers. However, the NIA-AA emphasizes that these ranges are guidelines and not cutoffs, and that setting a cutoff value is not recommended for the diagnosis of MCI [2]. AD can be conceptualized as a seamless biological and clinical continuum from the preclinical (clinically asymptomatic but with AD pathology) to clinical phase, in which cognitive impairment severity is not perceptibly different at the boundaries. It is still not known in which phase of AD pathology DMT should be initiated for optimum cost-effectiveness. Considering the degree of certainty of disease manifestation and the length of the treatment period, it is important to find ways to identify early-stage MCI patients at greater risk of conversion to AD [3].

The recent and rapid developments in artificial intelligence (AI) and machine learning (ML) are expected to bring about a drastic change in industrial structure as well. ML requires large and highly reliable datasets for training, and the Alzheimer’s disease Neuroimaging Initiative (ADNI) database appears to be most suitable for this purpose at present. ADNI studies began primarily in the United States, but have been extended world-wide, including to Europe, Australia, Japan, China, Korea, and Taiwan. An open database also enables researchers to compare performance among similar studies, which encourages competition and technological improvement. Since computer science is a realm where high performance is always required, it is also necessary to assess the performance of the latest ML paradigms. Jo et al. have reported the 16 studies that have shown high ML performance for AD diagnosis and/or prediction of MCI progression using the combination of positron emission tomography (PET) and magnetic resonance (MR) imaging (MRI) [4]. One study assessed the use of structural MRI alone and reported that the accuracy of AD diagnosis using 3D convolutional neural network (CNN), 2D CNN, and support vector machine (SVM) was 91.1% [5], 91.4% [6], and 93.0% [7], respectively; in two other studies, the accuracy of prediction of MCI progression using SVM was 74.7% and 75.0% [7,8]. Another recent study reported an almost 90% accuracy of ML-based MRI analysis for AD diagnosis, and this is considered the upper limit [9]. This may be because of the complex mixed-pathologies of the underlying disease or that the data used for training are not pathologically confirmed. Nevertheless, although these studies have demonstrated that ML performs well, it is important to note that they lack verification based on independent test data, and often do not consider metrics to maximize the overall accuracy of the classification model while maintaining generalizability.

Voxel-based morphometry (VBM) enables the analysis of shape-based characteristics of the brain by transforming an individual’s brain coordinates to that of a standard brain and converting volumetric information into a density map. Coordinate transformation to a standard brain enables the measurement of regional brain volume for each region of interest (ROI) that is designated in advance. Medial temporal atrophy is one of the biomarkers of AD, but several studies have reported the utility of more objective measurement [10]. For example, voxel-based specific regional analysis system for Alzheimer’s Disease (VSRAD) is a software developed to help clinicians diagnose AD based on the z-value of a ROI containing the hippocampus and its surroundings. Several methods have been reported for extracting regional brain volume features using VBM, reducing the number of features by principal components analysis [11] and probability distribution functions [12], and creating masks using pattern recognition [13] before transferring to SVM classifiers. Additionally, methods that create feature values using the brain atlas in advance [14,15], using multiple atlases [16,17], or using deep learning without VBM have also been reported [18-20].

We have developed software named Brain Anatomical Analysis using Diffeomorphic deformation (BAAD) (https://www.fil.ion.ucl. ac.uk/spm/ext/#BAAD), which integrates VBM and SVM to form an “AI” algorithm. BAAD evaluates individual brains using VBM, and the regularized features are then passed to a SVM to create an eminent algorithm. The purpose of this study is to validate the utility of this AI for the diagnosis of AD and for predicting the progression of MCI patients. For this purpose, we: 1) validated the AI utility for AD diagnosis and prediction of the progression of MCI patients in several test data cohorts for the application of this model in medical practice, 2) verified the superiority of the AI algorithm, which extracts features from the entire brain, by comparing with expert radiologists with and without the support of VSRAD indicating the degree of atrophy in the medial temporal structures, and 3) confirmed whether the AI can correctly diagnose AD, from a pathological perspective, based on cerebrospinal fluid (CSF) biomarker measurements, which was considered important given the complex pathology of the disease. We also discuss the potential contribution of the model to DMTs.

Participants and data source

The patient data for training and validating the proposed AI was obtained from 4 publicly available databases, namely ADNI, Japanese ADNI (JADNI), Australian Imaging, Biomarker and Lifestyle (AIBL), and Minimal Interval Resonance Imaging in Alzheimer’s disease (MIRIAD) databases. ADNI database consists of 4 phases, namely ADNI 1, ADNI GO, ADNI 2, and ADNI 3. Data of ADNI and AIBL subjects were extracted from the Laboratory of Neuroimaging (LONI) Image and Data Archive (ida.loni.usc.edu). JADNI data were extracted from the National Bioscience Database Center (Research ID: hum0043. v1). Detailed information of the database was reported by Iwatsubo et al. [21]. The ADNI was launched in 2003, and has been engaged since 2004 in longitudinal, multicenter cohort studies of healthy elderly individuals and individuals with MCI and early AD. Like the ADNI in North America, AIBL and JADNI were launched in 2006 and 2007, respectively. Details of the database and diagnostic criteria were reported by Petersen et al. [22]. The MIRIAD study was designed to investigate the feasibility of using MRI as an outcome measure in clinical trials for AD treatment. Details of the study were reported by Malone et al. [23].

We initially included 1446 subjects from the ADNI database-543 cognitively normal (NL), 544 MCI, and 359 AD participants. We classified MCI as progressive MCI (pMCI) or stable MCI (sMCI) based on whether it progressed to AD. Of the sMCI cases, 132 were excluded from further analyses because of the short observation period (<4 years) (Supplemental Figure 1). To reduce potential errors in clinical diagnosis, we also excluded 17 AD and 14 pMCI patients who were negative for brain amyloid beta (Aβ) deposition on florbetapir (AV-45) PET. To verify the generalization capability of the AI, we used other datasets of 519 subjects from AIBL, 535 subjects from JADNI, and 69 subjects from MIRIAD (Table 1).

Table 1: Demographic features of the databases.

Figure 1: Scatter plot between Z score of VSRAD and ADLS of SVMst. The horizontal axis represents the value of ADSL, and the vertical axis represents the z-value of VSRAD. The subjects in the lower right quadrant are subjects without medial temporal atrophy, but SVMst indicates those as AD spectrum. The upper left quadrant is a group of subjects with medial temporal atrophy, but SVMst indicates those as non-AD spectrum.

This study was approved by the Shiga University of Medical Science Research Review Board (approval number: 29-012) according to the Ethical Guidelines for Medical and Health Research Involving Human Subjects.

Image acquisition and processing

Brain structural MR images were acquired using 1.5 T (ADNI 1, AIBL, JADNI) and 3.0 T scanners (ADNI GO, ADNI 2) from several vendors such as Philips Medical Systems, Siemens, and GE Healthcare. All images were scanned under the ADNI protocol conditions with 3D-sagittal plane slices and the magnetization prepared rapid acquisition gradient echo (MPRAGE) sequence [24].

Voxel-based morphometry: MR images were processed using the BAAD software (version 4.4) which enables automated VBM. The details of a standard VBM procedure have been described elsewhere [25]. Coordinate transformation from native space to the Montreal Neurological Institute (MNI) space was applied using the Diffeomorphic Anatomical Registration Through Exponentiated Lie Algebra (DARTEL) algorithm [26]. The DARTEL templates were prepared based on 550 healthy control subjects from the Information eXtraction from Images (IXI) database. The segmented data were modulated for volume estimation using a Jacobian matrix. Total intracranial volume (TIV) was calculated as the sum of the gray matter, white matter, and CSF volumes.

Region of interest (ROI) analyses: We used multiple sets of adaptive ROIs to reduce the number of feature vectors for the SVM classifier instead of applying a voxel-based approach [27]. We used the MarsBar toolbox (http://marsbar.sourceforge.net) for ROI-wise analysis using Automated Anatomical Labeling atlases, Brodmann’s atlas, and the LONI Probabilistic Brain Atlas (LPBA40) in the MNI space. Each atlas has 108, 118, and 80 ROIs respectively. For z-score estimation, we used TIV as the confounding factor and subjects aged over 50 in the IXI database as a reference for age. Z-scores were derived using the following formula: ([control mean]-[individual value])/ (control standard deviation).

Machine learning: We expressed the probability of an AD brain as the AD likelihood score (ADLS), which represents the distance to the hyperplane and is obtained using the posterior probability function, Pr=(Y=k|X=x), where the probability Y is the class k given that the input variable X is x. The probability is transformed by the sigmoid function to squash the value within the range [0,1]; a larger value means a greater likelihood of AD. We used the radial basis function for the SVM kernel, and the values of parameters were optimized using the ADNI database. For training, we introduced the spatial-anatomical approach by using ROI z-values as feature vectors; leave-one-out cross validation was used to obtain an unbiased estimation and to reduce over-fitting.

MCI can be diagnosed based on prescribed criteria at the research level, but it may be difficult to implement the same complex protocol in clinical practice. For this reason, we labeled AD and pMCI as the AD spectrum and NL and sMCI as the non-AD spectrum for SVM learning. Half (n=642) of the 1283 subjects were randomly selected for SVM training (training dataset). To fine-tune the hyperparameters, 10% of the training dataset was used for validation (n=60). We named the created classifier SVMst. Since the assessment of cognitive ability was important for the diagnosis of dementia, we included the Mini- Mental State Examination (MMSE) score in SVM learning, and named the classifier SVMcog.

Test datasets and tasks for radiologists: To investigate the generality of our algorithm, the AIBL, JADNI, and MIRIAD databases were analyzed as test datasets, along with the remaining ADNI subjects who were not recruited for training (Table 1). There are several ways to measure hippocampal volume, including by manual delineation, automated techniques, and qualitative ratings. Previous reports have demonstrated that manual and automated measurements are well- correlated [28,29]. In this study, we used VSRAD (version advance 2) as an example of an automated technique. The Japanese Society of Neurology recommends using VSRAD for AD diagnosis in its diagnostic guidelines [30]. VSRAD is based on VBM methods using a preset ROI of medial temporal structures, including the entorhinal cortex, hippocampus, and amygdala, and represents the z-scores of these regions [31].

Two radiologists (RI and HK), blinded to patient diagnosis and characteristics other than the age and gender, independently reviewed structural MRI sets in random order. Both radiologists are board- certified experts in Japan with more than 20 years of clinical experience. MR images were randomly extracted from an ADNI dataset with 100 AD and 100 NL subjects. First, 10 AD and 10 NL subjects from the sets were extracted for training of the radiologists. Second, several days later, the radiologists were asked to diagnose AD or NL based on the 200 structural images. Third, once the diagnosis was completed, the radiologists were allowed to modify their initial diagnosis according to the VSRAD results.

CSF biomarker measurement: In the ADNI study, CSF biomarkers were measured as previously described [32]. Briefly, CSF was collected in tubes, frozen, and shipped to the ADNI Biomarker Core laboratory at the University of Pennsylvania Medical Center. Aβ_1-42, total tau (t-tau), and phosphorylated tau (p-tau) levels were measured using the Luminex 100 IS (Luminex Corporation, Austin, TX) or BioPlex 100 immunoassay platform (Bio Rad, Hercules, CA) and the INNOBIA AlzBio3 kit (Fujirebio Europe, Ghent, Belgium). The cutoff values for each biomarker were set according to published ADNI study group data [33] (192 pg/ml, 23 pg/ml, and 93 pg/ml for Aβ, p-tau, and t-tau, respectively). The t-tau/Aβ_1-42 and p-tau/Aβ_1-42 ratios were set to 0.39 and 0.1, respectively.

Amyloid PET: Amyloid PET images were obtained using different PET scanners, manufactured by GE, Siemens, and Philips. The dynamic 3D acquisition method was used, consisting of 4 frames of a 5-min scan in a 30-60 min interval after intravenous injection of 370 MBq (approximately 10 mCi) of 18 F-AV-45. To define the four cortical grey matter regions (frontal, anterior/posterior cingulate, lateral parietal, lateral temporal), brain MRI for each subject was segmented and parcellated using FreeSurfer (version 5.3.0). The standardized uptake value ratio (SUVR) was calculated by creating a conventional (non-weighted) average across the four main cortical regions and was normalized based on the entire cerebellum reference region. All data were downloaded from the LONI web site (https://ida.loni.usc.edu/ login.jsp). The SUVR cutoff value was defined as 1.11 according to the recommendation from UC Berkeley [34].

Statistics: Statistical analyses were performed using the JMP® software (version 14.3, SAS Institute, Cary, NC). Receiver operating characteristic (ROC) curves were created to evaluate each biomarker’s ability for disease diagnosis. The equations for accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F1, and Matthews correlation coefficient (MCC) are available elsewhere [35]. MCC is the geometric mean of the regression coefficients for binary classification problem, and it is robust against imbalanced classes. The level of statistical significance was set to p<0.05.

The AIBL study comprised more NL subjects than the ADNI and JADNI studies. The mean age of NL subjects was significantly different between the ADNI-JADNI, ADNI-MIRIAD, AIBL-JADNI, AIBL- MIRIAD, and ADNI-AIBL pairs (Student’s t-test). The mean age of AD patients was significantly different between the ADNI-MIRIAD, JADNI- MIRIAD, AIBL-MIRIAD, and ADNI-AIBL pairs (Student’s t-test). The mean MMSE score of NL subjects was significantly different between the AIBL-MIRIAD, AIBL-JADNI, and AIBL-ADNI pairs (Student’s t-test). The mean MMSE score of AD patients was significantly different in all database pairs (Student’s t-test); the databases in descending order of MMSE score are ADNI, JADNI, AIBL and MIRIAD. The ADNI studies targeted early-stage AD, while the MIRIAD study targeted more advanced stages (Table 1).

Of 544 MCI cases (260 sMCI and 284 pMCI) in the ADNI database, 52.2% (284 cases) progressed to AD during follow-up. The observation period for pMCI ranged from 6 to 108 months (mean [SD], 43.0 (23.6) months; median, 36 months). The observation period for the selected sMCI patients (n=128) ranged from 48 to 120 months (mean [SD], 63.0 (21.8) months; median, 60 months). Patients with pMCI in the ADNI database were most likely to progress to AD within 1 year-87.3% of cases progressed within 3 years and 95.8% within 4 years (Supplemental Figure 1). The apolipoprotein E (ApoE) allele ε4 number (0, 1, 2) was significantly different in all pairs except the AD-pMCI and sMCI-NL pairs (Steel-Dwass test) (Supplemental Figure 2).

Figure 2: ROC curves for prediction of MCI progression by SVMs. The subjects of the ADNI were selected for the test dataset. The cutoff values of SVMs were set to ADLS>0.5. The accuracy of SVMst and SVMcog in predicting MCI progression was 83.0% and 85.0%, respectively. Since the ADNI database was used for training, it was further examined using the JADNI database to verify its generality. Unfortunately, due to the short follow-up period in JADNI, NL was replaced by sMCI patients. For reference, we also validated the same combination in the ADNI. The accuracy of SVMst and SVM cog for classification of NL and pMCI in JADNI was 86.4% and 91.2%, respectively, not much different from 87.3% and 90.7% in ADNI. Numerical values in the graph indicate area under the receiver operating characteristic (ROC) curve and its area under the curve (AUC).

The relationship between SVMst ADLS and VSRAD z-score in the ADNI database (n=1314) is shown in Figure 1. Since VSRAD indicates the degree of atrophy in the medial temporal structures based on z-score, the subjects in the lower right orthant have less hippocampal atrophy, but SVMst indicates high AD probability. Inversely, subjects in the upper left orthant have medial temporal atrophy, but SVMst indicates low AD probability. Note that AD (red) and pMCI (orange) patients are predominant in the lower right quarter, while NL subjects (blue) and sMCI patients (light blue) are predominant in the upper left quarter. These results suggest that SVMst can presume AD spectrum even in a subject without medial temporal atrophy, and can presume healthy even in the subjects having medial temporal atrophy.

Results of CSF biomarker analysis are summarized in Supplemental Table 1. In terms of Aβ_1-42, t-tau, p-tau, t-tau/Aβ_1-42, and p-tau/Aβ_1-42, there were significant differences in all combinations except AD-pMCI and NL-sMCI (Wilcoxon test). The distribution of biomarker levels in each group is shown in Supplemental Figure 3. The concentration of Aβ_1-42 was bimodally distributed in NL and sMCI groups. The results of AD/NL classification and prediction of MCI progression based on CSF biomarker levels are summarized in Supplemental Table 2.

Table 2: Results of SVMs for AD diagnosis in several datasets.

Figure 3: Number of subjects per CSF biomarkers in groups classified by SVMst. In the SVMst-positive (ADLS>0.5) group, 72/75 (96.0%) of the MCI with Aβ-positive progressed to AD. Note that 92.2% of the subjects classified as SVMst-positive were Aβ-positive. In the SVMst-negative group, 46.6% of subjects were Aβ-positive, of which 19/39 (48.7%) of MCI progressed to AD. These results clearly show that among MCI with positive Aβ, those with positive SVMst are more likely to convert to AD. Numerical values in the figure indicate the number of subject in each group. “A” and “pT” indicate amyloid β and phosphorylated tau of the cerebrospinal fluid, respectively, and the signs in the parentheses indicate positive or negative for these biomarkers. The light gray in the figure indicates positive for amyloid-beta and the dark gray for negative as well.

AD-NL classification

The performance of the SVMs in the test datasets are summarized in Table 2. The cutoff values for SVMs were set to ADLS>0.5. The classification accuracy of SVMst in the test dataset ranged from 88.0% to 94.2%. With the addition of the MMSE score, the diagnostic accuracy of SVMcog became higher than that of SVMst, ranging from 92.5% to 100%. The diagnostic accuracy of the two radiologists was 57.5% and 70.0%, respectively, which were lower than the accuracy of SVMst of 90.5% (ADLS>0.5). By adding the VSRAD z-value information, the diagnostic accuracy of the radiologists improved to 70.0% and 73.0%, which was still lower than that of SVMst. The kappa coefficient, which is the coincidence rate between the diagnoses of the two radiologists, was poor (0.35; p=0.710, McNemar test) at the first diagnosis, but improved to 0.56 by referring to the VSRAD (p=0.016, McNemar test) (Supplemental Table 3).

Table 3: Cases of false negative and false positive with BAAD-AI.

Prediction of disease progression

To assess the ability of the two SVMs to predict MCI progression, we analyzed data from 138 pMCI and 64 sMCI subjects from the ADNI test dataset (Figure 2). Cutoff values for the SVMs were set to ADLS>0.5. The predictive accuracy of SVMst and SVMcog were 83.2% and 85.1%, respectively. To validate the versatility, we further examined this using the JADNI database. Unfortunately, due to short follow- up periods none of the sMCI patients met our criteria; therefore, we substituted NL for sMCI patients. For reference, we also examined the ADNI test dataset with the same combination. The accuracy of SVMst and SVM cog for classification of NL and pMCI in JADNI was 86.4% and 91.2%, respectively, not much different from 87.3% and 90.7% in ADNI. Although these results cannot be interpreted as the SVMs being substantially accurate for predicting MCI progression in the JADNI dataset, it could be expected that the SVMs would work well in it, since the accuracy did not differ between the ADNI test dataset and JADNI dataset.

With regard to prediction of disease progression among ADNI test dataset subjects who did not have dementia (NL and MCI), the PPV was 70.6% for SVMst and 72.7% for SVMcog when sensitivity was set greater than 90%. Likewise, the NPV was 95.6% for SVMst and 95.7% for SVMcog with sensitivity was greater than 90%. When restricted to MCI patients in this data set, the PPV for SVMst and SVMcog was 89.3% and 91.9%, respectively, and the corresponding NPV was 79.0% and 80.3% respectively, with sensitivity greater than 90%.

AD likelihood score and brain Aβ deposition

Since the cut-off value of CSF Aβ_1-42 concentration has been defined as 192 pg/ml in many previous studies [33,36-38], and the concentration is inversely correlated with brain Aβ deposition, we designated any subject with CSF Aβ_1-42, <192 pg/ml as Aβ positive. 92.2% of subjects in the ADNI test dataset who were classified as positive using the SVMst (ADLS>0.5) were Aβ positive (Figure 3). In the SVMst-positive group, the percentage of pMCI (n=72) cases among the Aβ-positive MCI patients (n=75) was 96.0%, or 97.3% of all pMCI patients in this class (n=74). In addition, two of five MCI patients (40.0%) progressed to AD, even though they were Aβ negative. In the SVMst-negative group, 46.6% of the subjects were Aβ positive, of which pMCI patients (n=19) accounted for 48.7% of the total MCI patients (n=39). In this group, only 13.4% (32/238) were AD or pMCI patients. Among only the Aβ- negative MCI patients (n=21), none progressed to AD. These results suggest that there is a strong association between Aβ accumulation and AD-like brain atrophy. Note that 79.8% (91/114) of the Aβ-positive MCI subjects progressed to AD within four years, and 79.1% (72/91) of them were SVMst-positive.

CSF biomarkers: in vivo diagnosis of AD pathology

We also interpreted our results by relating them to the ATN system recently proposed by the NIA-AA [39]. To speculate on the presence of AD pathology in vivo, we used Aβ_1-42, and p-tau CSF biomarker data from the ADNI database, representing extracellular amyloid plaques (low CSF Aβ_1-42) and intracellular neurofibrillary tangles (elevated CSF p-tau). In the SVMst-positive group, 156 of 180 subjects (86.7%) had AD pathology based on CSF biomarkers. Conversely, 86 of 238 subjects (36.1%) had AD pathology in the SVMst-negative group (Figure 3).

In the ADNI database, 91.7% and 90.5% subjects were consistent with AD pathology (positive for Aβ_1-42 and p-tau) based on CSF biomarkers in patients with clinically diagnosed AD and pMCI, respectively. In the ADNI test dataset, when SVMst was positive, 97.5% of AD and 91.9% of pMCI patients were consistent with AD pathology based on CSF biomarkers. Similarly, when SVMcog was positive, 94.4% of AD and 92.1% of pMCI patients were consistent with AD pathology based on CSF biomarkers, respectively (Supplemental Figure 4).

Figure 4: Progression-free survival curves in MCI patients. Symbols in parentheses indicate positive (+) or negative (-) results.

Progression-free survival analysis was performed to estimate the risk of conversion for each biomarker (Figure 4). The hazard ratios (HRs) are summarized in Supplemental Table 4. In case of subjects designated Aβ-positive based on CSF Aβ_1-42, the HR was 4.43, which was higher than VSRAD, but was lower than that of SVMst (2.52 and 4.86, respectively).

AI detects characteristic atrophic pattern in AD

We showed that SVMst and SVMcog can offer a critical opportunity for working on the diagnosis of AD and predicting disease progression in MCI patients. With the addition of the MMSE score, the diagnostic accuracy of SVMcog reached up to 95%, and the accuracy remained high for the other test-set cohorts as well. As far as we know, no studies have investigated the accuracy of AI or ML in several cohorts, including the JADNI. Medial temporal atrophy has long been accepted as a validated marker of AD and is estimated by radiologists visually or quantified using computer algorisms such as VSRAD. However, it is known that hippocampal involvement is not an essential factor in AD, and 10%-15% of cases of AD are hippocampal-sparing [39-41]. Therefore, relying solely on hippocampal atrophy is insufficient for AD diagnosis. As shown in Figure 1, it was apparent that the SVMst ADLS is not based solely on medial temporal atrophy, but on that of the whole brain. Tau pathology typically extends by neuron-to-neuron transmission, generally starting transentorhinally and extending medially [42], but it may extend laterally faster when Aβ is deposited in the temporal cortex. When tau pathology extends to the neocortex (Braak stage V/VI), which invariably affects cognition, patients start to exhibit AD symptoms [43]. Interestingly, several reports suggest that Aβ accelerates tau pathology extension [44,45]. Tau protein extracted from human brains with Aβ plaques was more seed-competent than that from brains without Aβ plaques [46]. Since tau pathology is a major driver of neurodegeneration in AD, and topological tau-PET distribution and subsequent brain atrophy are well correlated [47-49], our algorithm may be able to detect cell-to-cell transmission of tau pathology as a characteristic atrophic pattern in AD [50].

AI outperformed clinicians in MRI diagnosis of AD

In our model, all images were preprocessed using VBM and the features were standardized and simplified, that is, image noise removal, brain segmentation, coordinate transformation, brain volume adjustment by TIV and age, and finally SVM extraction of features to classify the brains in hyperspace was performed. To extract and integrate a large amount of information by surveying the whole brain is undoubtedly beyond the scope of human ability. In fact, the SVMs outperformed two expert radiologists in terms of AD diagnosis-the diagnostic accuracy of SVMst was 90.5%, whereas the mean accuracy of the two radiologists was 63.8%. It is noteworthy that the VSRAD z-score improved the accuracy of AD diagnosis by radiologists, but it was still lower than that of our AI model. Our algorithm utilized the whole brain with varying degrees of influence of various anatomic areas, and the diagnostic performance was apparently better than that of expert radiologists.

AI is not a panacea

We do not intend to claim that our algorithm is unique or can cover any type of cognitive disease, since the ADNI database is already selected and simplified due to the exclusion of patients with other types of dementia. We also acknowledge that the clinical use of AI is some way short of being a useful clinical tool that can replace an expert. However, considering the rapid increase in the number of dementia patients who need to be cared for by primary clinicians, we believe that our AI can help clinicians estimate the likelihood of the disease. The exclusion criteria in the ADNI study-‘exclude any significant neurologic disease other than AD’-may also be difficult even for experts to implement without complete knowledge of the pathological findings. Thus, it is important to preemptively work towards how clinicians can use AI to perform diagnoses reliably and cost-effectively in the practical setting.

If there is a possibility of other dementia diseases, single-photon emission computed tomography imaging of cerebral blood flow is one technique with considerable diagnostic value, particularly in differentiating AD from frontotemporal dementia [51,52]. Dopamine transporter imaging is a useful tool to diagnose parkinsonism and its related disorders if dementia with Lewy bodies (DLB) or DLB with mixed pathology is suspected.

Metaiodobenzylguanidine myocardial scintigraphy is a widely accepted tool for differentiating Parkinson’s disease or DLB from other Parkinson-related disorders.

Here, we would like to call attention to the users by presenting the limitations of our AI. We focused on the false-negative cases where ADLS was less than 0.2 but the clinical diagnosis was AD or pMCI, and the false-positive cases where the clinical diagnosis was NL or sMCI despite ADLS of 0.8 or higher. The cases that did not undergo amyloid PET were excluded. The results were shown in Table 3 and Figures 5 and 6. The two false-negative AD cases and two pMCI cases extracted here were all positive for amyloid PET. The MRIs of each representative case were shown in Figure 6A and 6B. In both cases, brain atrophy was not noticeable. False-positive cases were observed in 10 NL and 4 sMCI. The MRIs of the representative cases among these were shown in Figure 6C and 6D. All of the false positive cases should have been clinically diagnosed as NL or MCI due to their high MMSE scores. On the other hand, 5 out of 10 false-positive NL cases and 2 out of 4 sMCI cases were positive for brain amyloid on PET, indicating that follow-up is necessary for cases with high ADLS.

Figure 5: False negative cases with low ADLS and false positive cases with high ADLS This figure is derived from Figure 1. Cases that underwent AV-45 PET were plotted in the right side. False-negative cases with ADLS below 0.2 and false-positive cases with ADLS above 0.8 are circled by squares. A~D indicates the cases of MRI in Figure 6.

Figure 6: Illustrative MRI of the cases with false negative and false positive. Please refer to Table 3 for details of each case. Cases A and B had positive amyloid PET results and were clinically diagnosed as AD and MCI, respectively, despite ADLS less than 0.2. Case B was subsequently converted to AD. Cases C and D had ADLS greater than 0.8, respectively. Cases C and D had negative amyloid PET and were clinically diagnosed as NL and MCI, respectively; case D remained MCI for 8 years.

Concordance between clinical diagnosis and pathology is ~90% in ADNI

Along with the elucidation of the AD pathology, the NIA-AA research framework advocated a new biomarker-based grouping called the ATN system [38]. Cortical amyloid PET ligand binding or low CSF Aβ42 are biomarkers of Aβ plaques (A). Elevated CSF phosphorylated tau and cortical tau PET ligand binding are biomarkers of fibrillar tau (T). Elevated CSF t-tau, 18 F-labeled fluoro-2-deoxyglucose (FDG) PET hypometabolism, and brain atrophy on MRI are biomarkers of neurodegeneration or neuronal injury (N). According to this system, extracellular amyloid plaques (A) and intracellular neurofibrillary tangles (T) are histopathologic features of AD. It is interesting to note that 88% of clinically diagnosed AD and 86.1% of pMCI cases in the ADNI study were consistent with the results of in vivo AD pathology assessed using CSF biomarkers, since similar results were reported from the Consortium to Establish a Registry for Alzheimer’s Disease study, which reported that 87% patients with clinically diagnosed AD are neuropathologically confirmed on autopsy [53]. This demonstrates the excellent agreement of clinical diagnosis in the ADNI study, and also the limitation of maximum 90%-95% accuracy for AI or ML was trained based on clinical diagnosis in the study.

AI reflects Aβ deposition in the brain

In the SVMst-classified high AD likelihood (ADLS>0.5) group, 89.1% of the subjects were Aβ positive, 91.1% had AD or pMCI, and only 8.9% were NL or had sMCI. Of all MCI patients classified into high AD likelihood group, 92.5% had pMCI, whereas in the SVMst- classified low AD likelihood group, the percentage of subjects with and without Aβ deposition were 46.7% and 53.3 %, respectively. Of all MCI patients who were classified into low AD likelihood group, 65.8% had sMCI, and 87.5% were NL or had sMCI. Therefore, SVMst has enough potential as a primary screening test to select candidates for DMT before amyloid PET examination. We showed that the risk of disease progression from MCI to AD was better predicted by SVMst than by CSF biomarkers (Supplemental Table 3). This result is not surprising, since Aβ deposition occurs earlier in AD pathogenesis than neuronal injury. Therefore, it is important to ascertain whether the therapeutic window is still open after neuronal injury has already started. Unfortunately, we do not have an answer to this, and it would be actually difficult to broaden the scope of disease-modifying drugs to a subject before any symptoms occur.

Concerns about DMT for Aβ

Recently, the 221AD301 ENGAGE study reported that subjects who received the highest aducanumab dose had a significant reduction in cognitive decline with regard to the primary endpoint, the Clinical Dementia Rating-Sum of Boxes. The subjects were amyloid PET positive MCI patients with AD or mild AD and MMSE scores between 24 and 30. It is noteworthy that this study suggested that clearance of aggregated Aβ can reduce the cognitive decline due to AD pathology and, above all, that the DMT window is still open in MCI and even in early-stage AD. Nevertheless, although starting treatment earlier may be more effective, it should be noted that people with Aβ burden do not necessarily progress to AD. For example, according to NIA-AA criteria, the progression risk for isolated amyloid pathology is only 22% [54]. Therefore, not every Aβ-positive MCI patient is likely to be a definite candidate for Aβ modification therapy. Prior studies have reported Aβ-positive MCI patient conversion rates of 40% [55], 48% [56], 57% [57,58] and 71% [59]. The variation between these studies may depend on the speed of Aβ and tau pathology progression of the subjects rather than the follow-up period. A prospective amyloid and tau PET study showed that an antecedent rise in Aβ was associated with subsequent tau-related changes in the brains of clinically normal elderly subjects [60].

Collaboration between Aβ detection and MRI-AI for selection of DMT candidate

With the traditional approach of incorporating biomarkers after the appearance of clinical symptoms, it may be rather late to apply drug therapies and other interventions. According to the amyloid cascade hypothesis, amyloidosis induces or facilitates the spread of pathologic tau, pathologic tau is immediately proximate to neurodegeneration, and neurodegeneration is the proximate cause of cognitive decline. Since ‘A’ and ‘T’ proteinopathies define AD as a unique disease among the many that can lead to dementia, the most ideal time to apply DMT efficiently would be before ‘N’ but following ‘A’ and ‘T’. What is important is how soon the appearance of ‘N’ can be detected. In our study, the AI was better than CSF biomarkers at predicting progression (Figure 4). CSF sample procurement needs lumbar puncture and is less informative regarding anatomical distribution compared to neuronal imaging. For this reason, patients with Aβ deposition by PET and high risk for progression predicted by MRI-AI may be strong candidates for DMT.

Limitations of the Study

WThis study had several limitations. First, as is frequently discussed in studies using the ADNI database, patients with dementia not related to AD are already carefully excluded. Therefore, the clinical relevance of AD diagnosis by our model is restricted to scenarios where patients with other forms of dementia have been screened out. Second, the definition of sMCI in our study is based on the length of the follow-up period. For example, some MCI patients may have eventually progressed to AD by now. Therefore, prediction by the AI cannot be used for long-term preventive treatment. Since the criteria for extraction of MCI patients is weighted to the amnestic type in the ADNI study, our model cannot deal with non-amnestic types. Third, measurement of CSF biomarkers can have various issues. For example, there are biases and random variations in biomarker measurements both within and between laboratories, and the markers only reflect part of the pathology underlying AD [61].

Based on the results of this study, our AI may be a promising tool that can support clinicians in diagnosing AD and predicting its progression, but the possibility of dementia other than that related to AD should always be kept in mind. When SVMst suggests a high probability of AD spectrum, about 90% of the patients are Aβ positive. In case of MCI, hazard ratio of conversion to AD is 3.6 when the ADLS is more than 0.5. Therefore, it is advisable that these patients are carefully observed for more than three years, or that molecular PET is considered to confirm AD pathology for DMT.

DMTs are likely to succeed if initiated at the preclinical stage of AD; furthermore, biomarker driven strategies for selecting the target are critical. In this respect, our AI could help optimize selection algorithms in order to increase study power and decrease observational periods.

We appreciate Mr. Eiichi Iwanari and Mr. Takafumi Shiino for their work in programming BAAD software. We would like to thank Editage (www.editage.com) for English language editing.

MI: MR image processing, literature search, data analysis and writing; SAH: data interpretation and reviewed the manuscript; IR and HK: participated in MRI diagnosis and competed with AI; KT, YS and AN; data interpretation and clinically reviewed the manuscript; AS: study design, data analysis, and reviewed the manuscript.

All authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.