J.R. Bock1, J. Hara1,2, D. Fortier1, M.D. Lee3, R.C. Petersen4, W.R. Shankle1-3 The Alzheimer’s Disease Neuroimaging Initiative*
1. Medical Care Corporation, Newport Beach, USA; 2. Pickup Family Neurosciences Institute, Hoag Memorial Hospital Presbyterian, Newport Beach, USA; 3. Dept. of Cognitive Sciences, University of California, Irvine, USA; 4. Department of Neurology, Mayo Clinic, USA; *Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf
Corresponding Author: Junko Hara, Ph.D. Medical Care Corporation, 3900 W. Coast Hwy, Ste 310, Newport Beach, CA 92663, Phone: 949-478-7388, Email: junkoh@mccare.com
J Prev Alz Dis 2020;
Published online November 11, 2020, http://dx.doi.org/10.14283/jpad.2020.63
Abstract
Background: Recent Alzheimer’s disease (AD) trials have faced significant challenges to enroll pre-symptomatic or early stage AD subjects with biomarker positivity, minimal or no cognitive impairment, and likelihood to decline cognitively during a short trial period. Our previous study showed that digital cognitive biomarkers (DCB), generated by a hierarchical Bayesian cognitive process (HBCP) model, were able to distinguish groups of cognitively normal individuals with impending cognitive decline from those without. We generated DCBs using only baseline Auditory Verbal Learning Test’s wordlist memory (WLM) item response data from the Mayo Clinic Alzheimer’s Disease Patient Registry.
Objectives: To replicate our previous findings, using baseline ADAS-Cog WLM item response data from the Alzheimer’s Disease Neuroimaging Initiative, and compare DCBs to traditional approaches for scoring word-list memory tests.
Design: Classified decliner subjects (n = 61) as those who developed amnestic MCI or AD dementia within 3 years of normal baseline assessment and non-decliner (n = 442) as those who did not.
Measures: Evaluated the relative value of DCBs compared to traditional measures, using three analytic approaches to group differences: 1) logistic regression of summary scores per ADAS-Cog WLM task; 2) Bayesian modeling of summary scores; and 3) HBCP modeling to generate DCBs from item-level responses.
Results: The HBCP model produced posterior distributions of group differences, of which Bayes factor assessment identified three DCBs with notable group differences: Immediate Retrieval from Durable Storage, (BFds = 11.8, strong evidence); One-Shot Learning, (BFds = 4.5, moderate evidence); and Partial Learning (BFds = 2.9, weak evidence). In contrast, logistic regression of summary scores did not significantly discriminate between groups, and the Bayes factor assessment of modeled summary scores provided moderate evidence that the groups were equivalent (BFsd = 3.4, 3.1, 2.9, and 1.4, respectively).
Conclusions: This study demonstrated DCBs’ ability to distinguish , at baseline, between impending cognitive decline and non-decline groups where individuals in both groups were classified as cognitively normal. This validated findings from our previous study, demonstrating DCBs’ advantages over traditional approaches. This study warrants further refinement of the HBCP DCBs to predict impending cognitive decline in individuals and other factors associated with AD, such as physical biomarker load.
Key words: Wordlist memory test, digital cognitive biomarkers, preclinical Alzheimer’s disease, clinical trial, Bayesian modeling.
Introduction
The major socioeconomic and healthcare burdens imposed by Alzheimer’s disease (AD) have pushed the focus of clinical research dramatically toward prevention and treatment in pre-symptomatic stages (1, 2). This shift has been well-aligned with guidance from the FDA in support of earlier stage therapies and new measurement methodologies for establishing clinically meaningful effects of those therapies (3).
However, recent AD trials have faced significant challenges identifying and enrolling subjects who meet thresholds for AD biomarker positivity but who have not yet experienced notable cognitive deficits. Despite great efforts made to enroll subjects in pre-clinical or early stage AD, trial sponsors have seen screen failure rates as high as 80%, primarily driven by required biomarker thresholds in cognitively normal subjects, leading to significantly prolonged enrollment periods and increased costs (4-9).
This underscores an urgent need for better approaches to pragmatically and cost effectively identify subjects who: 1) are cognitive normal; 2) will decline cognitively within 1-3 years; and 3) are likely to have PET scan positive AD biomarkers. Having such predictive capabilities will accelerate enrollment of specific subjects and will also improve study design for potentially shorter trial durations. The present study focuses on predicting impending cognitive decline in cognitively normal subjects. Predicting positivity for AD biomarkers will be discussed in a future publication.
Many efforts have focused on developing more sensitive cognitive assessment tools (e.g., composite scoring), including PACC (10) and ADCOMS (11). While some recently validated assessment tools can outperform less sensitive tools developed to assess dementia severity, as a group they lack the capability to predict impending cognitive decline in cognitively normal subjects.
One such assessment approach could arise from the application of hierarchical Bayesian cognitive process (HBCP) models to item-response data from a wordlist memory (WLM) test. This approach can generate digital cognitive biomarkers (DCB) that correspond to underlying cognitive processes of encoding, storage, and retrieval into and from various states of learning and memory (Figure 1). Such underlying cognitive processes cannot be directly observed or measured, while DCBs can quantify these processes, providing insights into cognitive function that traditional assessment approaches cannot provide. The details of this HBCP model and the generation of DCBs have been previously discussed elsewhere (12).
Figure 1. Hierarchical Bayesian Cognitive Process Model
The HBCP model can quantify underlying cognitive processes that cannot be observed or measured using the traditional approaches, and provide significant insights into how each cognitive process is affected by different conditions. Parameter r corresponds to one-shot learning; a, partial leaning; v, consolidation; b, testing effect; t, immediate retrieval from transient storage; L1, immediate retrieval from durable storage; and L2, delayed retrieval from durable storage.
In our previous work, HBCP-generated DCBs demonstrated the ability to distinguish groups of individuals with impending cognitive decline from those who would remain cognitively normal, using baseline, item-response data from a WLM test. This study was conducted using Auditory-Verbal Learning Test (AVLT) item response data from the Mayo Clinic Alzheimer’s Disease Patient Registry (13). Subjects, including those with normal cognition at baseline who would progress to amnestic MCI and those who would progress to AD dementia, were compared to those who would not decline. Bayes factor assessment identified notable reductions in Immediate Retrieval from Durable Storage, L1 (BFds = 30.4), and Delayed Retrieval from Durable Storage, L2 (BFds > 100). This study also appeared to identify compensatory increases in One-shot Learning, r (BFds = 3.2); Partial Learning, a (BFds = 10.8); and Consolidation, v (BFds = 13.5). However, subsequent work with our HBCP model did not replicate this apparent compensatory effect (12).
The present study, using baseline data from a novel WLM dataset, was designed to replicate our previous findings of deficits in retrieval DCBs for a group of individuals with impending cognitive decline due to AD, and compared this outcome to those generated by traditional scoring approaches. Replicating these results supports the role of DCBs for accelerating the clinical trial recruitment process and could greatly benefit future decisions about clinical trial study designs.
Methods
We used baseline ADAS-Cog WLM item response data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI: www.adni-info.org) database (adni.loni.usc.edu). The ADNI was launched in 2003 as a public-private partnership with the primary goal to test whether serial magnetic resonance imaging, positron emission tomography, other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment and early AD.
From the ADNI dataset, we classified non-decliner subjects (n = 442) as those whose diagnosis remained normal for 3 or more years after normal baseline assessment and decliner subjects (n = 61) as those who developed amnestic MCI or AD dementia within 3 years of normal baseline assessment. Table 1 shows sample characteristics.
Three analytic approaches were compared to demonstrate the relative value of DCBs.
Table 1. ADNI Sample Characteristics
Logistic Regression
Traditional summary scores per ADAS-Cog task were assessed for group differences. Logistic regression modeling was performed with individual subjects’ summary scores for immediate free recall tasks 1 through 3 and for the delayed free recall task, each included as predictors of impending cognitive decline as the outcome.
Bayesian Modeling
A Bayesian model of summary scores was assessed. Gaussian distributions were fitted to individuals’ number of items recalled, from 1 to 10, and uniform distributions for probability of 0 items recalled, on each free recall task.
HBCP Model
The HBCP model was applied to non-decliner and decliner group item response data, aggregated across subjects within each group. The model estimated DCBs across WLM items with the Batchelder multinomial processing tree model of memory for each item’s recall pattern across the four free recall tasks (12).
Results
Logistic regression of summary scores generated β coefficients (Table 2) that did not significantly discriminate between groups, either individually or across the test as a whole.
Table 2. Summary Score Logistic Regression Analysis
Note. IFR = Immediate Free Recall; DFR = Delayed Free Recall; OR = Odds Ratio. χ2(498) = 2.18, pseudo-R2 = -.005, p = .702.
Bayes Factor assessment of fitted Gaussian distributions to each free recall task by summary score measurement provided moderate evidence that the groups were measurably equivalent (BFsd = 3.4, 3.1, 2.9, and 1.4, respectively; Figure 2).
Posterior distributions of Bayesian-modeled summary scores and posterior distributions of mean differences across ADAS-Cog tasks are presented, with Savage-Dickey density ratio Bayes factors calculated for mean differences against prior distributions of no change.
The HBCP model produced posterior distributions of group differences (Figure 3). Bayes Factor assessment identified three DCBs with notable group differences: Immediate Retrieval from Durable Storage, L1 (BFds = 11.8, strong evidence), One-shot Learning, r (BFds = 4.5, moderate), and Partial Learning, a (BFds = 2.9, weak).
Figure 3. Bayes Factors for HBCP DCB Notable Group Difference Parameters
Posterior distributions of DCB mean differences are presented against prior distributions of no change for three parameters with notable group differences, along with Savage-Dickey density ratio Bayes factors.
Discussion
The present study validated our previous findings by demonstrating the HBCP DCBs’ ability to distinguish a group of cognitively normal individuals with impending cognitive decline from a group that would remain cognitively normal. This study also showed DCBs’ advantages over the traditional approach of summary score assessments and their applicability for detection of impending cognitive decline in asymptomatic AD patients.
The HBCP DCBs have an advantage over composite or summary score approaches because of their ability to measure and quantify underlying cognitive processes (Figure 1). Among these processes, only some are affected in the cognitively normal or pre-clinical stages of AD (14), and each is affected differently as the disease progresses (15). The HBCP model can also be applied to any existing, well-validated, WLM test protocol (e.g., AVLT, ADAS-Cog, MCI Screen), so DCBs can be generated on WLM data from past academic studies and clinical trials to examine which processes were improved by particular AD therapies (16) and which were not, even when traditional outcome measures identified no overall differences. This will provide novel insights into efficacy and trial design, potentially targeting different cognitive or disease processes.
This study warrants further development of the HBCP DCBs to predict impending cognitive decline at the individual level and to predict other factors associated with AD, such as the identification of stage progression, the accumulation of biomarkers, and the presence of other cognition-impairing conditions.
Funding: This study was supported by the National Institute on Aging of the National Institutes of Health under Award Number R44AG065126. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Acknowledgement: Data collection and sharing for this project was funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: AbbVie, Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merck & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.
Conflicts of interest: JRB, JH, DF, and WRS are employees of Medical Care Corporation.
Ethical standards: IRB exemption status was obtained for this study from WIRB.
Open Access: This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
References
1. Cummings J, Lee G, Mortsdorf T, Ritter A, Zhong K. Alzheimer’s disease drug development pipeline: 2017. Alzheimers Dement (N Y). 2017;3(3):367-384.
2. Cummings J, Ritter A, Zhong K. Clinical Trials for Disease-Modifying Therapies in Alzheimer’s Disease: A Primer, Lessons Learned, and a Blueprint for the Future. J Alzheimers Dis. 2018;64(s1):S3-S22.
3. Food and Drug Administration. Early Alzheimer’s Disease: Developing Drugs for Treatment Guidance for Industry: DRAFT GUIDANCE. February 2018. https://www.fda.gov/media/110903/download
4. Grill JD. Recruiting to preclinical Alzheimer’s disease clinical trials through registries. Alzheimers Dement (N Y). 2017;3(2):205-212. PMID: 28439532.
5. Jansen WJ, Ossenkoppele R, Tijms BM, Fagan AM, Hansson O, Klunk WE, et al. Association of Cerebral Amyloid-β Aggregation With Cognitive Functioning in Persons Without Dementia. JAMA Psychiatry. 2018;75(1):84-95. PMID: 29188296.
6. Langbaum JB, Karlawish J, Roberts JS, Wood EM, Bradbury A, High N, Walsh TL, Gordon D, Aggarwal R, Davis P, Stowell C, Trisko L, Langlois CM, Reiman EM, Tariot PN. GeneMatch: A novel recruitment registry using at-home APOE genotyping to enhance referrals to Alzheimer’s prevention studies. Alzheimers Dement. 2019;15(4):515-524. PMID: 30772251.
7. Rafii MS, Aisen PS. Alzheimer’s Disease Clinical Trials: Moving Toward Successful Prevention. CNS Drugs. 2019;33(2):99–106. PMID: 30560544.
8. Wolz R, Schwarz AJ, Gray KR, Yu P, Hill DL; Alzheimer’s Disease Neuroimaging Initiative. Enrichment of clinical trials in MCI due to AD using markers of amyloid and neurodegeneration. Neurology. 2016;87(12):1235–1241. PMID: 27558378.
9. Vermunt L, Muniz-Terrera G, Ter Meulen L, Veal C, Blennow K, Campbell A, Carrié I, Delrieu J, Fauria K, Huesa Rodríguez G, Ingala S, Jenkins N, Molinuevo JL, Ousset PJ, Porteous D, Prins ND, Solomon A, Tom BD, Zetterberg H, Zwan M, Ritchie CW, Scheltens P, Luscan G, Brookes AJ, Visser PJ, IMI-EPAD collaborators. Prescreening for European Prevention of Alzheimer Dementia (EPAD) trial-ready cohort: impact of AD risk factors and recruitment settings. Alzheimers Res Ther. 2020;12(1):8. PMID: 31907067.
10. Donohue M.C., Sperling R.A., Salmon D.P., Rentz D.M., Raman R., Thomas R.G. AIBL, ADNI, ADCS, the preclinical Alzheimer cognitive composite: measuring amyloid-related decline. JAMA Neurol. 2014;71:961–970.
11. Wang J, Logovinsky V, Hendrix SB, Stanworth SH, Perdomo C, Xu L, et al. ADCOMS: a composite clinical outcome for prodromal Alzheimer’s disease trials. J Neurol Neurosurg Psychiatry. 2016;87(9):993-9.
12. Lee MD, Bock JR, Cushman I, Shankle WR. An application of multinomial processing tree models and Bayesian methods to understanding memory impairment. J. Math. Psych. 2020;95:102328.
13. Shankle WR, Hara J, Bock JR, Fortier D, Mangrola T, Lee MD, Alexander GE, Batchelder WH, Petersen RC, Kremers W. Using Graphical Hierarchical Bayesian Cognitive Process Models Applied to Common Memory Tests to Predict AD Pathology within Normal Subjects. 2018. Clinical Trial in Alzheimer’s Disease. Poster Presentation. Barcelona, November 2018.
14. Braak H, Feldengut S, Del Tredici K. [Pathogenesis and prevention of Alzheimer’s disease: when and in what way does the pathological process begin?]. Nervenarzt. 2013;84(4):477-482. Review. German. PMID: 23508204.
15. Halliday GM, McCann H. Human-based studies on alpha-synuclein deposition and relationship to Parkinson’s disease symptoms. Exp Neurol. 2008;209(1):12-21. PMID: 17706644.
16. Roy DS, Arons A, Mitchell TI, Pignatelli M, Ryan TJ, Tonegawa S. Memory Retrieval by Activating Engram Cells in Mouse Models of Early Alzheimer’s Disease. Nature. 2016;531(7595):508-12. doi: 10.1038/nature17172.