M. Manjavong1, A. Diaz2-4, M.T. Ashford2-4, A. Aaronson3,4, M.J. Miller2-4, J.M. Kang5, S. Mackin3,4,6, R. Tank7, B. Landavazo3,4, D. Truran2,3, S.T. Farias8, M. Weiner2,3,9,10, R. Nosheny3,4,6, for the Alzheimer’s Disease Neuroimaging Initiative
1. Division of Geriatric Medicine, Department of Internal Medicine, Faculty of Medicine, Khon Kaen University, Khon Kaen, 40002, Thailand; 2. Northern California Institute for Research and Education (NCIRE), 4150 Clement Street, 151NC, San Francisco, CA, 94121-1563, USA; 3. VA Advanced Imaging Research Center, Department of Veterans Affairs Medical Center, 4150 Clement Street, San Francisco, California, 94121, USA; 4. Department of Radiology and Biomedical Imaging, University of California San Francisco, 505 Parnassus Avenue, M-391., San Francisco, CA, 94143-0628, USA; 5. Department of Psychiatry, Gil Medical Center, Gachon University College of Medicine, 21, Namdong-daero, 774 beon-gil, Namdong-gu, Incheon, 21565, Republic of Korea; 6. Department of Psychiatry, University of California San Francisco San Francisco California, 94143, USA; 7. Dementia Research Centre, UCL Institute of Neurology, University College London, London, WC1E 6BT, United Kingdom; 8. Departments of Neurology, University of California Davis, 3160, Folsom Blvd, Sacramento, Davis, CA, 95817, USA; 9. Department of Neurology, University of California San Francisco, 505 Parnassus Ave, San Francisco, CA, 94143, USA; 10. Department of Medicine, University of California San Francisco, 533 Parnassus Ave, San Francisco, CA, 94143, USA
Corresponding Author: Rachel L. Nosheny, Ph.D., Assistant Professor, University of California, San Francisco, Department of Psychiatry, San Francisco VA Medical Center, 4150 Clement Street (114M), San Francisco, CA 94121, Tel: 650-468-0619, Fax: 415-668-2864, email: rachel.nosheny@ucsf.edu
J Prev Alz Dis 2024;6(11):1741-1750
Published online June 21, 2024, http://dx.doi.org/10.14283/jpad.2024.109
Abstract
BACKGROUND: The Everyday Cognition (ECog) 12-item scale, a functional decline measurement, can distinguish dementia from cognitively unimpaired (CU). Limited data compare ECog-12 performance by raters (self vs. informant) and scoring systems (average numeric vs. categorical grouping) to differentiate cognitive statuses.
OBJECTIVES: To evaluate the performance of ECog-12 in differentiation cognitive statuses.
DESIGN: A cross-sectional diagnostic test study.
SETTING AND PARTICIPANTS: Data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) study are analyzed. Participants were aged 55-90 years old divided into subgroups based on diagnostic criteria.
MEASUREMENTS: We evaluated ECog-12 performance across different diagnostic groups, such as CU vs cognitive impairment (CI; mild cognitive impairment (MCI), and dementia), and the association between ECog-12 and CI. This procedure was repeated for self- and partner (informant)-reports. Additionally, types of ECog scores were also assessed, where an average ECog score was calculated (continuous numeric) as well as a categorical grouping (“any occasional declined” or “any consistently declined”) based on item-level responses to ECog questions.
RESULTS: ECog-12 cut-off scores of 1.36 (self-reported) and 1.45 (partner-reported) distinguish CU from CI with AUC 0.7 and 0.78, respectively. Adding a memory-concern question improved self-reported-ECog AUC to 0.79. Self- and partner-reported “consistently-declined” ECog-12 categorical grouping provided AUC 0.69 and 0.78. The study partner reported ECog-12 showed a greater association with CI than self-reported, with odds ratios of 35.45 and 8.79, respectively.
CONCLUSION: Study partner-reported ECog scores performed better than self-reported ECog-12 in differentiating cognitive statuses, and a higher study partner reported ECog score was a higher prognostic risk for CI. A memory concern question could enhance self-reported ECog-12 performance. This further emphasizes the need to obtain data from study partners for research and clinical practice.
Key words: ECog-12, A short version ECog-12, Alzheimer’s disease, everyday cognition scale, dementia, MCI.
Introduction
The recent development and FDA approval of disease-modifying treatments for Alzheimer’s disease (AD) increases the need for brief, efficient, and highly scalable tools that can be used to assess cognitive and functional status. Past studies (1-3) have shown the utility of Subjective Cognitive Decline (SCD) measures to identify older adults at risk, including instruments of subjective cognitive change and memory concerns.
The Everyday Cognition Scale (ECog) (4) is an instrument that measures the decline in everyday cognitive and functional abilities that map to six cognitive domains. It includes both self-report and informant/study partner-report versions. The original ECog, which consists of 39 items, has good test-retest reliability (r = 0.82) and external validity (4-7). It could significantly predict cognitive impairment progression (5, 6). ECog scoring outputs are usually examined as either (1) calculated average ECog scores with optimal cut-off and/or (2) group categorical assignment based on item-level response to ECog questions, such as “any consistent SCD” or “any occasional SCD” (1). The participants will be considered as “any occasional SCD“ if any item in ECog-12 scores = 2, but none of the item scores ≥ 3. The participants would be considered to have “any consistent SCD” if any item on the ECog-12 was rated as 3 or higher (1). These scoring systems have shown utility in identifying those with cognitive impairment and those likely to undergo clinical progression (1).
In 2011, a shorter version of the ECog (ECog-12) was developed and validated (8) to reduce evaluation time while maintaining psychometric properties. Several studies have been conducted on the performance of the ECog-39 and ECog-12 scale (1, 5, 6). However, there has been no comparison between the continuous ECog-12 scale, which has an optimal cut-off point, and the categorical SCD defined by the ECog-12 scale. Additionally, there has been no comparison between the performance of ECog-12 from participants and their study partners to detect cognitive impairment at baseline. This is particularly important in participants with MCI or early AD who have positive amyloid pathology, as they represent a potential target population for an AD-modifying drug candidate. Since the ECog-12 consists of a simple, short questionnaire that is easy to administer in many settings (e.g., in clinic, remotely), we hypothesized that the ECog-12 scale might be useful as an efficient, first-step screening tool to identify older adults with cognitive impairment for AD research studies, clinical trials, and in clinical care settings.
The relationship between self- and study partner-report ECog has also been found to be associated with important outcomes related to AD, although there is some inconsistency in results, with some showing agreement between self- and study partner-report and others showing discordance to be important (5, 9-14). When comparing self-reported ECog scores of the cognitively impaired participants with those reported by their study partners, several studies found lower scores for self-reports (11, 15). These lower scores from self-report were associated with poorer performance on memory tests and more AD biomarker evidence, such as CSF-based amyloid or tau biomarkers and brain volume (11, 12). Lower associations with disease outcomes for self-reported ECog scores (10) may be due to a person’s poor insight into their own cognition and function, called anosognosia. Therefore, because of poor insight, the ECog-12 rated by study partners may be a more accurate measure of impairment than the ECog-12 rated by self-report. Moreover, in most studies evaluating the association with cognitive performance, the original ECog score (ECog-39) was used (2, 16). Data compared the association with cognitive impairment between self and study partner reported that the ECog-12 was limited.
In addition to subjective reports of functional decline, self-reporting a memory concern increases the likelihood of progression from SCD to MCI (1). Therefore, abnormalities from the ECog scale added to the self-report memory concern may increase the performance and prognostic risk to detect cognitively impaired patients.
The primary aim of this study was to test the hypothesis that the performance of the ECog-12 can accurately distinguish five different diagnostic groups, including 1) cognitively unimpaired (CU) from cognitively impaired (CI; those with MCI and AD) older adults, 2) CU vs MCI, 3) CU vs. AD, 4) AD vs. MCI, and 5) CI with confirmed brain amyloid pathology vs others (CU&CI without amyloid pathology). Diagnoses were provided by study physicians, serving as the gold standard. For the fifth group (detecting CI with confirmed brain amyloid pathology), we aimed to investigate the ability of ECog to identify those who would be suitable candidates to receive currently approved AD disease modifying therapeutics. This includes those with evidence for CI and elevated brain amyloid, but not those who are CU (regardless of amyloid status). In this study, an amyloid-PET scan or CSF biomarkers was used to detect brain amyloid pathology.
The secondary objectives were to compare performance between self and study-partner ECog-12, to evaluate multiple ECog scoring outputs (e.g., average score, categorized score), to examine whether different measurement parameters can enhance sensitivity to group discrimination, to examine the predictive power of self-reported memory concern when added to ECog-12, to compare the performance of ECog-12 and ECog-39 in detecting cognitive impairment, and to evaluate the association between ECog-12 and cognitive impairment.
Methods
This was a cross-sectional study to evaluate the performance of the self-and study partner-report short version of ECog (ECog-12) in detecting cognitive impairment at baseline. For the gold standard of cognitive impairment detection, this study used a final diagnosis at baseline from ADNI data, which was a clinical diagnosis from a medically qualified professional.
Subjects and study setting
Data used in this study were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). The ADNI was launched in 2004 as a public-private partnership led by Principal Investigator Michael W. Weiner, MD. The primary goal of ADNI has been to validate biomarkers for clinical trials, specifically whether serial magnetic resonance imaging (MRI), positron emission tomography (PET), biofluid-based biomarkers (genetics, CSF, plasma), and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCI) and mild dementia.
The participants (n = 1593) included in this study were all ADNI participants who had available self- and study partner report ECog scores. They were aged between 55-90 years and clinically diagnosed as CU (n = 666), MCI (n = 707), or mild dementia (n = 220) at baseline visit. Participants who had major psychiatric and neurological diseases were excluded from the study. The complete inclusion and exclusion criteria can be downloaded from https://adni.loni.usc.edu/methods/documents/.
Procedure
During the baseline visit, participants were required to complete the ECog-39 questionnaire. Their study partners were asked to respond to the same questionnaire in reference to the participant’s cognitive and functional status. Additionally, participants were asked if they were concerned about their memory or thinking abilities.
The clinicians evaluated the participants and diagnosed them as CU, MCI, or mild dementia during the same visit. The baseline characteristics were recorded, including age, gender, years of education, marital status, race, and APOE4 status.
ECog-12
The ECog-12 score in this study was derived from the ECog-39 data. The ECog-39 consists of 39 functional assessment questions correlated to specific neuropsychological domains: memory, language, visuospatial function, planning, organization, and divided attention. Participants and their study partners completed the ECog-39 separately. They were asked to rate changes in level of participants’ current functioning in the last 10 years. Then they could answer each question by providing scores ranging from 1-4, with 1 indicating no change in ability over 10 years, 2 indicating occasionally perform the task worse, 3 indicating consistently perform little worse on task than 10 years ago, and 4 indicating participants perform the task much worse than 10 years ago. Moreover, the respondents can select the “don’t know” option if they are uncertain about their answer. Based on the prior study, the ECog-12 data was derived by choosing 2 items per domain from ECog-39 (8). The questions of ECog-12 are shown in Appendix 1.
In this study, the ECog-12 was scored by both continuous and categorized methods. For the continuous score, we calculated the average ECog-12 score by the sum of the total score divided by the number of items answered (this accounts for items with missing responses or the rater indicating the “don’t know”). The minimum and maximum score were 1 and 4. For the categorical assignment, participants were grouped into 3 categories based on ECog item-level responses. The participants would be defined as “any occasional SCD” in the case any item of ECog-12 was rated = 2, but none of the items was rated > 3. The participants would be defined as having “any consistent SCD” in the case any item of ECog-12 was rated at least 3 (1). Participants whose item-level responses were all “1” were grouped as “stable/not declining”.
Memory concern question
Only participants were asked to answer whether they were concerned about their memory or not. The question was, “Are you concerned that you have a memory or other thinking problem?” and they can reply as “Yes” or “No”. Out of a total of 1593, 1378 participants responded to this question.
Diagnosis criteria
Cognitively unimpaired and cognitively impaired participants
In this study, participants were classified as CU, MCI, or dementia based on clinical judgment from study physicians at the baseline visit. The final diagnoses were given by the study physicians based on overall cognitive and functional assessment. The term “cognitively impaired participants” refers to MCI or mild dementia participants.
Cognitive impairment with positive amyloid pathology
We identified a group of participants who would be suitable candidates to receive new disease-modifying therapy for AD (17). The participants were diagnosed with MCI or mild dementia and had brain amyloid pathology evidence. The amyloid pathology evidence was evaluated by using an amyloid-PET scan or CSF biomarkers in cases where a PET scan was not available. During the baseline visit, an amyloid-PET scan and lumbar puncture for CSF analysis were performed. For the amyloid-PET scan, the standardized uptake value ratio (SUVR) was converted to the Centiloid scale (CL) to standardize differences caused by radiotracer types. Amyloid-PET scans where the CL was at least 18.5, indicates the presence of amyloid pathology (18). For participants who did not have PET scan results, the CSF biomarkers were used. CSF Aβ42 < 980 pg/mL and ptau181/Aβ42 ratio ≥ 0.025, indicating the presence of amyloid pathology (19-21).
Statistical analyses
The overall cohort was divided into subgroups based on diagnostic criteria (CN, MCI, or mild dementia), and descriptive statistics were tabulated for baseline ECog-12, participant age, education, gender, marital status, race, and APOE4 status. Baseline differences between groups were evaluated using independent Kruskal-Wallis tests for continuous variables and Pearson’s chi-squared tests for categorical factors, with a significance threshold α = 0.05.
Logistic regression was used to evaluate the performance of the ECog-12 score with and without a memory concern question to classify diagnostic groups. The models included differentiating between CU vs. Cognitive Impairment (CI; MCI and mild dementia), CU vs MCI, mild dementia vs. MCI, CI with confirmed brain amyloid pathology vs others (CU and CI without amyloid pathology), and CU vs. mild dementia. The cutoff points used to dichotomize ECOG-12 were calculated using the cutpointr package (22), using kernel smoothing methods to optimize Youden’s J index. AUCs and other summary statistics are reported based on ordinary logistic regression models.
To evaluate the association between ECog-12 and cognitive impairment, we first used a logistic regression model with ECog-12 score as the sole independent variable and fit for the dichotomous response variable. We also used a multivariable logistic regression model with other participant characteristics included as covariates. This procedure was repeated twice for the self-report and study partner-report versions of the ECog 12.
Results
Participants
A total of 1593 participants and 1571 study partners were included in this study. The median age of participants was 71.4 years old (IQR 66.7, 76.8). Among participants, 666 (41.8%) were CU, 707 (44.4%) were diagnosed with MCI, and 220 (13.8%) were diagnosed with mild dementia. The average ECog-12 score of participants ranged from 1 to 4, and the median score was 1.46 (IQR 1.18, 1.83), while the ECog-12 score of study partners raged from 1 to 3.92 and the median was 1.33 (IQR 1.08, 2). For the ECog-39 score, it ranged from 1 to 3.86 and from 1 to 3.95 for participants and study partners, respectively. The median score of ECog-39 in participants was 1.5 (IQR 1.24, 1.9), and in study partners was 1.35 (IQR 1.1, 2). Out of 1593 participants, 69% or 951 individuals responded that they had memory concerns. The participants’ baseline characteristics are shown in Table 1, separated by cognitive status: CU, MCI, and mild dementia.
Table 1. Baseline characteristics of participants
CU, cognitively unimpaired participants; MCI, Mild Cognitive Impairment; AD, Alzheimer’s disease; SCD, Subjective Cognitive Decline; CDR-SB, Clinical Dementia Rating Scale sum of box score; IQR, interquartile range. P-values represent differences between diagnostic groups based on the Kruskal-Wallis rank sum test for continuous variables or the Chi-square test for categorical variables. P-values < 0.05 indicate significant differences.
Performance of ECog-12
To differentiate cognitive impairment and cognitively unimpaired group
The performance of each cut-off point of the average ECog-12 score from both participants and study partners is shown in Appendix 2A. The cut-off point at > 1.36 of ECog-12 from participants provided the best Youden index (0.41) and AUC (0.7) to distinguish CU from cognitively impaired. For the average score from the study partner, ECog-12 > 1.45 showed the best Youden index (0.57) and AUC (0.78).
Table 2 and Appendix 2B demonstrated the performance of the categorized grouping ECog-12 and the average ECog-12 scores with optimal cut-off points from participants and study partners, and the ECog-12 scores from participants combined with memory concern in detecting cognitive impairment. The average score of ECog-12 with the optimal cut-off point of the study partner showed higher performance in detecting cognitive impairment than the average self-report score (AUC 0.78 and 0.7, respectively). For self-report ECog-12, adding the memory concern question increased the AUC to 0.79 (95%CI 0.77-0.82). For the categorized ECog-12 score, the participants’ score showed a fair performance in the detection of cognitive impairment, but the performance increased to moderate after adding on the memory concern question (AUC 0.73, 95%CI 0.7-0.75). Although “any consistent SCD” ECog-12 of study partners showed moderate performance in differentiation (AUC 0.78, 95%CI 0.76-0.8), it provided higher performance than those from participants.
Table 2. Performance of ECog-12 to differentiate cognitive statuses
CP, cutpoint; AUC, Area Under the Curve of the “Receiver Operating Characteristic” curve; pt, self-participant; sp, study partner; CI’ cognitive impaired participants; CU, cognitively unimpaired participants; * Dichotomous score on each cutpoint; ** Performance of ECog-12 combined with positive concern question compared with none of high ECog and memory concern; ***AUC derived from the performance of participants who had any consistent SCD ECog-12 score and positive memory concern complaint compared to those without any consistent SCD in ECog12 and memory concern complaint, as well as those with positive only one of each.
To differentiate MCI and cognitively unimpaired group
The performance of the ECog-12 scores from both participants and study partners in detecting MCI from normal cognition is shown in Table 2 and Appendix 3A and 3B. The optimal cut-off point for ECog-12 from participants was > 1.37 (Youden index 0.39 and AUC 0.7), while the optimal cut-off point of study partners’ ECog-12 was > 1.27 (Youden index 0.49 and AUC 0.75), the result was shown in Appendix 3A. The average ECog-12 of self-participants at the cut-point > 1.37 combined with memory concern had the best AUC (0.8, 95%CI 0.77-0.82). Excluding the memory concern question, the performance of an average ECog-12 from study partners showed better performance compared with participants’ ECog-12 in detection (AUC = 0.75 and 0.7, respectively). Similarly, the performance of “any consistent SCD” ECog-12 from study partners was higher than the participants’ (AUC = 0.74 and 0.68). After adding the memory concern to any consistent SCD ECog-12 of participants, the AUC was increased to 0.72 and 0.8, as shown in Table 2.
To differentiate mild dementia and MCI
The performance of the ECog-12 scores from both participants and study partners in detecting mild dementia from MCI is shown in Table 2 and Appendix 4A and 4B. All average and categorized ECog-12 scores from self-participants had poor performance in differentiation (AUC ranged from 0.49 to 0.57). However, the performance of average and any consistent SCD ECog-12 from study partners showed moderate to good (AUC 0.77) and fair (AUC 0.66) performance in detection.
To differentiate cognitive impairment with confirmed amyloid pathology participants (AD drug candidate participants) and others (non-AD drug candidate participants)
The performance of the ECog-12 scores from both participants and study partners in detecting cognitive impairment with confirmed amyloid pathology is shown in Table 2 and Appendix 5A and 5B. Similar to other results reported here, ECog-12 of study partners with both average score (AUC = 0.75) and any consistent SCD ECog-12 category (AUC = 0.74) showed better performance than self-report ECog (AUC = 0.65) in detecting drug candidate patients from others.
To differentiate mild dementia and cognitively unimpaired group
The performance of the ECog-12 scores from both participants and study partners in detecting mild dementia from normal cognition is shown in Appendix 6A and 6B. Both average and any consistent SCD ECog-12 scores from study partners had excellent performance in differentiating mild dementia from cognitively unimpaired (AUC 0.93 and 0.9, respectively), while the average and any consistent SCD ECog-12 scores of participants with and without memory concern showed only moderate performance (AUC ranged from 0.71-0.78).
Comparison of ECog scoring outputs
For all models, the ECog-12 and ECog-39 performed similarly in distinguishing diagnostic groups; the results of ECog-39 performance are shown in Appendix 7. Continuous scores from both self-report and study-partner report ECog demonstrated a similar performance as the categorized ECog scale in distinguishing most analytic groups, results as shown in Table 2. However, to distinguish between mild dementia and MCI participants, the continuous ECog-12 score of study partners performed better than the categorical score (AUC 0.77 and 0.66, respectively).
The association between ECog-12 and cognitive impairment is shown in Table 3. Higher self-report ECog-12 was associated with higher odds of cognitive impairment in univariable (crude odds ratio 10, 95%CI 7.4-13.76) and adjusted models (adjusted odds ratio 8.79, 95%CI 6.32-12.43). Higher study partner-report ECog-12 was associated with higher odds of cognitive impairment, with an adjusted odds ratio of 35.45 (95%CI 22.41-58.03)
Table 3. Association between ECog-12 and cognitive impairment (MCI and AD)
*Model1: association of average ECog-12 score from self-participants and cognitive impairment adjusted with age, gender, educational years, race, marital status, and APOE4 status; **Model2: association of average ECog-12 score from study partners and cognitive impairment adjusted with age, gender, educational years, race, marital status, and APOE4 status
Discussion
The major findings of this study were: First, the ECog-12 rated by study partners had higher performance than the ECog-12 rated by participants themselves to detect cognitive impairment. Second, adding a memory concern question improved the ability of self-report ECog to distinguish diagnostic groups. Third, average scores from both self-report and study-partner reports demonstrated a similar performance as the categorized scale in most analytic groups. However, to distinguish between mild dementia and MCI participants, the average ECog-12 score of study partners showed good performance, while the categorized ECog-12 from study partners and the ECog-12 scores from the participant’s data had low classification performance. Fourth, the average and “consistently declined” ECog-12 scores from study partners provide fair sensitivity to detect cognitively impaired participants who had positive amyloid pathology (AD modifying drug candidates), and the average ECog-12 from participants added to the memory concern question provided an excellent sensitivity with moderate AUC in detection. Fifth, the ECog-12 showed a comparable performance to the ECog-39.
Our study found that the study partner-report ECog 12 was better than the self-report ECog 12 score for distinguishing all diagnostic groups examined. AUC for study partner-report ECog ranged from 0.75-0.93, whereas AUC for self-report ECog ranges from 0.57 to 0.74. The difference between self- and study partner-report ECog was greatest in the CU vs mild dementia models (AUC 0.93 for study partner ECog, AUC 0.74 for self-report ECog). In addition, after adjusting for participants’ demographics, the ECog-12 from study partners showed a stronger association with cognitive impairment than self-report ECog [adjusted OR for study partner 35.45 (22.41-58.03) and for self-participant 8.79 (95%CI 6.32-12.43)], as shown in Table 3. These results are consistent with several prior cross-sectional studies that revealed informant-reported SCD was better than self-reported SCD in distinguishing different diagnostic groups (2). A study from the original ECog (ECog-39) showed that ECog-39, rated by study partners, discriminates cognitive statuses better than self-reports.(10) The higher accuracy of the study partners’ ECog-12 than the self-participants’ could be explained by the cognitively impaired participants tending to underestimate their cognitive and memory function, called anosognosia, which is the impairment of insight or denial into their illnesses (23-25). Compared to other existing scales from a prior systematic review and meta-analysis study (26), the Mini-Mental State Examination (MMSE) provided a combined sensitivity of 0.81 (95%CI 0.78-0.84) and specificity of 0.89 (95%CI 0.87-0.91) for detecting dementia, while study partner-reported ECog-12 at a cut of point > 1.67 from our study showed higher sensitivity (0.93) and specificity (0.95). For detecting MCI, the Montreal Cognitive Assessment (MoCA) outperformed ECog-12 with a sensitivity of 0.89 and specificity of 0.75 (26). However, the ECog-12 demonstrated superior advantages in the remote evaluation compared to the MoCA. A previous study conducted by ADNI and the Brain Health Registry demonstrated that Online Everyday Cognition is highly consistent with in-clinic Everyday Cognition (27).
Moreover, our results support the anosognosia hypothesis of the cognitively impaired participants. In the dementia group, the study partners rated higher average ECog-12 scores than participants. In contrast, the mean ECog-12 of the study partners was lower than the participants in CU and MCI groups. These results were similar to previous studies using ECog-39, which also found lower study partner scores in those groups (15, 28, 29). A previous study, from the Brain Health Registry data, evaluated the accuracy of online self- and study partner-reported ECog-39 to distinguish MCI from CU participants. The result of that study revealed that the mean ECog-39 score by online study partner report was higher than the self-report ECog-39 in the Aβ positive MCI group (mean ECog: study partner = 2.03 (SD 0.61) and self-report = 1.88 (0.67)). While the mean ECog-39 by study partner report was lower than the self-report in normal control and Aβ negative MCI (15).
Comparing the categorized and continuous average ECog-12 scale, our results demonstrated that the average score from both self- and study partner reports provided a similar performance as the categorized scale in all analytic groups except between dementia and MCI participants (AUC from any consistent SCD group and average ECog-12 of study partners were 0.66 and 0.77, respectively). These results were quite different from the study by Dr. van Harten et al. (2018), which evaluated the association between the risk of MCI progression and abnormal ECog scores. They reported that the average ECog score with optimal cut-point showed slightly lower strength of association with the risk of MCI progression compared to any consistent SCD (any response ≥ 3 compared to all responses < 3 of the ECog scale), hazard ratios 2.06 and 2.45, respectively (1). However, the methodology of prior and current studies was different. The current study was a cross-sectional study to detect cognitive impairment at baseline, but the prior was a longitudinal study to predict cognitive impairment progression.
The self-report memory concern question was also important. Our study showed that adding a memory concern question to ECog-12 from participants increased the accuracy of detecting cognitive impairment in most diagnostic groups. The results supported that participants of this study still have an awareness of their symptoms. These were similar to the prior population-based cohort study results that showed a subjective memory complaint associated with a faster cognitive decline (30) and could discriminate MCI and early AD from cognitively unimpaired participants (31). In addition, several longitudinal studies suggested that the subjective memory concern may be an early manifestation of AD and MCI (32-35). However, adding the memory concern question decreased accuracy in distinguishing mild dementia and MCI participants in the current study. That is because the presentation of memory concern is common in both mild dementia and MCI patients, particularly the amnestic MCI, which is the most common subtype (36, 37). Thus, adding a memory concern question couldn’t improve the performance of ECog-12 in distinguishing between dementia and MCI. In this study, only participants with mild dementia were included. If individuals with more severe stages of dementia had also been included, the results may have differed.
A novel aspect of this study was evaluating the ECog for identifying older adults who would be suitable candidates for AD therapeutics targeting amyloid, which have been approved for treatment for those with MCI and mild AD dementia. Our study showed that average and “consistently declined” ECog-12 scores from study partners provide fair sensitivity to detect participants with cognitive impairment who were amyloid positive (sensitivity 0.77 and 0.78, respectively). However, the average ECog-12 from participants added to the memory concern question provided an excellent sensitivity with moderate AUC to detect the drug-candidate participants (sensitivity 95% and AUC 0.71 see Appendix 5A and 5B). Therefore, the self-report ECog-12 plus memory concern question might be an appropriate and easy initial screening tool to identify individuals who may be candidates for AD disease-modifying medication.
Our study showed similar performance between average ECog-12 and ECog-39 from both self-participants and study partners among all diagnostic groups, as shown in Table 2 and Appendix 7. Thus, ECog-12 has comparable psychometric properties to the original version but with fewer items and a shorter evaluation time. These results were similar to a previous study, which demonstrated that ECog-12 had an excellent and comparable performance to ECog-39 in differentiating cognitive impairment from cognitively unimpaired (AUC; ECog-39 = 0.91, ECog-12 = 0.91) and between dementia from cognitively unimpaired (AUC; ECog-39 = 0.96, ECog-12 = 0.95) (8). Compared to the prior study in 2011, the cut points of Ecog-12 form study partners in our study to detect cognitive impairment are lower likely because participants in the current study have better cognitive performance than those in the previous one (median CDR-SB scores in MCI and mild dementia in the current study were 1 and 4.5, and in the prior study were 1.87 and 6.71) (8).
This is the first study to compare the performance of different scoring methods (average score vs. categorical grouping) of a short ECog version (Ecog-12) and to compare performance between self- and study partner-ECog-12 for detecting cognitive impairment at baseline (particularly AD medication candidate participants).
These findings support that ECog-12 can be used in multiple settings to facilitate dementia research, trials, and clinical care. It can be used to help screen participants for clinical trials and observational studies. It can also be used by clinicians to identify patients with possible cognitive impairment, who should undergo further screening. Additionally, the ECog-12 scale could be useful as an efficient, first-step screening tool to identify participants who are suitable candidates for AD disease-modifying therapy. We encourage using study partner-reported ECog-12, both the average score at the cutoff score > of 1.45 and a categorical grouping (any consistently declined) to efficiently screen and assess older adults for research studies, clinical trials, and clinical care. The self-reported ECog-12 can be used as well, but self-reported ECog-12 alone provided AUCs less than 0.8. Therefore, the memory concern question should be evaluated by adding to self-reported ECog-12 to increase diagnostic accuracy.
This study has several limitations. The ADNI study design may limit the generalizability of results to other populations and settings. ADNI is an observational study using a sample of convenience that lacks ethnocultural and educational diversity. We did not use separate “training” and “validation” cohorts or sub-samples to externally validate the cut-offs we established. Thus, the predictive performance of our models may be overestimate. Further studies are being planned to validate the results in other cohorts. The dementia participants included in this study had mild dementia, so the result could not be generalized to moderate and severe stages of dementia. The Ecog-12 in this study derived from the original version of ECog (ECog-39), so some items might not be updated and relevant to the current real-world activity (such as the capability of technology usage). The current ADNI4 study will evaluate the performance of ECog-12 derived from the new version of ECog (ECog-II) (38) in detecting cognitive impairment and may show better results (39). Other dementia subtypes and other AD stages of participants should be enrolled in future studies to generalize and expand the utility of this tool to other populations.
Funding sources: Data collection and sharing for the Alzheimer’s Disease Neuroimaging Initiative (ADNI) is funded by the National Institute on Aging (National Institutes of Health Grant U19 AG024904). The grantee organization is the Northern California Institute for Research and Education. In the past, ADNI has also received funding from the National Institute of Biomedical Imaging and Bioengineering, the Canadian Institutes of Health Research, and private sector contributions through the Foundation for the National Institutes of Health (FNIH) including generous contributions from the following: AbbVie, Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research &Development LLC.; Lumosity; Lundbeck; Merck & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics.
Acknowledgments: Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf
Conflicts of interest: MM – Dr. Manchumad Manjavong has no conflict of interest to declare. AD – Adam Diaz has no conflict of interest to declare. MTA – Dr. Miriam T. Ashford receives funding to her institution from NIH. AA – Anna Aaronson has no conflict of interest to declare. MJM – Dr. Melanie J. Miller has no conflict of interest to declare. JK – Dr. Jae Myeong Kang has no conflict of interest to declare. RSM – Dr. R. Scott Mackin has received research support from The National Institute of Mental Health, the National Institute of Aging, and Johnson and Johnson, during the past 2 years. RT – Dr. Rachana Tank has no conflict of interest to declare. BL – Bernard Landavazo has no conflict of interest to declare. DT – Diana Truran has no conflict of interest to declare. STF – Sarah Tomaszewski Farias received grant funding from the NIH, Alzheimer’s Association, and the California Dept of Public Health. MWW- Dr. Weiner reports grants from National Institutes of Health (NIH), grants from Department of Defense (DOD), grants from Patient-Centered Outcomes Research Institute (PCORI), grants from California Department of Public Health (CDPH), grants from University of Michigan, grants from Siemens, grants from Biogen, grants from Hillblom Foundation, grants from Alzheimer’s Association, grants from The State of California, grants from Johnson & Johnson, grants from Kevin and Connie Shanahan, grants from GE, grants from VUmc, grants from Australian Catholic University (HBI- BHR), grants from The Stroke Foundation, grants from Veterans Administration, personal fees from Acumen Pharmaceutical, personal fees from Cerecin, personal fees from Dolby Family Ventures, personal fees from Eli Lilly, personal fees from Merck Sharp & Dohme Corp., personal fees from National Institute on Aging (NIA), personal fees from Nestle/Nestec, personal fees from PCORI/PPRN, personal fees from Roche, personal fees from University of Southern California (USC), personal fees from NervGen, personal fees from Baird Equity Capital, personal fees from BioClinica, personal fees from Cytox, personal fees from Duke University, personal fees from Eisai, personal fees from FUJIFILM-Toyama Chemical (Japan), personal fees from Garfield Weston, personal fees from Genentech, personal fees from Guidepoint Global, personal fees from Indiana University, personal fees from Japanese Organization for Medical Device Development, Inc. (JOMDD), personal fees from Medscape, personal fees from Peerview Internal Medicine, personal fees from Roche, personal fees from T3D Therapeutics, personal fees from WebMD, personal fees from Vida Ventures, personal fees from The Buck Institute for Research on Aging, personal fees from China Association for Alzheimer’s Disease (CAAD), personal fees from Japan Society for Dementia Research, personal fees from Korean Dementia Society, outside the submitted work; and I hold stocks or options with Alzheon Inc., Alzeca, and Anven. RLN – Dr. Rachel L. Nosheny reports funding from the National Institutes of Health (grants to institution), California Department of Public Health (grants to institution), and Genentech Inc. (grants to institution).
Ethical standards: The ADNI protocols have been approved by all Institutional Review Boards of the participating institutions. The data used for these analyses were collected only from volunteers who provided written informed consent.
Open Access: This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
References
1. van Harten AC, Mielke MM, Swenson-Dravis DM, Hagen CE, Edwards KK, Roberts RO, et al. Subjective cognitive decline and risk of MCI: The Mayo Clinic Study of Aging. Neurology. 2018;91(4):e300-e12. doi: 10.1212/WNL.0000000000005863.
2. Nosheny RL, Jin C, Neuhaus J, Insel PS, Mackin RS, Weiner MW, et al. Study partner-reported decline identifies cognitive decline and dementia risk. Ann Clin Transl Neurol. 2019;6(12):2448-59. doi: 10.1002/acn3.50938.
3. Zlatar ZZ, Tarraf W, Gonzalez KA, Vasquez PM, Marquine MJ, Lipton RB, et al. Subjective cognitive decline and objective cognition among diverse U.S. Hispanics/Latinos: Results from the Study of Latinos-Investigation of Neurocognitive Aging (SOL-INCA). Alzheimers Dement. 2022;18(1):43-52. doi: 10.1002/alz.12381.
4. Farias ST, Mungas D, Reed BR, Cahn-Weiner D, Jagust W, Baynes K, et al. The measurement of everyday cognition (ECog): scale development and psychometric properties. Neuropsychology. 2008;22(4):531-44. doi: 10.1037/0894-4105.22.4.531.
5. Farias ST, Lau K, Harvey D, Denny KG, Barba C, Mefford AN. Early Functional Limitations in Cognitively Normal Older Adults Predict Diagnostic Conversion to Mild Cognitive Impairment. J Am Geriatr Soc. 2017;65(6):1152-8. doi: 10.1111/jgs.14835.
6. Lau KM, Parikh M, Harvey DJ, Huang CJ, Farias ST. Early Cognitively Based Functional Limitations Predict Loss of Independence in Instrumental Activities of Daily Living in Older Adults. J Int Neuropsychol Soc. 2015;21(9):688-98. doi: 10.1017/S1355617715000818.
7. Hsu JL, Hsu WC, Chang CC, Lin KJ, Hsiao IT, Fan YC, et al. Everyday cognition scales are related to cognitive function in the early stage of probable Alzheimer’s disease and FDG-PET findings. Sci Rep. 2017;7(1):1719. doi: 10.1038/s41598-017-01193-6.
8. Tomaszewski Farias S, Mungas D, Harvey DJ, Simmons A, Reed BR, Decarli C. The measurement of everyday cognition: development and validation of a short form of the Everyday Cognition scales. Alzheimers Dement. 2011;7(6):593-601. doi: 10.1016/j.jalz.2011.02.007.
9. Swinford CG, Risacher SL, Charil A, Schwarz AJ, Saykin AJ. Memory concerns in the early Alzheimer’s disease prodrome: Regional association with tau deposition. Alzheimers Dement (Amst). 2018;10:322-31. doi: 10.1016/j.dadm.2018.03.001.
10. Rueda AD, Lau KM, Saito N, Harvey D, Risacher SL, Aisen PS, et al. Self-rated and informant-rated everyday function in comparison to objective markers of Alzheimer’s disease. Alzheimers Dement. 2015;11(9):1080-9. doi: 10.1016/j.jalz.2014.09.002.
11. Bregman N, Kave G, Zeltzer E, Biran I, Alzheimer’s Disease Neuroimaging I. Memory impairment and Alzheimer’s disease pathology in individuals with MCI who underestimate or overestimate their decline. Int J Geriatr Psychiatry. 2020;35(5):581-8. doi: 10.1002/gps.5274.
12. Edmonds EC, Delano-Wood L, Galasko DR, Salmon DP, Bondi MW, Alzheimer’s Disease Neuroimaging I. Subjective cognitive complaints contribute to misdiagnosis of mild cognitive impairment. J Int Neuropsychol Soc. 2014;20(8):836-47. doi: 10.1017/S135561771400068X.
13. Gifford KA, Liu D, Hohman TJ, Xu M, Han X, Romano RR, 3rd, et al. A Mutual Self- and Informant-Report of Cognitive Complaint Correlates with Neuropathological Outcomes in Mild Cognitive Impairment. PLoS One. 2015;10(11):e0141831. doi: 10.1371/journal.pone.0141831.
14. Nakhla MZ, Cohen L, Salmon DP, Smirnov DS, Marquine MJ, Moore AA, et al. Self-reported subjective cognitive decline is associated with global cognition in a community sample of Latinos/as/x living in the United States. Journal of Clinical and Experimental Neuropsychology. 2021;43(7):663-76. doi: 10.1080/13803395.2021.1989381.
15. Nosheny RL, Jin C, Banh T, Ashford MT, Camacho MR, Mackin RS, et al. Remote identification of MCI using self- and study partner-report subjective cognitive decline in the Brain Health Registry. Alzheimer’s & Dementia. 2021;17(S6):e052337. doi: https://doi.org/10.1002/alz.052337.
16. Wells LF, Risacher SL, McDonald BC, Farlow MR, Brosch J, Gao S, et al. Measuring Subjective Cognitive Decline in Older Adults: Harmonization Between the Cognitive Change Index and the Measurement of Everyday Cognition Instruments. J Alzheimers Dis. 2022;87(2):761-9. doi: 10.3233/JAD-215388.
17. Cummings J, Apostolova L, Rabinovici GD, Atri A, Aisen P, Greenberg S, et al. Lecanemab: Appropriate Use Recommendations. The Journal of Prevention of Alzheimer’s Disease. 2023;10(3):362-77. doi: 10.14283/jpad.2023.30.
18. Farrell ME, Jiang S, Schultz AP, Properzi MJ, Price JC, Becker JA, et al. Defining the Lowest Threshold for Amyloid-PET to Predict Future Cognitive Decline and Amyloid Accumulation. Neurology. 2021;96(4):e619-e31. doi: 10.1212/WNL.0000000000011214.
19. Contador J, Perez-Millan A, Tort-Merino A, Balasa M, Falgas N, Olives J, et al. Longitudinal brain atrophy and CSF biomarkers in early-onset Alzheimer’s disease. Neuroimage Clin. 2021;32:102804. doi: 10.1016/j.nicl.2021.102804.
20. Libiger O, Shaw LM, Watson MH, Nairn AC, Umaña KL, Biarnes MC, et al. Longitudinal CSF proteomics identifies NPTX2 as a prognostic biomarker of Alzheimer’s disease. Alzheimer’s & Dementia. 2021;17(12):1976-87. doi: https://doi.org/10.1002/alz.12353.
21. Zicha S, Bateman RJ, Shaw LM, Zetterberg H, Bannon AW, Horton WA, et al. Comparative analytical performance of multiple plasma Abeta42 and Abeta40 assays and their ability to predict positron emission tomography amyloid positivity. Alzheimers Dement. 2022. doi: 10.1002/alz.12697.
22. Thiele C, Hirschfeld G. cutpointr: Improved Estimation and Validation of Optimal Cutpoints in R. Journal of Statistical Software. 2021;98(11):1 – 27. doi: 10.18637/jss.v098.i11.
23. Zilli B, Damasceno BP. Anosognosia in Alzheimer’s disease: A neuropsychological approach. Dement Neuropsychol. 2007;1(1):81-8. doi: 10.1590/S1980-57642008DN10100013.
24. Bastin C, Giacomelli F, Miévis F, Lemaire C, Guillaume B, Salmon E. Anosognosia in Mild Cognitive Impairment: Lack of Awareness of Memory Difficulties Characterizes Prodromal Alzheimer’s Disease. Frontiers in Psychiatry. 2021;12. doi: 10.3389/fpsyt.2021.631518.
25. Sevush S, Leve N. Denial of memory deficit in Alzheimer’s disease. Am J Psychiatry. 1993;150(5):748-51. doi: 10.1176/ajp.150.5.748.
26. Tsoi KK, Chan JY, Hirai HW, Wong SY, Kwok TC. Cognitive Tests to Detect Dementia: A Systematic Review and Meta-analysis. JAMA Intern Med. 2015;175(9):1450-8. doi: 10.1001/jamainternmed.2015.2152.
27. Howell T, Neuhaus J, Glymour MM, Weiner MW, Nosheny RL. Validity of Online Versus In-Clinic Self-Reported Everyday Cognition Scale. J Prev Alzheimers Dis. 2022;9(2):269-76. doi: 10.14283/jpad.2022.20.
28. Thabtah F, Spencer R, Ye Y. The correlation of everyday cognition test scores and the progression of Alzheimer’s disease: a data analytics study. Health Inf Sci Syst. 2020;8(1):24. doi: 10.1007/s13755-020-00114-8.
29. Bregman N, Kave G, Shiner T, Biran I, Alzheimer’s Disease Neuroimaging I. Dissociation in awareness of memory and language decline in Alzheimer’s disease. Int J Geriatr Psychiatry. 2019;34(4):548-54. doi: 10.1002/gps.5049.
30. Dhana A, DeCarli C, Dhana K, Desai P, Krueger K, Evans DA, et al. Association of Subjective Memory Complaints With White Matter Hyperintensities and Cognitive Decline Among Older Adults in Chicago, Illinois. JAMA Netw Open. 2022;5(4):e227512. doi: 10.1001/jamanetworkopen.2022.7512.
31. Choe YM, Byun MS, Lee JH, Sohn BK, Lee DY, Kim JW. Subjective memory complaint as a useful tool for the early detection of Alzheimer’s disease. Neuropsychiatr Dis Treat. 2018;14:2451-60. doi: 10.2147/NDT.S174517.
32. Jessen F, Wiese B, Bachmann C, Eifflaender-Gorfer S, Haller F, Kolsch H, et al. Prediction of dementia by subjective memory impairment: effects of severity and temporal association with cognitive impairment. Arch Gen Psychiatry. 2010;67(4):414-22. doi: 10.1001/archgenpsychiatry.2010.30.
33. Jessen F, Wolfsgruber S, Wiese B, Bickel H, Mosch E, Kaduszkiewicz H, et al. AD dementia risk in late MCI, in early MCI, and in subjective memory impairment. Alzheimers Dement. 2014;10(1):76-83. doi: 10.1016/j.jalz.2012.09.017.
34. Reid LM, Maclullich AM. Subjective memory complaints and cognitive impairment in older people. Dement Geriatr Cogn Disord. 2006;22(5-6):471-85. doi: 10.1159/000096295.
35. Reisberg B, Prichep L, Mosconi L, John ER, Glodzik-Sobanska L, Boksay I, et al. The pre-mild cognitive impairment, subjective cognitive impairment stage of Alzheimer’s disease. Alzheimers Dement. 2008;4(1 Suppl 1):S98-S108. doi: 10.1016/j.jalz.2007.11.017.
36. Caffo AO, Spano G, Tinella L, Lopez A, Ricciardi E, Stasolla F, et al. The Prevalence of Amnestic and Non-Amnestic Mild Cognitive Impairment and Its Association with Different Lifestyle Factors in a South Italian Elderly Population. Int J Environ Res Public Health. 2022;19(5). doi: 10.3390/ijerph19053097.
37. Rapp SR, Legault C, Henderson VW, Brunner RL, Masaki K, Jones B, et al. Subtypes of mild cognitive impairment in older postmenopausal women: the Women’s Health Initiative Memory Study. Alzheimer Dis Assoc Disord. 2010;24(3):248-55. doi: 10.1097/WAD.0b013e3181d715d5.
38. Farias ST, Weakley A, Harvey D, Chandler J, Huss O, Mungas D. The Measurement of Everyday Cognition (ECog): Revisions and Updates. Alzheimer Dis Assoc Disord. 2021;35(3):258-64. doi: 10.1097/WAD.0000000000000450.
39. Weiner MW, Veitch DP, Miller MJ, Aisen PS, Albala B, Beckett LA, et al. Increasing participant diversity in AD research: Plans for digital screening, blood testing, and a community-engaged approach in the Alzheimer’s Disease Neuroimaging Initiative 4. Alzheimers Dement. 2023;19(1):307-17. doi: 10.1002/alz.12797.
© The Authors 2024