L.M.P. Wesselman1, A.K. Schild2, A.M. Hooghiemstra1,3, D. Meiberth2, A.J. Drijver4, M.v. Leeuwenstijn-Koopman1, N.D. Prins1, S. Brennan5, P. Scheltens1, F. Jessen2,6, W.M. van der Flier1,7, S.A.M. Sikkes1,8
1. Alzheimer Center Amsterdam, Department of Neurology, Amsterdam Neuroscience, Vrije Universiteit Amsterdam, Amsterdam UMC, Amsterdam, The Netherlands; 2. Department of Psychiatry, University Hospital Cologne, Medical Faculty, Cologne, Germany; 3. Department of Medical Humanities, Amsterdam Public Health Research Institute, Amsterdam UMC, Vrije Universiteit Amsterdam, De Boelelaan 1089a, 1081 HV Amsterdam, The Netherlands; 4. Amsterdam UMC, Vrije Universiteit Amsterdam, Department of Neurology, The Netherlands; 5. The Adapt Centre, & The Institute of Neuroscience, Trinity College Dublin; 6. German Center for Neurodegenerative Disorders (DZNE), Bonn-Cologne, Germany; 7. Department of Epidemiology and Biostatistics, Amsterdam Neuroscience, Vrije Universiteit Amsterdam, Amsterdam UMC, Amsterdam, the Netherlands; 8. Clinical Developmental Psychology & Clinical Neuropsychology, Faculty of Behavioural and Movement Sciences (FGB), Vrije University Amsterdam, Amsterdam, the Netherlands.
Corresponding Author: Linda M.P. Wesselman, Alzheimer Center Amsterdam and Department of Neurology, Amsterdam Neuroscience, Amsterdam UMC, P.O. Box 7057, 1007 MB Amsterdam, the Netherlands, Telephone: +31-204440816; Fax: +31-204448529; E-mail: l.wesselman@amsterdamumc.nl
J Prev Alz Dis 2020;3(7):184-194
Published online March 2, 2020, http://dx.doi.org/10.14283/jpad.2020.9
Abstract
Background: Online programs targeting lifestyle have the potential to benefit brain health. We aimed to develop such a program for individuals with subjective cognitive decline (SCD). These individuals were reported to be at increased risk for dementia, and report both an intrinsic need for brain health information and motivation to participate in prevention strategies. Co-creation and user-evaluation benefits the adherence to and acceptance of online programs. Previously, we developed a prototype of the online program in co-creation with the users .
Objectives: We now aimed to evaluate the user-experiences of our online lifestyle program for brain health.
Design: 30-day user test; multi-method.
Setting: Participants were recruited in a memory clinic and (online) research registries in the Netherlands (Alzheimer Center Amsterdam) and Germany (Center for memory disorders, Cologne).
Participants: Individuals with SCD (N=137, 65±9y, 57% female).
Measurements: We assessed user-experiences quantitatively with rating daily advices and usefulness, satisfaction and ease of use questionnaires as well as qualitatively using telephone interviews.
Results: Quantitative data showed that daily advices were rated moderately useful (3.5 ±1.5, range 1-5 points). Participants (n=101, 78%) gave moderate ratings on the programs’ usability (3.7±1.3, max 7), ease of learning (3.6±1.9) and satisfaction (4.0±1.5), and marginal ratings on the overall usability (63.7±19.0, max 100). Qualitative data collected during telephone interviews showed that participants highly appreciated the content of the program. They elaborated that lower ratings of the program were mainly due to technical issues that hindered a smooth walk through. Participants reported that the program increased awareness of lifestyle factors related to brain health.
Conclusions: Overall user-experience of the online lifestyle program was moderate to positive. Qualitative data showed that content was appreciated and that flawless, easy access technique is essential. The heterogeneity in ratings of program content and in program use highlights the need for personalization. These findings support the use of online self-applied lifestyle programs when aiming to reach large groups of motivated at-risk individuals for brain health promotion.
Key words: Lifestyle, dementia, subjective cognitive decline, eHealth, prevention.
Abbreviations: SCD: subjective cognitive decline; MCI: mild cognitive impairment; SUS: System Usability Scale; USE: User Satisfaction and Ease of use.
Introduction
The World Health Organization (WHO) Global Action Plan on Dementia emphasized the need for campaigns to increase public awareness and understanding of dementia (1). Recent studies found that knowledge about prevention and treatment of dementia remains poor and that there is a need for adequate dementia prevention education (2, 3).
The body of evidence on the association between a healthy lifestyle and brain health keeps growing (4). Risk factors for dementia due to Alzheimer’s disease (AD), such as lifestyle factors, are suggested to be partly modifiable (5). A healthy lifestyle may therefore decrease the risk for AD dementia. Since the etiology of AD is complex and multifactorial, recommendations are made to target several risk factors simultaneously (6, 7). Indeed, a multifactorial intervention has been found to improve or maintain cognitive functioning in people at risk for dementia (8). However, this intervention was offered face-to-face, which is beneficial for program use because of personal contact, but is relatively expensive and limits possibilities to reach a larger group of individuals. Offering intervention programs online has an important advantage because it offers the opportunity to reach many users, in particular in remote areas (9).
Our international EuroSCD-project aimed to develop an online lifestyle program for brain health. Individuals with subjective cognitive decline (SCD) experience cognitive decline in absence of objective cognitive impairments. SCD has previously been reported to be a risk factor for dementia and AD (10, 11). Therefore, individuals with SCD might be an ideal target group for online interventions. This at-risk group might present at memory clinics, their GP or research registries, and was found to be motivated to participate in prevention strategies (12). Individuals at-risk might benefit most from prevention strategies aimed at optimizing brain health or preventing cognitive decline (13, 14).
Our recent review and meta-analysis on online lifestyle programs for brain health suggested that these programs could indeed benefit brain health (15). However, the programs that we reviewed were heterogeneous in content and set-up. Further, characteristics and the methods and results of evaluations of the programs were often not described consistently. More specifically, it was often unclear how user-participation was operationalized and thus how users were involved during the development of the programs (15). This is an important aspect during the development of online programs, because it is essential to involve future users during development. With the users’ input, a program will better fit the users’ needs, which benefits acceptance and adherence, and thereby the implementation of sustainable innovations (16). Previously, we investigated barriers and facilitators for the use of an online lifestyle program in individuals with SCD (12). We found that both program characteristics and personal factors need to be considered, with trustworthiness, user-friendliness, and personalization being important facilitators. We implemented these results during the development of an online lifestyle program for brain health. In co-creation with the users, we developed and adapted the program in multiple iterations. We now aimed to evaluate user-experiences of our online lifestyle program in Dutch and German individuals with SCD, using both quantitative and qualitative methods.
Methods
Project and study design
This study is part of the European ‘Subjective cognitive decline in preclinical Alzheimer’s Disease: European initiative on harmonization and on a lifestyle-based prevention strategy’ project (Euro-SCD; JPND_PS_FP-689-019), which aims to develop an online lifestyle program for individuals with SCD. The Euro-SCD project is a collaboration between the Alzheimer Center Amsterdam, the Netherlands (17), Hospital Clinic Barcelona, Spain, and the Center for memory disorders, University Hospital Cologne, Germany. The study was conducted in accordance with Good Clinical Practice (GCP) Guidelines, applicable national guidelines, and to the Declaration of Helsinki. The local ethical committees approved this study and all participants provided informed consent.
The current study was conducted in the Netherlands and Germany (Figure 1: study overview). First, we conducted a feasibility study in the Netherlands to evaluate practicalities and study procedures. This allowed us to improve the online program and optimize the planned study procedures. Subsequently, we performed a 30-day online user test in both the Netherlands and Germany to evaluate user-experiences.
Figure 1. Study overview
NOTE: This Figure illustrates the study overview. During the feasibility study, using an iterative process, the program was adapted and study procedures were optimized. The 30-day online user test was quantitatively evaluated with questionnaires, rating of daily advices and data log, and qualitatively by follow-up telephone interviews in a subsample of participants. USE: User Satisfaction and Ease of use questionnaire; SUS: System Usability Scale. a: conducted in the Netherlands, b: recruited via Dutch Brain Health Registry, c: recruited via Cologne Alzheimer dementia prevention registry.
Participants
Individuals with SCD were recruited through either a memory clinic or research registry:
1) memory clinic: we included individuals that visited the Alzheimer Center Amsterdam because of cognitive complaints. They underwent clinical work-up including clinical evaluation, neuropsychological assessment, and MRI scan. Although not mandatory, an informant was present in most cases during consults and assessments. When all clinical investigations were normal, and no cognitive disorder could be objectified, patients were labelled as having SCD ((17) i.e. clinical criteria for MCI, dementia or psychiatric disorder not fulfilled, no neurological diseases known to cause memory complaints (e.g. Parkinson’s disease, epilepsy), HIV, abuse of alcohol or other substances). Individuals were invited for study participation based on the following criteria: I) diagnosis of SCD II) age 50 years or older, and III) owning a smartphone, tablet or computer.
2) research registries: we included individuals that signed up for research registries, a) the Dutch Brain Health Registry (online register; www.hersenonderzoek.nl) which facilitates participant recruitment for neuroscience studies and is open for individuals of any age; b) the Cologne Alzheimer dementia prevention registry [Kölner Alzheimer Präventionsregister (KAP)], which is open for individuals of any age interested in the field of dementia. Through newsletters individuals receive information on research and are asked to participate in scientific studies. Individuals were invited for study participation based on the following criteria: I) self-reported experience of memory loss as assessed by either the question “Do you have memory complaints?” (Dutch registry) or the SCD interview (18) (German registry), II) age 50 years or older, III) no diagnosis of Alzheimer’s disease, another type of dementia or mild cognitive impairment, as assessed through self-report, and VI) owning a smartphone, tablet or computer to access the online lifestyle program. No informant information was available for the participants from the research registries.
Online lifestyle program
Hello Brain is a European Project (FP7 grant no 304867) led by Trinity College Dublin. Hello Brain comprises a website and app which are available in English French and German. The website www.hellobrain.eu shares information and videos about the brain, brain health and brain research. The App aims to support users to live a brain healthy life by giving daily suggestions called ‘brain buffs’. There are five brain buff categories: physical activity, social activity, mental activity, lifestyle (nutrition, smoking, alcohol) and attitude (referring to stress management and positive thinking; 30 brain buffs per category). Participants are instructed to read the brain buff and are encouraged to engage in the described activity. If the user cannot or does not want to conduct a specific brain buff, a new brain buff can be requested.
For the current project, a collaboration was started between the EuroSCD team and Trinity College Dublin. We first investigated the preferences and wishes for an online lifestyle program in an international group of users (12). Then, in collaboration with users and a technical party, we adapted the program HelloBrain (Dutch: HalloHersenen, German: HalloGehirn; Appendix 1: details and screenshots). The scientific content was translated and the modules were adapted in order for the interactive module to become the main module. Additional brain buffs were created by a team of brain researchers and added to the program (15 per category) in order to allow tailoring based on a personal profile. The overall lay-out of HelloBrain was changed to a calmer look-and-feel by applying the grey background, that was included in some of the original HelloBrain screens, to all screens while keeping the colorful details.
Feasibility study
After the above mentioned adaptations, we performed a feasibility study to evaluate accessibility and the study procedures, to collect qualitative feedback and optimize study procedures for the online user test. We used 4 iterations of user input and adaptations to create a version of the program that was ready to evaluate user-experiences in a 30-day user test.
Focus groups
In 4 focus groups (memory clinic + Dutch Brain Health Registry, total N=17: 67±6y, 65% female) the language and structure of the program was evaluated. Specific topics were hierarchy of screens (wireframe), language, lay-out, and the wording of reminders and instructions. Feedback was translated into technical and content-related adaptations, and passed on to the developers. We used an iterative process, meaning that after each focus group the program was adapted. In the next focus group the adapted version of the program was evaluated.
Technical pilot
We conducted a technical pilot to evaluate accessibility of the program. Accessibility was defined as the ability to log in to the website or the app independently, with devices at home. Participants (memory clinic, N=5: 61±8y, 80% female) received access to the program through the website or the app for 2 weeks. All technical issues raised by the participants were collected and adaptations were made.
Pilot test phase
To evaluate feasibility of the planned study procedures, we conducted a pilot test phase in which we included 43 SCD subjects (Dutch Brain Health Registry, 65±8y, 66% female). Participants received account information by email and were instructed to use the program for 30 days. Users were able to email the researchers and if necessary, we initiated contact by telephone. At the end of the test-period, participants received digital questionnaires by email to evaluate the procedure of sending online questionnaires, having participants filling out the questionnaires and adequate data collection.
30-day user test: user-experiences
Participants
Finally, we conducted a 30-day user test to evaluate user-experiences. Individuals from the Dutch Brain Health Registry and the Cologne Alzheimer dementia prevention registry were approached. These individuals were not involved in previous phases of the program development.
Procedures
Participants received account information and could access the program for 30 days. Participants were instructed to use the program on a daily basis and complete one brain buff each day. Besides the daily brain buff, participants could access the information on brain health as they liked. After the 30-day user test the participants received self-report questionnaires to evaluate the program. In addition, participants were asked whether they were willing to share their experiences during a telephone interview. Study procedures slightly differed between centers, because of characteristics of the research registers (online in the Netherlands, on paper after an in-person information session in Germany) and requirements of the Cologne ethical committee to send information via post instead of email.
Measures
Data log
During the online user test, log data regarding the usage of the program were collected. Log data entailed number of log ins, log outs, brain buffs completed, brain buffs passed, and page visits during the test period.
Usefulness of daily advices
After indicating that a brain buff was completed, participants were asked to provide a rating of the usefulness of the brain buff. This rating was illustrated with 1 to 5 stars. Participants were invited to leave a comment.
Usability, ease of learning and satisfaction
We used the User Satisfaction and Ease of use (USE) (19) and the System Usability Scale (SUS) questionnaire (20) to assess perceived user-experiences of the online program. The USE questionnaire includes items on usefulness (e.g. is the program perceived as useful, does it have value to the user), ease of learning (e.g. is it easy to learn how the program works) and satisfaction (e.g. does it fulfill the wishes and expectations of the user), with scores ranging from 1 to 7. We used the domain scores for usefulness, ease of learning and satisfaction, by averaging the scores of items per domain. The SUS questionnaire includes 10 items on usability (scores ranging from 1-5; e.g. degree of convenience when using the program). The SUS questionnaire includes both positive and negative items. Total SUS score (range 0-100) was calculated by subtracting 1 from positive items and inversing negative items (5 – item score), summing these scores and multiplying with 2.5 (20). For both questionnaires higher scores indicate better ratings.
Qualitative exploration of user-experiences
We held semi-structured telephone interviews to gain more insight in the questionnaire results and to discuss additional topics. We chose a random sample (N=30) from participants that indicated to be willing to participate in the telephone interview. Aspects that were deemed most important to improve, good and useful aspects of the program and communication during the user test were discussed. In case the questionnaire results needed clarification, the interviewer posed specific questions.
Frequency of Internet use
In the Dutch subsample, a question regarding frequency of internet use was included in the usability questionnaire. In a German subsample frequency of internet use was discussed during the in-depth interview.
Data analysis
Analyses of quantitative data were conducted using SPSS version 22. Descriptive methods were used to describe demographics, average ratings of daily advices per category, use of the program (data log) and user-experience scores (questionnaires) in means and standard deviation, or percentages. Analysis of variance was used to compare questionnaire scores of Dutch and German participants, and to compare the ratings between brain buff categories. P-values of ≤0.05 were considered significant. Qualitative data was collected during the telephone interviews. Every interview was summarized in a short report. All comments were summarized independently by two researchers (LW, AKS). Data was then structured by these researchers upon consensus, in order to identify themes that were of importance to the participants when using the program of when participating in this study.
Results
Feasibility study
Focus groups
We let the participants discuss terminology within the program. At first, we kept some English terms in the program. The participants proposed to use Dutch language only. We discussed which terms should be incorporated to replace the English terms. ‘Brain buff’ became ‘Oppepper’ (Dutch for ‘Boost’), and although the category name ‘Attitude’ also translates to the Dutch ‘Attitude’, participants preferred a different wording (‘Houding’; Dutch synonym for ‘Attitude’). Participants agreed with the order and hierarchy of the screens (the wireframe). Upon their input the button for instructions was enlarged and placed more prominently, and we added ‘Uitleg’ (Dutch for ‘Explanation’) underneath this circled question mark symbol. Participants mentioned that back-and-forth buttons needed to be more prominent, which we adapted accordingly, and the hierarchy of the current location should be visible. Therefore, so called ‘Breadcrumbs’ were added to the page. Breadcrumbs are a simple display of the current location in the program, and easy way to click to a location with higher hierarchy (e.g. Start page / Brain Health / Neuroplasticity). Participants mentioned that they would prefer more instructions when entering the main screen. Together with the technical party and participants we came up with the solution to add a highlighting instruction, which highlights and explains all parts of the screen one by one.
Technical pilot
Of the 5 participants that evaluated the accessibility of the program, nearly all (4/5) reported a smooth download and log in without any assistance. One participant was not able to log in, as a result of a problem with the internet browser. Together with the technical party, the issue was resolved. After log in, 2 participants reported several technical bugs, such as wrong linking between pages or not enough variation in the daily advices, which was caused by an algorithm error. These issues were solved by the technical party.
Pilot test phase
Sending and receiving the questionnaires digitally went well. Participants did not report problems filling out the digital questionnaires. Almost all communication was done via email and online questionnaires. Some participants liked the efficient communication and felt that they were skilled enough to work online, while others would have preferred personal contact throughout the test-phase and provide feedback by telephone. Some participants mentioned that they would have liked an ‘emergency hotline’ in order to have personal contact by telephone in case they would have needed help when using in the program. Based on participants’ suggestions, we made the instructions for the online user test more detailed.
30-day user test: user-experiences
Participants
In total, 137 SCD subjects (55 Netherlands, 82 Germany) were included in the online user test. Participants were on average 65.1±8.6 years of age, 57% female and participants completed 11.3±1.9 years of education. German participants had on average more years of education (12.6±1.4) compared to Dutch participants (10.2±1.9, p<.01). The majority of the participants reported to use the internet on a daily basis (>90% of a subsample; Dutch N=55, German N=15).
Data log
In total, 120 (88%) participants used the online lifestyle program during the 30-day test period, whereas 17 (12%) participants did not log in. On average, participants reported to have completed 31±31 daily advices and requested a different brain buff 23±40times during these 30 days. Participants switched between pages on average 117 times (from brain buff screen to informative module and back, or within the informative module).
Usefulness of brain buffs
In total, participants rated 3266 brain buffs with a mean score of 3.5 (±1.5, max 5). The mean ratings differed between categories (F(4,3261)=5,725, p=.000). In general, buffs in the Attitude (3.6±1.4) and Physical activity (3.6±1.4) categories were higher appreciated than Lifestyle advices (3.3±1.6, resp. p<.001 and p=.001). Brain buffs of all categories received scores ranging from 1 to 5 stars and rankings were accompanied by both positive and negative comments. While some participants really liked a brain buff (“It would be very easy and fun to do this every day”) others disliked the same brain buffs (“I have never liked this and I will not do this today”). This diversity in appreciation of the categories, is presented in Figure 2.
Figure 2. Variety in reported usefulness of brain buffs
NOTE: This figure illustrates the percentages of brain buffs that received 1 to 5 stars ratings per category, and presents a negative (1 star, left) and a positive (5 stars, right) quotes for each category for illustrative purposes
Usability, ease of learning and satisfaction
The questionnaire was completed by 101 participants (response rate 74%; 37 Netherlands, 64 Germany). Participants gave on average moderate scores on items of the USE questionnaire (max 7): usefulness 3.7±1.3, ease of learning 3.6±1.9 and satisfaction 4.0±1.5 points. Dutch participants rated the program higher on these 3 domains compared to German participants (Dutch: 4.1±1.3, 4.8±1.5, 4.4±1.4 vs. German: 3.4±1.3, 2.4±1.5, 3.7±1.6; p<.05). The average score for usability on the SUS questionnaire was 63.7±19 out of 100, which translates to ‘OK to good’ (21) and did not differ between Dutch and German participants. Figure 3 presents the heterogeneity of user-experience scores within the total group.
Figure 3. Heterogeneity in user-experiences
Qualitative exploration of user-experiences
Table 1 gives a summary of the qualitative feedback illustrated by quotes. Participants mentioned that they would prefer a personalized program, meaning that it would fit their specific preferences. For example, with content based on their current lifestyle and preferred lifestyle category. Some of the participants used the program mainly for information, while others mainly liked the interactive part. When asked what was most important to improve, participants mostly mentioned to optimize technical aspects of the program to ensure a smooth walk-through.
Most participants mentioned to highly appreciate the content of the program. They liked to have a platform available to read about the brain and brain health, and to have access to a trustworthy source of information. When specifically asked what they liked most about the program, participants reported that the program induced awareness of lifestyle factors that are related to brain health. While most participants knew that physical exercise is related to brain health, they were often not aware of the relation between nutrition or social activities and brain health. Some participants mentioned that the program was positive and induced motivation to live healthier. Others were stimulated to look at their current lifestyle, felt confirmation that they have a healthy lifestyle or were motivated to continue with current lifestyle changes.
Table 1. Summary of in-depth exploration of user-experiences, collected during telephone interviews
NOTE: This Table presents qualitative feedback, which was provided by 30 participants during the telephone interviews.
Discussion
We developed an online lifestyle program for brain health and found that its’ overall user-experience was moderate to positive. Qualitatively, participants reported to appreciate the content of the program and having a trustworthy source of information on lifestyle and brain health. Quantitative scores on usefulness and ease of learning showed room for improvement. We observed high heterogeneity in the preference of specific lifestyle topics, which emphasizes the need for personalization.
Content on the brain and brain health of the online program, as offered in the brain buffs and the information pages, was highly appreciated by the participants. Both the brain buffs and the information pages were reported to be interesting and useful. Many participants reported to have learned new things. Often they were not aware that all the lifestyle factors that were included in the program were associated to brain health. Previous studies into the attitudes towards prevention of AD and related dementias highlighted the need to improve the beliefs and attitudes towards dementia prevention (1, 3, 22). Our study showed that a tool with information on lifestyle and brain health can contribute to the awareness on modifiable risk factors of dementia.
Involving the users throughout the process of development of an online program is expected to benefit usability and thereby adherence to the program. Our recent review, however, showed that for online lifestyle programs aimed at brain health it was often unclear whether and how users were involved during development and evaluation of the program (15). For example, a study on adherence to lifestyle interventions for dementia prevention found that adherence was lowest for the unsupervised computer-based cognitive training compared to other supervised trainings (23). However, user-involvement during development and evaluation was not described and therefore it remains unclear whether this could have benefitted adherence rates. In this study we aimed to evaluate and optimize user-experiences. When a program will be implemented internationally, it is important to explore cultural differences. Our multinational participatory research design increases the quality of output and sustainability, but also ensures culturally appropriate research, which is of importance when developing an international application (24). As a next step, additional options to increase the impact of the program should be explored. It might be worthwhile to evaluate integration of persuasive technologies that aim to influence behavior and attitudes. If such technologies are used the right way, it is more likely that users reach health-related goals (25).
The heterogeneity in the ratings of brain buffs, the frequent requests for different brain buffs and the qualitative feedback emphasize the need for personalization. Personalization has also been identified as one of the principles to increase appreciation and overall adherence to an online intervention (26, 27). In the current version of the program, part of the content was personalized, since users could request a different brain buff and could access information as they wished. Participants mentioned that they would prefer to receive brain buffs based on their current lifestyle behavior. Further evaluation and integration of personalization options, such as adapting lifestyle advices based on current lifestyle habits, could improve user-experience and thereby adherence to the program.
Lessons learned from the qualitative input of the users, mainly entail the preference for tailoring based on current lifestyle behavior. In addition, participants mentioned different possible effects of the program. Therefore, it might be interesting to rethink the most appropriate outcome measures of future lifestyle-based interventions in SCD. While changes in lifestyle or brain health might seem obvious, effects on psychological well-being or fear for dementia could also be worth consideration.
The quantitative ratings evaluating user-experience were moderate, which was lower than we expected. We identified room for improvement, particularly in ease of learning. Meaning that additional adaptations are necessary to improve instructions and clarity within the program. Differences in ratings could have several reasons, such as education, cultural differences, differences in reporting and differences in digital skills – which we did not assess systematically. Regarding the 30-day user test, German participants reported difficulties when learning to use the program. Some of the technical difficulties occurred only in the German back-end. Although we fixed these technical issues and thoroughly tested the program, we could not rule out remaining minor issues, possibly contributing to the differences in these scores. In our previous study (12), we found that German individuals with SCD used the Internet less often compared to the Dutch participants. However, information on the current Dutch sample and a subset of the German sample showed that over 90% of the participants uses the internet on a daily basis and therefore frequency of internet use is unlikely to influence perceived difficulties. Further, we did not match participants from the feasibility study and 30-day user test. Therefore, we cannot rule out the influence of demographical differences on perceived usability and satisfaction during development and the actual test phase.
In summary, qualitative feedback on the programs’ content was positive, while quantitative feedback on program characteristics showed room for improvement. This discrepancy between the positive qualitative feedback and the moderate quantitative ratings emphasizes the importance of combining methods when evaluation usability of eHealth applications, which was also emphasized in a recent scoping review on methods of usability testing in the development of eHealth applications (28).
This study had some limitations. First, the development and feasibility study took place in the Netherlands, and not in Germany. However, we believe it is promising that participants with different nationalities appreciated the same program, strengthening feasibility to offer one program in multiple countries. Second, a selection bias might have occurred, a study on an online program could have attracted individuals with better digital skills. However, the digital literacy of the participants varied from limited to very skilled, which was also reflected in the variety of feedback regarding ‘ease-of-learning’. Since an online program will only be used by those able and willing to access a program online, the current participants seem representative for the actual target group. Third, we did not have detailed information on drop-outs and therefore cannot describe their characteristics. We did however encourage all individuals to complete the questionnaires and we interviewed individuals independent of their attitude towards the program. Fourth, the participants were recruited based on different SCD criteria between the memory clinic and the research registries. However, we deem the population representative for the heterogeneous populations that can be recruited via memory clinics and brain health registries, and results are generalizable as such. Finally, based on the data log we cannot make a distinction between merely clicking through the program and attentively reading pages. Therefore, we could not take frequency or duration of active participation into account. In the future, this information could be considered when evaluating user-experiences and lifestyle effects of the program.
The strengths of the study include the study design. This was a multicenter study conducted in Germany and the Netherlands. This international character contributes to the generalizability of the findings to other European populations. Results also suggest that although some differences were found, one online tool for multiple European countries would be feasible. Second, we involved the target population throughout the process of development and evaluation. Co-creation is expected to increase the extent to which the tool fits the users’ preferences and digital skills and thereby acceptation and impact of the innovation in further stages (29, 30). Therefore, the users’ input was crucial and resulted in an online lifestyle program fitting the needs and preferences of individuals with SCD. Third, we combined methods to assess user-experiences of the online lifestyle program. The quantitative and qualitative methods were found to complement each other. Finally, we focused on individuals with SCD, who do not show cognitive deficits but at group level are at increased risk for cognitive decline. Therefore, this group is of clinical relevance in the context of dementia prevention. Individuals with SCD also report a need for brain health information, which has not yet been fulfilled since trustworthy sources are still lacking. This group is willing to participate in prevention strategies, which was also observed during recruitment and led to a higher inclusion number than planned.
In conclusion, in this study we developed and evaluated an online lifestyle program for brain health in individuals with SCD. We found that the overall user-experience of our program was moderate to positive. Participants appreciated content on lifestyle and brain health. The variety in preferences for different categories highlighted the need for personalization. It was feasible to offer this online lifestyle program in an at-risk population with SCD. Online self-applied lifestyle programs seem useful when aiming to reach large groups of motivated at-risk individuals for brain health promotion.
Conflicts of interest: The authors state no conflicts of interest. Ms. Wesselman reports grants from JPND/ZonMw, grants from Stichting Equilibrio, during the conduct of the study; Dr. Schild reports grants from Bundesministerium für Bildung und Forschung, during the conduct of the study; Dr. Hooghiemstra has nothing to disclose; Mr. Meiberth reports grants from Bundesministerium für Bildung und Forschung, during the conduct of the study; Ms. Drijver has nothing to disclose; Ms. v Leeuwenstijn-Koopman has nothing to disclose; Dr. Prins has nothing to disclose; Dr. Brennan has nothing to disclose; Dr. Scheltens has nothing to disclose; Dr. Jessen reports grants from Bundesministerium für Bildung und Forschung, during the conduct of the study; Dr. van der Flier reports grants from JPND/ZonMw, during the conduct of the study; Dr. Sikkes reports grants from JPND/ZonMw, grants from Stichting Equilibrio, during the conduct of the study.
Funding: The project is supported through the following funding organizations under the aegis of JPND (www.jpnd.eu; JPND_PS_FP-689-019): Germany, Bundesministerium für Bildung und Forschung (BMBF grant number: 01ED1508), the Netherlands, ZonMw grant no. 733051043. It was additionally supported by a research grant from Stichting Equilibrio. W.M. van der Flier is recipient of a grant by Gieskes-Strijbis fonds. S. Sikkes is recipient of a ZonMw Off Road grant (grant no. 451001010).
Acknowledgements: We thank all participants for their contribution to this research project. We thank Roxelane BV. and specifically Rudolf Wolterbeek, Brian Fa Si Oen and Max Hasenaar for their contribution to this project representing the technical party within the collaboration. We thank Mark Dubbelman for data visualization. We thank the founders of HelloBrain.eu (Trinity College Dublin, supported by European Union’s Seventh Framework Program for research, grant no. 304867) for the fruitful collaboration. The website www.hellobrain.eu shares easy-to-understand information and animations about the brain, brain health and brain research. The freely available interactive app, Hello Brain Health, aims to support users to live a brain healthy life by giving daily suggestions called ‘brain buffs’. The app is available on the project website, the App Store and Google Play. The Alzheimer Center Amsterdam is supported by Alzheimer Nederland and Stichting VUmc fonds. Research of the Alzheimer Center Amsterdam is part of the neurodegeneration research program of Neuroscience Amsterdam. Wiesje van der Flier holds the Pasman chair. Hersenonderzoek.nl is funded by ZonMw-Memorabel (project no 73305095003), a project in the context of the Dutch Deltaplan Dementie, the Alzheimer’s Society in the Netherlands and the Brain Foundation Netherlands.
Open Access: This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
References
1. Petersen R C, Smith GE, Waring SC, et al. Mild cognitive impairment: clinical characterization and outcome. Archives of Neurology 1999;56:303-308.
2. Winblad B, Palmer K, Kivipelto M, et al. Mild cognitive impairment–beyond controversies, towards a consensus: report of the International Working Group on Mild Cognitive Impairment. Journal of Internal Medicine 2004;256:240-246.
3. Baumgart M, Snyder HM, Carrillo MC, et al. Summary of the evidence on modifiable risk factors for cognitive decline and dementia: a population-based perspective. Alzheimer’s & Dementia 2015;11:718-726.
4. Bourre J & Paquotte P. Seafood (wild and farmed) for the elderly: Contribution to the dietary intakes of iodine, selenium, DHA and vitamins B12 and D. The Journal of Nutrition, Health & Aging 2008;12:186-192.
5. Dangour A, Allen E, Elbourne D, et al. Fish consumption and cognitive function among older people in the UK: Baseline data from the opal study. The Journal of Nutrition, Health & Aging 2009;13:198-202.
6. Morris MC, Evans DA, Tangney CC, et al. Fish consumption and cognitive decline with age in a large community study. Archives of Neurology 2005;62:1849-1853.
7. Innis SM. Dietary (n-3) fatty acids and brain development. The Journal of Nutrition 2007;137:855-859.
8. Valenzuela A. Docosahexaenoic acid (DHA), an essential fatty acid for the proper functioning of neuronal cells: their role in mood disorders. International Journal of Fats and Oils 2009;60:203-212.
9. van de Rest O, Geleijnse JM, Kok FJ, et al. Effect of fish oil on cognitive performance in older subjects: a randomized, controlled trial. Neurology 2008;71:430-438.
10. Dangour AD, Allen E, Elbourne D, et al. Effect of 2-y n−3 long-chain polyunsaturated fatty acid supplementation on cognitive function in older people: a randomized, double-blind, controlled trial. The American Journal of Clinical Nutrition 2010;91:1725-1732.
11. Danthiir V, Hosking DE, Nettelbeck T, et al. An 18-mo randomized, double-blind, placebo-controlled trial of DHA-rich fish oil to prevent age-related cognitive decline in cognitively normal older adults. The American Journal of Clinical Nutrition 2018;107:754-762.
12. Moran C, di Palumbo AS, Bramham J, et al. Effects of a Six-Month Multi-Ingredient Nutrition Supplement Intervention of Omega-3 Polyunsaturated Fatty Acids, vitamin D, Resveratrol, and Whey Protein on Cognitive Function in Older Adults: A Randomised, Double-Blind, Controlled Trial. The Journal of Prevention of Alzheimer’s disease 2018;5:175-183.
13. Nilsson A, Radeborg K, Salo I, et al. Effects of supplementation with n-3 polyunsaturated fatty acids on cognitive performance and cardiometabolic risk markers in healthy 51 to 72 years old subjects: a randomized controlled cross-over study. Nutrition Journal 2012;11:99.
14. Witte AV, Kerti L, Hermannstädter HM, et al. Long-chain omega-3 fatty acids improve brain function and structure in older adults. Cerebral Cortex 2013;24:3059-3068.
15. Külzow N, Witte AV, Kerti L, et al. Impact of Omega-3 Fatty Acid Supplementation on Memory Functions in Healthy Older Adults. Journal of Alzheimer’s Disease 2016;51:713-725.
16. Quinn JF, Raman R, Thomas RG, et al. Docosahexaenoic acid supplementation and cognitive decline in Alzheimer disease: a randomized trial. JAMA 2010;304:1903-1911.
17. Freund-Levi Y, Eriksdotter-Jönhagen M, Cederholm T, et al. ω-3 fatty acid treatment in 174 patients with mild to moderate Alzheimer disease: OmegAD study: a randomized double-blind trial. Archives of Neurology 2006;63:1402-1408.
18. Kotani S, Sakaguchi E, Warashina S, et al. Dietary supplementation of arachidonic and docosahexaenoic acids improves cognitive dysfunction. Neuroscience Research 2006;56:159-164.
19. Chiu C-C, Su K-P, Cheng T-C, et al. The effects of omega-3 fatty acids monotherapy in Alzheimer’s disease and mild cognitive impairment: a preliminary randomized double-blind placebo-controlled study. Progress in Neuropsychopharmacology and Biological Psychiatry 2008;32:1538-1544.
20. Yurko-Mauro K, McCarthy D, Rom D, et al. Beneficial effects of docosahexaenoic acid on cognition in age-related cognitive decline. Alzheimer’s & Dementia 2010;6:456-464.
21. Vakhapova V, Cohen T, Richter Y, et al. Phosphatidylserine containing ω–3 fatty acids may improve memory abilities in non-demented elderly with memory complaints: A double-blind placebo-controlled trial. Dementia and Geriatric Cognitive Disorders 2010;29:467-474.
22. Mazereeuw G, Lanctot K, Chau S, et al. Effects of omega-3 fatty acids on cognitive performance: a meta-analysis. Neurobiology of Aging 2012;33:1482.e17-1482.e29.
23. Sinn N, Milte CM, Street SJ, et al. Effects of n-3 fatty acids, EPA v. DHA, on depressive symptoms, quality of life, memory and executive function in older adults with mild cognitive impairment: a 6-month randomised controlled trial. The British Journal of Nutrition 2012;107:1682-1693.
24. Lee L, Shahar S, Chin AV, et al. Docosahexaenoic acid-concentrated fish oil supplementation in subjects with mild cognitive impairment (MCI): a 12-month randomised, double-blind, placebo-controlled trial. Psychopharmacology 2013;225:605-612.
25. Rondanelli M, Opizzi A, Faliva M, et al. Effects of a diet integration with an oily emulsion of DHA-phospholipids containing melatonin and tryptophan in elderly patients suffering from mild cognitive impairment. Nutritional Neuroscience 2012;15:46-54.
26. Yacong B, Xueyuan Z, Youli W, et al. The n-3 Polyunsaturated Fatty Acids Supplementation Improved the Cognitive Function in the Chinese Elderly with Mild Cognitive Impairment: A Double-Blind Randomized Controlled Trial. Nutrients 2017;9:54.
27. Barberger-Gateau P, Samieri C, Feart C, et al. Dietary omega 3 polyunsaturated fatty acids and Alzheimer’s disease: interaction with Apolipoprotein E genotype. Current Alzheimer Research 2011;8:479-491.
28. Fjell AM & Walhovd KB. Structural brain changes in aging: courses, causes and cognitive consequences. Reviews in the Neurosciences 2010;21:187-222.
29. Chhetri JK, de Souto Barreto P, Cantet C, et al. Trajectory of the MAPT-PACC-preclinical Alzheimer cognitive composite in the Placebo Group of a Randomized Control Trial: results from the MAPT Study: lessons for Further Trials. The Journal of Prevention of Alzheimer’s disease 2018;5:31-35.
30. Berlau DJ, Corrada MM, Head E, et al. APOE ε2 is associated with intact cognition but increased Alzheimer pathology in the oldest old. Neurology 2009;72:829-834.
31. Randolph C. Repeatable Battery for the Assessment of Neuropsychological Status (RBANS). San Antonio: Psychological Corporation, 1998;.
32. Nelson H. The Nelson adult reading test (NART) manual. Windsor, England. NFER-Nelson 1982;530:531.
33. Mendoza JE, Apostolos GT, Humphreys JD, et al. Coin rotation task (CRT): a new test of motor dexterity. Archives of Clinical Neuropsychology 2009;24:287-292.
34. Pachana NA, Thompson, LW, Marcopulos BA, et al. California older adult Stroop test (COAST) development of a Stroop test adapted for geriatric populations. Clinical Gerontologist 2004;27:3-22.
35. Army US. Army individual test battery. Manual of directions and scoring. Washington, DC: War Department, Adjutant General’s Office, 1944.
36. Wechsler D. Wechsler Adult Intelligence Scale, 3rd ed. San Antonio, TX: Psychological Corporation, 1997.
37. Thornton A & Leathem, JM. Coin rotation task. The development of norms for New Zealand and the United States. Presented at 6th International and 11th National Congress of Clinical Psychology. Santiago De Compostela, Spain, (2013, June).
38. Sheikh JI & Yesavage JA. Geriatric Depression Scale (GDS): Recent evidence and development of a shorter version. Clinical Gerontologist: The Journal of Aging and Mental Health 1986;5:165-173.
39. Pachana NA, Byrne GJ, Siddle H, et al. Development and validation of the Geriatric Anxiety Inventory. International Psychogeriatrics 2007;19:103-114.
40. Ware Jr JE & Gandek B. The SF-36 Health Survey: Development and use in mental health research and the IQOLA Project. International Journal of Mental Health 1994;23:49-73.
41. Bartels C, Wegrzyn M, Wiedl A, et al. Practice effects in healthy adults: a longitudinal study on frequent repetitive cognitive testing. BMC Neuroscience 2010;11:118.
42. Ahl RE, Beiser A, Seshadri S, et al. Defining MCI in the Framingham Heart Study Offspring: Education vs. WRAT-based norms. Alzheimer Disease and Associated Disorders 2013;27:1-15.
43. Das UN. Nutritional factors in the pathobiology of human essential hypertension. Nutrition 2001;17:337-346.
44. Morris MC, Sacks F & Rosner B. Does fish oil lower blood pressure? A meta-analysis of controlled trials. Circulation 1993;88:523-533.
45. Sullivan PF, Neale MC & Kendler KS. Genetic epidemiology of major depression: review and meta-analysis. American Journal of Psychiatry 2000;157:1552-1562.
46. Deckers K, Boxtel MP, Schiepers OJ, et al. Target risk factors for dementia prevention: a systematic review and Delphi consensus study on the evidence from observational studies. International Journal of Geriatric Psychiatry 2015;30:234-246.