C. Sampaio
Chief Medical Officer, CHDI Management/CHDI Foundation Inc; Professor Clinical Pharmacology, Laboratorio de Farmacologia Clinica, Faculdade de Medicina de Lisboa; Partner of Thelonoius Mind, Consultants; Former Member of CHMP at EMA.
Corresponding Author: Cristina Sampaio, Chief Medical Officer, CHDI Foundation, Princeton, USA, Cristina.Sampaio@chdifoundation.org
J Prev Alz Dis 2023;3(10):333-335
Published online June 13, 2023, http://dx.doi.org/10.14283/jpad.2023.70
Alzheimer’s disease is a progressive neurodegenerative disorder that affects millions of people worldwide, causing a significant burden on patients, caregivers, and healthcare systems. The recent licensing in the USA of amyloid-targeted drugs, which have shown a modest positive effect in delaying cognitive deterioration, has generated a debate on whether these treatments have a clinically meaningful impact on disease progression. The central issue under discussion is whether the measurable effect of amyloid-targeting antibody therapy is worth the financial cost and patients’ burden associated with its administration.
The issue of clinical meaningfulness regarding small effect sizes in Alzheimer’s disease treatments is not new. Over two decades ago, when Anticholinesterease inhibitors (ACHEI) were first licensed worldwide, there were intense debates about their therapeutic value, considering the minor improvement detected after six months of treatment in ADAS-Cog – now established as around 2.7 points (95%CI -3.0 to -2.3) (1). At that time, most countries agreed to reimburse the treatment, but several constraints were implemented, such as prescription restrictions and guidelines for efficacy monitoring using what would today be called real-world data (2). Over time, most of these restrictions were lifted, and now there is a general consensus that ACHEIs and Memantine provided incremental value to Alzheimer’s disease therapeutics, impacting not just cognition but also functional and global outcomes. In retrospect, payers took a risk amid the lack of alternatives and immense societal pressure. However, it can now be argued that this risk paid off, as it brought value that was initially doubted.
Determining what constitutes a clinically meaningful effect size is more an art than a science, and it still falls under the category of concepts that are recognized only when they are seen, similar to the term’s origin in discussions about «pornography». There is no doubt about the clinically meaningful effects of nusinersen or zolgensma in Spinal Muscular Atrophy types 1 and 2 (3); however, controversy arises with borderline effect sizes. This controversy may be fueled by a genuinely small effect that is not inherently doubtful, as with lecanemab (4), or by small to moderate effect sizes established in small sample size clinical trials, which might not be real. The latter is the case surrounding AMX0035 for ALS (5).
There are straightforward definitions of clinical meaningfulness and methodologies to rigorously establish the value for a clinically meaningful change (6).
Clinically meaningful change, also known as a minimum clinical important difference (MCID), is a threshold value for which any change as large or larger is considered meaningful to patients, clinicians, or both, with patients’ perspectives taking precedence. Several methods exist to establish this:
1. Anchor-based methods: Compare the change in an outcome measure to an external criterion or «anchor» that is considered clinically meaningful, reflecting patients’ perspectives, usually through a global measure or patient-reported outcome (PRO).
2. Distribution-based methods: Estimate the magnitude of change in an outcome measure using statistical approaches based on data distribution.
3. Delphi technique: Assemble a panel of experts to reach consensus on what constitutes a clinically meaningful difference for a specific outcome measure.
4. Patient-reported outcomes (PROs): Identify the minimal change in the outcome that patients perceive as important or beneficial.
5. Clinical trials and responder analyses: Compare the proportion of patients achieving a predefined level of improvement or lower deterioration between treatment groups.
These methods can be circular; the anchor-based method is most appropriate for establishing the clinically meaningful difference (CMD) for a clinical outcome like a domain-specific rating scale, with ADAS-Cog being a good example. However, factors such as the better recognition and valuation of improvement by patients than the insidious, difficult-to-ascertain rate of deterioration should be considered. Changes in the rate of deterioration are even harder to perceive; therefore, when CMDs are determined for deterioration, they are usually larger than for improvement.
Furthermore, there are common misunderstandings about what MCID means and how it should be used. MCIDs should not be used as benchmarks for average differences between treated and control groups. This mistake can even be found in the correspondence published in the New England Journal of Medicine (NEJM) related to the lecanemab clinical trial paper (7). MCIDs are potentially relevant for each patient at an individual level, despite being derived from averaged values, and they may serve as a threshold to define what a responder is. A more appropriate approach that acknowledges the derivation of MCIDs from average values would involve creating a range of potential MCIDs. This is rarely done in a single publication; however, when there are multiple publications using different databases for generating MCIDs, they end up creating a range of possible values. The literature in this field is not very abundant; therefore, any single proposed MCID must be used judiciously.
Despite these difficulties, there is relatively recent data on MCIDs for the most common outcomes in AD clinical trials. Scott Andrews et al. (8) published the MCID for the Clinical Dementia Rating (CDR) Scale sum of boxes (SB) score, the MMSE, and the Functional Activities Questionnaire (FAQ) for patients on the AD continuum who visit various Alzheimer’s Disease Centers (ADCs) across the US. The study considered both anchor-based and distribution-based approaches, and it measured MCIDs for deterioration rather than improvement. An important limitation of this study is the temporal horizon. The authors used the interval between two consecutive visits, but they don’t specify how long this interval was. However, MCIDs for deterioration must take into account the time interval in which deterioration occurs. Losing 1 point in CDR-SB over 1 year, 2 years, or 5 years, for example, can have very different implications.
To illustrate this point, we can refer to the data generated in the context of Huntington’s disease. We calculated deterioration MCID for different endpoints using an anchor method (9). The Single Digit Modality Test (SDMT), which is the quasi-cognitive measure we have in the dataset, showed an MCID of 1.1 points for a 1-year temporal horizon, 1.7 points for 2 years, and 2.2 points for 3 years in participants in HD-ISS stage 2. This stage is roughly equivalent to minimal cognitive impairment in AD. This example highlights how the temporal horizon can impact the MCID values for cognitive measures.
What has been established for lecanemab, in a very well-designed and executed RCT (4), which currently serves as the best benchmark available and is the most recently licensed drug in the group, is that people treated for 18 months have a mean deterioration in CDR-SB of 1.21 points, while those on placebo deteriorate on average by 1.66 points. There was a statistically significant difference between the groups in favor of lecanemab, but both groups deteriorated more than the threshold for MCID, which is set at about 1 to 2 points. Consequently, there is ongoing controversy about whether the observed effect is worth enduring the cumbersome administration process, potential adverse reactions, and the cost of the treatment.
The key to resolving this controversy lies in future data, which can only be obtained if the drug is used. This creates a typical catch-22 situation.
At the time of licensing, the data available to determine if the measured effect surpasses the MCID threshold is often scarce. It seems the Alzheimer’s disease community, including stakeholders such as payers, are facing a conundrum similar to the early 2000s when ACHEIs were approved. They must make decisions now concerning reimbursement and market access based on limited data, hoping that the future will validate their choices. This is a routine process for investors, who make decisions daily in their portfolios. However, for patients who will be treated with newly approved amyloid-targeted drugs, these decisions directly impact their potential quality of life, which is an even more precious asset than those typically dealt with by investors.
Continuing with the stock market analogy, the decision on reimbursement and the extent of access to newly licensed drugs can be seen as a typical futures investment. In this case, it is a mix of long-term and growth investment. For the growth investment aspect, it is possible that as the drugs become more widely used, the benefits will become more apparent, though it is also likely that the expected adverse reactions (ARIA) will proportionally increase. In the long-term, benefits may gradually accumulate as more people under study have longer treatment exposures, particularly when they reach more than 5 years of follow-up.
So, what is currently established is that 18 months of treatment with Lacenemab deflects the progression on CDR-SB by 0.46 points. For simplification, we will count this as half a point. There are 3 possibilities for the progression lines after this 18-month landmark (variations on these are also possible but we will not discuss them) [Figure 1]:
1. They keep diverging, and the treatment arm accrues an advantage over the untreated group that progressively grows larger, like with compounded interest. These lines may diverge at different paces, which are unknown for the moment. Let’s assume they keep diverging at a rate of 0.46 points every 18 months. By the end of 5 years, the difference would become approximately 1.4 points. It is very likely that a larger proportion of treated people would have achieved an unequivocally clinically meaningful difference (CMD) than if they were not treated in this scenario. The «interest rate» on the treatment effect would be around 25% per year. We can calculate scenarios with lower or higher ‘interest rates’. For a 10% per year interest rate, the difference in CDR-SB points would be 0.76 points and 1.19 points after 5 and 10 years, respectively. For a 40% per year interest rate, it would be 2.5 points and 13.3 points at 5 and 10 years, respectively.
2. The progression lines keep diverging for a few more months, after which the effect gained doesn’t disappear but is maintained throughout the remainder of life. In this scenario, the value of the treatment will be predicated on how large the difference will be when the progression lines stop diverging.
3. The progression lines keep diverging for a while, after which the beneficial effect accrued starts to wane, and progressively the two groups converge until they become indistinguishable.
Figure 1. Different scenarios of disease progression under treated and untreated
This is obviously a very crude approximation for the modeling that needs to be done, and it only illustrates the potential benefit. The risk of ARIA will be an extremely important aspect of the decision-making process, just as logistics of administration and overall compliance metrics will also contribute.
The critical point is that these treatments will only be valuable if the rates of progression continue to diverge after 18 months, at least for some time, and the rate of adverse reactions doesn’t increase too much in a real-world practice context outside of clinical trials. To gather this information, more data needs to be accrued, ideally in real-world settings that can take various forms, such as large registry studies, data generated by Electronic Health Records, or a combination of methodologies.
It seems appropriate to me that payers should reimburse these drugs under certain restrictions, ensuring that each clinical use generates further data. This will allow decision-makers to revisit their decisions when it becomes possible to ascertain with reasonable precision how much treatment effect is accrued over longer time horizons and what the rate of adverse reactions is.
Conflict of interest: Dr C. Sampaio received consultancy honoraria from NeuroDerm, Pinteon Pharmaceuticals, Pharma2B, Biocodex, Inflictis. She is partner at Thelonius Mind, Consultants. And she receives salary from CHDIManagement/CHDI Foundation, Inc.
References
1. Birks J. Cholinesterase inhibitors for Alzheimer’s disease. Cochrane Database Syst Rev. 2006 Jan 25;2006(1):CD005593. doi: 10.1002/14651858.CD005593. PMID: 16437532; PMCID: PMC9006343.
2. Holden, M., & Kelly, C. (2002). Use of cholinesterase inhibitors in dementia. _Advances in Psychiatric Treatment,_ _8_(2), 89-96. doi:10.1192/apt.8.2.89
3. Hjartarson HT, Nathorst-Böös K, Sejersen T. Disease Modifying Therapies for the Management of Children with Spinal Muscular Atrophy (5q SMA): An Update on the Emerging Evidence. Drug Des Devel Ther. 2022 Jun 16;16:1865-1883. doi: 10.2147/DDDT.S214174. PMID: 35734367; PMCID: PMC9208376.
4. van Dyck CH, Swanson CJ, Aisen P, Bateman RJ, Chen C, Gee M, Kanekiyo M, Li D, Reyderman L, Cohen S, Froelich L, Katayama S, Sabbagh M, Vellas B, Watson D, Dhadda S, Irizarry M, Kramer LD, Iwatsubo T. Lecanemab in Early Alzheimer’s Disease. N Engl J Med. 2023 Jan 5;388(1):9-21. doi: 10.1056/NEJMoa2212948. Epub 2022 Nov 29. PMID: 36449413.
5. Mullard A. Amylyx’s ALS therapy secures FDA approval, as regulatory flexibility trumps underwhelming data. Nat Rev Drug Discov. 2022 Nov;21(11):786. doi: 10.1038/d41573-022-00171-6. PMID: 36216876.
6. Kallogjeri D, Spitznagel EL Jr, Piccirillo JF. Importance of Defining and Interpreting a Clinically Meaningful Difference in Clinical Research. JAMA Otolaryngol Head Neck Surg. 2020 Feb 1;146(2):101-102. doi: 10.1001/jamaoto.2019.3744. PMID: 31804662.
7. Correspondence to Lecanemab in Early Alzheimer Disease.N Engl J Med 2023; 388:1630-1632 DOI: 10.1056/NEJMc2301380.
8. Andrews JS, Desai U, Kirson NY, Zichlin ML, Ball DE, Matthews BR. Disease severity and minimal clinically important differences in clinical outcome assessments for Alzheimer’s disease clinical trials. Alzheimers Dement (N Y). 2019 Aug 2;5:354-363. doi: 10.1016/j.trci.2019.06.005. PMID: 31417957; PMCID: PMC6690415.
9. Hamilton JL, Mills JA, Stebbins GT, Long JD, Fuller RLM, Sathe S, Roché M, Sampaio C. Defining Clinical Meaningfulness in Huntington’s Disease. Mov Disord. 2023 May 5. doi: 10.1002/mds.29394. Epub ahead of print. PMID: 37147862.
© Serdi 2023