ORIGINAL ARTICLE

DEVELOPMENT OF A SWEDISH SHORT VERSION OF THE MONTREAL COGNITIVE ASSESSMENT FOR COGNITIVE SCREENING IN PATIENTS WITH STROKE

Tamar ABZHANDADZE, PhD^1,2, Erik LUNDSTRÖM, PhD³, Dongni BUVARP, PhD¹, Marie ERIKSSON, PhD⁴, Terence J. QUINN, PhD⁵ and Katharina S. SUNNERHAGEN, PhD^1,6

From the ¹Institute of Neuroscience and Physiology, The Sahlgrenska Academy, University of Gothenburg, ²Department of Occupational Therapy and Physiotherapy, Sahlgrenska University Hospital, Gothenburg, ³Department of Medical Sciences, Neurology, Akademiska Sjukhuset, Uppsala, ⁴Department of Statistics, USBE, Umeå University, Umeå, Sweden, ⁵Institute of Cardiovascular and Medical Sciences, University of Glasgow, Glasgow, UK and ⁶Neurocare, Rehabilitation Medicine, Sahlgrenska University Hospital, Gothenburg, Sweden

Objective: The primary objective was to develop a Swedish short version of the Montreal Cognitive Assessment (s-MoCA-SWE) for use with patients with stroke. Secondary objectives were to identify an optimal cut-off value for the s-MoCA-SWE to screen for cognitive impairment and to compare its sensitivity with that of previously developed short forms of the Montreal Cognitive Assessment.

Design: Cross-sectional study.

Subjects/patients: Patients admitted to stroke and rehabilitation units in hospitals across Sweden.

Methods: Cognition was screened using the Montreal Cognitive Assessment. Working versions of the s-MoCA-SWE were developed using supervised and unsupervised algorithms.

Results: Data from 3,276 patients were analysed (40% female, mean age 71.5 years, 56% minor stroke at admission). The suggested s-MoCA-SWE comprised delayed recall, visuospatial/executive function, serial 7, fluency, and abstraction. The aggregated scores ranged from 0 to 16. A threshold for impaired cognition ≤ 12 had a sensitivity of 97.41 (95% confidence interval, 96.64–98.03) and positive predictive value of 90.30 (95% confidence interval 89.23–91.27). The s-MoCA-SWE had a higher absolute sensitivity than that of other short forms.

Conclusion: The s-MoCA-SWE (threshold ≤ 12) can detect post-stroke cognitive issues. The high sensitivity makes it a potentially useful “rule-out” tool that may eliminate severe cognitive impairment in people with stoke.

LAY ABSTRACT

Stroke survivors have an increased risk of developing cognitive impairment, a common consequence of stroke. Therefore, many international guidelines recommend cognitive screening for all patients admitted to hospital with stroke. The Montreal Cognitive Assessment (MoCA) has been recommended as an appropriate cognitive test to be applied in stroke units. Although the administration of MoCA takes approximately 15 min, the screening can take longer in patients with acute stroke. Therefore, this study aimed to develop a Swedish short version of the Montreal Cognitive Assessment (s-MoCA-SWE) based on data from a large Swedish sample of acute and early subacute stroke survivors. The current study analysed data from 3,276 patients and suggest an s-MoCA-SWE that comprised the following tasks: delayed recall, visuospatial/executive function, serial 7, fluency, and abstraction. The s-MoCA-SWE could identify cognitive impairment in 97% of patients. In conclusion, s-MoCA-SWE has the potential to rule out severe cognitive impairment.

Key words: cognitive function; disorder; Montreal Cognitive Assessment; sensitivity; specificity; stroke.

 

 

Stroke survivors have an increased risk of developing cognitive impairment (1, 2), a common consequence of stroke, and a condition related to dependency and limited participation in activities of daily living (3). Failure to detect cognitive impairment may lead to missed opportunities for early intervention and adaptation to healthcare pathways. Hence, many international guidelines recommend cognitive screening for all patients admitted to hospital with stroke (1, 2).

Poststroke impairments and pre-morbid conditions can hinder comprehensive cognitive evaluation (4). Moreover, many traditional cognitive assessments can be time- and resource-demanding (1, 2). Therefore, screening tools for global cognition, such as the Montreal Cognitive Assessment (MoCA), have been recommended as appropriate cognitive tests to be applied in stroke units (1, 2). The MoCA is a feasible tool to screen cognitive function in people with mild to moderate stroke, administration takes approximately 10–15 min, it is easy to use, and its correlation with cognitive and functional outcomes has been reported (5). However, screening time can be longer in patients with acute stroke, especially if patients are medically unwell or have concomitant physical and sensory impairments (6). Therefore, several abbreviated versions of the MoCA have been devised to improve the utility of cognitive screening, including a telephone version of the MoCA (T-MoCA), a version recommended by the National Institute of Neurological Disorders and Stroke and the Canadian Stroke Network (NINDS-CSN), and the Short Form MoCA (SF-MoCA) (7–15). Validation studies of the available short versions of the MoCA were performed mainly with patients in the later stages of stroke (8) or non-stroke cohorts (7). Thus, there is a need to develop and evaluate a short form of the MoCA that is sensitive to detect cognitive impairment early in the stroke journey. In addition, a new instrument must be tested in the context in which it will be used, including in specific healthcare settings and languages.

Therefore, the aim of this study was to develop a Swedish short version of the Montreal Cognitive Assessment (s-MoCA-SWE) using data from a large Swedish sample of acute and early subacute stroke survivors (objective 1). To facilitate the use of the s-MoCA-SWE, a further aim of this study was to identify the optimal threshold for detecting cognitive impairment and compare the properties of the s-MoCA-SWE with those of previously developed short forms of MoCA (objective 2).

METHODS

The study protocol has been published previously (16). This study was reported in accordance with the Standards for Reporting Diagnostic Accuracy in Dementia and Cognitive Impairment guidelines (17).

Study design and sample

This cross-sectional study consisted of data from 2 cohorts: a Swedish registry cohort and the Efficacy oF Fluoxetine – a randomisEd Controlled Trial in Stroke (EFFECTS) trial cohort (18). The registry cohort comprised data from 2 sources: Väststroke and Riksstroke. Väststroke is a local stroke registry in Gothenburg, Sweden. All patients admitted for stroke at 3 stroke units at the Sahlgrenska University Hospital (catchment population 800,000) were registered. Using the patients’ unique personal identification number, the Väststroke register was linked with Riksstroke, a national stroke registry in Sweden (19). Data linkage allowed individual person-level access to Väststroke cognitive data and Riksstroke information on prehospital status and medical treatment. The registry data were extracted for the period from 1 November 2014 to 30 June 2019. Stroke patients from 35 centres in Sweden were enrolled in the EFFECTS trial (18). Data were collected from 20 October 2014 to 28 June 2019 (18).

The inclusion criteria for this study were complete MoCA data, age ≥ 18 years at stroke onset, and stroke diagnosis according to the International Classification of Diseases. The exclusion criterion was missing data on individual tasks of the MoCA.

Study procedures

The registry and EFFECTS datasets were aligned based on variables common to both datasets. Cognitive screening was performed during the acute and early subacute phases of stroke (20). Official training in the MoCA, created by copyright holders, was not mandated during the study period. However, all assessors had received specific training and had experience in administering the MoCA according to local standards.

Experienced occupational therapists working in stroke units administered the MoCA. The patients were approached at any time during their stay in the stroke units (median length of stay in the acute stroke units was 7 days). In the EFFECTS trial, the MoCA was administered by clinical study personnel or by local occupational therapists; cognitive function was screened 2 and 15 days after the onset of stroke and prior to randomization. In both datasets, decisions regarding suitability for screening were made by health professionals. The MoCA was not conducted if patients were unable to complete the test due to severe stroke complications, such as severe aphasia and/or delirium.

Study variables

The total MoCA score ranges from 0 to 30 (≤ 25 indicates cognitive impairment). One additional point is given for ≤ 12 years of education (21). In the current study dataset, the MoCA tasks were registered as follows (a maximum number of points indicates a perfect outcome): orientation (6 points [date and place]), delayed recall (5 points [free recall of 5 words]), visuospatial/executive function (5 points [trails test, cube, clock]), naming (3 points [naming of 3 animals]), digit span with 2 tasks (2 points [forward repetition of 5 digits and backward repetition of 3 digits]), sentence repetition (2 points [repetition of 2 sentences with different complexity]), abstraction tasks (2 points), serial 7 (3 points [100 − 7 serial subtraction task]), fluency (1 point [generating a minimum of 11 words or more in 60 s that begin with letter F), and attention (1 point [sequence of letters is read and patient has to tap their hand on letter A]).

Stroke severity at admission was assessed using the National Institutes of Health Stroke Scale (NIHSS) (22), minor stroke was defined as an NIHSS score of ≤ 3 (23).

Comparator index tests: previously developed short forms of the MoCA

There are many different versions of the short MoCA (7). Three versions are used in research and clinical practice; all 3 use the variables that were available in the current study dataset. The T-MoCA comprises the following tasks: delayed recall, digit span, attention (sequence of letters), serial 7, sentence repetition, fluency, abstraction, orientation (9). The total score on the T-MoCA consists of 22 points, with ≤ 18 points indicating impaired cognition (9); the NINDS-CSN (10, 12–14) includes delayed recall, fluency, and orientation. The total score consists of 12 points, with ≤ 9 points indicating impaired cognition (9, 11). The SF-MoCA contains delayed recall, serial 7, and orientation (7, 8) a total score of 14 points is possible, with ≤ 11 points indicating impaired cognition.

Statistical analysis

Detailed information is presented in Appendix S1.

Objective 1. To develop a Swedish short version of the Montreal Cognitive Assessment for cognitive screening in patients with stroke (s-MoCA-SWE). Two different methods were used to identify the most important MoCA tasks that should be included: principal component analysis (PCA) (24, 25) and boosted regression trees (BRT) (26, 27). The PCA helps to simplify information by reducing the dimensionality of the data without losing important information. BRT select tasks based on a reference standard instrument, which in this case was the complete MoCA.

PCA and BRT each resulted in a working version of a plausible short form: the s-MoCA-SWE/PCA and s-MoCA-SWE/BRT. Patients were then classified to cognitive impairment (no/yes) using separate binary logistic regression models that included the selected MoCA tasks as independent variables (expanded methods). Because the s-MoCA-SWE is a screening tool, greater weight was placed on reducing the number of false-negatives; therefore, sensitivity was used as the primary measure of accuracy (16).

Objective 2. To identify the optimal threshold for detecting cognitive impairment and compare the properties of the s-MoCA-SWE with those of previously developed short forms of the MoCA. The threshold for cognitive impairment on the s-MoCA-SWE was achieved by evaluating the coordinate points of the receiver operating characteristic curve and precision-recall curve.

Contingency tables were created for comparing the properties of the s-MoCA-SWE with those of previously developed short forms of the MoCA. A dichotomized s-MoCA-SWE (≤ 12 points for impaired cognition) was compared against a dichotomized T-MoCA (10), NINDS-CSN (9, 11), SF-MoCA (7, 8), and the complete MoCA (all at usual diagnostic thresholds). Sensitivity, specificity, positive and negative predictive values, positive and negative likelihood ratios, Youden’s index, and accuracy (overall probability that a patient is correctly classified) were calculated. The area under the receiver operating characteristic curve (AUC) for each test was evaluated. An AUC value of 0.8–0.9 was considered excellent and a value of more than 0.9 was considered outstanding (28).

Subgroup analyses were performed to evaluate the properties of the s-MoCA-SWE in 2 age groups (≤ 79 years and ≥ 80 years; 80 years is the median and mean age for dementia diagnosis in Sweden), stroke severity (minor stroke was defined as an NIHSS score ≤ 3 (22)), and previous stroke/transient ischaemic attack (16).

All statistical tests were two-tailed. The significance level was set at α = 5%. The data were analysed using SPSS (IBM Corp. Released 2018. IBM SPSS Statistics for Windows, version 28.0. Armonk, NY: IBM Corp.) and R, version 4.0.2 (R Core Development Group).

RESULTS

Participants

The registry cohort comprised 6,444 patients and the EFFECTS cohort included data on 1,500 patients. Of these, 3,276 patients had full MoCA data and were included in the study (Fig. 1). Included patients (n = 3,276) compared with excluded patients (n = 4,668) had a lower median age, 73 years vs 77 years (p < 0.001), and a lower proportion of female patients (p < 0.001) (Table SI).

Fig. 1. Flow diagram of the study sample selection. EFFECTS: Efficacy oF Fluoxetine – a randomisEd Controlled Trial in Stroke; MoCA: Montreal Cognitive Assessment; OT: occupational therapist.

Of the 3,276 patients included in the study, cognitive impairment, defined as a raw MoCA score ≤ 25, was present in 66% of the study sample (Tables I, SII and SIII).

Table I. Characteristics of the total study sample (n = 3,276) of the combined registry and EFFECTS cohorts

Characteristics	Pooled data n = 3,276	Registry cohort n = 1,990	EFFECTS cohort n = 1,286
Female sex, n (%)	1,311 (40)	825 (42)	486 (38)
Age, years, mean (SD)	71.5 (12.1)	71.8 (13.4)	71.0 (10.8)
Median (minimum–maximum, [IQR])	73 (18–100 [15])	74 (18–100 [16])	72.3 (21–95 [14])
Lived alone before stroke, n (%)	1,341 (45)	870 (51)	471 (37)
Independent in ADL before stroke, n (%)	2455 (85)	1,218 (76)	1,237 (96)
Having diabetes, n (%)	604 (19)	342 (18)	263 (21)
Previous TIA or/and stroke, n (%)	357 (12)	131 (8)	226 (18)
Stroke severity at admission, NIHSS, Mean (SD)	4.7 (5.3)	5.3 (6.4)	4.2 (3.4)
Median (minimum-maximum [IQR])	3 (0–33 [5])	2 (0–33 [8])	3 (0–20 [4])
Minor stroke, NIHSS ≤ 3 p, n (%)	1,544 (56)	848 (57)	696 (54)
Stroke type, n (%)
Intracerebral haemorrhage	299 (9)	151 (8)	148 (12)
Ischaemic stroke (IS)	2,933 (91)	1,796 (92)	1,137 (88)
Stroke, not specified	7 (< 0.1)	6 (0.3)	1 (0.1)
Reperfusion treatment for IS, n (%)
Thrombolysis	483 (15)	212 (11)	271 (22)
Thrombectomy	180 (6)	102 (6)	78 (7)
Discharge destination from the stroke units, n (%)
Own, or relatives’ home	1,763 (62)	1,016 (66)	747 (58)
Residential or long-term NHS home	411 (15)	365 (23)	48 (4)
Other hospital	541 (19)	164 (11)	377 (29)
Other	121 (4)	6 (0.4)	115 (9)
MoCA score (1–31 p)a, Mean (SD)	23.2 (4.8)	23.5 (4.6)	22.8 (5.2)
Median (IQR)	24 (6)	25 (6)	24 (7)
Impaired cognitive function (≤ 25 p), n (%)	2,006 (61)	1,189 (60)	817 (64)
MoCA score, (raw score 0–30 p)b, Mean (SD)	22.7 (4.9)	23.0 (4.7)	22.2 (5.3)
Median (IQR)	24 (6)	24 (6)	23 (7)
Impaired cognitive function (≤ 25 p), n (%)	2,159 (66)	1,281 (64)	878 (68)
+ 1 p for ≤ 12 years of education, n (%)	1,699 (52)	981 (49)	718 (56)
aMoCA scores with an extra point for ≤ 12 years of education; bMoCA score without an extra point for education.
Proportion of missing data: Living arrangements before stroke, 9%; ADL before stroke, 11%; Having diabetes, 1%; Previous TIA/stroke, 9%; Stroke type, 1%; Stroke severity, 15%; Discharge destination, 13%. The sum of observation in registry and EFFECTS cohorts may vary due to missing data.
SD: standard deviation; IQR: interquartile range; ADL: activities of daily living; TIA: transient ischaemic attack; NIHSS: National Institutes of Health Stroke scale; NHS: nursing homes; MoCA: Montreal Cognitive Assessment; EFFECTS: Efficacy oF Fluoxetine – a randomisEd Controlled Trial in Stroke.

Development and determination of the s-MoCA-SWE Principal component analysis

The data were suitable for detecting underlying components using principal component analyses (Bartlett’s test of sphericity, p < 0.0001; Kaiser–Meyer–Olkin’s measure was 0.88). The MoCA variable “Tap on A” had a communality value of ≤ 0.3 and was therefore excluded from further analysis (Table SIV). The model that used the varimax rotation method included 9 tasks of the MoCA. Two components were selected (Fig. S1, Table SV). These components were composed of 5 tasks of the MoCA having a loading value ≥ 0.6 (Table SVI). The s-MoCA-SWE/PCA was created by summing the scores of the tasks with the highest loading values: orientation, delayed recall, visuospatial/executive function, repetition, and digit span (Fig. 2). The aggregate score sum was 20, ranging from 0 to 20, with higher values indicating better cognitive function.

Fig. 2. Bar diagrams showing the selected tasks of the Montreal Cognitive Assessment (MoCA) by principal component analysis (PCA) and boosted regression tree analyses (BRT). Sample size, 3276 patients. s-MoCA-SWE: Swedish short version of the MoCA.

Boosted regression trees. The final BRT model was selected based on the best AUC value. The following MoCA tasks were selected (task’s relative influence value × 100): delayed recall (35.1), visuospatial/executive function (28.7), serial 7 (8.1), fluency (9.5), and abstraction (6.2) (Fig. 2). The aggregate score sum of these tasks was 16, ranging from 0 to 16, with a higher value indicating better cognitive function.

s-MoCA-SWE version

The sensitivity of the s-MoCA-SWE/PCA was 94% (SD 0.03); that of the s-MoCA-SWE/BRT was 95% (SD 0.03). The s-MoCA-SWE/BRT was therefore selected for further analysis, and thus the final version of the s-MoCA-SWE included the following tests: delayed recall, visuospatial/executive function, serial 7, fluency, and abstraction.

The Cut-off value of the index test s-MoCA-SWE for classifying patients with cognitive impairment

The receiver operating characteristic and precision-recall curves showed that an s-MoCA-SWE cut-off value of ≤ 12 had the highest sensitivity and precision for identifying patients with cognitive impairment (Fig. 3). This cut-off value remained unchanged after adjustment to an additional point allocated for education.

Fig. 3. Characteristics of the different cut-off points of the Swedish short version of the MoCA (s-MoCA-SWE), based on the boosted regression tree analyses (BRT) model. Reference standard: dichotomized Montreal Cognitive Assessment (MoCA).

Evaluation of the s-MoCA-SWE in relation to previously developed short forms of the MoCA

Descriptive information on dichotomized short forms of the MoCA and dichotomized reference standard instrument (a MoCA score of ≤ 25 indicates impaired cognition) is shown in Table II.

Table II. Cross tables with dichotomized short forms of the Montreal Cognitive Assessment and reference standard test (n = 3,276)

Index tests (score range)	Cut-off points for impaired and normal cognitive function	Reference Standard Test: Montreal Cognitive Assessment, score range 0–30 points.
		Impaired, ≤ 25 p	Normal, ≥ 26 p
s-MoCA-SWE (0–16)	Impaired, ≤ 12 p	2,103	226
	Normal, ≥ 13 p	56	891
T-MoCA (0–22)	Impaired, ≤ 18 p	2,028	169
	Normal, ≥ 19 p	131	948
NINDS-CSN (0–12)	Impaired, ≤ 9 p	1,935	349
	Normal, ≥ 10 p	224	768
SF-MoCA (0–14)	Impaired, ≤ 11p	1,877	275
	Normal, ≥ 12 p	282	842
s-MoCA-SWE: Swedish short version of the MoCA; T-MoCA: telephone version of the MoCA validated for telephone use; NINDS-CSN: National Institute of Neurological Disorders and Stroke and the Canadian Stroke Network MoCA; SF-MoCA: Short form MoCA.

The data from Table II were further used to calculate psychometric properties of the s-MoCA-SWE and previously developed short forms of the MoCA. The s-MoCA-SWE could correctly identify impaired cognition in 97.41% of cases (sensitivity value). However, among the patients who tested positive for impaired cognition according to the s-MoCA-SWE cut-off value of ≤ 12, the probability of having a true impairment was 90.30% (precision value) (Table III).

Table III. Psychometric properties of the s-MoCA-SWE, and previously developed short forms of the Montreal Cognitive Assessment (reference standard instrument: the dichotomized raw score of the MoCA, ≤ 25 impaired cognition; n = 3,276)

	Index Tests (score range [cut-off point for impaired cognition])
	s-MoCA-SWE (0–16 [≤ 12])	T-MoCA (0–12 [≤ 18])	NINDS-CSN (0–12 [≤ 9])	SF-MoCA (0–12 [≤ 11])
AUC	0.98 (0.98–1.00)	0.97 (0.96–0.97)	0.91 (0.90–0.92)	0.91 (0.90–0.92)
Sensitivity/recall	97.41 (96.64–98.03)	93.93 (92.84–94.90)	89.62 (88.26–90.88)	86.94 (85.44–88.33)
Specificity	79.77 (77.29–82.09)	84.87 (82.63–86.92)	68.76 (65.95–71.47)	75.38 (72.74–77.88)
PLR	4.81 (4.28–5.41)	6.21 (5.40–7.14)	2.87 (2.63–3.13)	3.53 (3.18–3.92)
NLR	0.03 (0.03–0.04)	0.07 (0.06–0.08)	0.15 (0.13–0.17)	0.17 (0.15–0.19)
PPV/precision	90.30 (89.23–91.27)	92.31 (91.11–93.39)	84.72 (83.18–86.17)	87.22 (85.74–88.60)
NPV	94.09 (92.46–95.38)	87.86 (85.76–89.75)	77.42 (74.69–79.99)	74.91 (72.27–77.42)
Youden’s Index	0.77	0.79	0.58	0.62
Accuracy	91.39 (90.38–92.33)	90.84 (89.80–91.81)	82.51 (81.16–83.80)	83.00 (81.67–84.27)
Data source: Cross tables with dichotomized tests based on short forms of the MoCA and reference standard test as reported in Table II.
MoCA: Montreal Cognitive Assessment; s-MoCA-SWE: Swedish short version of the MoCA; T-MoCA: telephone version of the MoCA, validated for telephone use; NINDS-CSN: National Institute of Neurological Disorders and Stroke and the Canadian Stroke Network MoCA; SF-MoCA: Short form MoCA; PLR: positive likelihood ratio; NLR: negative likelihood ratio; PPV: positive predictive value; NPV: negative predictive value; AUC: area under the receiving operating curve, results calculated using the total score of the index tests.

Subgroup analyses

The median score of the s-MoCA-SWE was significantly higher in patients aged ≤ 79 years, patients with minor stroke (NIHSS score ≤ 3), and in those without previous stroke and/or transient ischaemic attack (Table IV).

Table IV. Subgroup analyses showing the performance of the s-MoCA-SWE, ≤ 12 indicates impaired cognition

	N	s-MoCA-SWE (range 0–16)
		Mean (SD)	Median (IQR)	Minimum–Maximum	p-value (Mann–Whitney U)	Z score for Mann–Whitney U
Age					< 0.001	–14.67
18–79 years	2,424	10.5 (3.6)	11 (5)	0–16
80–100 years	853	8.5 (3.5)	9 (5)	0–16
Stroke severity					< 0.001	–6.13
Minor stroke, NIHSS ≤ 3	1,544	10.3 (3.5)	11 (5)	0–16
Moderate/severe stroke, NIHSS ≥ 4	1,224	9.4 (3.8)	10 (5)	0–16
Previous stroke/TIA					0.002	–3.09
No	2,625	10.0 (3.6)	11 (5)	0–16
Yes	357	9.4 (3.7)	10 (5)	0–16
MoCA: Montreal Cognitive Assessment; s-MoCA-SWE: Swedish short version of the MoCA; SD: standard deviation; IQR: interquartile range; NIHSS: National Institutes of Health Stroke scale; TIA: transient ischaemic attack.
Missing data, n (%): stroke severity, 508 (16%), previous stroke/TIA, 294 (9).

DISCUSSION

A short version of the MoCA was developed with high sensitivity to identify possible cognitive impairment early after stroke and a cut-off was determined that has good discriminative value between patients with or without cognitive impairment. The s-MoCA-SWE comprises cognitive tests that could be grouped under the cognitive domains of visuospatial abilities (a clock drawing and a cube copy task), executive functioning (trail making test, fluency, and abstraction), short-term memory (delayed recall test), and one concentration task (serial 7 task). These cognitive domains are frequently impaired in stroke survivors.

All of the tasks included in the s-MoCA-SWE (delayed recall, visuospatial/executive function, serial 7, fluency, and abstraction) have previously been identified as important contributors to the formation of various other short forms of the MoCA and for identifying people with cognitive impairment (7, 29–31). However, the combinations of test tasks in our short version are different. There are several explanations for this finding. First, it was not possible to analyse the MoCA at the individual tasks level because aggregated scores were recorded for certain tasks; for example, visuospatial/executive function (5 tasks) and orientation (6 tasks); secondly, different reference standard instruments have been used in previous studies; and thirdly, different analytical methods were used in those studies (7, 29–31).

The s-MoCA-SWE/PCA and the s-MoCA-SWE/BRT selected 5 tasks each and the subtests’ ability to detect possible cognitive impairment were 94% and 95%, respectively. Based on the decision criteria defined in this study, as well as its published protocol (16), the BRT model was chosen as it had the highest sensitivity. This decision has a clinical implication with regards to planning the patients’ discharge from the stroke units. As cognitive impairment early after stroke is very common (3, 32), and the length of stay at stroke units has decreased, it is necessary for a rapid screening tool to have a high sensitivity.

The cut-off of ≤ 12 for the s-MoCA-SWE was chosen to indicate cognitive impairment and demonstrated high sensitivity compared with previously developed short forms of the MoCA (8). The false-positive rate was 20.2%, whereas the false-negative rate was much lower, at 2.6%. A false-positive classification of patients can be accepted during the acute and subacute phases of stroke, but false-negative classification can have adverse consequences, as it was possible to miss patients with possible cognitive impairment. However, a false-positive test result is not completely benign and could be associated with psychological harm, increased test burden, and healthcare costs. This problem of balancing the sensitivity and specificity is common to all screening tools (33). We propose that the s-MoCA-SWE be used as a brief screening tool that can be used as a “rule-out” test for severe cognitive impairment.

The rapid nature of the s-MoCA-SWE should make it feasible for use even in busy acute stroke units. As with most traditional screening tests, impairments in motor, visual, or speech function could still complicate testing. The tool may not be suited to the most severe stroke presentations; however, brief cognitive screening is unlikely to have clinical value in this patient group (4).

This study has several strengths and limitations. It was based on a large sample of patients from several acute stroke and rehabilitation centres in Sweden. However, many patients were excluded from the analyses because of missing MoCA data. In most cases, it could be assumed that the data were not missing at random, as clinicians do not administer the MoCA to patients with a history of dementia, aphasia, or limb weakness. Cognitive screening using rapid screening tools is not relevant in patients with these histories (4, 34). Moreover, the MoCA was not administered in situations where the occupational therapist reported a shortage of time. Therefore, it can be assumed that the administration of the MoCA is time consuming. It is hoped that the s-MoCA-SWE will improve the proportion of cognitive screenings performed in stroke units.

Many different versions of short MoCAs exist and they cannot be used interchangeably (8). Other short MoCAs were developed or tested 3–18 months post-stroke, in rehabilitation or stroke prevention clinics. Only 1 study evaluated the feasibility of the NINDS-CSN in patients in the acute stage of stroke, but only in the ischaemic stroke population (13). The current study sample included patients from 35 acute stroke, geriatric, and rehabilitation units in Sweden, with patients who had different types, severity, and resulting impairments of stroke. While the results of this study may not apply to every stroke admission, they may be valid for those patients suited to screening, such as those with very mild to moderate stroke with a short length of hospital stay.

The s-MoCA-SWE (≤ 12) had good sensitivity for identifying patients with cognitive impairment. However, it is important to be aware that this short version refers to the full MoCA as a reference standard instrument. Nonetheless, the MoCA is an imperfect reference standard. In a recent study, 78% of the patients classified as cognitively intact by the MoCA had some form of post-stroke cognitive impairment on a comprehensive neuropsychological assessment within 3 months after stroke onset (35). Therefore, studies testing the s-MoCA-SWE against clinical diagnostic reference standards may also be useful. Data on premorbid cognitive function, lesion side, and localization were not available in this study. This can be regarded as a limitation of the current study.

The statistical methods used in the study should also be addressed. Different approaches were used for developing the s-MoCA-SWE. With PCA, the internal consistency between tasks is relevant, and there is no outcome. With BRT, tasks that have high importance in relation to the outcome are selected. In both approaches, the selected tasks should be used at an aggregated level rather than at the domain level for screening of global cognition. Moreover, optimism-corrected AUC or similar approaches were not used; therefore, external validation would be important to strengthen the current results.

In conclusion, these results suggest that the s-MoCA-SWE can be used as a brief tool for screening cognitive function in patients with acute and early subacute stroke. A threshold of ≤ 12 points shows good discriminative ability between patients who are cognitively intact and those who are cognitively impaired.

ACKNOWLEDGEMENTS

We acknowledge Väststroke, Riksstroke, and EFFECTS for allowing us to use their data. We also acknowledge all the healthcare professionals involved in collecting and registering data.

This work was supported by the Swedish Research Council (grant number VR2017-00946), Swedish Heart and Lung Foundation, Swedish Brain Foundation, Promobilia, and Swedish State under an agreement between the Swedish government and county councils, ALF agreement ALFGBG-965653), Swedish National Stroke Association, Local Research and Development Board of Gothenburg and Södra Bohuslän, Greta and Einar Asker’s Foundation, Rune and Ulla Amlöv’s Foundation for Neurological Research, Hjalmar Svensson’s Research Foundation, P-O Ahl & J-B Wennerström’s Foundation, Herbert & Karin Jacobssons Foundation, Sahlgrenska University Hospital's Foundation, Stroke Invest: Stroke research foundation Väst, Gun and Bertil Stohne’s Foundation, and the Iris Foundation.

Registry

The study was approved by the regional ethics review board of Gothenburg (approval number 346-16) and the Swedish Ethical Review Authority (amendment 2019-04299). The EFFECTS trial was approved by the Stockholm Ethics Committee (approval number 2013/1265-31/2 dated 30 September 2013).

Informed consent

According to the Swedish Data Protection Authority, the handling of data generated within the framework of quality registers represents an exception to the general rule that requires written informed consent. The Personal Data Act (Swedish law #1998:204, issued 29 April 1998) allows data from medical charts to be collected for clinical purposes and quality control without written or oral informed consent. Patients from EFFECTS provided written and oral consent to use the collected data for research purposes.

Study data

According to Swedish regulations (https://etikprovning.se/for-forskare/ansvar/), the data for this study cannot be publicly shared, for ethical and legal reasons. Researchers can request access to the data by emailing the principal investigator, Katharina S. Sunnerhagen, at ks.sunnerhagen@neuro.gu.se (registry data) and Erik Lundström at erik. lundstrom@neuro.uu.se (EFFECTS).

REFERENCES

1. Lanctôt KL, Lindsay MP, Smith EE, Sahlas DJ, Foley N, Gubitz G, et al. Canadian stroke best practice recommendations: mood, cognition and fatigue following stroke, 6th edition update 2019. Int J Stroke 2020; 15: 668–688. DOI: 10.1177/1747493019847334
2. Verdelho A, Wardlaw J, Pavlovic A, Pantoni L, Godefroy O, Duering M, et al. Cognitive impairment in patients with cerebrovascular disease: a white paper from the links between stroke ESO Dementia Committee. Eur Stroke J 2021; 6: 5–17. DOI: 10.1177/23969873211000258
3. Stolwyk RJ, Mihaljcic T, Wong DK, Chapman JE, Rogers JM. Poststroke cognitive impairment negatively impacts activity and participation outcomes: a systematic review and meta-analysis. Stroke 2021; 52: 748–760. DOI: 10.1161/STROKEAHA.120.032215
4. Elliott E, Drozdowska BA, Taylor-Rowan M, Shaw RC, Cuthbertson G, Quinn TJ. Who is classified as untestable on brief cognitive screens in an acute stroke setting? Diagnostics (Basel) 2019; 9: 10.3390. DOI: 10.3390/diagnostics9030095
5. Chiti G, Pantoni L. Use of Montreal Cognitive Assessment in patients with stroke. Stroke 2014; 45: 3135–3140. DOI: 10.1161/STROKEAHA.114.004590
6. Lees RA, Hendry Ba K, Broomfield N, Stott D, Larner AJ, Quinn TJ. Cognitive assessment in stroke: feasibility and test properties using differing approaches to scoring of incomplete items. Int J Geriatr Psychiatry 2017; 32: 1072–1078. DOI: 10.1002/gps.4568
7. Horton DK, Hynan LS, Lacritz LH, Rossetti HC, Weiner MF, Cullum CM. An abbreviated Montreal Cognitive Assessment (MoCA) for dementia screening. Clin Neuropsychol 2015; 29: 413–425. DOI: 10.1080/13854046.2015.1043349
8. McDicken JA, Elliott E, Blayney G, Makin S, Ali M, Larner AJ, et al. Accuracy of the short-form Montreal Cognitive Assessment: systematic review and validation. Int J Geriatr Psychiatry 2019; 34: 1515–1525. DOI: 10.1002/gps.5162
9. Pendlebury ST, Welch SJ, Cuthbertson FC, Mariz J, Mehta Z, Rothwell PM. Telephone assessment of cognition after transient ischemic attack and stroke: Modified telephone interview of cognitive status and telephone Montreal Cognitive Assessment versus face-to-face Montreal Cognitive Assessment and neuropsychological battery. Stroke 2013; 44: 227–229. DOI: 10.1161/STROKEAHA.112.673384
10. Wittich W, Phillips N, Nasreddine ZS, Chertkow H. Sensitivity and specificity of the Montreal Cognitive Assessment modified for individuals who are visually impaired. J Vis Impair Blind 2010; 104: 360–368. DOI:10.1177/0145482X1010400606
11. Cameron JD, Gallagher R, Pressler SJ, McLennan SN, Ski CF, Tofler G, et al. Sensitivity and specificity of a five-minute cognitive screening test in patients with heart failure. J Card Fail 2016; 22: 99–107. DOI: 10.1016/j.cardfail.2015.08.343
12. Kaur D, Kumar G, Singh AK. Quick screening of cognitive function in Indian multiple sclerosis patients using Montreal Cognitive Assessment test-short version. Ann Indian Acad Neurol 2013; 16: 585–589. DOI: 10.4103/0972-2327.120478
13. Lim JS, Oh MS, Lee JH, Jung S, Kim C, Jang MU, et al. Prediction of post-stroke dementia using NINDS-CSN 5-minute neuropsychology protocol in acute stroke. Int Psychogeriatr 2017; 29: 777–784. DOI: 10.1017/S1041610216002520
14. Lin H-F, Chern C-M, Chen H-M, Yeh Y-C, Yao S-C, Huang M-F, et al. Validation of NINDS-VCI neuropsychology protocols for vascular cognitive impairment in Taiwan. PloS One 2016: 11: e0156404. DOI: 10.1371/journal.pone.0156404
15. Wei J, Jin X, Chen B, Liu X, Zheng H, Guo R, et al. Comparative study of two short-form versions of the Montreal Cognitive Assessment for screening of post-stroke cognitive impairment in a Chinese population. Clin Interv Aging 2020; 15: 907–914. DOI: 10.2147/CIA.S248856
16. Abzhandadze T, Lundström E, Buvarp D, Eriksson M, Quinn TJ, Sunnerhagen KS. Development of a short-form Swedish version of the Montreal Cognitive Assessment (s-MoCA-SWE): protocol for a cross-sectional study. BMJ Open 2021; 11: e049035. DOI: 10.1136/bmjopen-2021-049035
17. Noel-Storr AH, McCleery JM, Richard E, Ritchie CW, Flicker L, Cullum SJ, et al. Reporting standards for studies of diagnostic test accuracy in dementia: The STARDdem Initiative. Neurology 2014; 83: 364–373. DOI: 10.1212/WNL.0000000000000621
18. Lundström E, Isaksson E, Näsman P, Wester P, Mårtensson B, Norrving B, et al. Safety and efficacy of fluoxetine on functional recovery after acute stroke (EFFECTS): a randomised, double-blind, placebo-controlled trial. Lancet Neurol 2020; 19: 661–669. DOI: 10.1016/S1474-4422(20)30219-2
19. Asplund K, Eriksson M, Riks-Stroke Collaboration. Inflammation, poststroke depression and statins. Int J Stroke 2011; 6: 567–568. DOI: 10.1111/j.1747-4949.2011.00691.x
20. Bernhardt J, Hayward KS, Kwakkel G, Ward NS, Wolf SL, Borschmann K, et al. Agreed definitions and a shared vision for new standards in stroke recovery research: The Stroke Recovery and Rehabilitation Roundtable taskforce. Int J Stroke 2017; 12: 444–450. DOI: 10.1177/1747493017711816
21. Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, Collin I, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 2005; 53: 695–699. DOI: 10.1111/j.1532-5415.2005.53221.x
22. Kasner SE, Chalela JA, Luciano JM, Cucchiara BL, Raps EC, McGarvey ML, et al. Reliability and validity of estimating the NIH stroke scale score from medical records. Stroke 1999; 30: 1534–1537. DOI: 10.1161/01.str.30.8.1534
23. Fischer U, Baumgartner A, Arnold M, Nedeltchev K, Gralla J, De Marchis GM, et al. What is a minor stroke? Stroke 2010; 41: 661–666. DOI: 10.1161/STROKEAHA.109.572883
24. Child D. Essentials of factor analysis. New York: Continuum; 2006.
25. Kline P. An easy guide to factor analysis. London: Routledge; 1994.
26. Kuhn M, Silge J. Tidy Modeling with R. "O'Reilly Media, Inc."; 2022 Jul 12.
27. Chen T, Guestrin C. Xgboost: A scalable tree boosting system. InProceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining 2016; Aug 13, (pp. 785–794).
28. Mandrekar JN. Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol 2010; 5: 1315–1316. DOI: 10.1097/JTO.0b013e3181ec173d
29. Roalf DR, Moore TM, Wolk DA, Arnold SE, Mechanic-Hamilton D, Rick J, et al. Defining and validating a short form Montreal Cognitive Assessment (s-MoCA) for use in neurodegenerative disease. J Neurol Neurosurg Psychiatry 2016; 87: 1303–1310. DOI: 10.1136/jnnp-2015-312723
30. Bezdicek O, Červenková M, Moore TM, Stepankova Georgi H, Sulc Z, Wolk DA, et al. Determining a short form Montreal Cognitive Assessment (s-MoCA) Czech version: validity in mild cognitive impairment Parkinson’s disease and cross-cultural comparison. Assessment 2020; 27: 1960–1970. DOI: 10.1177/1073191118778896
31. Cecato JF, Martinelli JE, Izbicki R, Yassuda MS, Aprahamian I. A subtest analysis of the Montreal Cognitive Assessment (MoCA): which subtests can best discriminate between healthy controls, mild cognitive impairment and Alzheimer’s disease? Int Psychogeriatr 2016; 28: 825–832. DOI: 10.1017/S1041610215001982
32. Abzhandadze T, Reinholdsson M, Sunnerhagen KS. NIHSS is not enough for cognitive screening in acute stroke: a cross-sectional, retrospective study. Sci Rep 2020; 10: 534–534. DOI: 10.1038/s41598-019-57316-8
33. Lees R, Selvarajah J, Fenton C, Pendlebury ST, Langhorne P, Stott DJ, et al. Test accuracy of cognitive screening tests for diagnosis of dementia and multidomain cognitive impairment in stroke. Stroke 2014; 45: 3008–3018. DOI: 10.1161/STROKEAHA.114.005842
34. Abzhandadze T, Buvarp D, Lundgren-Nilsson Å, Sunnerhagen KS. Barriers to cognitive screening in acute stroke units. Sci Rep 2021; 11: 19621. DOI: 10.1038/s41598-021-98853-5
35. Chan E, Khan S, Oliver R, Gill SK, Werring DJ, Cipolotti L. Underestimation of cognitive impairments by the Montreal Cognitive Assessment (MoCA) in an acute stroke unit population. J Neurol Sci 2014; 343: 176–179. DOI: 10.1016/j.jns.2014.05.005

Supplementary material has been published as submitted. It has not been copyedited, typeset or checked for scientific content by Journal of Rehabilitation Medicine

Table SI, SII, SIII, SIV, SV, SVI

Fig S1

Appendix S1