Test-retest stabilitet og reliabilitet Neuropsykologiske test Test-retest stabilitet og reliabilitet Anders Gade Gade

S-81 data 20 personer retestet efter 3 måneder Erik Lykke Mortensen Hanne Udesen Gade

AGENDA-projekt 77 personer retestet efter 4 uger (38 efter 10 mg escitalopram dgl; 39 placebo) Ulla Knorr .. & Helena Dysgaard, Vibe Bredsdorf & Peter Kristian Jacobsen Gade

ACD-gruppen (Agenda-Cimbi-Depad) Helene Carstensen and Vibe Bredsdorff (primarily Agenda (with Peter Kristian Jacobsen)) Gry Zornhagen (Cimbi) and Eva Meldal (all 3) Sofie Folke Kasper Eskelund (primarily Depad together Julie Nyvang) Gade

FReJA – prospektiv neuropsykologisk undersøgelse 2002, 2005, 2010 Gade

FReJA – prospektiv neuropsykologisk undersøgelse 2002, 2005, 2010 Tom Teasdale Korrelationer: 2002-2005 2002-2010 2005-2010 N: 38 31 35 (Indtil videre ikke renset for genbærere) 2002 gennemsnit: alder 53; uddannelsesscore 11.8; DART 28) Gade

Test-retest reliabilitet Reliabilitet handler grundlæggende om målefejl ”A given score is thought to consist of two parts: the true score and the error. The true score is the actual value of a given individual on the construct if it were to measured completely accurately by the test. The error is the value of the extraneous influences on the measurement process”. Franzen, 1989, p.15 Gade

Hvad skal vi kræve i test-retest korrelationskoefficienter? ”At MATRICS consensus meetings, high test-retest reliability was considered the most important test feature in a clinical trial”. ”The committee considered an r value of 0.70 to be acceptable for test-retest reliability for clinical trials”. Nuechterlein et al., 2008 Gade

Hvad skal vi kræve i test-retest korrelationskoefficienter? We here present findings from three new studies to show that (a) the BPP has a satisfactory Test-Retest reliability, r = 0.77 Teasdale et al., 2011 BPP som gruppeprøve ved alder 19.2 år WAIS med individuel administration 4 år senere BPP – full scale IQ korrelation: 0.82 Mortensen et al., 1990 Gade AG# a444a

Domæne # test faktor r enkelttest mean r Abstraktion 2 .82 .60 ”Basisbatteri” og S-81 – test-retest korrelationer i enkeltprøver og i ”faktorskorer” Domæne # test faktor r enkelttest mean r Abstraktion 2 .82 .60 Hukommelsesspændvidde 3 .89 .71 Verbal indlæring 3 .56 .47 Genkendelse 5 .87 .68 Visuomotorisk tempo 3 .56 .55 Visuo-spatiale prøver 4 .70 .41 PASAT 2 .51 .38 Visuel syntese 2 .80 .68 Total BB 13 .77 .51 Mean total af 7 faktorer 24 .90 .54 QS 50 ord SDMT vis.gestalter Gade

Eksempel på brug af test-retest data: SDMT Test 1 Test 2 Pearson r SE St. difference 95% 45.3 48.8 .74 3.9 10.8 95% konfidensintervaller Skore Forbedring Forværring +/- 7.6 + 14.3 - 7.3 Gade

”Reliable change index” Temkin et al. (1999). JINS, 5, 357-369 Bird et al. (2003), Br.J.Clin.Psychol., 42, 407-425 Recognition Memory Words (QS 50 ord) 1. us. 2. us. r London 46.1 46.5 .69 S-81 46.6 47.2 .70 Gade

Indlæring: .14 Retention: .27 Test reliabilitet - Visuelle Gestalter Pearson r Indlæring: .14 Retention: .27 Gade

Gade

Reys komplekse figur 3 og/eller 30 minutters retention? Begge er OK reliable ved 4 ugers retest: r = .80 / .85 Korrelation mellem præstation ved 3 og 30 min: .96 / .97 Gade

”Intelligence” DART (reading test) Memory Rey Auditory Verbal .. Rey Complex Figure Construction Block design Visuomotor speed SDMT Trail Making A & B Language Boston Naming Test Familiar faces (naming) Gade

Attentional control Stroop Set shifting Wisconsin Card Sorting ”Frontal lobe tests” Attentional control Stroop Set shifting Wisconsin Card Sorting Fluency Animals S-words Working memory Letter-number sequencing Decision making Iowa Gambling Task Gade

NART / DART Moray House Test: 11 år NART: 79 år 45 demens: r= 0.60 464 NC: r= 0.63 McGurn et al. (2004), Neurology, 62, 1184-6 #2197j / x105a Gade

Danish Adult Reading Test Examples of items GAGE GIN MONTAGE ... NEUTRUM LANCERE POMERANS MARKIS KASEIN CLOU … HUGUENOT ECLAT OBSKUR Gade

Danish Adult Reading Test Examples of items GAGE GIN MONTAGE ... NEUTRUM LANCERE POMERANS MARKIS KASEIN CLOU … HUGUENOT ECLAT OBSKUR Test-retest reliabilitet (AGENDA & FReJA) Alle (N: 77): 0.93 Placebo (N: 39): 0.93 2002-2005: 0.93 2002-2010: 0.92 2005-2010: 0.92 Gade

Visuomotorisk tempo SDMT .74 .83 .89 / .90 / .94 S-81 AGENDA FReJA SDMT .74 .83 .89 / .90 / .94 Trails A .43 .67 .67 / .58 / .47 3) Trails B .47 .68 .80 / .78 / .83 Faktorscore .56 .88 / .84 / .87 I faktorscore sammen med Rey og Blokmønstre: AGENDA .91

”Findings demonstrated modest reliability Eksekutive tests ”Findings demonstrated modest reliability of individual EF measures but very high stability of a latent EF construct”. Ettenhofer et al., 2006 Gade

Eksekutive tests - S-81 Sorteringsprøve r: .37 WCST, antal fejl i alt r: .45 Gade

Eksekutive tests - AGENDA r alle placebo Ordmobilisering – S .48 .42 Ordmobilisering – dyr .69 .84 Bogstav-tal sekvensering .50 .57 Stroop inkongruente .81 .80 Eksekutive faktor .79 .84 Eksekutiv faktor i FReJA: .89 / .72 / .80 Gade

Stroop - neutral XXX Gade

Stroop - kongruent blå grøn rød gul Gade

Stroop - inkongruent gul rød blå grøn Gade

Stroop inkongruent (sekunder) minus Stroop kongruent (sekunder) Stroop subtraktion = Stroop inkongruent (sekunder) minus Stroop kongruent (sekunder) ”Test-retest reliability coefficients were high for standard … Stroop tasks when reliability was measured using response latencies alone. However, test-retest coefficients were unacceptably low when reliability estimates were calculated using difference scores” Strauss et al., 2005 (Stroop kongruent r .71; inkongruent .79; subtraktion .46) Stroop inkongruent r .81 / .80 Stroop subtraktion r .65 / .60 Gade

eksekutive tests senere ….. ? Grundig gennemgang af eksekutive tests senere ….. ? Fx på kvartalsmøde i 2012 Gade

Konklusioner Nogle af vore prøver – men ikke alle - har glimrende reliabilitet 2. Brug viden om prøvernes reliabilitet i din udvælgelse af prøver 3. Brug viden om prøvernes reliabilitet i dine konklusioner, så du ikke udtaler dig mere sikkert, end dine data berettiger til 4. Faktorskorer er generelt mere pålidelige end enkelttests. Hvis du bruger prøver med ringe reliabilitet – især hukommelse og eksekutive funktioner – så brug mere end én Gade

Gade