Test-Retest Reliability of the SRET
Justin Dainer-Best
2017
Determining optimal parameters of the Self Referent Encoding Task: A large-scale examination of self-referent cognition and depression
This file is one of a series of supplemental explanatory documents for the study “Determining optimal parameters of the Self Referent Encoding Task: A large-scale examination of self-referent cognition and depression”. Data and code are located at doi: 10.18738/T8/XK5PXX, and websites with visual R Markdown explanations are located and navigable on the paper’s github pages website.
Data description
This file examines test-retest reliability using the MTurk sample, which includes participants who completed the task at two timepoints, one week apart.
The web version of this file focuses on the summary data; the R Markdown file includes the code. If you are viewing this as an HTML file, and wish to see the code, please download the R Markdown file from the Texas Data Repository.
Functions for comparisons
There are 167 participants who had good data on all of the variables of interest, for whom we can compare a variety of indices. We created some functions for making these comparisons, allowing us to repeat the comparisons for each of our variables of interest. These variables were those selected using the best subsets procedures described in the manuscript.
Based on our best subsets models, we tested the following variables: (1) number of negative words endorsed (num.neg.endorsed), (2) drift rate (v) to positive words, (3) inter-trial-variability of starting point (szr) , (4) drift rate (v) to negative words, (5) inter-trial-variability of non-decisional components (st0), (6) relative starting point (zr) for negative words, and (7) threshold separation (a).
Predictive models
For the valenced components, we tested interactions between valence and time. For all variables, we tested for a main effect of time.
Testing interactions between valence and time:
These models tested for whether each variable of interest included an interaction between session (time) and valence (positive/negative words). If there was an interaction, it returns the results of that model. There were no interactions, and thus each returns the t-value for session and valence, as well as information about model fit.
For endorsements:
No significant interaction between valence and session.
Chisq(1) = 1.815, p = 0.178
The model without an interaction has a marginal r-squared of 0.5375384 and a conditional r-squared of 0.942
lmer interaction model had beta = -0.521, t = -1.349
lmer model without interaction: beta.valence = 15.219, t = 15.44
lmer model without interaction: beta.session = -0.213, t = -1.098
Returning lme4 model WITHOUT interaction.For drift rate:
No significant interaction between valence and session.
Chisq(1) = 0.708, p = 0.4
The model without an interaction has a marginal r-squared of 0.5122916 and a conditional r-squared of 0.91
lmer interaction model had beta = 0.112, t = 0.842
lmer model without interaction: beta.valence = 4.102, t = 14.499
lmer model without interaction: beta.session = 0.063, t = 0.94
Returning lme4 model WITHOUT interaction.For relative starting point
No significant interaction between valence and session.
Chisq(1) = 0.01, p = 0.919
The model without an interaction has a marginal r-squared of 0.2196777 and a conditional r-squared of 0.463
lmer interaction model had beta = 0.002, t = 0.101
lmer model without interaction: beta.valence = 0.14, t = 9.853
lmer model without interaction: beta.session = -0.002, t = -0.292
Returning lme4 model WITHOUT interaction.Testing change over time with t-tests
The t-test to see change in num.neg.endorsed found that t(332) = -0.07 p = 0.942The t-test to see change in num.pos.endorsed found that t(331) = 0.54 p = 0.589The t-test to see change in v.pos found that t(329.6) = -0.5 p = 0.617The t-test to see change in v.neg found that t(332) = -0.03 p = 0.974The t-test to see change in zr.neg found that t(331.9) = 0.23 p = 0.819The t-test to see change in szr found that t(332) = 0.75 p = 0.456The t-test to see change in st0 found that t(328.4) = -0.14 p = 0.89The t-test to see change in a found that t(330.9) = -0.14 p = 0.886Correlations between sessions
These tests show the corelations within valence for each comparison of interest between timepoints. They include boot-strapped 95% confidence intervals (with 10,000 iterations) on Pearson’s r.
Endorsements over time:
This comparison has the following correlations between sessions for positive:
r = 0.872, 95% CI [0.804, 0.928], t(165) = 22.88, p < .001
And, this comparison has the following correlations between sessions for negative:
r = 0.883, 95% CI [0.828, 0.927], t(165) = 24.19, p < .001Drift rate over time:
This comparison has the following correlations between sessions for positive:
r = 0.832, 95% CI [0.779, 0.876], t(165) = 19.29, p < .001
And, this comparison has the following correlations between sessions for negative:
r = 0.798, 95% CI [0.715, 0.863], t(165) = 16.99, p < .001Relative starting point over time:
This comparison has the following correlations between sessions for positive:
r = 0.252, 95% CI [0.074, 0.419], t(165) = 3.35, p < .001
And, this comparison has the following correlations between sessions for negative:
r = 0.37, 95% CI [0.241, 0.494], t(165) = 5.12, p < .001szr:
This comparison has the following correlations between sessions:
r = -0.134, 95% CI [-0.291, 0.024], t(165) = -1.73, p = 0.085st0:
This comparison has the following correlations between sessions:
r = 0.413, 95% CI [0.207, 0.604], t(165) = 5.82, p < .001a:
This comparison has the following correlations between sessions:
r = 0.555, 95% CI [0.443, 0.657], t(165) = 8.57, p < .001Example of plots of change over time
