How to bootstrap respecting within-subject information?
r, resampling, statistics-bootstrap
Solution
Just modify your call to `boot()` like this:
data.boot <- boot(data, boot.huber, 1999, strata=data$Subject)
`?boot` provides this description of the `strata=` argument, which does exactly what you are asking for:
strata: An integer vector or factor specifying the strata for multi-sample problems. This may be specified for any simulation, but is ignored when ‘sim = "parametric"’. When ‘strata’ is supplied for a nonparametric bootstrap, the simulations are done within the specified strata.
Additional note:
To confirm that it's working as you'd like, you can call `debugonce(boot)`, run the call above, and step through the debugger until the object `i` (whose rows contain the indices used to resample rows of `data` to create each bootstrap resample) has been assigned, and then have a look at it.
debugonce(boot)
data.boot <- boot(data, boot.huber, 1999, strata=data$Subject)
# Browse[2]>
## [Press return 34 times]
# Browse[2]> head(i)
# [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14]
# [1,] 9 10 11 16 9 14 15 16 9 2 15 16 1 10
# [2,] 9 14 7 12 5 6 15 4 13 6 11 16 13 6
# [3,] 5 10 15 16 9 6 3 4 1 2 15 12 5 6
# [4,] 5 10 11 4 9 6 15 16 9 14 11 16 5 2
# [5,] 5 10 3 4 1 10 15 16 9 6 3 8 13 14
# [6,] 13 10 3 12 5 10 3 4 5 14 7 16 5 14
# [,15] [,16]
# [1,] 7 8
# [2,] 11 16
# [3,] 3 16
# [4,] 3 8
# [5,] 7 8
# [6,] 7 12
(You can enter `Q` to leave the debugger at any time.)
Problem
This is the first time I post to this forum, and I want to say from the start I am not a skilled programmer. So please let me know if the question or code were unclear! I am trying to get the 95% confidence interval (CI) for an interaction (that is my test statistic) by doing bootstrapping. I am using the package "boot". My problem is that for every resample, I would like the randomization to be done within subjects, so that observations from different subjects are not mixed. Here is the code to generate a dataframe similar to mine. As you can see, I have two within-subjects factors ("Num" and "Gram" and I am interested in the interaction between both): ``` Subject = rep(c("S1","S2","S3","S4"),4) Num = rep(c("singular","plural"),8) Gram = rep(c("gram","gram","ungram","ungram"),4) RT = c(657,775,678,895,887,235,645,916,930,768,890,1016,590,978,450,920) data = data.frame(Subject,Num,Gram,RT) ``` This is the code I used to get the empirical interaction value: ``` summary(lm(RT ~ Num*Gram, data=data)) ``` As you can see, the interaction between my two factors is -348. I want to get a bootstrap confidence interval for this statistic, which I can generate using the "boot" package: ``` # You need the following packages install.packages("car") install.packages("MASS") install.packages("boot") library("car") library("MASS") library("boot") #Function to create the statistic to be boostrapped boot.huber <- function(data, indices) { data <- data[indices, ] #select obs. in bootstrap sample mod <- lm(RT ~ Num*Gram, data=data) coefficients(mod) #return coefficient vector } #Generate bootstrap estimate data.boot <- boot(data, boot.huber, 1999) #Get confidence interval boot.ci(data.boot, index=4, type=c("norm", "perc", "bca"),conf=0.95) #4 gets the CI for the interaction ``` My problem is that I think the resamples should be generated without mixing the individual subjects observations: that is, to generate the new resamples, the observations from subject 1 (S1) should be shuffled within subject 1, not mixing them with the observations from subjects 2, etc... I don't know how "boot" is doing the resampling (I read the documentation but don't understand how the function is doing it) Does anyone know how I could make sure that the resampling procedure used by "boot" respects subject level information? Thanks a lot for your help/advice!