Is there a way to 'compress' an lm() object for later prediction?
compression, memory, r
Solution
You can use `biglm` to fit your models, a `biglm` model object is smaller than a lm model object. You can use `predict.biglm` create a function that you can pass the newdata design matrix to, which returns the predicted values.
Another option is to use `saveRDS` to save the files, which appear to be slightly smaller, as they have less overhead, being a single object, not like save which can save multiple objects.
library(biglm)
m <- lm(log(Volume)~log(Girth)+log(Height), trees)
mm <- lm(log(Volume)~log(Girth)+log(Height), trees, model = FALSE, x =FALSE, y = FALSE)
bm <- biglm(log(Volume)~log(Girth)+log(Height), trees)
pred <- predict(bm, make.function = TRUE)
save(m, file = 'm.rdata')
save(mm, file = 'mm.rdata')
save(bm, file = 'bm.rdata')
save(pred, file = 'pred.rdata')
saveRDS(m, file = 'm.rds')
saveRDS(mm, file = 'mm.rds')
saveRDS(bm, file = 'bm.rds')
saveRDS(pred, file = 'pred.rds')
file.info(paste(rep(c('m','mm','bm','pred'),each=2) ,c('.rdata','.rds'),sep=''))
# size isdir mode mtime ctime atime exe
# m.rdata 2806 FALSE 666 2013-03-07 11:29:30 2013-03-07 11:24:23 2013-03-07 11:29:30 no
# m.rds 2798 FALSE 666 2013-03-07 11:29:30 2013-03-07 11:29:30 2013-03-07 11:29:30 no
# mm.rdata 2113 FALSE 666 2013-03-07 11:29:30 2013-03-07 11:24:28 2013-03-07 11:29:30 no
# mm.rds 2102 FALSE 666 2013-03-07 11:29:30 2013-03-07 11:29:30 2013-03-07 11:29:30 no
# bm.rdata 592 FALSE 666 2013-03-07 11:29:30 2013-03-07 11:24:34 2013-03-07 11:29:30 no
# bm.rds 583 FALSE 666 2013-03-07 11:29:30 2013-03-07 11:29:30 2013-03-07 11:29:30 no
# pred.rdata 1007 FALSE 666 2013-03-07 11:29:30 2013-03-07 11:24:40 2013-03-07 11:29:30 no
# pred.rds 995 FALSE 666 2013-03-07 11:29:30 2013-03-07 11:27:30 2013-03-07 11:29:30 no
Problem
Is there a way to 'compress' an object of class lm, so that I can save it to the disk and load it up later for use with predict.lm? I have an lm object that ends up being ~142mb upon saving, and I have a hard time believing that predict.lm needs all of the original observations / fitted values / residuals etc. to make a linear prediction. Can I remove information so that the saved model is smaller? I have tried setting some of the variables (fitted.values, residuals, etc.) to NA, but it seems to have no effect on the saved file size.