Is there a way to 'compress' an lm() objec

Question

Is there a way to 'compress' an object of class lm, so that I can save it to the disk and load it up later for use with predict.lm?

I have an lm object that ends up being ~142mb upon saving, and I have a hard time believing that predict.lm needs all of the original observations / fitted values / residuals etc. to make a linear prediction. Can I remove information so that the saved model is smaller?

I have tried setting some of the variables (fitted.values, residuals, etc.) to NA, but it seems to have no effect on the saved file size.

Answer 1

You can use biglm to fit your models, a biglm model object is smaller than a lm model object. You can use predict.biglm create a function that you can pass the newdata design matrix to, which returns the predicted values.

Another option is to use saveRDS to save the files, which appear to be slightly smaller, as they have less overhead, being a single object, not like save which can save multiple objects.

 library(biglm)
 m <- lm(log(Volume)~log(Girth)+log(Height), trees)
 mm <- lm(log(Volume)~log(Girth)+log(Height), trees, model = FALSE, x =FALSE, y = FALSE)
 bm <- biglm(log(Volume)~log(Girth)+log(Height), trees)
 pred <- predict(bm, make.function = TRUE)
 save(m, file = 'm.rdata')
 save(mm, file = 'mm.rdata')
 save(bm, file = 'bm.rdata')
 save(pred, file = 'pred.rdata')
 saveRDS(m, file = 'm.rds')
 saveRDS(mm, file = 'mm.rds')
 saveRDS(bm, file = 'bm.rds')
 saveRDS(pred, file = 'pred.rds')

 file.info(paste(rep(c('m','mm','bm','pred'),each=2) ,c('.rdata','.rds'),sep=''))
#             size isdir mode mtime               ctime               atime               exe
#  m.rdata    2806 FALSE  666 2013-03-07 11:29:30 2013-03-07 11:24:23 2013-03-07 11:29:30  no
#  m.rds      2798 FALSE  666 2013-03-07 11:29:30 2013-03-07 11:29:30 2013-03-07 11:29:30  no
#  mm.rdata   2113 FALSE  666 2013-03-07 11:29:30 2013-03-07 11:24:28 2013-03-07 11:29:30  no
#  mm.rds     2102 FALSE  666 2013-03-07 11:29:30 2013-03-07 11:29:30 2013-03-07 11:29:30  no
#  bm.rdata    592 FALSE  666 2013-03-07 11:29:30 2013-03-07 11:24:34 2013-03-07 11:29:30  no
#  bm.rds      583 FALSE  666 2013-03-07 11:29:30 2013-03-07 11:29:30 2013-03-07 11:29:30  no
#  pred.rdata 1007 FALSE  666 2013-03-07 11:29:30 2013-03-07 11:24:40 2013-03-07 11:29:30  no
#  pred.rds    995 FALSE  666 2013-03-07 11:29:30 2013-03-07 11:27:30 2013-03-07 11:29:30  no

Answer 2

Turns out I solved my own problem. Using the following:

model<-lm(form,data=ct,model=FALSE,x=FALSE,y=FALSE)

reduced the size of my model substantially.

Answer 3

A couple of things:

1. This question really is a duplicate.

2. In the narrow sense model=FALSE as was already answered in another question.

3. In a wider sense, predict(fit, newdata) really just does a matrix-vector multiplication so you could save just the vector of predictions and multiply it with a matrix.

4. There are alternate fitting functions. Below is an example from fastLm() in RcppArmadillo which also happens to be faster.

See below for an illustration.

R> library(RcppArmadillo)
Loading required package: Rcpp
R> flm <- fastLm(Volume ~ Girth, data=trees)
R> predict(flm, newdata=trees[1:5,])             ## can predict as with lm()
[1]  5.10315  6.62291  7.63608 16.24803 17.26120
R> object.size(flm)                              ## tiny object size ...
3608 bytes
R> stdlm <- lm(Volume ~ Girth, data=trees)
R> object.size(stdlm)                            ## ... compared to what lm() has
20264 bytes
R> stdlm <- lm(Volume ~ Girth, data=trees, model=FALSE)
R> object.size(stdlm)                            ## ... even when model=FALSE
15424 bytes
R>