Error with predict and glm.predict in R

Question

The Problem

I trained a linear regression in R to predict this.target from city, variables in data frame data. This trainig is done on a subset of the data, which is specified by train.index.

model = glm('data[, this.target] ~ data$city', data = data, subset = train.index)

I am trying to test this model on the held out data, which is specified by test.index.

predictions = predict(model, data[test.index, ])

For whatever reason, this second step creates an error and a warning.

Error in model.frame.default(Terms, newdata, na.action = na.action, xlev =
object$xlevels) : invalid type (NULL) for variable 'data$city' In addition:
Warning message: 'newdata' had 22313 rows but variables found have 0 rows

My Analysis

data$city is a factor of 4 levels, but it seems like R reads it as an "invalid type (NULL)" even though none of the observations in this variable are NULL.

Moreover, it seems like R reads the rows but not the columns of the training set correctly. dim(data[test.index, ]) yields a vector with 22313 and 12.

Answer 1

Thanks to joran's comment, I was able to find the solution to my problem. joran pointed out that formulas should not include subsetting operations.

As it turns out, this subsetting was allowing model fitting to proceed as normal, but it caused model predicting to balk with the error and warning described above. By removing subsetting from my formula definition, both model fitting and predicting ran without problem.

Answer 2

This solution is for this error but not for the main question cause it is difficult to follow .

The solution is making a variable let's say x; x=as.data.frame(testset) and pass it to predict as

classifier = glm(formula = Survived ~ .,
                 family = binomial,
                 data = training_set)
x = as.data.frame(test_set)
prob_pred = predict(classifier, type = 'response', newdata =x)
y_pred = ifelse(prob_pred > 0.5, 1, 0)

Predicting the Test set results

In glm or rpart (library for classification based on decision tree), the test set should be data frame and sometimes the pre-processing steps change your data type (the data you want to predict) should be data frame, if it is not, the error

Error in model.frame.default(Terms, newdata, na.action = na.action, xlev = object$xlevels): 'data' must be a data.frame, not a matrix or an array

will appear.