Retrieve list of training features names from clas

Question

Is there a way to retrieve the list of feature names used for training of a classifier, once it has been trained with the fit method? I would like to get this information before applying to unseen data. The data used for training is a pandas DataFrame and in my case, the classifier is a RandomForestClassifier.

Answer 1

Based on the documentation and previous experience, there is no way to get a list of the features considered at least at one of the splitting.

Is your concern that you do not want to use all your features for prediction, just the ones actually used for training? In this case I suggest to list the feature_importances_ after fitting and eliminate the features that does not seem relevant. Then train a new model with only the relevant features and use those features for prediction as well.

Answer 2

I have a solution which works but is not very elegant. This is an old post with no existing solutions so I suppose there are not any.

Create and fit your model. For example

model = GradientBoostingRegressor(**params)
model.fit(X_train, y_train)

Then you can add an attribute which is the 'feature_names' since you know them at training time

model.feature_names = list(X_train.columns.values)

I typically then put the model into a binary file to pass it around but you can ignore this

joblib.dump(model, filename)
loaded_model = joblib.load(filename)

Then you can get the feature names back from the model to use them when you predict

f_names = loaded_model.feature_names
loaded_model.predict(X_pred[f_names])

Answer 3

You don't need to know which features were selected for the training. Just make sure to give, during the prediction step, to the fitted classifier the same features you used during the learning phase.

The Random Forest Classifier will only use the features on which it makes its splits. Those will be the same as those learnt during the first phase. Others won't be considered.

If the shape of your test data is not the same as the training data it will throw an error, even if the test data contains all the features used for the splits of you decision trees.

What's more, since Random Forests make random selection of features for your decision trees (called estimators in sklearn) all the features are likely to be used at least once.

---

However, if you want to know the features used, you can just call the attributes n_features_ and feature_importances_ on your classifier once fitted.

You can look here to see how you can retrieve the names of the most important features you used.