I want to find the parameters of ParamGridBuilder
that make the best model in CrossValidator in Spark 1.4.x,
In Pipeline Example in Spark documentation, they add different parameters (numFeatures
, regParam
) by using ParamGridBuilder
in the Pipeline. Then by the following line of code they make the best model:
val cvModel = crossval.fit(training.toDF)
Now, I want to know what are the parameters (numFeatures
, regParam
) from ParamGridBuilder
that produces the best model.
I already used the following commands without success:
cvModel.bestModel.extractParamMap().toString()
cvModel.params.toList.mkString("(", ",", ")")
cvModel.estimatorParamMaps.toString()
cvModel.explainParams()
cvModel.getEstimatorParamMaps.mkString("(", ",", ")")
cvModel.toString()
Any help?
Thanks in advance,
This is the ParamGridBuilder()
There are 3 stages in pipeline. It seems we can assess parameters as the following:
However, there is no parameter in the last stage, logiscRegression.
We can also get weight and intercept parameter from logistregression like the following:
Full exploration: http://kuanliang.github.io/2016-06-07-SparkML-pipeline/
source
I am working with Spark Scala 1.6.x and here is a full example of how i can set and fit a
CrossValidator
and then return the value of the parameter used to get the best model (assuming thattraining.toDF
gives a dataframe ready to be used) :You can do the same for any parameter or any other type of model.