Overloaded method value apply with alternatives:

问题:

I am new to spark and I was trying to define a schema for a json data and ran into the following error in (spark-shell,

<console>:28: error: overloaded method value apply with alternatives:
  (fields: Array[org.apache.spark.sql.types.StructField])org.apache.spark.sql.types.StructType <and>
  (fields: java.util.List[org.apache.spark.sql.types.StructField])org.apache.spark.sql.types.StructType <and>
  (fields: Seq[org.apache.spark.sql.types.StructField])org.apache.spark.sql.types.StructType
 cannot be applied to (org.apache.spark.sql.types.StructField, org.apache.spark.sql.types.StructField)
       val schema = StructType(Array(StructField("type", StructType(StructField("name", StringType,   true), StructField("version", StringType,   true)), true) :: StructField("value", StructType(StructField("answerBlacklistedEntities", StringType,   true) :: StructField("answerBlacklistedPhrase", StringType,   true) :: StructField("answerEntities", StringType,   true) :: StructField("answerText", StringType,   true) :: StructField("blacklistReason", StringType,   true) :: StructField("blacklistedDomains", StringType,   true) :: StructField("blacklistedEntities", ArrayType(StringType, true), true) :: StructField("customerId", StringType,   true) :: StructField("impolitePhrase", StringType,   true) :: StructField("isResponseBlacklisted", BooleanType,   true) :: StructField("queryString", StringType,   true) :: StructField("utteranceDomains", StringType,   true) :: StructField("utteranceEntities", ArrayType(StringType, true), true) :: StructField("utteranceId", StructType(StructField("identifier", StringType, true)), true)) :: Nil)))

Can anybody guide me to what's going on here? :) I'd really appreciate your help!

回答1:

This happens because of this:

val schema = StructType(Array(StructField("type", 
   StructType(StructField("name", StringType,   true), ...))

You create StructType and pass StructField as an argument, while it should be a sequence of StructFields:

val schema = StructType(Array(StructField("type", 
   StructType(Array(StructField("name", StringType,   true), ...)) ...)