我想选择几列,加几列或鸿沟,一些列的空间填充,并将它们存储以新的名称作为别名。 例如,在SQL应该是这样的:
select " " as col1, b as b1, c+d as e from table
我怎样才能在星火实现这一目标?
我想选择几列,加几列或鸿沟,一些列的空间填充,并将它们存储以新的名称作为别名。 例如,在SQL应该是这样的:
select " " as col1, b as b1, c+d as e from table
我怎样才能在星火实现这一目标?
您也可以使用本机DF功能以及。 例如,给定:
import org.apache.spark.sql.functions._
val df1 = Seq(
("A",1,5,3),
("B",3,4,2),
("C",4,6,3),
("D",5,9,1)).toDF("a","b","c","d")
选择列如下:
df1.select(lit(" ").as("col1"),
col("b").as("b1"),
(col("c") + col("d")).as("e"))
为您提供了预期的结果:
+----+---+---+
|col1| b1| e|
+----+---+---+
| | 1| 8|
| | 3| 6|
| | 4| 9|
| | 5| 10|
+----+---+---+
在Spark-SQL,你可以做同样的方式。
import org.apache.spark.sql.functions._
val df1 = Seq(
("A",1,5,3),
("B",3,4,2),
("C",4,6,3),
("D",5,9,1)).toDF("a","b","c","d")
df1.createOrReplaceTempView("table")
df1.show()
val df2 = spark.sql("select ' ' as col1, b as b1, c+d as e from table ").show()
输入:
+---+---+---+---+
| a| b| c| d|
+---+---+---+---+
| A| 1| 5| 3|
| B| 3| 4| 2|
| C| 4| 6| 3|
| D| 5| 9| 1|
+---+---+---+---+
输出:
+----+---+---+
|col1| b1| e|
+----+---+---+
| | 1| 8|
| | 3| 6|
| | 4| 9|
| | 5| 10|
+----+---+---+