如何在Spark Scala中从列数据类型中提取字符串？

Question

我有一个函数，它接受一个字符串参数，并对其进行“匹配”以确定返回值，如下所示-

编辑（完成功能）：

 def getSubscriptionDaysFunc(account_status:Column, created_at: org.apache.spark.sql.Column, updated_at: org.apache.spark.sql.Column):org.apache.spark.sql.Column = {
account_status match {
    case "expired" =>datediff(updated_at,created_at)
    case "cancelled" =>datediff(updated_at,created_at)
    case "active" =>datediff(updated_at, current_date())
    case default => null 
} }  

df.withColumn("subscription_days", getSubscriptionDaysFunc($"account_status",$"created_at",$"updated_at"))

val getSubscriptionDaysFunc = udf((account_status:String, created_at: org.apache.spark.sql.Column, updated_at: org.apache.spark.sql.Column):Column =>  {
account_status match {
case "expired" =>datediff(updated_at,created_at)
case "cancelled" =>datediff(updated_at,created_at)
case "active" => datediff(updated_at, current_date())
case default => null
} })

Answer 1

我认为您要做的是实现一个UDF：

import org.apache.spark.sql.functions.udf

val getSubscriptionDaysFunc = udf((account_status:String) =>  {
  account_status match {
    case "expired" =>//some logic
    case "cancelled" =>//some logic
    case "active" =>//some logic
    case default => null
  } 
})

df.withColumn("subscription_days", getSubscriptionDaysFunc($"account_status"))