创建JSON类型的模式并在Scala中使用Spark读取它[错误：无法解析jsontostructs]

Question

我有一个JSON文件，如下所示：

{"Codes":[{"CName":"012","CValue":"XYZ1234","CLevel":"0","msg":"","CType":"event"},{"CName":"013","CValue":"ABC1234","CLevel":"1","msg":"","CType":"event"}}

我想为此创建模式，如果JSON文件是空的({})，它应该是空字符串。

但是，当我使用df.show时，df输出如下：

[[012, XYZ1234, 0, event, ], [013, ABC1234, 1, event, ]]

val schemaF = ArrayType(
  StructType(
    Array(
      StructField("CName", StringType),
      StructField("CValue", StringType),
      StructField("CLevel", StringType),
      StructField("msg", StringType),
      StructField("CType", StringType)
    )
  )
)

val df1 = df.withColumn("Codes",from_json('Codes, schemaF))

有人能告诉我为什么和如何解决这个问题吗？

Answer 1

val schema =
      StructType(
        Array(
          StructField("CName", StringType),
          StructField("CValue", StringType),
          StructField("CLevel", StringType),
          StructField("msg", StringType),
          StructField("CType", StringType)
        )

      )
    val df0 = spark.read.schema(schema).json("/path/to/data.json")