时间戳未从胶水加载到红移表中

Question

我在csv文件中有yyyy-mm-dd xx:xx:xx格式的时间戳，该文件存储在S3中，但是当我使用timestamp数据类型使用胶水加载到Redshift数据库时，timestamp列为NULL。该格式似乎是有效的，但为了以防万一，我还尝试了yyyymddxxxxx和yymddxx:xxx格式。

1,2016 Summer,2016-06-22 00:00:00

Line |    Term     |    Date
-----+-------------+------------
   1 | 2016 Summer |  

Line |    Term     |        Date
-----+-------------+---------------------
   1 | 2016 Summer | 2016-06-22 00:00:00

似乎这应该是一个简单的任务，但我不能把它做对，所以如果其他人能发现我的错误，那将非常感谢。

代码：

val datasource37 = glueContext.getCatalogSource(database = "data", tableName = "term", redshiftTmpDir = "", transformationContext = "datasource37").getDynamicFrame()
val applymapping37 = datasource37.applyMapping(mappings = Seq(("id", "bigint", "id", "bigint"), ("name", "string", "name", "varchar(256)"), ("date", "timestamp", "date_start", "timestamp")), caseSensitive = false, transformationContext = "applymapping37")
val resolvechoice37 = applymapping37.resolveChoice(choiceOption = Some(ChoiceOption("make_cols")), transformationContext = "resolvechoice37")
val dropnullfields37 = resolvechoice37.dropNulls(transformationContext = "dropnullfields37")
val datasink37 = glueContext.getJDBCSink(catalogConnection = "dataConnection", options = JsonOptions("""{"dbtable": "term", "database": "data"}"""), redshiftTmpDir = args("TempDir"), transformationContext = "datasink37").writeDynamicFrame(dropnullfields37)

Answer 1

我最终从string->timestamp映射，它工作了。Glue让它从timestamp->timestamp自动映射，所以我认为它是正确的。

例如：

val applymapping37 = datasource37.applyMapping
                     (mappings = Seq(("id", "bigint", "id", "bigint"),
                     ("name", "string", "name", "varchar(256)"), 
                     ("date", "string", "date_start", "timestamp")),
                     caseSensitive = false, transformationContext = "applymapping37")