配置SQL-在多个分区之间

Question

我有一张按年、月、日划分的蜂巢表

CREATE TABLE t1 (
...
 )
PARTITIONED BY (
year INT,
month INT,
day INT'
)
STORED AS PARQUET;

val formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd")
val runDate = date
val runDay = LocalDate.parse(runDate.toString, formatter)
val runDayMinus7 = runDay.minusDays(7)

Answer 1

我创建了next方法来解决这个问题。两个参数，日期和天数

  def dayFilter(date: LocalDate, days: Int): Column = {
   days match {
    case _ if days < 0 => lit(false)
    case 0 => (col("year") === date.getYear and
     col("month") === date.getMonthValue and
     col("day") === date.getDayOfMonth)
    case _ => (col("year") === date.getYear and
     col("month") === date.getMonthValue and
     col("day") === date.getDayOfMonth) or dayFilter(date.minusDays(1), days - 1)
   }
 }

如果我们导入Java LocalDate，那么我们需要GetMonthValue。以防万一，我们使用joda.time然后getMonthOfYear