用scala表示嵌套结构

Question

| rowkey |  orderid  |      name   |    amount    |     supplier      |   account

| rowkey1|id0: 1001  |id1: "apple" |  id1: 1000   | id3: "fruits, inc"|
                     |id2: "apple2"|  id2: 1200   |                   | 

| rowkey2|id4: 1002  |id5: "orange"|  id5: 5000   |                   | 

| rowkey3|id6: 1003  |id7: "pear"  |  id7: 500    |                   |id10: 77777
                     |id8: "pear2"  |  id8: 350    |                   | 
                     |id9: "pear3"  |  id9: 500    |                   | 

从另一个角度看这3行：

    rowkey1, (orderid, id0, 1001), (name, id1, "apple"), (amount, id1, 1000), (name, id2, "apple2"), (amount, id2,1200), (supplier, id3, "fruit inc.")
    rowkey2, (orderid, id4, 1002), (name, id5, "orange"), (amount, id5,5000)
    rowkey3, (orderid, id6, 1003), (name, id7, "pear"), (amount, id7,500),(name, id8, "pear2"), (amount, id8,350),(name, id9, "pear3"), (amount, id9, 250), (account, id10, 777777)

编辑：注意表有2000列，是否可以动态创建一个类（或向类添加属性），例如在Scala中从外部文件加载字段名和类型？我知道case类仅限于22个字段

Edit2:还要注意，任何属性都可以有多行（rowkey除外），即orderid、name、amount、supplier、account和1995+其他列，所以为所有这些属性创建单独的“singleline”类是不可行的，我正在寻找最通用的解决方案。

case class ColumnLine(
  id: Int,
  value: Option[Any]
)
case class Column(
  colname: String,
  coltype: String,
  lines: Option[List[ColumnLine]]
)
case class Row (
  rowkey:String,
  columns:Map[String,Column] //colname -> Column
)
case class Table (
  name:String,
  rows:Map[String,Row] //rowkey -> Row
)

编辑3：好吧，这是“快速和肮脏”的方式，但似乎工作，它扫描10M记录在我的笔记本电脑在大约15秒

import scala.util.control.Breaks._

object hello{

def main(args: Array[String]) {
    val n = 10000000
    def uuid = java.util.UUID.randomUUID.toString
    val row: Row = new Row(uuid, List(
                Column("orderid", "String", List(Single("id2",Some(uuid)))),
                Column("name", "String", List(Single("id2",Some("apple")),Single("id3",Some("apple2")))),
                Column("amount", "Int", List(Single("id2",Some(1000)),Single("id3",Some(1200)))),
                Column("supplier", "String", List(Single("id4",Some("fruits.inc")))),
                Column("account", "Int", List(Single("id10",Some(7777))))
                           )
            )
    println(new java.util.Date)
    val table: List[Row]= List.fill(n)(row)
    table.par.filter(row=> gt(row, "amount",500))
    .filter(row=> eq(row, "supplier","fruits.inc"))
    .filter(row=> eq(row, "account", 7777))
    //.foreach(println)
    println(new java.util.Date)

}

def eq (row:Row, colname: String, colvalue:Any): Boolean = {
    var res:Boolean = false
    val col:Column = getCol(row,colname) 
    breakable{ 
        for (line <- col.lines){ 
            if (line.value.getOrElse()==colvalue){
                res = true
                break
            }
        }
    }
    return res
}

def gt (row:Row, colname: String, colvalue:Int): Boolean = {
        var res:Boolean = false
        val col:Column = getCol(row,colname)
        breakable{
                for (line <- col.lines){
                        if (line.value.getOrElse().asInstanceOf[Int]>colvalue){
                                res = true
                                break
                        }
                }
        }
        return res
}

def getCol(row: Row, colname: String) : Column =
  row.columns.filter(_.colname==colname).head

case class Single(id: String, value: Option[Any])

case class Column(
  colname: String,
  coltype: String,
  lines: List[Single]
)

case class Row(
   rowkey: String,
   columns: List[Column]
)

}

Answer 1

有很多办法。例如，您可以定义以下内容：

case class OrderLine(
  name:String,
  amount:Int,
  supplier:Option[String],
  account:Option[String]
)

case class Order(
  rowkey:String,
  orderid:String,
  orders:Seq[OrderLine]
)

然后（这只是为了创建上面的示例；从一个文件中读取2000行，当然会有所不同，但您明白了）：

   val myOrders: Seq[Order] =
     Seq(
       Order("rowkey1", "1001", Seq(
         OrderLine("apple", 1000, Some("fruits, inc"), None),
         OrderLine("apple2", 1200, None, None)
       )),
       Order("rowkey2", "1002", Seq(
         OrderLine("orange", 5000, None, None)
       )),
       Order("rowkey3", "1003", Seq(
         OrderLine("pear", 500, None, Some("77777")),
         OrderLine("pear", 350, None, None),
         OrderLine("pear", 500, None, None)
       ))
     )

从外部文件加载数据的代码取决于外部文件的结构。基本上，我将创建一个从文件中读取orderline的函数，以及一个读取orderline的函数（反过来，它使用该函数读取orderline)。这些将是将2000行组装成内存中数据结构的基本构建块。