通过Spark查询Cassandra表

Question

我正在尝试通过以下步骤通过 spark-1.6.0 和 scala-2.11.7 从 Cassandra 2.0.17 表中获取值

1. 已启动cassandra--服务cassandr启动
2. 已启动spark--sbin/start all。sh
3. 规定的火花标度-箱/火花壳-罐子火花-盒-连接器_2.10-1.5.0-M1.jar

在 Scala 中执行了这些命令

import org.apache.spark.SparkContext
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext._

sc.stop

val conf = new SparkConf(true).set("spark.cassandra.connection.host","127.0.0.1")

val sc=new SparkContext("local[2]","test",conf)

import com.datastax.spark.connector._

直到现在一切都很好，但当我执行-

val rdd=sc.cassandraTable("tutorialspoint","emp")

它给了我以下错误

error: bad symbolic reference. A signature in CassandraTableScanRDD.class refers to term driver
in package com.datastax which is not available.
It may be completely missing from the current classpath, or the version on
the classpath might be incompatible with the version used when compiling CassandraTableScanRDD.class.
error: bad symbolic reference. A signature in CassandraTableScanRDD.class refers to term core
in value com.datastax.driver which is not available.
It may be completely missing from the current classpath, or the version on
the classpath might be incompatible with the version used when compiling CassandraTableScanRDD.class.
error: bad symbolic reference. A signature in CassandraTableScanRDD.class refers to term core
in value com.datastax.driver which is not available.
It may be completely missing from the current classpath, or the version on
the classpath might be incompatible with the version used when compiling CassandraTableScanRDD.class.

添加了来自cassandra lib的jar来激发和引用它。我的使用java版本1.8.0_72

我错过了一些东西？

Answer 1

您使用的驱动程序与您的 scala 和 Spark 版本不兼容。您使用的是 scala-2.11.7，但此驱动程序适用于 scala 2.10。同样对于火花，此驱动程序支持火花 1.5.x。