Spark Streaming Kafka with Java11 scala代码问题

Question

我在JDK 11中使用scalaSpark Streaming Kafka。但是我得到了下面的错误。

线程“main”java中出现异常。lang.NoSuchMethodError：scala。Predef美元。refArrayOps（[Ljava/lang/Object；）Lscala/collection/mutable/ArrayOps；

下面是我正在使用的代码。

val conf = new SparkConf().setMaster("local[*]").setAppName("KafkaExample")
      .set("spark.mongodb.input.uri", "mongodb://127.0.0.1/db.table_data")
      .set("spark.mongodb.output.uri", "mongodb://127.0.0.1/db.table_data")
      .set("spark.driver.allowMultipleContexts", "false")
      .set("spark.ui.enabled", "false")

val kafkaParams = Map("metadata.broker.list" -> "localhost:9092")
    val topics = List("topic").toSet
    val lines = KafkaUtils.createDirectStream[String, String, StringDecoder, StringDecoder](
      ssc, kafkaParams, topics)
    

下面是我的pom。xml

<dependency>
          <groupId>org.apache.spark</groupId>
          <artifactId>spark-core_2.12</artifactId>
          <version>2.4.8</version><!--<version>2.3.2</version>-->
      </dependency>
      <dependency>
          <groupId>org.apache.spark</groupId>
          <artifactId>spark-sql_2.12</artifactId>
          <version>2.4.8</version>
      </dependency>
      <dependency>
          <groupId>org.apache.spark</groupId>
          <artifactId>spark-streaming_2.12</artifactId>
          <version>2.4.8</version><!--<version>2.3.2</version>-->
          <!--<scope>provided</scope>-->
      </dependency>
      <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming-kafka-0-10 -->
      <dependency>
          <groupId>org.apache.spark</groupId>
          <artifactId>spark-streaming-kafka_2.11</artifactId>
          <version>1.6.1</version>
      </dependency>
     
      <dependency>
          <groupId>org.mongodb.spark</groupId>
          <artifactId>mongo-spark-connector_2.12</artifactId>
          <version>2.4.3</version>
      </dependency>
      <!-- https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-core -->
      <dependency>
          <groupId>com.fasterxml.jackson.core</groupId>
          <artifactId>jackson-core</artifactId>
          <version>2.10.0</version>
      </dependency>

问题出现在KafkaUtils.createDirectStream（）中，我检查了maven和scala兼容性JDK 11与scala2.12及更高版本兼容，我们对spack-Streing-kafkajar没有任何maven依赖。请让我知道我的分析是否错误，我应该为JDK 11使用spack-Streing-kafka

Answer 1

>

正如所评论的，您正在混合Scala版本（代理Scala版本无关紧要），并且您也在混合Spark版本