当前位置: 首页 > 知识库问答 >
问题:

使用sbt程序集生成fat jar时build.sbt中的重复数据删除错误

邢博学
2023-03-14
    java.lang.RuntimeException: deduplicate: different file contents found in the following:
    /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.transaction/orbits/javax.transaction-1.1.1.v201105210645.jar:META-INF/ECLIPSEF.RSA
    /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.servlet/orbits/javax.servlet-3.0.0.v201112011016.jar:META-INF/ECLIPSEF.RSA
    /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.mail.glassfish/orbits/javax.mail.glassfish-1.4.1.v201005082020.jar:META-INF/ECLIPSEF.RSA
    /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.activation/orbits/javax.activation-1.1.0.v201105071233.jar:META-INF/ECLIPSEF.RSA
            at sbtassembly.Plugin$Assembly$.sbtassembly$Plugin$Assembly$$applyStrategy$1(Plugin.scala:253)
            at sbtassembly.Plugin$Assembly$$anonfun$15.apply(Plugin.scala:270)
            at sbtassembly.Plugin$Assembly$$anonfun$15.apply(Plugin.scala:267)
            at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
            at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
            at scala.collection.Iterator$class.foreach(Iterator.scala:727)
            at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
            at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
            at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
            at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251)
            at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105)
            at sbtassembly.Plugin$Assembly$.applyStrategies(Plugin.scala:272)
            at sbtassembly.Plugin$Assembly$.x$4$lzycompute$1(Plugin.scala:172)
            at sbtassembly.Plugin$Assembly$.x$4$1(Plugin.scala:170)
            at sbtassembly.Plugin$Assembly$.stratMapping$lzycompute$1(Plugin.scala:170)
            at sbtassembly.Plugin$Assembly$.stratMapping$1(Plugin.scala:170)
            at sbtassembly.Plugin$Assembly$.inputs$lzycompute$1(Plugin.scala:214)
            at sbtassembly.Plugin$Assembly$.inputs$1(Plugin.scala:204)
            at sbtassembly.Plugin$Assembly$.apply(Plugin.scala:230)
            at sbtassembly.Plugin$Assembly$$anonfun$assemblyTask$1.apply(Plugin.scala:373)
            at sbtassembly.Plugin$Assembly$$anonfun$assemblyTask$1.apply(Plugin.scala:370)
            at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
            at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:42)
            at sbt.std.Transform$$anon$4.work(System.scala:64)
            at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
            at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
            at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
            at sbt.Execute.work(Execute.scala:244)
            at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
            at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
            at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
            at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)
            at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
            at java.util.concurrent.FutureTask.run(FutureTask.java:166)
            at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
            at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
            at java.util.concurrent.FutureTask.run(FutureTask.java:166)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
            at java.lang.Thread.run(Thread.java:724)
    [error] (*:assembly) deduplicate: different file contents found in the following:
    [error] /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.transaction/orbits/javax.transaction-1.1.1.v201105210645.jar:META-INF/ECLIPSEF.RSA
    [error] /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.servlet/orbits/javax.servlet-3.0.0.v201112011016.jar:META-INF/ECLIPSEF.RSA
    [error] /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.mail.glassfish/orbits/javax.mail.glassfish-1.4.1.v201005082020.jar:META-INF/ECLIPSEF.RSA
    [error] /Users/ccd/.ivy2/cache/org.eclipse.jetty.orbit/javax.activation/orbits/javax.activation-1.1.0.v201105071233.jar:META-INF/ECLIPSEF.RSA
    [error] Total time: 8 s, completed 19/09/2014 11:50:50 AM
logLevel := Level.Warn

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.11.2")

这是我的build.sbt文件

import AssemblyKeys._ // put this at the top of the file

name := "Spark_sbt2"

version := "1.1"

scalaVersion := "2.10.4"

libraryDependencies += "org.apache.spark" %% "spark-core" % "1.1.0"

libraryDependencies += "org.apache.spark" %% "spark-mllib" % "1.1.0"

libraryDependencies += "org.apache.spark" %% "spark-streaming" % "1.1.0"

libraryDependencies += "org.apache.httpcomponents" % "httpclient" % "4.3.5"

libraryDependencies += "org.restlet.jse" % "org.restlet" % "2.2.2"

libraryDependencies += "org.restlet.jse" % "org.restlet.ext.jackson" % "2.2.2"

resolvers += "Akka Repository" at "http://repo.akka.io/releases/"

resolvers += "Restlet" at "http://maven.restlet.org"

assemblySettings

jarName in assembly := "spark-assembly.jar"

我不知道该怎么修好它。如果有人能帮忙,我很感激!

共有1个答案

呼延明朗
2023-03-14

问题是sbt-assembly无法选择要拾取的meta-inf/eclipsef.rsa文件。尝试此操作排除所有此类文件:

excludedFiles in assembly <<=
(excludedFiles in assembly) {
  (old) => (bases) =>
    old(bases) ++ (bases flatMap (base =>
      (base / "META-INF/ECLIPSEF.RSA").get))
}

如果您真的需要JAR中的RSA密钥:

assembledMappings in assembly <<= (assembledMappings in assembly) map { (old: File => Seq[(File, String)]) =>
  (f: File) => old(f) :+ (file("path/to/choosen/rsa/key"), "META-INF/ECLIPSEF.RSA")
}

(或者不要在excludedFiles中提及适当的基本内容)

 类似资料:
  • 然而,当我运行foreach循环时,它运行了几分钟就崩溃了 最初的数据库mydb有0.173GB,现在是0.368GB 你知道出什么问题了吗? 所以这次看起来很管用,但为什么'mydb'变大了呢?

  • [错误]不是有效的项目ID:程序集 [错误]应为“:”(如果选择配置) [错误]不是有效的键:程序集

  • 本文向大家介绍Oracle删除重复的数据,Oracle数据去重复,包括了Oracle删除重复的数据,Oracle数据去重复的使用技巧和注意事项,需要的朋友参考一下 Oracle  数据库中查询重复数据: select * from employee group by emp_name having count (*)>1;  Oracle  查询可以删除的重复数据 select t1.* from

  • 本文向大家介绍Mysql删除重复的数据 Mysql数据去重复,包括了Mysql删除重复的数据 Mysql数据去重复的使用技巧和注意事项,需要的朋友参考一下 MySQL数据库中查询重复数据 select * from employee group by emp_name having count (*)>1; Mysql  查询可以删除的重复数据 select t1.* from employee

  • 我在Talend创建了一个作业,其中我必须生成包含用tRowGenerator生成的数据的文件,以及其他源:SQL Server数据库和分隔文件。 问题是我有相同主键的重复文件。我只想获得100条记录(420行):对于每个随机生成的UUID,我将获得42行,以此类推,但相反,我将获得相同的行10次(它重复了10次)

  • sbt版本为0.13。9和scala 2.11。7/我知道以前版本的sbt依赖于scala 2.10-仍然是这样吗?我有一个Java项目,为它添加了一个程序集。项目目录中的sbt文件(根据本版本sbt组件的sbt组件说明): 我运行了sbt重新加载/清理以及编译。但是,当我尝试运行程序集时,会出现以下异常: 在使用sbt为我的java项目创建可运行的jar之前,我遗漏了什么步骤? 如果需要的话,我