我目前在让NiFi和Zookeeper使用Kerberos进行身份验证时遇到了一些问题。任何帮助都将不胜感激。
当我尝试用Kerberos配置启动NiFI时,它只是在启动过程中关闭。
我正在使用外部ZooGuard集群(不是嵌入式集群)。根据NiFi管理指南,我已将以下内容添加到我的配置文件中:
$NIFI\U HOME/conf/bootstrap。形态:
java.arg.15=-Djava.security.auth.login.config=/opt/nifi/conf/zookeeper-jaas.conf
$NIFI_HOME/conf/nifi.properties:
nifi.zookeeper.auth.type=sasl
nifi.zookeeper.kerberos.removeHostFromPrincipal=true
nifi.zookeeper.kerberos.removeRealmFromPrincipal=true
$NIFI\U HOME/conf/zookeeper jaas。形态:
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/opt/nifi/conf/nifi.keytab"
storeKey=true
useTicketCache=false
principal="HTTP/nifi-2.nifi.nifi4.svc.cluster.local@MYDOMAIN.NET";
};
我也使用初始化keytab:
kinit -kt /opt/nifi/conf/nifi.keytab HTTP/nifi-2.nifi.nifi4.svc.cluster.local@MYDOMAIN.NET
当我启动NiFi时,我在它关闭之前得到以下堆栈跟踪:
2020-04-21 18:41:43,736 WARN [main] org.eclipse.jetty.webapp.WebAppContext Failed startup of context o.e.j.w.WebAppContext@504497fa{nifi-api,/nifi-api,file:///opt/nifi/work/jetty/nifi-web-api-1.10.0.war/webapp/,UNAVAILABLE}{./work/nar/framework/nifi-framework-nar-1.10.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.10.0.war}
org.apache.nifi.web.NiFiCoreException: Unable to start Flow Controller.
at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:88)
at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:959)
at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:553)
at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:924)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:365)
at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1497)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1459)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:854)
at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:278)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:406)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
at org.eclipse.jetty.server.Server.start(Server.java:418)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at org.eclipse.jetty.server.Server.doStart(Server.java:382)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:952)
at org.apache.nifi.NiFi.<init>(NiFi.java:158)
at org.apache.nifi.NiFi.<init>(NiFi.java:72)
at org.apache.nifi.NiFi.main(NiFi.java:301)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flowService': FactoryBean threw exception on object creation; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flowController': FactoryBean threw exception on object creation; nested exception is java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 87
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:185)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1640)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1086)
at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:55)
... 37 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flowController': FactoryBean threw exception on object creation; nested exception is java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 87
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:185)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1640)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1086)
at org.apache.nifi.spring.StandardFlowServiceFactoryBean.getObject(StandardFlowServiceFactoryBean.java:48)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178)
... 43 common frames omitted
Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 87
at java.base/java.lang.String.checkBoundsBeginEnd(String.java:3319)
at java.base/java.lang.String.substring(String.java:1874)
at org.apache.nifi.controller.leader.election.CuratorACLProviderFactory$SaslACLProvider.<init>(CuratorACLProviderFactory.java:49)
at org.apache.nifi.controller.leader.election.CuratorACLProviderFactory$SaslACLProvider.<init>(CuratorACLProviderFactory.java:40)
at org.apache.nifi.controller.leader.election.CuratorACLProviderFactory.create(CuratorACLProviderFactory.java:37)
at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.createClient(CuratorLeaderElectionManager.java:389)
at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.determineLeaderExternal(CuratorLeaderElectionManager.java:343)
at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.getLeader(CuratorLeaderElectionManager.java:240)
at org.apache.nifi.controller.FlowController.<init>(FlowController.java:680)
at org.apache.nifi.controller.FlowController.createClusteredInstance(FlowController.java:413)
at org.apache.nifi.spring.FlowControllerFactoryBean.getObject(FlowControllerFactoryBean.java:65)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178)
... 50 common frames omitted
2020-04-21 18:41:43,983 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=97ms
2020-04-21 18:41:43,984 INFO [main] o.e.j.s.h.C._nifi_content_viewer No Spring WebApplicationInitializer types detected on classpath
2020-04-21 18:41:44,010 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@5618fc1f{nifi-content-viewer,/nifi-content-viewer,file:///opt/nifi/work/jetty/nifi-web-content-viewer-1.10.0.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.10.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-content-viewer-1.10.0.war}
2020-04-21 18:41:44,044 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=22ms
2020-04-21 18:41:44,046 WARN [main] o.e.j.webapp.StandardDescriptorProcessor Duplicate mapping from / to default
2020-04-21 18:41:44,046 INFO [main] o.e.j.s.h.ContextHandler._nifi_docs No Spring WebApplicationInitializer types detected on classpath
2020-04-21 18:41:44,071 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@60b1ff3b{nifi-docs,/nifi-docs,file:///opt/nifi/work/jetty/nifi-web-docs-1.10.0.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.10.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-docs-1.10.0.war}
2020-04-21 18:41:44,091 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=11ms
2020-04-21 18:41:44,093 INFO [main] o.e.j.server.handler.ContextHandler._ No Spring WebApplicationInitializer types detected on classpath
2020-04-21 18:41:44,116 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@374c3975{nifi-error,/,file:///opt/nifi/work/jetty/nifi-web-error-1.10.0.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.10.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-error-1.10.0.war}
2020-04-21 18:41:44,137 INFO [main] o.e.jetty.util.ssl.SslContextFactory x509=X509@5fb8db6c(nifi-2,h=[nifi-2-fix, nifi-2, test.gdn.network, test-api.gdn.network, nifi.gdn.network],w=[]) for SslContextFactory@4556a9a7[provider=null,keyStore=file:///opt/nifi/conf/nifi-2.p12,trustStore=file:///opt/nifi/conf/truststore.jks]
2020-04-21 18:41:44,149 INFO [main] o.eclipse.jetty.server.AbstractConnector Started ServerConnector@4d81e83a{SSL,[ssl, http/1.1]}{nifi-2:8443}
2020-04-21 18:41:44,149 INFO [main] org.eclipse.jetty.server.Server Started @38713ms
2020-04-21 18:41:44,150 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down.
org.apache.nifi.web.NiFiCoreException: Unable to start Flow Controller.
at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:88)
at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:959)
at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:553)
at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:924)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:365)
at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1497)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1459)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:854)
at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:278)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:406)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
at org.eclipse.jetty.server.Server.start(Server.java:418)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at org.eclipse.jetty.server.Server.doStart(Server.java:382)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:952)
at org.apache.nifi.NiFi.<init>(NiFi.java:158)
at org.apache.nifi.NiFi.<init>(NiFi.java:72)
at org.apache.nifi.NiFi.main(NiFi.java:301)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flowService': FactoryBean threw exception on object creation; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flowController': FactoryBean threw exception on object creation; nested exception is java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 87
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:185)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1640)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1086)
at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:55)
... 37 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flowController': FactoryBean threw exception on object creation; nested exception is java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 87
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:185)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1640)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1086)
at org.apache.nifi.spring.StandardFlowServiceFactoryBean.getObject(StandardFlowServiceFactoryBean.java:48)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178)
... 43 common frames omitted
Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 87
at java.base/java.lang.String.checkBoundsBeginEnd(String.java:3319)
at java.base/java.lang.String.substring(String.java:1874)
at org.apache.nifi.controller.leader.election.CuratorACLProviderFactory$SaslACLProvider.<init>(CuratorACLProviderFactory.java:49)
at org.apache.nifi.controller.leader.election.CuratorACLProviderFactory$SaslACLProvider.<init>(CuratorACLProviderFactory.java:40)
at org.apache.nifi.controller.leader.election.CuratorACLProviderFactory.create(CuratorACLProviderFactory.java:37)
at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.createClient(CuratorLeaderElectionManager.java:389)
at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.determineLeaderExternal(CuratorLeaderElectionManager.java:343)
at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.getLeader(CuratorLeaderElectionManager.java:240)
at org.apache.nifi.controller.FlowController.<init>(FlowController.java:680)
at org.apache.nifi.controller.FlowController.createClusteredInstance(FlowController.java:413)
at org.apache.nifi.spring.FlowControllerFactoryBean.getObject(FlowControllerFactoryBean.java:65)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178)
... 50 common frames omitted
2020-04-21 18:41:44,150 INFO [Thread-0] org.apache.nifi.NiFi Initiating shutdown of Jetty web server...
如果我删除Kerberos配置,那么NiFi将运行得很好(尽管没有使用Kerberised Zookeeper)。我尝试过用一个全新的动物园管理员来运行这个程序,以防与以前的非kerberised节点发生某种冲突,但运气不佳。
对NiFi/Zookeeper有更多了解的人能帮我理解这个错误吗?
事先非常感谢,
哈利
您仍然需要在nifi中设置kerberos服务主体。属性:
nifi.kerberos.service.principal
管理指南中关于ZK客户端的留档应该更新,但它假设NiFi本身已经“kerberated”,这将设置服务主体。
分布式 Apache HBase 安装依赖于正在运行的 ZooKeeper 集群。所有参与节点和客户端都需要能够访问正在运行的 ZooKeeper 集合。 Apache HBase 默认为您管理 ZooKeeper“集群”。它将启动和停止 ZooKeeper 集合作为 HBase 启动/停止过程的一部分。您还可以独立于 HBase 管理 ZooKeeper 集合,只需将 HBase 指向它应该使用
我正在为Storm开发代码,这是开发人员的一个示例。我的问题是,当您从IDE Eclipse运行此代码时,不会建立Storm和ZooKeer之间的连接。ZooKeer在2181中运行,也设置在storm.yaml. 我的例外是: 有什么想法或程序要执行吗?是否从配置代码中传递?和我一样?
我必须采用4.3.1单solr索引并将其迁移到solrcloud 5.2.1 新的5.2.1体系结构是2个碎片,每个碎片有1个主碎片和1个从碎片(副本)。我的步骤是: 设置新的单分片solrcloud 5.2.1 我也有配置在一个单一的ZooKeeper(我知道1 ZK不推荐)。 一切正常,碎片正在运行,副本包含数据,我可以从新的solrcloud 5.2.1 2碎片中查询数据。 当我添加新文档时
注意:这将只显示有关使用Java消费者API的消费者(非基于ZooKeeper的消费者)的信息。 此工具还适用于基于ZooKeeper的使用者: bin/kafka-consumer-groups.sh--zookeeper localhost:2181--description--group my-group null PS:这似乎没有出现在2.5文档中
假设我有 3 台 Kafka 服务器。服务器 1 zoopkeeper1 服务器 2 zoopkeeper2 服务器 3 zoopkeeper3 在集群配置中,zoopkeepers 会发生什么?它们是为每个服务器单独维护的,还是会在群集配置中同步其数据?
在Zookeeper和代理身份验证上启用SASL时,我面临以下错误。 以下配置在JAAS文件中给出,该文件作为KAFKA_OPTS传递,将其作为JVM参数:- Kafka经纪人的服务器。属性设置了以下额外字段:- Zookeeper属性如下所示: