当前位置: 首页 > 知识库问答 >
问题:

如果动物园管理员设置好了,Kafka就不会启动。acl设置为true

萧芷阳
2023-03-14

我有一套kerberize Zookeeper和kerberize Kafka,zookeeper.set.acl设置为false时效果很好。当我尝试以参数设置为true启动Kafka时,我在zookeeper日志中得到了以下信息:

Nov 12 13:36:26 <zk host> docker:zookeeper_corelinux_<zk host>[1195]: [2019-11-12 13:36:26,625] INFO Client attempting to establish new session at /<kafka ip>:54272 (org.apache.zookeeper.server.ZooKeeperServer)
Nov 12 13:36:26 <zk host> docker:zookeeper_corelinux_<zk host>[1195]: [2019-11-12 13:36:26,631] INFO Established session 0x3007c8bcb5c0000 with negotiated timeout 6000 for client /<kafka ip>:54272 (org.apache.zookeeper.server.ZooKeeperServer)
Nov 12 13:36:26 <zk host> docker:zookeeper_corelinux_<zk host>[1195]: [2019-11-12 13:36:26,775] INFO Successfully authenticated client: authenticationID=kafka/<kafka host>@REALM;  authorizationID=kafka/<kafka host>@REALM. (org.apache.zookeeper.server.auth.SaslServerCallbackHandler)
Nov 12 13:36:26 <zk host> docker:zookeeper_corelinux_<zk host>[1195]: [2019-11-12 13:36:26,778] INFO Setting authorizedID: kafka (org.apache.zookeeper.server.auth.SaslServerCallbackHandler)
Nov 12 13:36:26 <zk host> docker:zookeeper_corelinux_<zk host>[1195]: [2019-11-12 13:36:26,778] INFO adding SASL authorization for authorizationID: kafka (org.apache.zookeeper.server.ZooKeeperServer)
Nov 12 13:36:26 <zk host> docker:zookeeper_corelinux_<zk host>[1195]: [2019-11-12 13:36:26,807] ERROR Missing AuthenticationProvider for sasl (org.apache.zookeeper.server.PrepRequestProcessor)
Nov 12 13:36:26 <zk host> docker:zookeeper_corelinux_<zk host>[1195]: [2019-11-12 13:36:26,808] INFO Got user-level KeeperException when processing sessionid:0x3007c8bcb5c0000 type:create cxid:0x4 zxid:0x100000005 txntype:-1 reqpath:n/a Error Path:/brokers/ids Error:KeeperErrorCode = InvalidACL for /brokers/ids (org.apache.zookeeper.server.PrepRequestProcessor)
Nov 12 13:36:26 <zk host> docker:zookeeper_corelinux_<zk host>[1195]: [2019-11-12 13:36:26,829] INFO Processed session termination for sessionid: 0x3007c8bcb5c0000 (org.apache.zookeeper.server.PrepRequestProcessor)

Kafka和佐克佩尔都在docker中跑步(使用Confluent的图片)

以下是Zookeeper配置(通过环境变量传入):

"ZOOKEEPER_AUTHPROVIDER_1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider",
"KAFKA_OPTS=-Djava.security.auth.login.config=/etc/zookeeper/secrets/zookeeper_jaas.conf -Dzookeeper.kerberos.removeHostFromPrincipal=true -Dzookeeper.kerberos.removeRealmFromPrincipal=true",
"ZOOKEEPER_SERVER_ID=1",
"ZOOKEEPER_REQUIRECLIENTAUTHSCHEME=SASL",
"KAFKA_JMX_HOSTNAME=<zk host>",
"ZOOKEEPER_INIT_LIMIT=10",
"ZOOKEEPER_JASSLOGINRENEW=3600000",
"ZOOKEEPER_LOG4J_PROP=DEBUG,ROLLINGFILE",
"ZOOKEEPER_MAX_CLIENT_CNXNS=0",
"ZOOKEEPER_SERVERS=0.0.0.0:2888:3888;zookeeper2:2888:3888;zookeeper3:2888:3888",
"ZOOKEEPER_DATA_DIR=/data/zookeeper",
"ZOOKEEPER_CLIENT_PORT=2181",
"KAFKA_JMX_PORT=55554"

动物园管理员JAAS:

Server {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    storeKey=true
    doNotPrompt=true
    useTicketCache=false
    keyTab="/etc/zookeeper/secrets/kfkzkp.keytab"
    principal="zookeeper/<zk host>@REALM";
};

这是Kafka配置:

"KAFKA_ZOOKEEPER_SET_ACL=true",
"KAFKA_DEFAULT_REPLICATION_FACTOR=3",
"KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL=GSSAPI",
"KAFKA_ADVERTISED_LISTENERS=SASL_SSL://<kafka host>:9092",
"KAFKA_OPTS=-Djava.security.auth.login.config=/etc/kafka/secrets/kafka_server_jaas.conf",
"KAFKA_ZOOKEEPER_CONNECT=zookeeper1:2181,zookeeper2:2181,zookeeper3:2181",
"KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND=true",
"KAFKA_SSL_CLIENT_AUTH=required",
"KAFKA_CONFLUENT_SUPPORT_METRICS_ENABLE=False",
"KAFKA_LOG_DIRS=/data/kafka",
"KAFKA_SASL_KERBEROS_SERVICE_NAME=kafka",
"KAFKA_SSL_TRUSTSTORE_FILENAME=root-ca-certificate.jks",
"KAFKA_JMX_HOSTNAME=<kafka host>",
"KAFKA_MIN_INSYNC_REPLICAS=2",
"KAFKA_JMX_PORT=55555",
"KAFKA_SSL_KEY_CREDENTIALS=redacted",
"KAFKA_AUTHORIZER_CLASS_NAME=kafka.security.auth.SimpleAclAuthorizer",
"KAFKA_SUPER_USERS=User:superuser;User:me",
"KAFKA_SSL_KEYSTORE_FILENAME=<kafka host>.jks",
"KAFKA_SSL_KEYSTORE_CREDENTIALS=redacted",
"KAFKA_SSL_TRUSTSTORE_CREDENTIALS=redacted",
"KAFKA_AUTO_CREATE_TOPICS_ENABLE=true",
"KAFKA_SASL_ENABLED_MECHANISMS=GSSAPI,PLAIN",
"KAFKA_LISTENERS=SASL_SSL://<kafka host>:9092",
"KAFKA_SECURITY_INTER_BROKER_PROTOCOL=SASL_SSL",

Kafka·JAAS:

// Zookeeper client authentication
Client {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    storeKey=true
    doNotPrompt=true
    useTicketCache=false
    serviceName=kafka
    keyTab="/etc/kafka/secrets/kfkzkp.keytab"
    principal="kafka/<kafka host>@REALM";
};

我看这个已经有一段时间了,已经浏览了谷歌上的大部分相关内容(包括stackoverflow的一些链接)。欢迎提出任何建议。

共有1个答案

厉熠彤
2023-03-14

想出来了。由于某些原因,有些变量没有从环境中正确拾取。我昨天通过ZOOKEEPER_KERBEROS_REMOVEREALMFROMPRINCIPAL(和REMOVEHOSTFROMPRONCIPAL)注意到了这一点。所以我试着移动这些

"ZOOKEEPER_AUTHPROVIDER_1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider",
"ZOOKEEPER_REQUIRECLIENTAUTHSCHEME=SASL",

进入

KAFKA_OPTS: "-Djava.security.auth.login.config=/etc/zookeeper/secrets/zookeeper_jaas.conf -Dzookeeper.kerberos.removeHostFromPrincipal=true -Dzookeeper.kerberos.removeRealmFromPrincipal=true -Dzookeeper.authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider -Dzookeeper.requireClientAuthScheme=sasl"

这就解决了。

 类似资料:
  • 分布式 Apache HBase 安装依赖于正在运行的 ZooKeeper 集群。所有参与节点和客户端都需要能够访问正在运行的 ZooKeeper 集合。 Apache HBase 默认为您管理 ZooKeeper“集群”。它将启动和停止 ZooKeeper 集合作为 HBase 启动/停止过程的一部分。您还可以独立于 HBase 管理 ZooKeeper 集合,只需将 HBase 指向它应该使用

  • 我目前在让NiFi和Zookeeper使用Kerberos进行身份验证时遇到了一些问题。任何帮助都将不胜感激。 当我尝试用Kerberos配置启动NiFI时,它只是在启动过程中关闭。 我正在使用外部ZooGuard集群(不是嵌入式集群)。根据NiFi管理指南,我已将以下内容添加到我的配置文件中: $NIFI\U HOME/conf/bootstrap。形态: $NIFI_HOME/conf/nif

  • 假设我有 3 台 Kafka 服务器。服务器 1 zoopkeeper1 服务器 2 zoopkeeper2 服务器 3 zoopkeeper3 在集群配置中,zoopkeepers 会发生什么?它们是为每个服务器单独维护的,还是会在群集配置中同步其数据?

  • 在Zookeeper和代理身份验证上启用SASL时,我面临以下错误。 以下配置在JAAS文件中给出,该文件作为KAFKA_OPTS传递,将其作为JVM参数:- Kafka经纪人的服务器。属性设置了以下额外字段:- Zookeeper属性如下所示:

  • 我在用拉斯贝里圆周率和拉斯比安。我想用Kafka把数据从相机流到我的手机上。我从Kafka网站下载了这个包,里面包含Zookeeper和Kafka: https://www.apache.org/dyn/closer.cgi?path=/kafka/2.4.1/kafka2.12-2.4.1.tgz 我的假设是Kafka联系不到动物园管理员,但我真的不知道。 有没有办法我可以测试动物园管理员是否在

  • 如果我部署war,我将得到以下异常: 原因:org.apache.camel.resolveEndpointFailedException:无法解析endpoint:kafka://localhost:9092?serializerClass=kafka.serializer.stringencoder&topic=checking&zookeePerhost=localhost&zookeePe