ApacheIgnite TcpDiscoveryKubernetesIpFinder在vNet中的Azure Kubernetes群集中失败

Question

vNet IP Address space: 10.106.8.0/22
Apache Ignite version: 2.9.1
Kubernetes version: 1.19.7
Service CIDR: 10.0.0.0/16
DNS Service IP: 10.0.0.10
Docker bridge CIDS: 172.17.0.1/16

[Error] [org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi] Failed to get registered addresses from IP finder on start

[06:04:20] [Error] [org.apache.ignite.internal.util.typedef.G] Blocked system-critical thread has been detected. This can lead to cluster-wide undefined behaviour [workerName=tcp-client-disco-msg-worker, threadName=tcp-client-disco-msg-worker-#4-#35, blockedFor=13s]
[06:04:20] [Warn] [] Possible failure suppressed accordingly to a configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=Unmodifia[72901-default-config.xml][1]bleSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=tcp-client-disco-msg-worker, igniteInstanceName=null, finished=false, heartbeatTs=1614578647003]]]
[06:04:20] [Warn] [org.apache.ignite.internal.processors.cache.CacheDiagnosticManager] Page locks dump:

NAME            READY   STATUS    RESTARTS   AGE
cohortstore-0   1/1     Running   0          3d6h
cohortstore-1   1/1     Running   0          3d6h
cohortstore-2   1/1     Running   0          3d6h

kubectl -n cohort-store get svc
NAME                                 TYPE           CLUSTER-IP     EXTERNAL-IP    PORT(S)                                          AGE
cohortstore-load-balancer-internal   LoadBalancer   10.0.113.146   10.106.8.255   8080:31417/TCP,10800:32719/TCP,10900:31208/TCP   29h

kubectl get Pods-N队列-前门

NAME                               READY   STATUS    RESTARTS   AGE
cohortfrontdoor-665f99bb6b-tdl5z   1/1     Running   0          72m

客户端XML SpringConfig文件

<?xml version="1.0" encoding="UTF-8"?>
<!--
        Configuration example with Kubernetes IP finder and Ignite persistence enabled.
        WAL files and database files are stored in separate disk drives.
    -->
<beans
    xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="
            http://www.springframework.org/schema/beans
            http://www.springframework.org/schema/beans/spring-beans.xsd">
    <bean class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="clientmode" value="true"/>
        <property name="failureDetectionTimeout" value="5000"/>
        <property name="clientFailureDetectionTimeout" value="10000"/>
        <property name="cacheConfiguration">
            <list>
                <bean class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="ephi"/>
                    <property name="cacheMode" value="PARTITIONED"/>
                    <property name="writeSynchronizationMode" value="FULL_SYNC"/>
                    <property name="backups" value="0"/>
                </bean>
            </list>
        </property>
        <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="networkTimeout" value="10000" />
                <property name="localPort" value="47500" />
                <property name="ipFinder">
                    <!--
                            Enables Kubernetes IP finder and setting custom namespace and service names.
                        -->
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
                        <property name="namespace" value="cohort-store"/>
                        <property name="serviceName" value="cohortstore-load-balancer-internal"/>
                    </bean>
                </property>
                <property name="socketTimeout" value="300" />
            </bean>
        </property>
        <property name="communicationSpi">
            <bean class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
                <property name="slowClientQueueLimit" value="1000"/>
            </bean>
        </property>
    </bean>
</beans>

Answer 1

在我执行clusterRoleBinding之后，错误消失了（更多信息请参见下面）。我想，必须这样做，因为我的Ignite服务器节点和think客户端节点位于两个不同的K8S名称空间中&有两个不同的服务帐户。

但现在，下面是我在服务器节点上看到的一个大循环...

INFO: TCP discovery accepted incoming connection [rmtAddr=/10.106.8.32, rmtPort=43883]
Mar 02, 2021 12:02:44 AM org.apache.ignite.logger.java.JavaLogger info
INFO: TCP discovery spawning a new thread for connection [rmtAddr=/10.106.8.32, rmtPort=43883]
Mar 02, 2021 12:02:44 AM org.apache.ignite.logger.java.JavaLogger info
INFO: Started serving remote node connection [rmtAddr=/10.106.8.32:43883, rmtPort=43883]
Mar 02, 2021 12:02:44 AM org.apache.ignite.logger.java.JavaLogger info
INFO: Initialized connection with remote client node [nodeId=59a5ce6f-2d0d-4abb-aaf5-b2b9f51f7e44, rmtAddr=/10.106.8.32:43883]
Mar 02, 2021 12:02:44 AM org.apache.ignite.logger.java.JavaLogger info
INFO: Finished serving remote node connection [rmtAddr=/10.106.8.32:43883, rmtPort=43883
Mar 02, 2021 12:02:46 AM org.apache.ignite.logger.java.JavaLogger info

clusterrolebinding信息

k describe clusterrolebinding cohortstore-RoleBinding
Name:         cohortstore-RoleBinding
Labels:       app.kubernetes.io/managed-by=Helm
Annotations:  meta.helm.sh/release-name: cohortstore
              meta.helm.sh/release-namespace: cohort-store
Role:
  Kind:  ClusterRole
  Name:  cohortstore-Role
Subjects:
  Kind            Name                  Namespace
  ----            ----                  ---------
  ServiceAccount  cohortstore.com       cohort-store
  ServiceAccount  cohort-frontdoor.com  cohort-frontdoor