连接到cloudera impala环境时发生Kerberos错误

Question

连接到kerberized hadoop环境时错误：[Simba]ImpalaJdbcDriver无法连接到服务器：[Simba]ImpalaJdbcDriver Kerberos身份验证失败。

我已经在virtualbox中安装了cloudera quickstart vm，启用了kerberos，编写了连接到imapala db的java代码，并得到了kerberos身份验证失败的错误。

public static void main(String[] args) throws Exception {

        Configuration conf = new Configuration();
        conf.set("hadoop.security.authentication", "Kerberos");
        UserGroupInformation.setConfiguration(conf);
        UserGroupInformation ugi = UserGroupInformation
                .loginUserFromKeytabAndReturnUGI("hdfs/quickstart.cloudera@CLOUDERA", "hdfs.keytab");

        Class.forName("com.cloudera.impala.jdbc41.Driver");
        Connection conn = (Connection) ugi.doAs(new PrivilegedExceptionAction<Object>() {
            public Object run() {
                Connection tcon = null;
                try {
                    tcon = DriverManager.getConnection(
                            "jdbc:impala://quickstart.cloudera:21050;AuthMech=1;KrbHostFQDN=quickstart.cloudera;KrbRealm=CLOUDERA;KrbServiceName=hdfs");
                    System.out.println("Connected!");
                } catch (SQLException e) {
                    e.printStackTrace();
                }
                return tcon;
            }
        });

        Statement stmt = conn.createStatement();

        String sql = "show tables";
        System.out.println("Running: " + sql);
        ResultSet res = stmt.executeQuery(sql);
        while (res.next()) {
            System.out.println(res.getString(1));
        }
    }

我已启用调试模式，我得到的异常：

    ...
    Client Principal = hdfs/quickstart.cloudera@CLOUDERA
    Server Principal = hdfs/quickstart.cloudera@CLOUDERA
    Session Key = EncryptionKey: keyType=23 keyBytes (hex dump)=
    0000: 8A B3 79 07 A5 06 05 9F   CE 37 84 8A 15 2E 7E B5  ..y......7......


    Forwardable Ticket true
    Forwarded Ticket false
    Proxiable Ticket false
    Proxy Ticket false
    Postdated Ticket false
    Renewable Ticket false
    Initial Ticket false
    Auth Time = Sun Jun 23 11:52:03 PDT 2019
    Start Time = Sun Jun 23 11:52:03 PDT 2019
    End Time = Mon Jun 24 11:52:03 PDT 2019
    Renew Till = null
    Client Addresses  Null 
    >>> KrbApReq: APOptions are 00100000 00000000 00000000 00000000
    >>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType
    Krb5Context setting mySeqNumber to: 925793988
    Created InitSecContextToken:
    0000: 01 00 6E 82 02 2E 30 82   02 2A A0 03 02 01 05 A1  ..n...0..*......

    0220: 4A 3E 74 0A 67 B6 5E 16   3B B8 1D FB 91 75 53 33  J>t.g.^.;....uS3
    0230: 76 5E 40 81                                        v^@.

    java.sql.SQLException: [Simba][ImpalaJDBCDriver](500164) Error initialized or created transport for authentication: [Simba][ImpalaJDBCDriver](500169) Unable to connect to server: [Simba][ImpalaJDBCDriver](500591) Kerberos Authentication failed..
        at com.cloudera.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
        at com.cloudera.hivecommon.api.HiveServer2ClientFactory.createClient(Unknown Source)
        at com.cloudera.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown Source)
        at com.cloudera.impala.core.ImpalaJDBCConnection.establishConnection(Unknown Source)
        at com.cloudera.jdbc.core.LoginTimeoutConnection.connect(Unknown Source)
        at com.cloudera.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
        at com.cloudera.jdbc.common.AbstractDriver.connect(Unknown Source)
        at java.sql.DriverManager.getConnection(DriverManager.java:571)
        at java.sql.DriverManager.getConnection(DriverManager.java:233)
        at ImpalaJDBC$1.run(ImpalaJDBC.java:64)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
    Caused by: com.cloudera.support.exceptions.GeneralException: [Simba][ImpalaJDBCDriver](500164) Error initialized or created transport for authentication: [Simba][ImpalaJDBCDriver](500169) Unable to connect to server: [Simba][ImpalaJDBCDriver](500591) Kerberos Authentication failed..
        ... 13 more
    Caused by: java.lang.RuntimeException: [Simba][ImpalaJDBCDriver](500169) Unable to connect to server: [Simba][ImpalaJDBCDriver](500591) Kerberos Authentication failed.
        at com.cloudera.hivecommon.api.HiveServerPrivilegedAction.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:356)
        at com.cloudera.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
        at com.cloudera.hivecommon.api.HiveServer2ClientFactory.createClient(Unknown Source)
        at com.cloudera.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown Source)
        at com.cloudera.impala.core.ImpalaJDBCConnection.establishConnection(Unknown Source)
        at com.cloudera.jdbc.core.LoginTimeoutConnection.connect(Unknown Source)
        at com.cloudera.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
        at com.cloudera.jdbc.common.AbstractDriver.connect(Unknown Source)
        at java.sql.DriverManager.getConnection(DriverManager.java:571)
        at java.sql.DriverManager.getConnection(DriverManager.java:233)
        at ImpalaJDBC$1.run(ImpalaJDBC.java:64)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
        at ImpalaJDBC.main(ImpalaJDBC.java:60)
    Caused by: org.apache.thrift.transport.TTransportException
        at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
        at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
        at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:178)
        at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:258)
        at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
        ... 17 more

Answer 1

在Impala重新启动后，我的web项目遇到了同样的问题。将连接池设置为在空闲时释放所有连接，并让下一个查询重新申请连接。这时，就不会出现这个问题了。