无法获取远程数据（被动模式、TLS隐式、客户端证书+登录/PWD身份验证）

Question

我尝试用apache-commons-net-3.7.2（隐式TLS，使用客户端证书+登录/密码的双因素身份验证）与FTP服务器连接。

我可以验证自己，进入被动模式，但客户端没有成功连接到服务器，以便通过数据套接字获取数据。

我可以连接自己，在同一台计算机上，与WinSCP（相同的设置），我已经激活WinSCP日志查看协议细节，我已经调整了我的源代码与相同的选项。我可以通过ProtocolCommandListener验证我的协议是否正常。

下面是WinSCP日志

连接并浏览根目录
Chiffrement ssl/TLS Implicite
版本最小TLS/ssl：TLS 1.0
版本最大TLS/ssl：TLS 1.2
réutiliser L'id de session TLS/sl pour le connexions de données activé
（对激活的数据连接重用ssl会话id）

我使用与我的Java相同的证书（带有WinSCP的PFX格式，JCEKS用于Java，即Java 8）

. 2021-01-07 15:36:03.031 --------------------------------------------------------------------------
. 2021-01-07 15:36:03.032 WinSCP Version 5.17.9 (Compilation 10905) (OS 10.0.19041 - Windows 10 Enterprise)
. 2021-01-07 15:36:03.032 Configuration: HKCU\Software\Martin Prikryl\WinSCP 2\
. 2021-01-07 15:36:03.032 Log level: Normal
. 2021-01-07 15:36:03.032 Local account: FANLESS\damie
. 2021-01-07 15:36:03.032 Working directory: C:\Users\damie\Desktop
. 2021-01-07 15:36:03.032 Process ID: 22344
. 2021-01-07 15:36:03.033 Command-line: "C:\Program Files (x86)\WinSCP\WinSCP.exe" "xxxxxx" /Desktop /UploadIfAny /log="d:\temp\debugftp.log" /loglevel=0
. 2021-01-07 15:36:03.033 Time zone: Current: GMT+1, Standard: GMT+1 (Paris, Madrid), DST: GMT+2 (Paris, Madrid (heure d’été)), DST Start: 28/03/2021, DST End: 31/10/2021
. 2021-01-07 15:36:03.033 Login time: jeudi 7 janvier 2021 15:36:03
. 2021-01-07 15:36:03.033 --------------------------------------------------------------------------
. 2021-01-07 15:36:03.033 Session name: XXX@XXX (Site)
. 2021-01-07 15:36:03.034 Host name: XXX (Port: 999)
. 2021-01-07 15:36:03.034 User name: XXX (Password: Yes, Key file: No, Passphrase: No)
. 2021-01-07 15:36:03.034 Transfer Protocol: FTP
. 2021-01-07 15:36:03.034 Ping type: Dummy, Ping interval: 30 sec; Timeout: 15 sec
. 2021-01-07 15:36:03.034 Disable Nagle: No
. 2021-01-07 15:36:03.034 Proxy: None
. 2021-01-07 15:36:03.034 Send buffer: 262144
. 2021-01-07 15:36:03.034 UTF: Auto
. 2021-01-07 15:36:03.034 FTPS: Implicit TLS/SSL [Client certificate: Yes]
. 2021-01-07 15:36:03.034 FTP: Passive: Yes [Force IP: Auto]; MLSD: Auto [List all: Auto]; HOST: Auto
. 2021-01-07 15:36:03.034 Session reuse: Yes
. 2021-01-07 15:36:03.034 TLS/SSL versions: TLSv1.0-TLSv1.2
. 2021-01-07 15:36:03.034 Local directory: D:\Local\Documents, Remote directory: /, Update: Yes, Cache: Yes
. 2021-01-07 15:36:03.034 Cache directory changes: Yes, Permanent: Yes
. 2021-01-07 15:36:03.034 Recycle bin: Delete to: No, Overwritten to: No, Bin path: 
. 2021-01-07 15:36:03.034 Timezone offset: 0h 0m
. 2021-01-07 15:36:03.034 --------------------------------------------------------------------------
. 2021-01-07 15:36:03.381 Connexion à XXX:999...
. 2021-01-07 15:36:03.382 Connecté à XXX:999, négociation de la connexion SSL...
. 2021-01-07 15:36:03.382 Server asks for authentication with a client certificate.
. 2021-01-07 15:36:03.382 Verifying certificate for "RESEAU GDS" with fingerprint 58:12:03:0e:b3:dd:cc:d9:c4:a1:ce:a0:4a:d8:83:77:46:0b:9e:4c and 20 failures
. 2021-01-07 15:36:03.382 Certificate common name "XXX" matches hostname
. 2021-01-07 15:36:03.463 Certificate verified against Windows certificate store
. 2021-01-07 15:36:03.463 Using TLSv1.2, cipher TLSv1.2: AES256-GCM-SHA384, 2048 bit RSA, AES256-GCM-SHA384       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(256) Mac=AEAD
. 2021-01-07 15:36:03.479 Connexion SSL établie. En attente du message de bienvenue...
< 2021-01-07 15:36:03.479 220 w2003-ftp X2 WS_FTP Server 7.7(73371983)
> 2021-01-07 15:36:03.479 USER XXX
< 2021-01-07 15:36:03.483 331 Enter password
> 2021-01-07 15:36:03.483 PASS ********************
< 2021-01-07 15:36:03.533 230-User logged in
< 2021-01-07 15:36:03.533  Bonjour Serveur FTP de ReseauGDS
< 2021-01-07 15:36:03.533 230 User logged in
> 2021-01-07 15:36:03.533 SYST
< 2021-01-07 15:36:03.548 215 UNIX
> 2021-01-07 15:36:03.548 FEAT
< 2021-01-07 15:36:03.567 211-Extensions supported
< 2021-01-07 15:36:03.567     SIZE
< 2021-01-07 15:36:03.567     XMD5
< 2021-01-07 15:36:03.567     XSHA1
< 2021-01-07 15:36:03.567     XSHA256
< 2021-01-07 15:36:03.567     XSHA512
< 2021-01-07 15:36:03.567     XQUOTA
< 2021-01-07 15:36:03.567     LANG EN, ES, FR, GE
< 2021-01-07 15:36:03.567     MDTM
< 2021-01-07 15:36:03.567     MLST size*;type*;perm*;create*;modify*;
< 2021-01-07 15:36:03.567     REST STREAM
< 2021-01-07 15:36:03.567     TVFS
< 2021-01-07 15:36:03.567     UTF8
< 2021-01-07 15:36:03.568     AUTH SSL;TLS-P;
< 2021-01-07 15:36:03.568     PBSZ
< 2021-01-07 15:36:03.568     PROT C;P;
< 2021-01-07 15:36:03.568 211 end
> 2021-01-07 15:36:03.568 OPTS UTF8 ON
< 2021-01-07 15:36:03.584 200 Command OPTS succeed
> 2021-01-07 15:36:03.584 PBSZ 0
< 2021-01-07 15:36:03.598 200 PBSZ=0
> 2021-01-07 15:36:03.598 PROT P
< 2021-01-07 15:36:03.615 200 PRIVATE data channel protection level set
. 2021-01-07 15:36:03.664 Connecté
. 2021-01-07 15:36:03.664 --------------------------------------------------------------------------
. 2021-01-07 15:36:03.664 Using FTP protocol.
. 2021-01-07 15:36:03.664 Doing startup conversation with host.
> 2021-01-07 15:36:03.679 PWD
< 2021-01-07 15:36:03.694 257 "/" is current directory
. 2021-01-07 15:36:03.694 Changing directory to "/".
> 2021-01-07 15:36:03.695 CWD /
< 2021-01-07 15:36:03.723 250 Command CWD succeed
. 2021-01-07 15:36:03.723 Getting current directory name.
> 2021-01-07 15:36:03.723 PWD
< 2021-01-07 15:36:03.740 257 "/" is current directory
. 2021-01-07 15:36:03.787 Listage du répertoire en cours...
> 2021-01-07 15:36:03.787 TYPE A
< 2021-01-07 15:36:03.802 200 Transfer mode set to ASCII
> 2021-01-07 15:36:03.802 PASV
< 2021-01-07 15:36:03.817 227 Entering Passive Mode (192,168,4,122,235,142).
. 2021-01-07 15:36:03.817 Server sent passive reply with unroutable address 192.168.4.122, using host address instead.
> 2021-01-07 15:36:03.817 MLSD
. 2021-01-07 15:36:03.817 Connexion à 83.XXX.XXX.XXX:60302...
< 2021-01-07 15:36:03.845 150 Transferring directory
. 2021-01-07 15:36:03.846 Session ID reused
. 2021-01-07 15:36:03.847 Using TLSv1.2, cipher TLSv1.2: AES256-GCM-SHA384, 2048 bit RSA, AES256-GCM-SHA384       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(256) Mac=AEAD
. 2021-01-07 15:36:03.847 Connexion SSL établie
< 2021-01-07 15:36:03.869 226 Transfer completed
. 2021-01-07 15:36:03.869 Data connection closed
. 2021-01-07 15:36:03.869 size=0;type=cdir;create=20200623122630;modify=20201020142437; .
. 2021-01-07 15:36:03.869 size=0;type=pdir;create=20160308132406;modify=20200812101755; ..
. 2021-01-07 15:36:03.869 size=0;type=dir;create=20201020141336;modify=20201208104858; Mensuel
. 2021-01-07 15:36:03.869 size=0;type=dir;create=20201020141315;modify=20210107105026; Quotidien
. 2021-01-07 15:36:03.869 Listage du répertoire réussi
. 2021-01-07 15:36:03.870 ..;D;0;1899-12-30T01:00:00.000Z;0;"" [0];"" [0];---------;0
. 2021-01-07 15:36:03.870 Mensuel;D;0;2020-12-08T10:48:58.000Z;3;"" [0];"" [0];---------;0
. 2021-01-07 15:36:03.870 Quotidien;D;0;2021-01-07T10:50:26.000Z;3;"" [0];"" [0];---------;0
. 2021-01-07 15:36:03.893 Startup conversation with host finished.

Java代码摘录如下。您可以看到，为了输出协议细节，我使用了ProtocolCommandListener

    public static void main(String[] args) throws SocketException, IOException, GeneralSecurityException {

        String hostname = args[0];
        int port = Integer.parseInt(args[1]);
        String login = args[2];
        String pwd = args[3];
        
        System.setProperty("jdk.tls.useExtendedMasterSecret", "false");     

        //CompliantFTPSClient client = new CompliantFTPSClient("TLS", true);
        FTPSClient client = new FTPSClient("TLS", true);
        
        File clientCertStore = new File(
                "path to store");
        KeyManager keyManager = KeyManagerUtils.createClientKeyManager("JCEKS", clientCertStore, "",
                "alias", "");
        client.setKeyManager(keyManager);

        client.connect(hostname, port);
        int reply = client.getReplyCode();

        if (!FTPReply.isPositiveCompletion(reply)) {
            client.disconnect();
            System.err.println("FTP server refused connection.");
            System.exit(1);
        } else {
            if (client.login(login, pwd)) {

                client.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.err), true));
                client.sendCommand("OPTS", "UTF8 ON");
                client.sendCommand("PBSZ", "0");
                client.sendCommand("PROT", "P");

                client.enterLocalPassiveMode();

                System.out.println("Current directory : "+client.printWorkingDirectory());

                System.out.println(client.listHelp());

                FTPFile[] remoteFiles = client.mlistDir();

                reply = client.getReplyCode();
                System.out.println("fichiers : "+reply);
                System.out.println("nombre : "+remoteFiles.length);

                System.out.println("LOGOUT");
                client.logout();
            } else {
                System.out.println("echec login");
            }
        }
    }
}

下面的Java详细日志（system.err和system.out）：

janv. 07, 2021 8:31:50 PM sun.security.ssl.SSLLogger log
AVERTISSEMENT: Signature algorithm, ed25519, is not supported by the underlying providers
janv. 07, 2021 8:31:50 PM sun.security.ssl.SSLLogger log
AVERTISSEMENT: Signature algorithm, ed448, is not supported by the underlying providers
janv. 07, 2021 8:31:50 PM sun.security.ssl.SSLLogger log
INFOS: No available application protocols
janv. 07, 2021 8:31:50 PM sun.security.ssl.SSLLogger log
AVERTISSEMENT: Ignore impact of unsupported extension: renegotiation_info
OPTS UTF8 ON
200 Command OPTS succeed
PBSZ 0
200 PBSZ=0
PROT P
200 PRIVATE data channel protection level set
PWD
257 "/" is current directory
Current directory : /
HELP
214-The following commands are implemented
    ABOR  ACCT  ALLO* APPE  CDUP  CWD   DELE  FEAT+ HELP
    HOST+ LANG+ LIST  MDTM+ MLST+ MKD   MODE  NLST  NOOP
    OPTS+ PASS  PASV  PORT  PWD   QUIT  REIN  REST  RETR
    RMD   RNFR  RNTO  SITE  SIZE  SMNT  STAT  STOR  STOU
    STRU* SYST  TYPE  USER  XCUP  XCRC  XCWD  XMD5  XMKD
    XPWD  XRMD  XSIGN XSHA1 XSHA256 XSHA512 XQUOTA
214 Help complete

214-The following commands are implemented
    ABOR  ACCT  ALLO* APPE  CDUP  CWD   DELE  FEAT+ HELP
    HOST+ LANG+ LIST  MDTM+ MLST+ MKD   MODE  NLST  NOOP
    OPTS+ PASS  PASV  PORT  PWD   QUIT  REIN  REST  RETR
    RMD   RNFR  RNTO  SITE  SIZE  SMNT  STAT  STOR  STOU
    STRU* SYST  TYPE  USER  XCUP  XCRC  XCWD  XMD5  XMKD
    XPWD  XRMD  XSIGN XSHA1 XSHA256 XSHA512 XQUOTA
214 Help complete
PASV
227 Entering Passive Mode (192,168,4,122,236,242).
[Replacing PASV mode reply address 192.168.4.122 with 83.206.162.234]
MLSD
150 Transferring directory

MLSD命令从不回复。

问题可能出在ssl会话id重用上？这似乎是apache-commons-net的一个已知问题，参见https://www.javaer101.com/en/article/970693.html

代码：

public class CompliantFTPSClient extends FTPSClient {

    public CompliantFTPSClient(String protocol, boolean isImplicit) {
        super(protocol, isImplicit);
    }

    @Override
    protected void _prepareDataSocket_(Socket socket) throws IOException {
        if (socket instanceof SSLSocket) {
            // Control socket is SSL
            final SSLSession session = ((SSLSocket) _socket_).getSession();
            if (session.isValid()) {
                final SSLSessionContext context = session.getSessionContext();
                context.setSessionCacheSize(0);
                try {
                    final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
                    sessionHostPortCache.setAccessible(true);
                    final Object cache = sessionHostPortCache.get(context);
                    final Method method = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
                    method.setAccessible(true);
                    method.invoke(cache, String
                            .format("%s:%s", socket.getInetAddress().getHostName(), String.valueOf(socket.getPort()))
                            .toLowerCase(Locale.ROOT), session);
                    method.invoke(cache, String
                            .format("%s:%s", socket.getInetAddress().getHostAddress(), String.valueOf(socket.getPort()))
                            .toLowerCase(Locale.ROOT), session);
                } catch (NoSuchFieldException e) {
                    System.err.println("WARN : No field sessionHostPortCache in SSLSessionContext");
                    e.printStackTrace(System.err);
                } catch (Exception e) {
                    System.err.println("WARN : " + e.getMessage());
                }
            } else {
                System.err.println(String.format("SSL session %s for socket %s is not rejoinable", session, socket));
            }
        }
    }
}

我想做的是：只需递归地浏览服务器上的目录，列出文件，下载一些文件，这类事情通常我可以在SFTP或SCP中几分钟内完成。但在使用此API的FTPS中，这只是痛苦。

Answer 1

通过向服务器发送protp命令，数据连接加密不匹配。

由于只使用ftp.sendcommand将命令发送到服务器，因此该命令只影响服务器。而不是客户端（Apache Commonftpclient)。因此，当启动数据连接时，服务器等待TLS/SSL交换，但客户端没有响应，因为它只需要普通数据。这就是连接挂起的原因。

您必须调用ftpclient.execprot（“p”）。它在内部调用ftp.sendcommand（“prot”,“p”），但也在客户端启用加密。

WinSCP还重用会话ID。如果您的服务器需要这样做，请参阅:
如何使用相同的TLS会话通过数据连接连接到FTPS服务器？