无法读取文档URL：无法读取整个标题；6字节读取；应为32字节

Question

String url = "http://prevention.cancer.gov/sites/default/files/uploads/clinical_trial/Master-DMP-Template.doc";
InputStream inputStream = new URL(urlString).openStream();
HWPFDocument doc = new HWPFDocument(inputStream);
WordExtractor extractor = new WordExtractor(doc);
String text = extractor.getText();

尝试2：有趣的部分是下载文件（只是在浏览器地址栏中粘贴URL），然后执行类似的代码在本地读取文档确实有效：

InputStream inputStream = new FileInputStream("C:\\Users\\me\\Downloads\\Master-DMP-Template (2).doc");
HWPFDocument doc = new HWPFDocument(inputStream);
WordExtractor extractor = new WordExtractor(doc);
System.out.println(extractor.getText());

尝试3：现在是最奇怪的部分。我想文件需要先下载。因此，我首先使用Java下载了它，然后在本地执行前面的代码来读取文档。像第一个案例一样失败！

final String url = "http://prevention.cancer.gov/sites/default/files/uploads/clinical_trial/Master-DMP-Template.doc";
String localPath  = FileUtils.downloadFile("C:\\Users\\me\\Downloads", url);
InputStream inputStream = new FileInputStream(localPath);
HWPFDocument doc = new HWPFDocument(inputStream);
WordExtractor extractor = new WordExtractor(doc);
System.out.println(extractor.getText());

public static String downloadFile(String targetDir, String sourceUrl) throws IOException {
    sourceUrl = StringUtils.removeEnd(sourceUrl, "/");
    String fileName = sourceUrl.substring(sourceUrl.lastIndexOf("/") + 1);
    String targetPath = targetDir + FileUtils.SEPARATOR + fileName;
    InputStream in = new URL(sourceUrl).openStream();
    Files.copy(in, Paths.get(targetPath), StandardCopyOption.REPLACE_EXISTING);
    System.out.println("Downloaded " + sourceUrl + " to " + targetPath);
    return targetPath;
}

你知道这是怎么回事吗？

public static void main(String[] args) {
    try {
        if (args.length > 0 && args[0].equals("disableCertValidation")) {
            SSLUtil.disableCertificateValidation(); // redirect is https
        }
        final String stringURL = "https://prevention.cancer.gov/sites/default/files/uploads/clinical_trial/Master-DMP-Template.doc";
        URL url = new URL(stringURL);
        HttpURLConnection con = (HttpURLConnection) url.openConnection();
        int responseCode = con.getResponseCode();
        System.out.println("Response code: " + responseCode); //301 Moved Permanently
        InputStream in = con.getInputStream();
        HWPFDocument doc = new HWPFDocument(in);
        WordExtractor extractor = new WordExtractor(doc);
        String text = extractor.getText();
        System.out.println(text);
        in.close();
    } catch (IOException e) {
        e.printStackTrace();
    }
}

int responseCode = con.getResponseCode();

以下异常失败：javax.net.ssl.sslhandShakeException：sun.security.validator.validatoreXception：PKIX路径构建失败：sun.security.provider.certPath.SunCertPathBuilderException：找不到请求目标的有效证书路径

当运行带有disableCertificateValidation参数的代码时，响应代码为404，我得到以下异常：

java.io.fileNotFoundException:https://prevention.cancer.gov/sites/default/files/uploads/clinical_trial/master-dmp-template.doc at sun.reflect.nativeconstructoraccessorimpl.newinstance0（原生方法）at sun.reflect.nativeconstructoraccessorimpl.newinstance0（原生方法）at sun.reflect.nativeconstructoraccessorimpl.newinstance（原生方法）at lconnection.java:1890)在sun.net.www.protocol.http.http.httpurlconnection$10.run(httpurlconnection.java:1885)在sun.net.www.protocol.http.http.httpurlconnection.getChainedException(httpurlconnection.java:1885)在sun.net.www.protocol.http.httppurlconnection.getInputStream(httpurlconnection.java:1441)在sun.net.www.protocol.https.httpsurlconnectionimpl.getInputStream(httpsurlconnectionimpl.java:254)在com.keywords.control.util.testhtmlparser.main(Testhtmlparser.java:472)由:java.io.filenotfoundexception:https://prevention.cancer.gov/sites/default/files/uploads/clinical_trial/master-dmp-template.doc在sun.net.www.protocol.https.httpsurlconnectionimpl.getresponseCode(httpsurlconnectionimpl.java:480)在com.keywords.control.util.testhtmlparser.main(Testhtmlparser.java:470)

有什么想法吗？

Answer 1

对URL的初始HTTP请求导致一个重定向301永久移动。所以我们需要处理这个并读取新的位置。

完整示例：

import java.io.InputStream;
import java.net.URL;
import java.net.HttpURLConnection;

import org.apache.poi.hwpf.HWPFDocument;
import org.apache.poi.hwpf.extractor.WordExtractor;

public class OpenHWPFFromURL {

 public static void main(String[] args) throws Exception {

  String stringURL = "http://prevention.cancer.gov/sites/default/files/uploads/clinical_trial/Master-DMP-Template.doc";

  URL url = new URL(stringURL);
  HttpURLConnection con = (HttpURLConnection)url.openConnection();

  int responseCode = con.getResponseCode();
  System.out.println(responseCode); //301 Moved Permanently

  if (responseCode != HttpURLConnection.HTTP_OK) {
   if (responseCode == HttpURLConnection.HTTP_MOVED_TEMP
       || responseCode == HttpURLConnection.HTTP_MOVED_PERM
       || responseCode == HttpURLConnection.HTTP_SEE_OTHER) {
    url = new URL(con.getHeaderField("Location")); //get new location
    con = (HttpURLConnection)url.openConnection();
   }   
  }

  InputStream in = con.getInputStream();
  HWPFDocument doc = new HWPFDocument(in);
  WordExtractor extractor = new WordExtractor(doc);
  String text = extractor.getText();

  System.out.println(text);

 }
}

注意：简单地将httpurlconnection.setfollredirects设置为true（也是默认值），如果重定向还更改了协议（例如，从http设置为https)的话，将没有帮助。这里的情况也是如此。因此，我们需要手动获取新的位置，如我的代码所示。