Java JSoup提取URL时出错

Question

public class TestSl {
    public static void main(String[] args) throws IOException {
        Document doc = Jsoup.connect("https://stackoverflow.com/questions/11970938/java-html-parser-to-extract-specific-data").get();
        Elements spans = doc.select("span[class=hidden-text]");
        for (Element span: spans) {
            System.out.println(span.text());
        }
    }
}

线程“main”org.jsoup.HttpStatusException中出现异常：提取URL时出现HTTP错误。status=403,url=java Htmlhtml" target="_blank">解析器提取特定数据？在org.jsoup.helper.httpconnection$response.execute(httpconnection.java:590)在org.jsoup.helper.httpconnection$response.execute(httpconnection.java:540)在org.jsoup.helper.httpconnection.execute(httpconnection.java:227)在org.jsoup.helper.httpconnection.execute(httpconnection.java:216)在testsl.main(testsl.java:19)

我做错了什么，我该如何解决？

Answer 1

设置用户代理标头：

.userAgent("Mozilla")

示例：

Document document = Jsoup.connect("https://stackoverflow.com/questions/11970938/java-html-parser-to-extract-specific-data").userAgent("Mozilla").get();
Elements elements = document.select("span.hidden-text");
for (Element element : elements) {
  System.out.println(element.text());
}

堆栈交换