Java：无法从URL获取html[重复]

Question

我正在尝试从URL读取html文件。我的代码可以与大多数站点工作，除了一些站点，比如http://dota2.gamepedia.com/dota_2_wiki。我猜我需要设置java代理什么的？...

    try {
        URL webPage = new URL("http://dota2.gamepedia.com/Dota_2_Wiki");

        URLConnection con = webPage.openConnection();
        con.setConnectTimeout(5000);
        con.setReadTimeout(5000);

        BufferedReader in = new BufferedReader(
                            newInputStreamReader(con.getInputStream()));

        String inputLine;
        while ((inputLine = in.readLine()) != null)
            System.out.println(inputLine);

        in.close();
    }
    catch (MalformedURLException exc){exc.printStackTrace();}
    catch (IOException exc){exc.printStackTrace();}

java.io.IOException: Server returned HTTP response code: 403 for URL: http://dota2.gamepedia.com/Dota_2_Wiki
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1838)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1439)
at com.Popov.Main.main(Main.java:17)

错误代码403：我如何访问它？顺便说一句，它可以在浏览器中正常工作

Answer 1

您的问题很可能是因为没有正确设置用户代理。给你们这些喜欢香草爪哇的人。这些是密码

private void sendGet() throws Exception {

    String url = "http://dota2.gamepedia.com/Dota_2_Wiki";

    URL obj = new URL(url);
    CookieHandler.setDefault(new CookieManager(null, CookiePolicy.ACCEPT_ALL));
    HttpURLConnection con = (HttpURLConnection) obj.openConnection();

    con.setRequestMethod("GET");
    con.setRequestProperty("User-Agent", USER_AGENT);

    int responseCode = con.getResponseCode();
    System.out.println("\nSending 'GET' request to URL : " + url);
    System.out.println("Response Code : " + responseCode);

    BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
    String inputLine;
    StringBuffer response = new StringBuffer();

    while ((inputLine = in.readLine()) != null) {
        response.append(inputLine);
    }
    in.close();

    System.out.println(response.toString());

}

请注意，您还需要设置cookie，因为当我尝试不使用它时，代码会给我很多重定向循环