Jsoup 无法登录以建立 Cookie

Question

因此，我已经阅读了所有关于创建会话和自动登录的jsoup文章。阅读有关cookie问题的问题。因此，使用所有这些示例，我创建了一些JAVA代码来登录到特定站点。但每次我登录或尝试登录时，我都会被重定向回登录页面。

1. 使用loginForm我获得所需的cookie
2. 然后我使用Forms操作URL发布我的登录信息
3. 然后我应该登录，但它是重新定向到登录。我可以在loginDoc对象解析中看到这一点。

问题是，我在尝试登录此特定站点时做错了什么？

package com.auction.estimates.utils.scraper;

import java.io.IOException;
import java.net.URLEncoder;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;

import org.jsoup.Connection;
import org.jsoup.Connection.Response;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;

import com.google.api.client.http.GenericUrl;
import com.google.api.client.http.HttpRequest;
import com.google.api.client.http.HttpRequestFactory;
import com.google.api.client.http.HttpResponse;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.http.javanet.NetHttpTransport;

public class GatherData {

public static void main(String[] args) throws IOException {

    Response loginForm = Jsoup
            .connect(
                    "https://www.rbauction.com/myaccount?redirect=/home/auth")
            .userAgent(
                    "Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Firefox/31.0")
            .maxBodySize(0)
            .timeout(600000)
            .method(Connection.Method.GET)
            .execute();

    Document formDoc = loginForm.parse();
    Elements forms = formDoc.select("form");
    Element elemForm = forms.first();
    String urlAction = elemForm.attr("action");

    Response login = Jsoup
            .connect(
                    "https://www.rbauction.com" + urlAction)
            .data("_58_redirect", "%2Fhome%2Fauth&")
            .data("_58_rememberMe", "false")
            .data("_58_breakpoint","null")
            .data("_58_login", "myEmail")
            .data("_58_password", "myPass")
            .data("_58_rba-login-submit", "Sign+In")
            .cookies(loginForm.cookies())
            .userAgent(
                    "Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Firefox/31.0")
            .followRedirects(true)
            .method(Connection.Method.POST).execute();

    Document loginDoc = login.parse();
    Map<String, String> loginCookies = login.cookies();

    String url = "https://www.rbauction.com/2006-caterpillar-303-5ccr?invId=4322220&id=ar";
    Document doc = Jsoup
            .connect(url)
            .cookies(loginCookies)
            .userAgent(
                    "Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Firefox/31.0")
            .followRedirects(true).get();

    Elements links = doc.select("a[href]");

    System.out.println("What do we have.");
}

}

html prettyprint-override">	<form name="_58_fm" method="post" action="/myaccount?p_auth=rlOiteI3&amp;p_p_id=58&amp;p_p_lifecycle=1&amp;p_p_state=normal&amp;p_p_mode=view&amp;p_p_col_id=column-1&amp;p_p_col_count=2&amp;saveLastPath=0&amp;_58_struts_action=%2Flogin%2Flogin" id="yui_3_4_0_6_1415034856663_348">
		<input type="hidden" value="/home/auth" id="_58_redirect" name="_58_redirect">	   
		<input type="hidden" value="true" id="_58_rememberMe" name="_58_rememberMe">
		<input type="hidden" value="null" id="_58_breakpoint" name="_58_breakpoint">
		
		<div class="rba-input-line">
			<label for="_58_login">Email address</label>
			<input type="text" tabindex="1" autocomplete="off" maxlength="75" value="" name="_58_login" id="_58_login" class="rba-input ">
		</div>
		
		<div class="rba-input-line" id="yui_3_4_0_6_1415034856663_347">
			<label for="_58_password">Password</label> 
			<a class="rba-label-link float-right" target="_top" href="/myaccount/forgot-password">Forgot password?</a>
			<input type="password" autofocus="true" tabindex="2" autocomplete="off" maxlength="75" value="" name="_58_password" id="_58_password" class="rba-input rba-form-autofocus">
		</div>
		
		<div class="rba-input-line" id="rba-remember-me">
			<span class="rba-ico-inline-text-24">
				<input type="checkbox" tabindex="3" name="_58_chkRememberMe" id="_58_chkRememberMe" checked="checked" class="rba-input-checkbox">								
				<label for="_58_chkRememberMe">
					<span class="faux-a">Remember me</span>	
						&nbsp;<span class="rba-sprite rba-ico rba-ico-24 rba-ico-24-exclaimation">
						</span>
				</label>
			</span>
		</div>
		
		<div class="button-stack button-stack-horizontal-right hf-bottom-margin">
			<input type="submit" tabindex="4" title="Sign In" value="Sign In" id="_58_rba-login-submit" class="rba-button rba-btn-lvl-1">
		</div>
	</form>

Answer 1

所以我根本无法使用自动登录。但是这种数据收集可能每两周进行一次。所以为了解决这个问题，我登录了Mozilla，提取了我当前所有的rbauctions.com cookie。然后，我尝试设置JSESSIONID和USER并传递cookies。那也失败了。所以我手动将所有31个cookies及其内容复制并粘贴到一个HashMap中。

Map<String, String> loginCookies = new HashMap<String, String>();
    loginCookies.put("COOKIE_SUPPORT", "true");
    loginCookies.put("__CT_Data", "gpv=9&apv_9545_www09=9");
    loginCookies.put("WRUID", "0");
    etc......

在那之后，我可以访问该网站并删除我的数据。这并不理想，但至少我可以得到我的数据。

Answer 2

你的代码给了我

HTTP error fetching URL. Status=404, URL=https://www.rbauction.com/home/auth&

如果您仅更改:

.data（“_58_redirect”， “/home/auth

它工作正常！

如果没有，请检查您的用户/凭证。