当使用requests.session登录www.researchgate.net时，我得到了一个“403禁止”，警告说“您的浏览器不接受cookie。“

Question

我正在用Python为https://www.reseachgate.net编写一个网络爬虫程序。当我尝试使用requests.session登录时，我得到了一个“403禁止”页面，警告说“您的浏览器不接受cookie。使用此站点需要cookie。”如何解决这个问题？

import requests
from bs4 import BeautifulSoupter 

headers = {
            'User-Agent':'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:32.0) Gecko/20100101 Firefox/32.0'
        }
session = requests.Session()

params = {'login': '######@std.uestc.edu.cn', 'password': '#######'}
s = session.post("https://www.researchgate.net/application.Login.html", data = params, headers = headers)
print s.cookies.get_dict()
print s.text
s = session.get("https://www.researchgate.net/home")
print BeautifulSoup(s.text).title

以下是我得到的：

 <div class="error-page">
     <h1>
         403 Forbidden            </h1>

     <h3>Your browser doesn't accept cookies. Cookies are required to use this site.</h3>

Answer 1

原来我在登录表单中漏掉了一些字段。

以下是我更新的代码：

import requests
from bs4 import BeautifulSoup

session = requests.Session()
loginpage = session.get("https://www.researchgate.net/application.Login.html")
request_token = BeautifulSoup(loginpage.text).form.find("input",{"name":"request_token"}).attrs["value"]
print request_token
params = {"request_token":request_token,
          "invalidPasswordCount":"0",
          'login': 'my_email', 
          'password': 'my_password',
          "setLoginCookie":"yes"
          }
session.post("https://www.researchgate.net/application.Login.html", data = params)
s = session.get("https://www.researchgate.net/search.Search.html?type=researcher&query=zhang")
print BeautifulSoup(s.text).title