Post请求返回405

Question

我必须做一个帖子，但我得到错误405

在本网站->http://177.66.89.34:8079/transparencia/#我需要迭代页面顶部的2个下拉菜单的选项。

首先，我会在“escolha o exercício”旁边加上选项：然后，我会在“escolha a entidade”旁边加上选项：

用下面的代码，我在“EscolhaoExercício：”上面写了一篇文章

# -*- coding: utf-8 -*-
import scrapy

class ScpiSpider(scrapy.Spider): # classe abstrata
    start_urls = ['http://177.66.89.34:8079/Transparencia']

def parse(self, response):
    anos_exercicios = response.xpath("//table[@id='cmbExercicio_DDD_L_LBT']//td/text()").extract()

    for ano in anos_exercicios:
        formadata = {"Scriptmanager1": "UpdatePanel1|cmbExercicio",
                         "cmbExercicio_VI": ano,
                        "cmbExercicio": ano,
                        "__EVENTTARGET": "cmbExercicio",
                        "__VIEWSTATE": response.xpath("//input[@id='__VIEWSTATE']/@value").get(),
                        "__VIEWSTATEGENERATOR": response.xpath("//input[@id='__VIEWSTATEGENERATOR']/@value").get(),
                        "__EVENTVALIDATION": response.xpath("//input[@id='__EVENTVALIDATION']/@value").get(),
                         "__ASYNCPOST": "true"}
        headers = {
            'origin': "http://177.125.200.195:8079",
            'x-requested-with': "XMLHttpRequest",
            'cache-control': "no-cache",
            'x-microsoftajax': "Delta=true",
            'user-agent': "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36",
            'accept': "*/*"
        }
        yield scrapy.FormRequest(url=self.start_urls[0], formdata=formadata, callback=self.parse_entidade,
                                 dont_filter=True, headers=headers)

def parse_entidade(self, response):
    print(response)

我希望在parse_entidade中输入代码，但收到[scraphy.spidermiddlewares.httperror]信息：忽略响应<405HTTP://177.66.89.34:8079/transparencia>:HTTP状态代码未处理或不允许

Answer 1

只需在start_url中添加一个'/'就能起到效果：

start_urls = ['http://177.66.89.34:8079/Transparencia/']

像这样启动它时得到的输出如下：

2019-08-07 09:44:43 [scrapy.core.engine] DEBUG: Crawled (200) <GET http://177.66.89.34:8079/Transparencia/> (referer: None)
2019-08-07 09:44:43 [scrapy.core.engine] DEBUG: Crawled (200) <POST http://177.66.89.34:8079/Transparencia/> (referer: http://177.66.89.34:8079/Transparencia/)
2019-08-07 09:44:43 [scrapy.core.engine] DEBUG: Crawled (200) <POST http://177.66.89.34:8079/Transparencia/> (referer: http://177.66.89.34:8079/Transparencia/)
<200 http://177.66.89.34:8079/Transparencia/>
<200 http://177.66.89.34:8079/Transparencia/>
2019-08-07 09:44:44 [scrapy.core.engine] DEBUG: Crawled (200) <POST http://177.66.89.34:8079/Transparencia/> (referer: http://177.66.89.34:8079/Transparencia/)
<200 http://177.66.89.34:8079/Transparencia/>
2019-08-07 09:44:44 [scrapy.core.engine] DEBUG: Crawled (200) <POST http://177.66.89.34:8079/Transparencia/> (referer: http://177.66.89.34:8079/Transparencia/)
<200 http://177.66.89.34:8079/Transparencia/>
2019-08-07 09:44:45 [scrapy.core.engine] INFO: Closing spider (finished)