亚马逊下一页选项选择。在网页中[重复]

Question

我正试着在网页上收集亚马逊的产品信息。我有代码，它将打开一个web驱动程序，搜索产品名称，并导航到产品页面的第一页。它只能收集第一页的数据，但如何移动到下一页收集相同的数据。以下是我的代码:

import time
import json
import re
import numpy as np
from bs4 import BeautifulSoup
from selenium import webdriver
import urllib.request
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.common.keys import Keys
import pandas as pd


temp = []


def init_driver():
    driver = webdriver.Chrome(executable_path = "C:\\Users\\Desktop\\chromedriver")
    driver.wait = WebDriverWait(driver, 10)
    return driver


def get_asin(driver):

    driver.get("https://www.amazon.in")
    print ('Getting the URL')
    HTML = driver.page_source
    search_button = driver.find_element_by_id("twotabsearchtextbox")
    search_button.send_keys("Mobiles")
    select_button = driver.find_element_by_class_name("nav-input")
    select_button.click()
    HTML1=driver.page_source
    soup = BeautifulSoup(HTML1, "html.parser")


    styles = soup.find_all('li')
    #print(styles)
    #print(type(styles))
    ASIN=[]
    for link in styles:
        if link.has_attr('data-asin'):
            ASIN.append(link['data-asin'])

    return(ASIN)
    #print(ASIN)


if __name__ == "__main__":
    driver = init_driver()
    ASIN_NO = get_asin(driver)
    #time.sleep(3)
    #print ('opening search page')
    #for i in range(0,len(ASIN_NO)):
        #scrape(driver,ASIN_NO[i])

    print (ASIN_NO)
    time.sleep(5)

我已经尝试了以下两种语法，以显示错误：

select_button = driver.find_element_by_id('pagnNextString')
select_button.click()

日志中的异常：

WebDriverException:消息：未知错误：元素…在点（778606）处不可单击。其他元素将收到点击：

select_button = driver.find_element_by_class_name('srSprite pagnNextArrow')
select_button.click()

InvalidSelectorException:消息:无效选择器:不允许复合类名

请用正确的方法帮助。提前感谢。

Answer 1

我认为你必须最大化窗口，因为元素是不可见的，这就是为什么问题元素是不可点击的

driver.maximize_window()

USe 下一个按钮的 x 路径（对于无效的探测器问题）

.//*[@id='nav-search']/form/div[2]/div/input

我对python没有太多的了解。这是java代码在我的系统中运行良好

WebDriver driver=new FirefoxDriver();
driver.get("https://www.amazon.in");
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
WebElement search_txt=driver.findElement(By.xpath("//*[@id='twotabsearchtextbox']"));
search_txt.sendKeys("Mobiles");
driver.manage().window().maximize();
driver.findElement(By.xpath(".//*[@id='nav-search']/form/div[2]/div/input")).click();
WebElement select_btn=driver.findElement(By.xpath("//*[@id='pagnNextString']"));
select_btn.click();

Answer 2

要能够单击Next按钮，您可以使用下面的代码：

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

next_button = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.ID, "pagnNextString")))
next_button.location_once_scrolled_into_view
next_button.click()

这应该允许您等到按钮出现在页面上，向下滚动到它并成功单击