错误:“元素当前不可见，因此可能无法与selenuim交互”

Question

我想抓取谷歌playstore搜索结果完全呈现的网页。

完全呈现的页面具有所有搜索项，而未呈现的页面仅具有20项。（请参阅https://play.google.com/store/search?q=best

我试图用selenium抓取页面，但得到了下面的错误信息。

Traceback (most recent call last):
File "play_test_2.py", line 25, in test_play_test2
driver.find_element_by_id("show-more-button").click()
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webelement.py", line 65, in click
self._execute(Command.CLICK_ELEMENT)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webelement.py", line 385, in _execute
return self._parent.execute(command, params)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
self.error_handler.check_response(response)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 166, in check_response
raise exception_class(message, screen, stacktrace)
ElementNotVisibleException: Message: Element is not currently visible and so may not be interacted with
Stacktrace:
at fxdriver.preconditions.visible (file:///var/folders/8_/n90htn1d0_j4h7l9yt04chl80000gn/T/tmpErWdUz/extensions/[e-mail]/components/command-processor.js:8959:5)
at DelayedCommand.prototype.checkPreconditions_ (file:///var/folders/8_/n90htn1d0_j4h7l9yt04chl80000gn/T/tmpErWdUz/extensions/[e-mail]/components/command-processor.js:11618:1)
at DelayedCommand.prototype.executeInternal_/h (file:///var/folders/8_/n90htn1d0_j4h7l9yt04chl80000gn/T/tmpErWdUz/extensions/[e-mail]/components/command-processor.js:11635:11)
at fxdriver.Timer.prototype.setTimeout/<.notify          (file:///var/folders/8_/n90htn1d0_j4h7l9yt04chl80000gn/T/tmpErWdUz/extensions/[e-mail]/components/command-processor.js:548:5)

以下代码由Selenuim IDE编写。

# -*- coding: utf-8 -*-
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import NoAlertPresentException
import unittest, time, re

class PlayTest2(unittest.TestCase):
    def setUp(self):
        self.driver = webdriver.Firefox()
        self.driver.implicitly_wait(30)
        self.base_url = "https://play.google.com/"
        self.verificationErrors = []
        self.accept_next_alert = True

    def test_play_test2(self):
        driver = self.driver
        driver.get(self.base_url + "/store/search?q=best&c=apps")
        driver.find_element_by_id("gbqfb").click()
        driver.find_element_by_id("show-more-button").click()
        driver.find_element_by_id("show-more-button").click()

    def is_element_present(self, how, what):
        try: self.driver.find_element(by=how, value=what)
        except NoSuchElementException, e: return False
        return True

    def is_alert_present(self):
        try: self.driver.switch_to_alert()
        except NoAlertPresentException, e: return False
        return True

    def close_alert_and_get_its_text(self):
        try:
            alert = self.driver.switch_to_alert()
            alert_text = alert.text
            if self.accept_next_alert:
                alert.accept()
            else:
                alert.dismiss()
            return alert_text
        finally: self.accept_next_alert = True

    def tearDown(self):
        self.driver.quit()
        self.assertEqual([], self.verificationErrors)

if __name__ == "__main__":
    unittest.main()

我认为出现错误是因为谷歌商店搜索结果页面有“显示更多”按钮，当它满足某些特定条件时会显示，例如向下和向上滚动，然后再次向下滚动。

如何解决这个问题并抓取google搜索结果页面？

Answer 1

test_play_test2呼叫

driver.find_element_by_id（显示更多按钮）

两次。< br >我认为在第一次单击之后，selenium无法找到相同的元素，因此失败了。只需从这个方法中删除最近的一行。

更新:你完全正确，按钮的问题是在页面滚动后才出现的。因此，我们必须滚动窗口。以下javascript命令很有帮助

（（JavascriptExecutor） driver）.executeScript（“window.scrollTo（0，document.body.scrollHeight）;”）;

一个命令是不够的，我们已经做了好几次:

for (int i=0; i<5; i++ ){
    ((JavascriptExecutor) driver).executeScript("window.scrollTo(0,document.body.scrollHeight);");
    try{
        (new WebDriverWait(driver, 5/*sec*/))
                        .until(ExpectedConditions.visibilityOf(element));
        break;
    }
    catch (org.openqa.selenium.TimeoutException e){
    }
}
if(element.isDisplayed()){
    element.click();
}

抱歉，我在这个例子中使用了Java代码，但我希望这能让你明白。