使用Selenium擦除时出现StaleElementReferenceException问题

Question

我正在尝试完全加载此页面：https://candidat.pole-emploi.fr/offres/emploi/horticulteur/s1m1

我设置了一行代码来处理cookie弹出。

然后，我设置了一些行来单击加载更多结果按钮，以便加载完整的html，然后将其打印出来。

但单击一次后，我发现一条错误消息：

StaleElementReferenceException: stale element reference: element is not attached to the page document

我不知道这意味着什么，也不知道如何修复它

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
import time

options = webdriver.ChromeOptions()
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')

site = 'https://candidat.pole-emploi.fr/offres/emploi/horticulteur/s1m1'
wd = webdriver.Chrome("C:\Program Files (x86)\chromedriver.exe", options=options)
wd.get(site)

time.sleep(10)

wait = WebDriverWait(wd, 10)

# click cookies popup
wd.find_element_by_xpath('//*[(@id = "description")]//*[contains(concat( " ", @class, " " ), concat( " ", "tc-open-privacy-center", " " ))]').click()

time.sleep(10)

# click show more button until no more results to load
while True:
    try:
        more_button = wait.until(EC.visibility_of_element_located((By.LINK_TEXT, 'AFFICHER LES 20 OFFRES SUIVANTES'))).click()
    except TimeoutException:
        break

time.sleep(10)

print(wd.page_source)
print("Complete")

time.sleep(10)
wd.quit()

Answer 1

尝试使用execute\u script方法，我认为这是解决此类问题最可靠的方法。

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException, NoSuchElementException
import time

options = webdriver.ChromeOptions()
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')

site = 'https://candidat.pole-emploi.fr/offres/emploi/horticulteur/s1m1'
wd = webdriver.Chrome("C:\Program Files (x86)\chromedriver.exe", options=options)
wd.get(site)

time.sleep(10)

wait = WebDriverWait(wd, 10)

# click cookies popup
wd.find_element_by_xpath('//*[(@id = "description")]//*[contains(concat( " ", @class, " " ), concat( " ", "tc-open-privacy-center", " " ))]').click()

time.sleep(10)

# click show more button until no more results to load
while True:
    try:
        wait.until(EC.visibility_of_element_located((By.LINK_TEXT, 'AFFICHER LES 20 OFFRES SUIVANTES')))
        more_button = wd.find_element_by_link_text('AFFICHER LES 20 OFFRES SUIVANTES')
        wd.execute_script('arguments[0].click()', more_button)
        #print('clicked')
    except (TimeoutException, NoSuchElementException):
        break

time.sleep(10)

print(wd.page_source)
print("Complete")

time.sleep(10)
wd.quit()

Answer 2

StaleElementReferenceException：stale元素引用：元素未附加到页面文档

指示对元素的引用现在已“过时”——该元素不再显示在页面的DOM上。出现这种预期的原因可能是您的DOM更新或刷新了。例如，在执行类似于单击的操作后，您的DOM可能会更新或刷新。此时，当您试图在DOM上查找元素时，您将遇到此错误。

您必须在更新或刷新的DOM中重新找到该元素

       try:  
            more_button = wait.until(EC.visibility_of_element_located((By.LINK_TEXT, 'AFFICHER LES 20 OFFRES SUIVANTES'))).click()  
     except StaleElementReferenceException:
            more_button = WebDriverWait(self.driver, 10).until(EC.visibility_of_element_located((By.LINK_TEXT, 'AFFICHER LES 20 OFFRES SUIVANTES')))
            more_button.click()

Answer 3

有许多方法可以处理陈旧的元素引用。

一个是像尝试重新单击web元素在一个while循环。

您的link\u文本看起来也有误，请使用以下xpath：

# click cookies popup
driver.find_element_by_xpath('//*[(@id = "description")]//*[contains(concat( " ", @class, " " ), concat( " ", "tc-open-privacy-center", " " ))]').click()

time.sleep(10)

# click show more button until no more results to load
while True:
    try:
        more_button = wait.until(EC.visibility_of_element_located((By.XPATH, "//a[starts-with(@onclick,'tagDeClick') and contains(@href,'/offres/emploi.rechercheoffre:afficherplusderesultats')]")))
        ActionChains(driver).move_to_element(more_button).perform()
        attempts = 0
        while attempts < 2 :
            try:
                more_button.click()
                break
            except StaleElementReferenceException as exception:
                print(exception.msg)
            attempts = attempts  + 1

    except TimeoutException:
        break

time.sleep(10)

print(driver.page_source)
print("Complete")

time.sleep(10)

输出：

stale element reference: element is not attached to the page document
  (Session info: chrome=94.0.4606.81)

如果您在logs中看到这一点，并且您不希望看到这一点，则必须注释print（exception.msg）。

进口：

from selenium.webdriver.common.action_chains import ActionChains