Selenium python代码在交互式地图上单击按钮以防止网页刮擦不起作用

Question

我正在尝试使用Python中的Selenium从交互式地图中web刮取数据。我一直有困难与代码单击某些按钮以获得数据。第一个“点击”很好，但第二个不起作用。我试过调整时间，但什么都不起作用。我想能够做第二次点击。如有任何帮助，不胜感激，谢谢。下面是我的代码：

!apt-get update # to update ubuntu to correctly run apt install
!apt install chromium-chromedriver
!cp /usr/lib/chromium-browser/chromedriver /usr/bin
import sys
sys.path.insert(0,'/usr/lib/chromium-browser/chromedriver')
from selenium import webdriver
from selenium.webdriver.common.keys import Keys  
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import Select
import os
from bs4 import BeautifulSoup
import re
import pandas as pd
import sys
import re
import csv
import time
import shutil
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
wd = webdriver.Chrome('chromedriver',chrome_options=chrome_options) 

from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium import webdriver
import time
wd.maximize_window()
wd.get("https://hazards.geoplatform.gov/portal/apps/MapSeries/index.html?appid=ddf915a24fb24dc8863eed96bc3345f8")
wd.execute_script("arguments[0].click();", WebDriverWait(wd, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="nav-bar"]/div/div[1]/ul/li[4]/a'))))
wd.execute_script("arguments[0].click();", WebDriverWait(wd, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="widgets_About_Widget_34"]/div/div/div/div[10]/font/a'))))

Answer 1

您的代码有几个问题。

1. 阻止您访问第二个元素的主要障碍是因为它位于iframe中，所以要访问它，首先必须切换到该iframe。
2. 您的定位器不好。
3. 在别无选择之前不要单击JavaScript中的元素。
4. 在无头模式下，必须定义屏幕大小。
5. wd.maximize_window()用于普通模式，而不是headless。
   所以代码应该如下所示:

import sys
sys.path.insert(0,'/usr/lib/chromium-browser/chromedriver')
from selenium import webdriver
from selenium.webdriver.common.keys import Keys  
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import Select
import os
from bs4 import BeautifulSoup
import re
import pandas as pd
import sys
import re
import csv
import time
import shutil
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--window-size=1920,1080')
chrome_options.add_argument('--disable-dev-shm-usage')
wd = webdriver.Chrome('chromedriver',chrome_options=chrome_options) 

from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium import webdriver
import time
#wd.maximize_window()
wd.get("https://hazards.geoplatform.gov/portal/apps/MapSeries/index.html?appid=ddf915a24fb24dc8863eed96bc3345f8")
wait.until(EC.visibility_of_element_located((By.XPATH, "//li[contains(@class,'entry  visible')]//a[contains(text(),'Data Download Tool')]"))).click()
wait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it(driver.find_element_by_css_selector("div[class='mainMediaContainer active']>iframe")))
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div[title='Download']"))).click()