网页加载在selenium和到达结束，但不包含div内的所有元素

Question

这是网站。https://www.talabat.com/uae/top-selling.大约有100种产品，只有30种被加载。我试图获取所有链接和页面，但只显示30个产品，单击webdriver中的某个位置，然后加载其余产品。如何打印所有产品的链接？提前谢谢！！

from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

from bs4 import BeautifulSoup

HOME_PAGE_URL = "https://www.talabat.com/uae/top-selling"
PATIENCE_TIME = 60
LOAD_MORE_XPATH = '//*[@id="comment-ajx"]/div'

driver = webdriver.Chrome(executable_path='C:\\Users\\Mansi Dhingra\\Downloads\\chromedriver.exe')
driver.get(HOME_PAGE_URL)
soup=BeautifulSoup(driver.page_source)

driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
# sleep for 30s
res=[]

results = driver.find_elements_by_xpath("/html/body/div[3]/ui-view/section/div/div[2]/div/div[2]/div/div[2]")

html_code = driver.find_element_by_tag_name("section").text
print(html_code)
for res in results:
    link=res.find_elements_by_tag_name('a')
    for x in link:
        product_link = x.get_attribute("href")

        print(product_link)
print(results)

Answer 1

主要的一点是，selenium在页面加载所有项目之前读取页面，您需要等待。

只读文档：https://selenium-python.readthedocs.io/waits.html

选择适合您的最佳条件，并为之努力。