刮除包含：：的网页之前

Question

我的问题是当使用bs4刮超文本标记语言不能刮包含像::before。

我想知道该公司在一个页面中为哪些可持续发展目标做出了贡献。https://www.unglobalcompact.org/participation/report/cop/create-and-submit/active/395091但是复选标记在源代码中是不可见的。

我应该怎么做或者我可以用什么从网站上刮掉它？

Answer 1

这里根本不需要::bef或e::部分。选择的和未选择的元素有不同的类-选择的有selected_question，未选择的advanced_question。

您可以使用以下方法对其进行分析：

from bs4 import BeautifulSoup
import requests


url = "https://www.unglobalcompact.org/participation/report/cop/create-and-submit/active/395091"
response = requests.get(url)

soup = BeautifulSoup(response.content, "lxml")

questions = soup.select("ul.questionnaire > li.question_group")
for question in questions:
    question_text = question.get_text(strip=True)
    print(question_text)

    answers = question.find_next_siblings("li")
    for answer in answers:
        answer_text = answer.get_text(strip=True)
        is_selected = "selected_question" in answer.get("class", [])

        print(answer_text, is_selected)
    print("-----")

将打印：

Which of the following Sustainable Development Goals (SDGs) do the activities described in your COP address? [Select all that apply]
SDG 1: End poverty in all its forms everywhere False
SDG 2: End hunger, achieve food security and improved nutrition and promote sustainable agriculture False
SDG 3: Ensure healthy lives and promote well-being for all at all ages True
SDG 4: Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all False
...

请注意为所选答案打印的True。

我还注意到，如果选择html.parser作为解析器，则此代码不能正常工作。