如何在python中从html表中刮取数据

Question

<tr class="even">
<td><strong><a href='../eagleweb/viewDoc.jsp?node=DOC186S8881'>DEED<br/>
2016002023</a></strong></td>
<td><a href='../eagleweb/viewDoc.jsp?node=DOC186S8881'><b> Recording Date: </b>01/12/2016 08:05:17 AM&nbsp;&nbsp;&nbsp;<b>Book Page: </b> <table cellspacing=0 width="100%"><tr><td width="50%"  valign="top"><b>Grantor:</b> ARELLANO ISAIAS</td><td width="50%"  valign="top"><b>Grantee:</b> ARELLANO ISAIAS, ARELLANO ALICIA</td></tr></table>
<b>Number Pages:</b> 3<br></a></td>
<td></td>
<td></td></tr>

我对python和刮擦是新手，请帮助我如何从这个表中刮擦数据。对于登录，请转到公共登录，然后输入收件人和收件人日期。

数据模型：数据模型具有以下特定顺序和大小写的列：“record_date”、“doc_number”、“doc_type”、“role”、“name”、“apn”、“transfer_amount”、“county”和“state”。“角色”列可以是“授权人”，也可以是“授权人”，这取决于名称分配的位置。如果设保人和受赠人有多个名称，请为每个名称添加新的行，并复制记录日期、文档编号、文档类型、角色和APN。

https://crarecords.sonomacounty.ca.gov/recorder/eagleweb/docsearchresults.jsp？searchid=0

Answer 1

您发布的html不包含数据模型中列出的所有列字段。但是，对于它确实包含的字段，这将生成一个python字典，您可以从该字典获取数据模型的字段：

import urllib.request
from bs4 import BeautifulSoup

url = "the_url_of_webpage_to_scrape" # Replace with the URL of your webpage

with urllib.request.urlopen(url) as response:
    html = response.read()

soup = BeautifulSoup(html, 'html.parser')

table = soup.find("tr", attrs={"class":"even"})

btags = [str(b.text).strip().strip(':') for b in table.find_all("b")]

bsibs = [str(b.next_sibling.replace(u'\xa0', '')).strip() for b in table.find_all('b')]

data = dict(zip(btags, bsibs))

data_model = {"record_date": None, "doc_number": None, "doc_type": None, "role": None, "name": None, "apn": None, "transfer_amount": None, "county": None, "state": None}

data_model["record_date"] = data['Recording Date']
data_model['role'] = data['Grantee']

print(data_model)

产出：

{'apn': None,
 'county': None,
 'doc_number': None,
 'doc_type': None,
 'name': None,
 'record_date': '01/12/2016 08:05:17 AM',
 'role': 'ARELLANO ISAIAS, ARELLANO ALICIA',
 'state': None,
 'transfer_amount': None}

您可以这样做：

print(data_model['record_date']) # 01/12/2016 08:05:17 AM
print(data_model['role'])        # ARELLANO ISAIAS, ARELLANO ALICIA