从服务器链接下载压缩的csv文件并读入pandas

Question

我一直试图使用请求库从服务器主机URL下载压缩csv。

我还尝试使用这里建议的zipfile库：在python中读取csv压缩文件

并尝试在read_csv中设置编码和压缩。

下面是用于非zipped服务器文件的代码：

response = requests.get(url, auth=HTTPBasicAuth(un, pw), stream=True, verify = False)
dfs = pd.read_csv(response.raw)

request = get(url, auth=HTTPBasicAuth(un, pw), stream=True, verify=False)
zip_file = ZipFile(BytesIO(request.content))
files = zip_file.namelist()
with gzip.open(files[0], 'rb') as csvfile:
    csvreader = csv.reader(csvfile)
    for row in csvreader:
        print(row)

Answer 1

以下是一种方法：

import pandas as pd
import requests
from requests.auth import HTTPBasicAuth
from zipfile import ZipFile
import io

# Example dataset
url = 'https://www.stats.govt.nz/assets/Uploads/Retail-trade-survey/Retail-trade-survey-September-2020-quarter/Download-data/retail-trade-survey-september-2020-quarter-csv.zip'

response = requests.get(url, auth=HTTPBasicAuth(un, pw), stream=True, verify=False)
with ZipFile(io.BytesIO(response.content)) as myzip:
    with myzip.open(myzip.namelist()[0]) as myfile:
        df = pd.read_csv(myfile)

print(df)

如果要读取多csv压缩文件中的特定csv，请将myzip.namelist()[0]替换为要读取的文件。如果不知道其名称，可以使用print(ZipFile(io.bytesio(response.content)))检查zip文件内容