如何从7z压缩的文本文件中读取？

Question

我想从csv（文本）文件逐行读取（在Python 2.7中），该文件是7z压缩的。我不想解压缩整个（大）文件，而是流线。

我尝试pylzma.decompressobj()失败。我收到数据错误。请注意，此代码尚未逐行读取：

input_filename = r"testing.csv.7z"
with open(input_filename, 'rb') as infile:
    obj = pylzma.decompressobj()
    o = open('decompressed.raw', 'wb')
    obj = pylzma.decompressobj()
    while True:
        tmp = infile.read(1)
        if not tmp: break
        o.write(obj.decompress(tmp))
    o.close()

输出：

    o.write(obj.decompress(tmp))
ValueError: data error during decompression

Answer 1

这将允许您迭代行。它部分源自我在另一个问题的答案中找到的一些代码。

在该时间点（pylzma-0.5.0），该py7zlib模块未实现允许将存档成员作为字节流或字符流读取的API-其ArchiveFile类仅提供了一次read()解压缩并返回成员中未压缩数据的功能。鉴于此，最好的办法是通过Python生成器使用该缓冲区作为缓冲区迭代地返回字节或行。

下面是后者的操作，但是如果问题是存档 成员 文件本身很大，则可能没有帮助。

下面的代码应在Python 3.x和2.7中均可使用。

import io
import os
import py7zlib


class SevenZFileError(py7zlib.ArchiveError):
    pass

class SevenZFile(object):
    @classmethod
    def is_7zfile(cls, filepath):
        """ Determine if filepath points to a valid 7z archive. """
        is7z = False
        fp = None
        try:
            fp = open(filepath, 'rb')
            archive = py7zlib.Archive7z(fp)
            _ = len(archive.getnames())
            is7z = True
        finally:
            if fp: fp.close()
        return is7z

    def __init__(self, filepath):
        fp = open(filepath, 'rb')
        self.filepath = filepath
        self.archive = py7zlib.Archive7z(fp)

    def __contains__(self, name):
        return name in self.archive.getnames()

    def readlines(self, name, newline=''):
        r""" Iterator of lines from named archive member.

        `newline` controls how line endings are handled.

        It can be None, '', '\n', '\r', and '\r\n' and works the same way as it does
        in StringIO. Note however that the default value is different and is to enable
        universal newlines mode, but line endings are returned untranslated.
        """
        archivefile = self.archive.getmember(name)
        if not archivefile:
            raise SevenZFileError('archive member %r not found in %r' %
                                  (name, self.filepath))

        # Decompress entire member and return its contents iteratively.
        data = archivefile.read().decode()
        for line in io.StringIO(data, newline=newline):
            yield line


if __name__ == '__main__':

    import csv

    if SevenZFile.is_7zfile('testing.csv.7z'):
        sevenZfile = SevenZFile('testing.csv.7z')

        if 'testing.csv' not in sevenZfile:
            print('testing.csv is not a member of testing.csv.7z')
        else:
            reader = csv.reader(sevenZfile.readlines('testing.csv'))
            for row in reader:
                print(', '.join(row))