写入CSV时将列添加到CSV

Question

我正在用csv快速编写以下数据：

name first file parsed  
STEP ID  ELEMENT_ID  Fatigue SW  Fatigue F1  Fatigue F3
Step 10  10000       1.30E-07    1.51E-06    2.15E-06

当我完成解析第一个文件并启动第二个文件时，我想添加更多列，如下所示：

name first file parsed                                   name first file

parsed
STEP ID ELEMENT_ID Fatigue SW Fatigue F1 Fatigue F3 Fatigue SW
Fatigue F1 Fatigue F3
Step 10 10000 1.30E-07 1.51E-06 2.15E-06 1.30E-07
1.51E-06 2.15E-06

我正在读取的文件大小为2Gb，因此我无力创建列表，我需要在解析时编写。

有什么建议？

Answer 1

您不能将列添加到现有的CSV文件中；恐怕您将不得不重写整个文件。

您可以使用以下上下文管理器来简化文件的替换：

from contextlib import contextmanager
import io
import os


@contextmanager
def inplace(filename, mode='r', buffering=-1, encoding=None, errors=None,
            newline=None, backup_extension=None):
    """Allow for a file to be replaced with new content.

    yields a tuple of (readable, writable) file objects, where writable
    replaces readable.

    If an exception occurs, the old file is restored, removing the
    written data.

    mode should *not* use 'w', 'a' or '+'; only read-only-modes are supported.

    """

    # move existing file to backup, create new file with same permissions
    # borrowed extensively from the fileinput module
    if set(mode) & set('wa+'):
        raise ValueError('Only read-only file modes can be used')

    backupfilename = filename + (backup_extension or os.extsep + 'bak')
    try:
        os.unlink(backupfilename)
    except os.error:
        pass
    os.rename(filename, backupfilename)
    readable = io.open(backupfilename, mode, buffering=buffering,
                       encoding=encoding, errors=errors, newline=newline)
    try:
        perm = os.fstat(readable.fileno()).st_mode
    except OSError:
        writable = open(filename, 'w' + mode.replace('r', ''),
                        buffering=buffering, encoding=encoding, errors=errors,
                        newline=newline)
    else:
        os_mode = os.O_CREAT | os.O_WRONLY | os.O_TRUNC
        if hasattr(os, 'O_BINARY'):
            os_mode |= os.O_BINARY
        fd = os.open(filename, os_mode, perm)
        writable = io.open(fd, "w" + mode.replace('r', ''), buffering=buffering,
                           encoding=encoding, errors=errors, newline=newline)
        try:
            if hasattr(os, 'chmod'):
                os.chmod(filename, perm)
        except OSError:
            pass
    try:
        yield readable, writable
    except Exception:
        # move backup back
        try:
            os.unlink(filename)
        except os.error:
            pass
        os.rename(backupfilename, filename)
        raise
    finally:
        readable.close()
        writable.close()
        try:
            os.unlink(backupfilename)
        except os.error:
            pass

将此与csv模块一起使用以添加列：

with inplace(csvfilename, 'rb') as (infh, outfh):
    reader = csv.reader(infh)
    writer = csv.writer(outfh)

    for row in reader:
        row += ['new', 'column']
        writer.writerow(row)