BrokenPipeError: [Errno 32] Broken pipe

BrokenPipeError: [Errno 32] Broken pipe

前言：今天在训练yolov5.6.1版本，突然出现BrokenPipeError: [Errno 32] Broken pipe错误。

一、 运行命令python train.py 出现如下错误

Traceback (most recent call last):
  File "train.py", line 643, in <module>
    main(opt)
  File "train.py", line 539, in main
    train(opt.hyp, opt, device, callbacks)
  File "train.py", line 237, in train
    prefix=colorstr('test: '))[0]
  File "D:\liufq\yolov5-6.1\utils\datasets.py", line 122, in create_dataloader
    collate_fn=LoadImagesAndLabels.collate_fn4 if quad else LoadImagesAndLabels.collate_fn), dataset
  File "D:\liufq\yolov5-6.1\utils\datasets.py", line 134, in __init__
    self.iterator = super().__iter__()
  File "E:\Anaconda3\envs\yolov550\lib\site-packages\torch\utils\data\dataloader.py", line 359, in __iter__
    return self._get_iterator()
  File "E:\Anaconda3\envs\yolov550\lib\site-packages\torch\utils\data\dataloader.py", line 305, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "E:\Anaconda3\envs\yolov550\lib\site-packages\torch\utils\data\dataloader.py", line 918, in __init__
    w.start()
  File "E:\Anaconda3\envs\yolov550\lib\multiprocessing\process.py", line 112, in start
    self._popen = self._Popen(self)
  File "E:\Anaconda3\envs\yolov550\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "E:\Anaconda3\envs\yolov550\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "E:\Anaconda3\envs\yolov550\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
    reduction.dump(process_obj, to_child)
  File "E:\Anaconda3\envs\yolov550\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
BrokenPipeError: [Errno 32] Broken pipe

二、对比之前的版本的代码，没有发现错误。在错误里面dataloader.py报错。

def create_dataloader(path, imgsz, batch_size, stride, single_cls=False, hyp=None, augment=False, cache=False, pad=0.0,
                      rect=False, rank=-1, workers=8, image_weights=False, quad=False, prefix='', shuffle=False):
    if rect and shuffle:
        LOGGER.warning('WARNING: --rect is incompatible with DataLoader shuffle, setting shuffle=False')
        shuffle = False
    with torch_distributed_zero_first(rank):  # init dataset *.cache only once if DDP
        dataset = LoadImagesAndLabels(path, imgsz, batch_size,
                                      augment=augment,  # augmentation
                                      hyp=hyp,  # hyperparameters
                                      rect=rect,  # rectangular batches
                                      cache_images=cache,
                                      single_cls=single_cls,
                                      stride=int(stride),
                                      pad=pad,
                                      image_weights=image_weights,
                                      prefix=prefix)

    batch_size = min(batch_size, len(dataset))
    nd = torch.cuda.device_count()  # number of CUDA devices
    nw = min([os.cpu_count() // max(nd, 1), batch_size if batch_size > 1 else 0, workers])  # number of workers
    sampler = None if rank == -1 else distributed.DistributedSampler(dataset, shuffle=shuffle)
    loader = DataLoader if image_weights else InfiniteDataLoader  # only DataLoader allows for attribute updates
    return loader(dataset,
                  batch_size=batch_size,
                  shuffle=shuffle and sampler is None,
                  num_workers=nw,
                  sampler=sampler,
                  pin_memory=True,
                  collate_fn=LoadImagesAndLabels.collate_fn4 if quad else LoadImagesAndLabels.collate_fn), dataset

三、查看官方的Dataload函数

num_workers (int, optional) – how many subprocesses to use for data loading. 0 
means that the data will be loaded in the main process. (default: 0)

该参数是指在进行数据集加载时，启用的线程数目。

四、在参数里面设置 num_works = 0

问题解决。

train: weights=, cfg=./models/yolov5s-se-ghost.yaml, data=data\PV_Data\PV.yaml, hyp=data\hyps\hyp.scratch-low.yaml, epochs=300, batch_size=16, imgsz=640, rect=False, resume=False, nosave=False, noval=False, noautoanchor=False, evolve=None, bucket=, cache=None, image_weights=False, device=, multi_scale=False, single_cls=False, optimizer=SGD, sync_bn=False, workers=0, project=runs\train, name=exp, exist_ok=False, quad=False, cos_lr=False, label_smoothing=0.0, patience=100, freeze=[0], save_period=-1, local_rank=-1, entity=None, upload_dataset=False, bbox_interval=-1, artifact_alias=latest
github: skipping check (not a git repository), for updates see https://github.com/ultralytics/yolov5
YOLOv5  2022-2-22 torch 1.9.0+cu102 CUDA:0 (GeForce RTX 2080 Ti, 11264MiB)

hyperparameters: lr0=0.01, lrf=0.01, momentum=0.937, weight_decay=0.0005, warmup_epochs=3.0, warmup_momentum=0.8, warmup_bias_lr=0.1, box=0.05, cls=0.5, cls_pw=1.0, obj=1.0, obj_pw=1.0, iou_t=0.2, anchor_t=4.0, fl_gamma=0.0, hsv_h=0.015, hsv_s=0.7, hsv_v=0.4, degrees=0.0, translate=0.1, scale=0.5, shear=0.0, perspective=0.0, flipud=0.0, fliplr=0.5, mosaic=1.0, mixup=0.0, copy_paste=0.0
Weights & Biases: run 'pip install wandb' to automatically track and visualize YOLOv5  runs (RECOMMENDED)
TensorBoard: Start with 'tensorboard --logdir runs\train', view at http://localhost:6006/
Overriding model.yaml nc=2 with nc=3

                 from  n    params  module                                  arguments
  0                -1  1      3520  models.common.Conv                      [3, 32, 6, 2, 2]
  1                -1  1     18560  models.common.Conv                      [32, 64, 3, 2]
  2                -1  1      3440  models.common.GhostBottleneck           [64, 64, 3, 1]
  3                -1  1     18784  models.common.GhostBottleneck           [64, 128, 3, 2]
  4                -1  3     32928  models.common.GhostBottleneck           [128, 128, 3, 1]
  5                -1  1      2184  models.common.SElayer                   [128, 16]
  6                -1  1     66240  models.common.GhostBottleneck           [128, 256, 3, 2]
  7                -1  3    115008  models.common.GhostBottleneck           [256, 256, 3, 1]
  8                -1  1      8464  models.common.SElayer                   [256, 16]
  9                -1  1   1180672  models.common.Conv                      [256, 512, 3, 2]
 10                -1  1    656896  models.common.SPPF                      [512, 512, 5]
 11                -1  1     33312  models.common.SElayer                   [512, 16]
 12                -1  1    142208  models.common.GhostBottleneck           [512, 512, 3, 1]
 13                -1  1      1024  models.common.DWConv                    [512, 256, 1, 1]
 14                -1  1         0  torch.nn.modules.upsampling.Upsample    [None, 2, 'nearest']
 15           [-1, 8]  1         0  models.common.Concat                    [1]
 16                -1  1      5120  models.common.DWConv                    [512, 256, 3, 1]
 17                -1  1     38336  models.common.GhostBottleneck           [256, 256, 3, 1]
 18                -1  1       512  models.common.DWConv                    [256, 128, 1, 1]
 19                -1  1         0  torch.nn.modules.upsampling.Upsample    [None, 2, 'nearest']
 20           [-1, 5]  1         0  models.common.Concat                    [1]
 21                -1  1      2560  models.common.DWConv                    [256, 128, 3, 1]
 22                -1  1     10976  models.common.GhostBottleneck           [128, 128, 3, 1]
 23                -2  1      1408  models.common.DWConv                    [128, 128, 3, 2]
 24          [-1, 18]  1         0  models.common.Concat                    [1]
 25                -1  1     38336  models.common.GhostBottleneck           [256, 256, 3, 1]
 26                -2  1      2816  models.common.DWConv                    [256, 256, 3, 2]
 27          [-1, 13]  1         0  models.common.Concat                    [1]
 28                -1  1    142208  models.common.GhostBottleneck           [512, 512, 3, 1]
 29      [22, 25, 28]  1     21576  models.yolo.Detect                      [3, [[10, 13, 16, 30, 33, 23], [30, 61, 62, 45, 59, 119], [116, 90, 156, 198, 373, 326]], [128, 256, 512]]
E:\Anaconda3\envs\yolov550\lib\site-packages\torch\_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at  ..\aten\src\ATen\native\BinaryOps.cpp:467.)
  return torch.floor_divide(self, other)
Model Summary: 412 layers, 2547088 parameters, 2547088 gradients, 5.5 GFLOPs

Scaled weight_decay = 0.0005
optimizer: SGD with parameter groups 73 weight (no decay), 82 weight, 82 bias