使用model_main.py训练目标检测失败，Windows致命异常：访问违规

Question

我试图用model_main.py文件训练目标检测模型。我可以在ubuntu环境下训练这个没有任何问题，但是现在移动到赢10（因为我在那台电脑上有一个GeForece 1080Ti），现在我有麻烦了。训练可以开始并做得很好，直到我得到这些错误的第一个检查点（我也可以重新启动，它从最后一个检查点连续训练，但在保存下一个检查点后再次失败...）：

所以从…运行这个命令。。。\models master\research\object\u检测文件夹

python model_main.py --model_dir=training --pipeline_config_path=training/faster_rcnn_inception_v2_pets.config -–num_train_steps=20000 --sample_1_of_n_eval_examples=2 --alsologtostderr

产生这个：

INFO： tensorflow：将46040的检查点保存到训练中\model.ckpt.我0307 10:01:21.055022 8112basic_session_run_hooks.py:606]将46040的检查点保存到训练中\model.ckpt.警告：tensorflow： from C：\用户\Zsetszko21\Anaconda3\envs\tf_env_Ti\lib\site-包\tensorflow\python\训练\saver.py:960：remove_checkpoint（tensorflow.python.training.checkpoint_management）已弃用，并将在未来版本中删除。更新说明：使用标准文件API删除带有此前缀的文件。W0307 10:01:22.363223 8112deprecation.py:323]从C：\用户\Zsetszko21\Anaconda3\envs\tf_env_Ti\lib\site-包\tensorflow\python\训练\saver.py:960：remove_checkpoint（来自tensorflow.python.training.checkpoint_management）已弃用，并将在未来版本中删除。更新说明：使用标准文件API删除带有此前缀的文件。Windows致命异常：访问违规

线程0x000023b4（最近的调用优先）：文件"C：\用户\Zsetszko21\Anaconda3\envs\tf_env_Ti\lib\threading.py"，等待文件"C：\用户\Zsetszko21\Anaconda3\envs\tf_env_Ti\lib\queue.py"中的第296行，获取文件"C：\用户\Zsetszko21"中的第170行\anaconda3\envs\tf_env_Ti\lib\site-包\tensorflow\python\总结\作家\event_file_writer.py"，运行文件"C：\用户\Zsetszko21\Anaconda3\envs\tf_env_Ti\lib\threading.py"中的第159行，_bootstrap_inner文件"C：\用户\Zsetszko21\Anaconda3\envs\tf_env_Ti\lib\<-plhd--4/>"，_bootstrap第890行

我的配置文件：

# Faster R-CNN with Inception v2, configured for Oxford-IIIT Pets Dataset.
# Users should configure the fine_tune_checkpoint field in the train config as
# well as the label_map_path and input_path fields in the train_input_reader and
# eval_input_reader. Search for "PATH_TO_BE_CONFIGURED" to find the fields that
# should be configured.

model {
  faster_rcnn {
    num_classes: 2
    image_resizer {
      keep_aspect_ratio_resizer {
        min_dimension: 600
        max_dimension: 1024
      }
    }
    feature_extractor {
      type: 'faster_rcnn_inception_v2'
      first_stage_features_stride: 16
    }
    first_stage_anchor_generator {
      grid_anchor_generator {
        scales: [0.25, 0.5, 1.0, 2.0]
        aspect_ratios: [0.5, 1.0, 2.0]
        height_stride: 16
        width_stride: 16
      }
    }
    first_stage_box_predictor_conv_hyperparams {
      op: CONV
      regularizer {
        l2_regularizer {
          weight: 0.0
        }
      }
      initializer {
        truncated_normal_initializer {
          stddev: 0.01
        }
      }
    }
    first_stage_nms_score_threshold: 0.0
    first_stage_nms_iou_threshold: 0.7
    first_stage_max_proposals: 300
    first_stage_localization_loss_weight: 2.0
    first_stage_objectness_loss_weight: 1.0
    initial_crop_size: 14
    maxpool_kernel_size: 2
    maxpool_stride: 2
    second_stage_box_predictor {
      mask_rcnn_box_predictor {
        use_dropout: true
        dropout_keep_probability: 0.95
        fc_hyperparams {
          op: FC
          regularizer {
            l2_regularizer {
              weight: 0.0
            }
          }
          initializer {
            variance_scaling_initializer {
              factor: 1.0
              uniform: true
              mode: FAN_AVG
            }
          }
        }
      }
    }
    second_stage_post_processing {
      batch_non_max_suppression {
        score_threshold: 0.0
        iou_threshold: 0.6
        max_detections_per_class: 100
        max_total_detections: 300
      }
      score_converter: SOFTMAX
    }
    second_stage_localization_loss_weight: 2.0
    second_stage_classification_loss_weight: 1.0
  }
}

train_config: {
  batch_size: 1
  optimizer {
    momentum_optimizer: {
      learning_rate: {
          exponential_decay_learning_rate {
          initial_learning_rate: 0.00200000018999
          decay_steps: 1000
          decay_factor: 0.989999988079
        }
        #manual_step_learning_rate {
        #  initial_learning_rate: 0.0002
        #  schedule {
        #    step: 100000
        #    learning_rate: .002
        #  }

        #}

      }
      momentum_optimizer_value: 0.9
    }
    use_moving_average: false
  }
  gradient_clipping_by_norm: 10.0
  fine_tune_checkpoint:  "A:\\PPEVision\\trainer\\models-master\\research\\object_detection\\faster_rcnn_inception_v2_coco_2018_01_28\\model.ckpt"
  from_detection_checkpoint: true
  load_all_detection_checkpoint_vars: true
  # Note: The below line limits the training process to 200K steps, which we
  # empirically found to be sufficient enough to train the pets dataset. This
  # effectively bypasses the learning rate schedule (the learning rate will
  # never decay). Remove the below line to train indefinitely.
  num_steps: 200000
  data_augmentation_options {
    random_horizontal_flip {
    }
  }
}


train_input_reader: {
  tf_record_input_reader {
    input_path: "A:\\PPEVision\\trainer\\models-master\\research\\object_detection\\train.record"
  }
  label_map_path: "A:\\PPEVision\\trainer\\models-master\\research\\object_detection\\training\\labelmap.pbtxt"
}

eval_config: {
  metrics_set: "coco_detection_metrics"
  num_examples: 1000
  num_visualizations: 1000
  visualization_export_dir: "A:\\PPEVision\\trainer\\models-master\\research\\object_detection\\eval"
  eval_interval_secs: 120
}

eval_input_reader: {
  tf_record_input_reader {
    input_path: "A:\\PPEVision\\trainer\\models-master\\research\\object_detection\\test.record"
  }
  label_map_path: "A:\\PPEVision\\trainer\\models-master\\research\\object_detection\\labelmap.pbtxt"
  shuffle: true
  num_readers: 1
}

我还添加了这些行，以防止任何OOMmodel_main.py：

  session_config = tf.ConfigProto()
  session_config.gpu_options.per_process_gpu_memory_fraction = 0.7    # replace this field with whichever real number you prefer
                                                                    # also gives a workaround to specify RAM usage
  config = tf.estimator.RunConfig(model_dir=FLAGS.model_dir, session_config=session_config)

我的规格：

Win10 Geforce GTX 1080Ti 11Gb 32Gb RAM i5-7500 3Ghz CPU Tensorflow 1.14-gpu使用conda env创建------------------------------------------------------------------------------------------NVIDIA-SMI 442.50驱动程序版本：442.50 CUDA版本：10.2-------------------------------------------------------------------------------------gpu名称TCC/WDDM总线Id Disp。A |挥发性解吸剂。ECC | |风扇温度性能Pwr：使用率/上限|内存使用率| GPU Util Compute M.| |=============================================================================================================================================0 GeForce GTX 108。。。WDDM | 00000000:01:00.0 On | N/A | 23%36C P8 13W/250W | 449MiB/11264MiB | 0%默认值|-------------------------------------------------------------------------------------------------------------

Answer 1

我也有类似的问题，看来我不得不用label_map.pbtxt.txt.代替label_map.pbtxt，也许用 /.代替