neutron-openvswitch-agent 服务启动失败

1.openvswitch-agent.log日志报错信息，重启neutron-openvswitch-agent服务成功，但是服务状态无法启动。

2019-06-13 12:30:22.510 4208 INFO neutron.agent.securitygroups_rpc [req-8edf01ca-a3fc-43c6-bdce-44083eb1a518 - - - - -] Refresh firewall rules
2019-06-13 12:30:22.646 4208 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-8edf01ca-a3fc-43c6-bdce-44083eb1a518 - - - - -] Configuration for devices up [] and devices down [] complet
ed.
2019-06-13 12:30:49.938 4208 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-8edf01ca-a3fc-43c6-bdce-44083eb1a518 - - - - -] Agent caught SIGTERM, quitting daemon loop.
2019-06-13 12:30:50.049 4208 ERROR neutron.agent.linux.async_process [-] Error received from [ovsdb-client monitor Interface name,ofport,external_ids --format=json]: None
2019-06-13 12:30:50.049 4208 ERROR neutron.agent.linux.async_process [-] Process [ovsdb-client monitor Interface name,ofport,external_ids --format=json] dies due to the error: None
2019-06-13 12:30:50.060 4208 INFO oslo_rootwrap.client [req-8edf01ca-a3fc-43c6-bdce-44083eb1a518 - - - - -] Stopping rootwrap daemon process with pid=4241
2019-06-13 12:30:51.791 8239 INFO neutron.common.config [-] Logging enabled!
2019-06-13 12:30:51.792 8239 INFO neutron.common.config [-] /usr/bin/neutron-openvswitch-agent version 8.1.2
2019-06-13 12:30:51.800 8239 WARNING oslo_config.cfg [-] Option "prevent_arp_spoofing" from group "AGENT" is deprecated for removal.  Its value may be silently ignored in the future.
2019-06-13 12:30:51.924 8239 INFO oslo_rootwrap.client [-] Spawned new rootwrap daemon process with pid=8279
2019-06-13 12:30:52.013 8239 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-f04c0a10-18f2-45d1-93d7-6056f75a81f2 - - - - -] Mapping physical network physnet1 to bridge br-ex
2019-06-13 12:30:52.124 8239 INFO neutron.agent.l2.extensions.manager [req-f04c0a10-18f2-45d1-93d7-6056f75a81f2 - - - - -] Loaded agent extensions: []
2019-06-13 12:30:52.256 8239 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_dvr_neutron_agent [req-f04c0a10-18f2-45d1-93d7-6056f75a81f2 - - - - -] L2 Agent operating in DVR Mode with MAC fa:16:3f:d9:0a:
54
2019-06-13 12:30:52.519 8239 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] Agent has just been revived. Doing a full sync.
2019-06-13 12:30:52.753 8239 CRITICAL neutron [req-f04c0a10-18f2-45d1-93d7-6056f75a81f2 - - - - -] KeyError: 6
2019-06-13 12:30:52.753 8239 ERROR neutron Traceback (most recent call last):
2019-06-13 12:30:52.753 8239 ERROR neutron   File "/usr/bin/neutron-openvswitch-agent", line 10, in <module>
2019-06-13 12:30:52.753 8239 ERROR neutron     sys.exit(main())
2019-06-13 12:30:52.753 8239 ERROR neutron   File "/usr/lib/python2.7/site-packages/neutron/cmd/eventlet/plugins/ovs_neutron_agent.py", line 20, in main
2019-06-13 12:30:52.753 8239 ERROR neutron     agent_main.main()
2019-06-13 12:30:52.753 8239 ERROR neutron   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/main.py", line 49, in main
2019-06-13 12:30:52.753 8239 ERROR neutron     mod.main()
2019-06-13 12:30:52.753 8239 ERROR neutron   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/ovs_ofctl/main.py", line 36, in main
2019-06-13 12:30:52.753 8239 ERROR neutron     ovs_neutron_agent.main(bridge_classes)
2019-06-13 12:30:52.753 8239 ERROR neutron   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 2119, in main
2019-06-13 12:30:52.753 8239 ERROR neutron     agent = OVSNeutronAgent(bridge_classes, cfg.CONF)
2019-06-13 12:30:52.753 8239 ERROR neutron   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 274, in __init__
2019-06-13 12:30:52.753 8239 ERROR neutron     self._restore_local_vlan_map()
2019-06-13 12:30:52.753 8239 ERROR neutron   File "/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 349, in _restore_local_vlan_map
2019-06-13 12:30:52.753 8239 ERROR neutron     self.available_local_vlans.remove(local_vlan)
2019-06-13 12:30:52.753 8239 ERROR neutron KeyError: 6
2019-06-13 12:30:52.753 8239 ERROR neutron
2019-06-13 12:30:52.756 8239 INFO oslo_rootwrap.client [req-f04c0a10-18f2-45d1-93d7-6056f75a81f2 - - - - -] Stopping rootwrap daemon process with pid=8279
2019-07-18 02:56:51.491 8710 INFO neutron.common.config [-] Logging enabled!
2019-07-18 02:56:51.497 8710 INFO neutron.common.config [-] /usr/bin/neutron-openvswitch-agent version 8.1.2
2019-07-18 02:56:51.506 8710 WARNING oslo_config.cfg [-] Option "prevent_arp_spoofing" from group "AGENT" is deprecated for removal.  Its value may be silently ignored in the future.
2019-07-18 02:56:51.747 8710 INFO oslo_rootwrap.client [-] Spawned new rootwrap daemon process with pid=8968
2019-07-18 02:56:51.951 8710 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-7210695f-4b45-4c29-b965-2cef22a592a8 - - - - -] Mapping physical network physnet1 to bridge br-ex
2019-07-18 02:56:52.076 8710 INFO neutron.agent.l2.extensions.manager [req-7210695f-4b45-4c29-b965-2cef22a592a8 - - - - -] Loaded agent extensions: []

2.查看计算节点br-ex流表

[root@compute-se122 ~]# ovs-ofctl dump-flows br-ex
NXST_FLOW reply (xid=0x4):
cookie=0x9aec1bf161d6039e, duration=59133.523s, table=0, n_packets=3, n_bytes=416, idle_age=37761, priority=1,in_port=1 actions=resubmit(,2)
cookie=0x9aec1bf161d6039e, duration=59125.488s, table=0, n_packets=2, n_bytes=433, idle_age=37761, priority=1,in_port=2 actions=resubmit(,4)
cookie=0x9aec1bf161d6039e, duration=59125.463s, table=0, n_packets=0, n_bytes=0, idle_age=59125, priority=1,in_port=3 actions=resubmit(,4)
cookie=0x9aec1bf161d6039e, duration=59133.521s, table=0, n_packets=0, n_bytes=0, idle_age=59133, priority=0 actions=drop
cookie=0x9aec1bf161d6039e, duration=59133.518s, table=2, n_packets=1, n_bytes=42, idle_age=37766, priority=1,arp,dl_dst=ff:ff:ff:ff:ff:ff actions=resubmit(,21)
cookie=0x9aec1bf161d6039e, duration=59133.515s, table=2, n_packets=2, n_bytes=374, idle_age=37761, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20)
cookie=0x9aec1bf161d6039e, duration=59133.512s, table=2, n_packets=0, n_bytes=0, idle_age=59133, priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,22)
cookie=0x9aec1bf161d6039e, duration=59133.511s, table=3, n_packets=0, n_bytes=0, idle_age=59133, priority=0 actions=drop
cookie=0x9aec1bf161d6039e, duration=59131.764s, table=4, n_packets=2, n_bytes=433, idle_age=37761, priority=1,tun_id=0x64 actions=mod_vlan_vid:1,resubmit(,10)
cookie=0x9aec1bf161d6039e, duration=59133.509s, table=4, n_packets=0, n_bytes=0, idle_age=59133, priority=0 actions=drop
cookie=0x9aec1bf161d6039e, duration=59133.507s, table=6, n_packets=0, n_bytes=0, idle_age=59133, priority=0 actions=drop
cookie=0x9aec1bf161d6039e, duration=59133.505s, table=10, n_packets=2, n_bytes=433, idle_age=37761, priority=1

ovs执行emer-reset命令，清空流表

[root@compute-se122 ~]# ovs-vsctl emer-reset

3.重启 neutron-openvswitch-agent 后，服务正常，流表正常

[root@compute-se122 ~]# ovs-ofctl dump-flows br-ex
NXST_FLOW reply (xid=0x4):
cookie=0x884e11be28aa5750, duration=9215.670s, table=0, n_packets=1252, n_bytes=111040, idle_age=1140, priority=2,in_port=2 actions=resubmit(,1)
cookie=0x884e11be28aa5750, duration=9215.833s, table=0, n_packets=0, n_bytes=0, idle_age=9215, priority=0 actions=NORMAL
cookie=0x884e11be28aa5750, duration=9215.664s, table=0, n_packets=16413, n_bytes=4798109, idle_age=0, priority=1 actions=resubmit(,3)
cookie=0x884e11be28aa5750, duration=9215.659s, table=1, n_packets=1252, n_bytes=111040, idle_age=1140, priority=0 actions=resubmit(,2)
cookie=0x884e11be28aa5750, duration=9211.452s, table=2, n_packets=0, n_bytes=0, idle_age=9211, priority=4,in_port=2,dl_vlan=1 actions=mod_vlan_vid:109,NORMAL
cookie=0x884e11be28aa5750, duration=9211.394s, table=2, n_packets=0, n_bytes=0, idle_age=9211, priority=4,in_port=2,dl_vlan=3 actions=mod_vlan_vid:105,NORMAL
cookie=0x884e11be28aa5750, duration=9211.362s, table=2, n_packets=0, n_bytes=0, idle_age=9211, priority=4,in_port=2,dl_vlan=4 actions=mod_vlan_vid:100,NORMAL
cookie=0x884e11be28aa5750, duration=9211.322s, table=2, n_packets=0, n_bytes=0, idle_age=9211, priority=4,in_port=2,dl_vlan=5 actions=mod_vlan_vid:128,NORMAL
cookie=0x884e11be28aa5750, duration=9211.184s, table=2, n_packets=0, n_bytes=0, idle_age=9211, priority=4,in_port=2,dl_vlan=9 actions=mod_vlan_vid:104,NORMAL
cookie=0x884e11be28aa5750, duration=9211.138s, table=2, n_packets=0, n_bytes=0, idle_age=9211, priority=4,in_port=2,dl_vlan=2 actions=mod_vlan_vid:117,NORMAL
cookie=0x884e11be28aa5750, duration=9211.088s, table=2, n_packets=0, n_bytes=0, idle_age=9211, priority=4,in_port=2,dl_vlan=6 actions=mod_vlan_vid:126,NORMAL
cookie=0x884e11be28aa5750, duration=9211.050s, table=2, n_packets=0, n_bytes=0, idle_age=9211, priority=4,in_port=2,dl_vlan=7 actions=mod_vlan_vid:155,NORMAL
cookie=0x884e11be28aa5750, duration=9210.813s, table=2, n_packets=0, n_bytes=0, idle_age=9210, priority=4,in_port=2,dl_vlan=8 actions=mod_vlan_vid:103,NORMAL
cookie=0x884e11be28aa5750, duration=9210.782s, table=2, n_packets=0, n_bytes=0, idle_age=9210, priority=4,in_port=2,dl_vlan=10 actions=mod_vlan_vid:157,NORMAL
cookie=0x884e11be28aa5750, duration=9181.622s, table=2, n_packets=1248, n_bytes=110700, idle_age=1140, priority=4,in_port=2,dl_vlan=11 actions=mod_vlan_vid:106,NORMAL
cookie=0x884e11be28aa5750, duration=9215.653s, table=2, n_packets=4, n_bytes=340, idle_age=9182, priority=2,in_port=2 actions=drop
cookie=0x884e11be28aa5750, duration=9215.626s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:12:04:3b actions=output:2
cookie=0x884e11be28aa5750, duration=9215.616s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:14:b9:ac actions=output:2
cookie=0x884e11be28aa5750, duration=9215.605s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:34:c7:25 actions=output:2
cookie=0x884e11be28aa5750, duration=9215.595s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:45:d3:97 actions=output:2
cookie=0x884e11be28aa5750, duration=9215.585s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:47:05:ea actions=output:2
cookie=0x884e11be28aa5750, duration=9215.574s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:55:90:99 actions=output:2
cookie=0x884e11be28aa5750, duration=9215.564s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:5c:0f:3e actions=output:2
cookie=0x884e11be28aa5750, duration=9215.554s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:5f:60:aa actions=output:2
cookie=0x884e11be28aa5750, duration=9215.544s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:7b:6f:a7 actions=output:2
cookie=0x884e11be28aa5750, duration=9215.534s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:9d:0d:7f actions=output:2
cookie=0x884e11be28aa5750, duration=9215.524s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:a3:a0:7b actions=output:2
cookie=0x884e11be28aa5750, duration=9215.513s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:a6:cc:f0 actions=output:2
cookie=0x884e11be28aa5750, duration=9215.503s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:b1:4f:f9 actions=output:2
cookie=0x884e11be28aa5750, duration=9215.492s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:b5:c8:12 actions=output:2
cookie=0x884e11be28aa5750, duration=9215.482s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:b9:52:49 actions=output:2
cookie=0x884e11be28aa5750, duration=9215.472s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:c4:68:0a actions=output:2
cookie=0x884e11be28aa5750, duration=9215.461s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:c7:4e:7a actions=output:2
cookie=0x884e11be28aa5750, duration=9215.451s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:ce:c1:03 actions=output:2
cookie=0x884e11be28aa5750, duration=9215.441s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:d8:1e:65 actions=output:2
cookie=0x884e11be28aa5750, duration=9215.431s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:e8:f8:53 actions=output:2
cookie=0x884e11be28aa5750, duration=9215.420s, table=3, n_packets=0, n_bytes=0, idle_age=9215, priority=2,dl_src=fa:16:3f:f8:38:99 actions=output:2
cookie=0x884e11be28aa5750, duration=9215.649s, table=3, n_packets=16413, n_bytes=4798109, idle_age=0, priority=1 actions=NORMAL

查看neutron-openvswitch-agent服务状态

[root@compute-se122 ~]# systemctl status neutron-openvswitch-agent
● neutron-openvswitch-agent.service - OpenStack Neutron Open vSwitch Agent
   Loaded: loaded (/usr/lib/systemd/system/neutron-openvswitch-agent.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2020-01-20 11:16:15 CST; 2h 34min ago
Main PID: 26577 (neutron-openvsw)
   CGroup: /system.slice/neutron-openvswitch-agent.service
           ├─26577 /usr/bin/python2 /usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/openvs...
           ├─26586 sudo neutron-rootwrap-daemon /etc/neutron/rootwrap.conf
           ├─26587 /usr/bin/python2 /usr/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf
           ├─26675 sudo neutron-rootwrap /etc/neutron/rootwrap.conf ovsdb-client monitor Interface name,ofport,external_ids --format=json
           ├─26677 /usr/bin/python2 /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf ovsdb-client monitor Interface name,ofport,external_ids --format=json
           └─26680 /bin/ovsdb-client monitor Interface name,ofport,external_ids --format=json
Jan 20 11:16:22 compute-se122 ovs-ofctl[26824]: ovs|00001|ofp_util|INFO|normalization changed ofp_match, details:
Jan 20 11:16:22 compute-se122 ovs-ofctl[26824]: ovs|00002|ofp_util|INFO| pre: in_port=18,nw_proto=58,tp_src=136
Jan 20 11:16:22 compute-se122 ovs-ofctl[26829]: ovs|00001|ofp_util|INFO|normalization changed ofp_match, details:
Jan 20 11:16:22 compute-se122 ovs-ofctl[26829]: ovs|00002|ofp_util|INFO| pre: in_port=467,nw_proto=58,tp_src=136
Jan 20 11:16:22 compute-se122 ovs-ofctl[26834]: ovs|00001|ofp_util|INFO|normalization changed ofp_match, details:
Jan 20 11:16:22 compute-se122 ovs-ofctl[26834]: ovs|00002|ofp_util|INFO| pre: in_port=456,nw_proto=58,tp_src=136
Jan 20 11:16:22 compute-se122 ovs-ofctl[26834]: ovs|00003|ofp_util|INFO|post: in_port=456
Jan 20 11:16:51 compute-se122 ovs-vsctl[29656]: ovs|00001|vsctl|INFO|Called as /bin/ovs-vsctl --timeout=10 --oneline --format=json -- set Port qvo6105d110-42 other_config:segmentation_id=106 ot...work_type=vlan
Jan 20 11:16:51 compute-se122 ovs-vsctl[29863]: ovs|00001|vsctl|INFO|Called as /bin/ovs-vsctl --timeout=10 --oneline --format=json -- set Port qvo6105d110-42 other_config:segmentation_id=106 ot...work_type=vlan
Jan 20 11:16:51 compute-se122 ovs-vsctl[30081]: ovs|00001|vsctl|INFO|Called as /bin/ovs-vsctl --timeout=10 --oneline --format=json -- set Port qvo6105d110-42 tag=11
Hint: Some lines were ellipsized, use -l to show in full.