登录
当前位置： 首页 > 工具软件 > rawip4j > 使用案例 >

Set ip IPv6 env (by quqi99)

公良渝
2023-12-01

基础 - ND协议的三个位

ND协议包中有三个位(Auto, Managed, Other)：

  • M bit (Managed Address Configuration), M bit如果是1,表示Clients要另外再去跟DHCPv6要IPv6 Prefix
  • O bit (Other Configuration), O bit如果是1,表示Clients要去跟DHCPv6要DNS(RDNSS)等其他信息
    这样：
  • slaas, Stateless autoconfiguration, A=1, M=0, O=0, 主机將只得到Router給的 Prefix, 无法取得DNS等资讯, 其他必须自己填写.
  • dhcpv6-stateful, A=0, M=1, O=1, 所有信息（IPv6 prefix, DNS等)都通过DHCPv6获得,客戶端主要使用UDP port 546, 而服務器端使用 UDP port 547
  • dhcpv6-stateless,A=1, M=0, O=1, 除了使用RA裡面的Prefix,其他如DNS等等信息会由DHCPv6 取得.

基础 - Neutron IPv6

ND协议包中有三个位(Auto, Managed, Other)：

  • A bit, 是否使用Router RA来配置IPv6 Prefix前缀. M bit, 置位时使用托管的DHCPv6
  • Server来配置IPv6 address, 不置位时都是使用radvd来提供 IPv6 prefix Other, 是否由DHCPv6
  • Server提供Other如DHCP信息

Neutron中有两个重要属性来支持IPv6 (ipv6_address_mode 与 ipv6_ra_mode). 若配置了ipv6_ra_mode, 将由radvd来模拟IPv6 Router, 如果不设置由使用external IPv6 router. ipv6_address_mode可为：

  • slaas, A=1, M=0, O=0, radvd提供IPv6 prefix，radvd不托管DHCPv6 server, 不提供DHCP信息. client自己填
  • dhcpv6-stateless, A=1, M=0, O=1, 由radvd提供IPv6 prefix, radvd指定外部DHCPv6去提供DHCP信息
  • dhcpv6-stateful, A=0, M=1, O=1, 所有信息(IPv6 prefix, DNS等)都由DHCPv6而不是radvd提供

eg: 下面是创建一个使用外部IPv6路由器并使用dhcpv6-stateless的例子:
neutron net-create --provider:network_type flat --provider:physical_network physnet1 --router:external=True ext_net
neutron subnet-create ext_net --name external-subnet-v6 --ip_version 6 --ipv6_address_mode dhcpv6-stateless --allocation-pool start=2001:db8:0:1::2,end=2001:db8:0:1:ffff:ffff:ffff:ffff 2001:db8:0:1::/64

基础 - Ubuntu中手工配置IPv6的注意点

Ubuntu中配置IPv6可以采用network-manager, 也可采用在/etc/network/interface中手工配置, 也可以使用最新的netplan. 这里描述的是采用手工配置的方法.
先看一个遇到的实际问题:

下面配置不work
iface eth0 inet6 auto
   # use SLAAC to get global IPv6 address from the router
   # we may not enable ipv6 forwarding, otherwise SLAAC gets disabled
   up sleep 5
   dhcp 1
   autoconf 1
   accept_ra 2

下列配置work
iface eth0 inet6 static
address 2001:192:168:99::135
   gateway 2001:192:168:99::1
   netmask 64
且改成network-manager也work, 这是为什么呢?

测试方法是:
#it will flush link-local address as well
#ip addr flush br-eth0
# avoid the error: can't get a link-local address
sudo ip link set dev eth0 down
sudo ip link set dev eth0 up
ifdown br-eth0
ifup --force --verbose br-eth0

采用"ifup --force --verbose br-eth0"命令看到的错误是"can’t get a link-local address".
为什么static模式与network-manager模式没有这个错误呢? 原来是这两者默认执行了:

echo 0 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6

并且之前Linux网桥br-eth0上一直没有IPv6地址的原因也是这个, 且上面"sudo ip link set dev eth0 up"这句也会自动设置disable_ipv6=0, 但不会对br-eth0作同样的设置.
所以添加"up echo 0 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6"后问题解决, 完整配置如下:

root@node1:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet manual
auto br-eth0
iface br-eth0 inet static
    address 192.168.99.124/24
    gateway 192.168.99.1
    bridge_ports eth0
    dns-nameservers 192.168.99.1
    bridge_stp on
    bridge_fd 0
    bridge_maxwait 0
    up echo -n 0 > /sys/devices/virtual/net/$IFACE/bridge/multicast_snooping
# for stateless it's 'inet6 auto', for stateful it's 'inet6 dhcp'
iface br-eth0 inet6 auto
    #iface eth0 inet6 static
    #address 2001:192:168:99::135                                                                                            
    #gateway 2001:192:168:99::1
    #netmask 64
    # use SLAAC to get global IPv6 address from the router
    # we may not enable ipv6 forwarding, otherwise SLAAC gets disabled
    # sleep 5 is due a bug and 'dhcp 1' indicates that info should be obtained from dhcpv6 server for stateless
    up echo 0 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6
    up sleep 5
    autoconf 1
    accept_ra 2
    dhcp 1

此外, 最好设置accept_ra=2, 因为经常会遇到自动配置的IPv6地址丢失或者不能获取的问题。一般情况是都是启用了IPv6转发功能(sudo sysctl -w net.ipv6.conf.all.forwarding=1)引起的。
为了配置IPv6 address和default gateway, client/host都会默认去listen或者solicit RA广播, 并且host作为router时会忽略RA, 这由accept_ra设置:

  • 0 Do not accept RouterAdvertisements.
  • 1 Accept Router Advertisements if forwarding is disabled.
  • 2 Overrule forwarding behavior. Accept Router Advertisements even if forwarding is enabled.
sudo sysctl -w net.ipv6.conf.all.forwarding=1
sudo sysctl -w net.ipv6.conf.all.accept_ra=2
sudo sysctl -w net.ipv6.conf.br-lan.disable_ipv6=0
#echo 0 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6

IPv6中的防火墙

IPv6 Router端:

# Clear all ip6tables rules
ip6tables -t nat -X
ip6tables -t nat -P PREROUTING ACCEPT
ip6tables -t nat -P POSTROUTING ACCEPT
ip6tables -t nat -P OUTPUT ACCEPT
ip6tables -t mangle -F
ip6tables -t mangle -X
ip6tables -t mangle -P PREROUTING ACCEPT
ip6tables -t mangle -P INPUT ACCEPT
ip6tables -t mangle -P FORWARD ACCEPT
ip6tables -t mangle -P OUTPUT ACCEPT
ip6tables -t mangle -P POSTROUTING ACCEPT
ip6tables -F
ip6tables -X
ip6tables -P FORWARD ACCEPT
ip6tables -P INPUT ACCEPT
ip6tables -P OUTPUT ACCEPT
ip6tables -t raw -F
ip6tables -t raw -X
ip6tables -t raw -P PREROUTING ACCEPT
ip6tables -t raw -P OUTPUT ACCEPT

# Default DROP rules
ip6tables -P INPUT   DROP
ip6tables -P OUTPUT  ACCEPT
ip6tables -P FORWARD DROP

# Allow established connections
ip6tables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
ip6tables -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT

# For IPv6
# it's not required due to ipv6-icmp 
# sudo ip6tables -A INPUT -p udp --dport 547 -j ACCEPT  
#ip6tables -A INPUT -p icmpv6 --icmpv6-type echo-request -j ACCEPT
ip6tables -A INPUT -p ipv6-icmp -j ACCEPT
ip6tables -A FORWARD -p ipv6-icmp -j ACCEPT

# Ajust MTU
ip6tables -t mangle -A POSTROUTING -o pppoe-wan -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

IPv6 Client端:

ip6tables -t filter -A INPUT -p udp -m udp --sport 547 --dport 546 -j ACCEPT
ip6tables -t filter -A INPUT -p ipv6-icmp -j ACCEPT

# or security group
https://bugs.launchpad.net/neutron/+bug/1335984
openstack security group rule create $secgroup --protocol udp --dst-port 546 --ethertype IPv6
openstack security group rule create $secgroup --protocol icmpv6 --ethertype IPv6

# Flow based firewall
hard_timeout=0,idle_timeout=0,priority=4,udp,tp_dst=546/0xffff,table=32,tp_src=547/0xffff,nw_src=fe80::f816:3eff:fea3:ec40,actions=learn(table=33,priority=5,hard_timeout=120,eth_type=0x800,nw_proto=17,NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],NXM_OF_IP_SRC[]=NXM_OF_IP_DST[],NXM_OF_UDP_SRC[]=NXM_OF_UDP_DST[], NXM_OF_UDP_DST[]=NXM_OF_UDP_SRC[],output:NXM_OF_IN_PORT[]),normal

另外, 别忘了禁用掉ufw或者SELinux之类的.

sudo ufw disable

Statefull DHCPv6

采用isc-dhcp-server搭建DHCPv6 Server:

https://jochen.kirstaetter.name/dhcpv6-ipv6-in-your-local-network/
hua@t440p:~$ ip addr show eth0 |grep inet6 |grep global
    inet6 2001:192:168:99::430/128 scope global
echo 'Acquire::ForceIPv4 "true";' | sudo tee /etc/apt/apt.conf.d/99force-ipv4
sudo apt install isc-dhcp-server
grep -v ^# /etc/dhcp/dhcpd6.conf
sudo cp /etc/dhcp/dhcpd6.conf /etc/dhcp/dhcpd6.conf_bak
sudo bash -c 'cat >/etc/dhcp/dhcpd6.conf' <<EOF
authoritative;
default-lease-time 14400; 
max-lease-time 86400;
log-facility local7;
subnet6 2001:192:168:99::/64 {
    option dhcp6.name-servers 2001:4860:4860::8888, 2001:4860:4860::8844;
    option dhcp6.domain-search "quqi.com";
    range6 2001:192:168:99::100 2001:192:168:99::199;
    range6 2001:192:168:99::/64 temporary;
}
EOF
sudo touch /var/lib/dhcp/dhcpd6.leases
sudo /usr/sbin/dhcpd -6 -d -cf /etc/dhcp/dhcpd6.conf eth0
sudo chown dhcpd:dhcpd /var/lib/dhcp/dhcpd6.leases
sudo service isc-dhcp-server6 restart

然后记得照上节说的设置DHCPv6 Server与Client上的防火墙规则. 接着在另一台机器上作client测试:

# need to use 'inet6 dhcp' in client side for statefull DHCPv6
iface br-eth0 inet6 dhcp
    #iface eth0 inet6 static
    #address 2001:192:168:99::135
    #gateway 2001:192:168:99::1
    #netmask 64
    # use SLAAC to get global IPv6 address from the router
    # we may not enable ipv6 forwarding, otherwise SLAAC gets disabled
    # sleep 5 is due a bug and 'dhcp 1' indicates that info should be obtained from dhcpv6 server for stateless
    up echo 0 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6
    up sleep 5
    autoconf 1
    accept_ra 2
    dhcp 1
    
# test command
dhclient -6 -d br-eth0

# verity
hua@node1:~$ sudo tcpdump -ni eth0 ip6 host fe80::d5a3:10a3:6161:5b2e
12:44:00.868609 IP6 fe80::d5a3:10a3:6161:5b2e.547 > fe80::fa32:e4ff:febe:87cd.546: dhcp6 advertise
12:44:01.946548 IP6 fe80::d5a3:10a3:6161:5b2e.547 > fe80::fa32:e4ff:febe:87cd.546: dhcp6 reply
root@node1:~# cat /etc/resolv.conf 
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 192.168.99.1
#nameserver 211.136.17.107
#nameserver 114.114.114.114
#nameserver 223.5.5.5
nameserver 2001:4860:4860::8888
nameserver 2001:4860:4860::8844
nameserver 2001:db8::1:6a1:51ff:fe8a:2ca7
search quqi.com lan

另外, 使用BIND9的例子可参见 - https://jochen.kirstaetter.name/enabling-dns-for-ipv6-infrastructure/

SLAAC (Stateless Address Auto Configuration)

radvd来提供RA部分, SLAAC只有RA部分. RA只能设置IPv6 prefix与DNS (RDNSS).
Historically the software package radvd was commonly used for just the RA-part of this. But dnsmasq offers a more complete setup.

sudo sysctl -w net.ipv6.conf.all.forwarding=1
sudo ip addr add 2001:db8:0:1::1/64 dev eth0
sudo apt-get install radvd
$ cat /etc/radvd.conf 
    interface eth0
    {
       AdvSendAdvert on;
       prefix 2001:db8:0:1::/64
       {
            AdvOnLink on;
            AdvAutonomous on;
       };
       #Send DNS Server setting
       #RDNSS fd5d:12c9:2201:1::2{
    };
sudo /etc/init.d/radvd restart
sudo ip6tables -F

neutron subnet-create --ip-version=6 --name=ext-v6-subnet --gateway 2001:db8:0:1::1 --allocation-pool start=2001:db8:0:1::5,end=2001:db8:0:1:ffff:ffff:ffff:fffe --disable-dhcp ext_net 2001:db8:0:1::/64
neutron net-create private
neutron subnet-create --ip-version=6 --name=private_v6_subnet --ipv6-address-mode=slaac --ipv6-ra-mode=slaac private 2001:db8:0:2::/64
neutron router-interface-add provider-router private_v6_subnet

SLAAS with Stateless DHCPv6

Stateless意味着:

  • radvd提供RA (AdvManagedFlag=off)
  • client使用radvd RA提供的IPv6 prefix配置IPv6 address
  • client的其他信息如DNS等从DHCPv6获得
sudo bash -c 'cat > /etc/dhcp/dhcpd6.conf' <<EOF
interface eth0
{
    AdvSendAdvert on;
    MinRtrAdvInterval 30;
    MaxRtrAdvInterval 100;
    AdvManagedFlag off;
    AdvOtherConfigFlag on;
    prefix 2001:192:168:99::/64
    {
        AdvOnLink on;
        AdvAutonomous on;
        AdvRouterAddr off;
    };
};
EOF

sudo bash -c 'cat > /etc/radvd.conf' <<EOF
default-lease-time 600;
max-lease-time 7200;
log-facility local7;
option dhcp6.name-servers 2001:4860:4860::8888;
option dhcp6.domain-search "";
subnet6 2001:192:168:99::/64 {
}
EOF

SLAAS with Statefull DHCPv6

Statefull意味着:

  • radvd不提供RA (AdvManagedFlag=on)
  • client使用DHCPv6去配置IPv6 address
  • client的其他信息如DNS等也从DHCPv6获得
sudo bash -c 'cat > /etc/dhcp/dhcpd6.conf' <<EOF
interface eth0
{
    AdvSendAdvert on;
    MinRtrAdvInterval 30;
    MaxRtrAdvInterval 100;
    AdvManagedFlag on;
    AdvOtherConfigFlag on;
    prefix 2001:192:168:99::/64
    {
        AdvOnLink on;
        AdvAutonomous on;
        AdvRouterAddr off;
    };
};
EOF

sudo bash -c 'cat >/etc/dhcp/dhcpd6.conf' <<EOF
authoritative;
default-lease-time 14400; 
max-lease-time 86400;
log-facility local7;
subnet6 2001:192:168:99::/64 {
    option dhcp6.name-servers 2001:4860:4860::8888, 2001:4860:4860::8844;
    option dhcp6.domain-search "quqi.com";
    range6 2001:192:168:99::100 2001:192:168:99::199;
    range6 2001:192:168:99::/64 temporary;
}
EOF

bastion上安装radvd产生的后果

bastion是openstack over openstack测试环境中underlying openstack提供的一台虚机, 用于安装juju继续部署上层openstack.
忽然在bastion上运行juju add-model或者juju deploy偶尔(不是次次)将会出现下列问题:

ERROR failed to create environ: authentication failed.: authentication failed
caused by: requesting token: failed executing the request http://10.230.19.53:5000/v3/auth/tokens
caused by: Post http://10.230.19.53:5000/v3/auth/tokens: dial tcp 10.230.19.53:5000: i/o timeout

刚开始怀疑snap会自动升级juju版本到最新版本的原因, 其实不是. 接着发现从bastion连controller (juju ssh 0 -m controller)时偶尔会断线, 接着使用下列命令发现偶尔有timeout出现确认了这个假设.

while true; do nc -w 1 -vz 10.230.19.53 5000; sleep 1; done

接着在underlying openstack中看到了下列日志:

1000s of [484018.924930] IPv6: qr-0166579a-c8: IPv6 duplicate address 2001:192:168:99:f816:3eff:fec9:783 used by fa:16:3e:c9:07:83 detected!

才得知是在bastion上运行了radvd的原因, 一种原因是tenant network (zhhuabj_admin_net)做了HA的, 各vrrp节点上的keepalived由于没有配置这个未被管理的IPv6网段, 所以各vrrp节点上都被分配了相同的IPv6地址(2001:192:168:99:f816:3eff:fec9:783), 这样会引起keepalived不快会发生迁移导致tenant网络不稳定. 解决办法设想的是再创建一个non-HA的tenant network, 然后在这个network上开一台虚机上跑radvd, 上层的openstack环境也使用这个network.
但这里似乎不是这原因, 就是bastion上分配了ipv6地址导致连juju时ipv4/ipv6 fallback似乎有点问题导致网络时而中断.

另外, security group会禁止在虚机上提供DHCPv6服务(禁止从547到546的traffic),也有MAC/IP匹配防欺骗的rules等. 所以如果想使用这个openstack-over-openstack环境上做openstack使用外部stateless router的实验的话, 最好通过底层juju的use-default-secgroup=false (https://github.com/juju/juju/blob/2.5/provider/openstack/config.go#L20)禁用底层的security group.

之前还遇到了一个在bastion上安装radvd造成无法登录bastion的问题, 那是因为在l3-agent上分配了两个网卡, 并且将这两个网卡都plug进了一个网桥中, 结果这个网卡从radvd处发生广播风暴导致无法登录bastion.

实际案例

用户想根据外部物理IPv6 stateless router定义的IPv6网络, 虚机分配了IP, 但是网卡上去没配置.
下面我们在一个openstack over openstack环境上模拟这种测试.
首先, 我们不能直接在bastion上安装radvd, 因为这会导致basion上也分配到IPv6后造成bastion和juju通信有问题. 所以想要在openstack over openstack环境上继续安装radvd模拟外部路由器的话需要做特殊处理. 那就是创建一个单独的ipv6 tenant router, 在这个router里创建一个虚机安装radvd, 并且在上层openstack环境里使用flat定义IPv6网络(这样它就和radvd在同一个IPv6网络了).
1, 第一步, 需要创建一个单独的router, 并且要确保定义network时禁用underlying的security group.

source ~/novarc
#openstack router create --centralized --no-ha --description "for radvd" zhhuabj_router_radvd
openstack router create zhhuabj_router_radvd
# remember to disable security group for underlying openstack network
openstack network create --disable-port-security radvd-net
openstack subnet create --subnet-range 10.10.0.0/24 --network radvd-net --allocation-pool start=10.10.0.50,end=10.10.0.100 --gateway 10.10.0.1 radvd-subnet
openstack router add subnet zhhuabj_router_radvd radvd-subnet
openstack router set --external-gateway ext_net zhhuabj_router_radvd

2, 第二步, 创建一个虚机安装radvd来模拟外部路由器

source ~/novarc
openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey
openstack server create --image auto-sync/ubuntu-bionic-18.04-amd64-server-20190122-disk1.img --flavor m1.small --key-name mykey --network=radvd-net ip6router
openstack floating ip create ext_net
openstack server add floating ip ip6router 10.230.65.104

# Setting up the external stateless IPv6 router
ssh ubuntu@10.230.65.104 -v
sudo sysctl -w net.ipv6.conf.all.forwarding=1
sudo ip6tables -F
sudo ufw disable
sudo apt install -y radvd
sudo bash -c 'cat > /etc/radvd.conf' <<EOF
interface ens3
{
    AdvSendAdvert on;
    MinRtrAdvInterval 30;
    MaxRtrAdvInterval 100;
    AdvManagedFlag off;
    AdvOtherConfigFlag on;
    prefix 2001:192:168:99::/64
    {
        AdvOnLink on;
        AdvAutonomous on;
        AdvRouterAddr off;
    };
};
EOF
sudo systemctl restart radvd.service

注意, 这里虽然做的是stateless IPv6的实验, 但这里并不需要安装DHCPv6服务器, 因为openstack已经帮我们创建好了:

nobody   22883     1  0 06:45 ?        00:00:00 dnsmasq --no-hosts --no-resolv --strict-order --except-interface=lo --pid-file=/var/lib/neutron/dhcp/0aed25cf-b578-460d-9f4c-10d2fbe40179/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/0aed25cf-b578-460d-9f4c-10d2fbe40179/host --addn-hosts=/var/lib/neutron/dhcp/0aed25cf-b578-460d-9f4c-10d2fbe40179/addn_hosts --dhcp-optsfile=/var/lib/neutron/dhcp/0aed25cf-b578-460d-9f4c-10d2fbe40179/opts --dhcp-leasefile=/var/lib/neutron/dhcp/0aed25cf-b578-460d-9f4c-10d2fbe40179/leases --dhcp-match=set:ipxe,175 --bind-interfaces --interface=ns-9b88221a-86 --dhcp-range=set:tag0,2001:192:168:99::,static,64,86400s --dhcp-option-force=option:mtu,1500 --dhcp-lease-max=16777216 --conf-file=/etc/neutron/dnsmasq.conf --domain=openstacklocal
ubuntu@juju-a09725-xenial-mitaka-5:~$ sudo cat /var/lib/neutron/dhcp/0aed25cf-b578-460d-9f4c-10d2fbe40179/opts
tag:tag0,option6:domain-search,openstacklocalubuntu@juju-a09725-xenial-mitaka-5:~$ sudo cat /var/lib/neutron/dhcp/0aed25cf-b578-460d-9f4c-10d2fbe40179/addn_hosts
2001:192:168:99:f816:3eff:fe1f:95ba	host-2001-192-168-99-f816-3eff-fe1f-95ba.openstacklocal host-2001-192-168-99-f816-3eff-fe1f-95ba
2001:192:168:99:f816:3eff:fe6f:121f	host-2001-192-168-99-f816-3eff-fe6f-121f.openstacklocal host-2001-192-168-99-f816-3eff-fe6f-121f

下面为了知识完备性, 如果一定要自己去创建外部Stateless DHCPv6的话, 可以这样:

sudo apt install -y isc-dhcp-server
sudo bash -c 'cat > /etc/dhcp/dhcpd6.conf' <<EOF
default-lease-time 600;
max-lease-time 7200;
log-facility local7;
option dhcp6.name-servers 2001:4860:4860::8888;
option dhcp6.domain-search "";
subnet6 2001:192:168:99::/64 {
}
EOF
sudo service isc-dhcp-server6 restart

第三步, 创建上层openstack

./generate-bundle.sh --series xenial --release mitaka
juju add-model xenial-mitaka
juju deploy ./b/openstack.yaml
~/stsstack-bundles/openstack/novarc

第四步, 因为上层openstack环境需要定义flat network, 所以它实际使用了br-data, 那么需要为计算节点与网络节点的br-data添加一个外部网络.
ens3(VM) tap55f99960-ba(compute) -> qbr55f99960-ba -> qvb55f99960-ba(qbr) -> qvo55f99960-ba(br-int) -> br-data -> ens7 - > external

source ~/novarc 
nova interface-attach $(nova list |grep $(juju ssh neutron-gateway/0 -- hostname) |awk '{print $2}') --net-id=$(neutron net-show radvd-net -c id -f value)
juju ssh neutron-gateway/0 -- sudo ovs-vsctl add-port br-data ens7
juju ssh neutron-gateway/0 -- sudo ifconfig ens7 up
nova interface-attach $(nova list |grep $(juju ssh nova-compute/0 -- hostname) |awk '{print $2}') --net-id=$(neutron net-show radvd-net -c id -f value)
juju ssh nova-compute/0 -- sudo ovs-vsctl add-port br-data ens7
juju ssh nova-compute/0 -- sudo ifconfig ens7 up

source ~/stsstack-bundles/openstack/novarc
neutron net-create --provider:network_type flat --provider:physical_network physnet1 --router:external=True ipv6_net
neutron subnet-create ipv6_net --name ipv6_subnet --ip_version 6 --ipv6_address_mode dhcpv6-stateless --allocation-pool start=2001:192:168:99::2,end=2001:192:168:99:ffff:ffff:ffff:ffff 2001:192:168:99::/64
openstack router create myrouter
openstack router add subnet myrouter ipv6_subnet
#openstack router set --external-gateway ext_net myrouter

第五步,创建一个虚机用以充当IPv6 client, 这里注意一定要为Ipv6定义flat的网络, 另外, 额外定义了一个ipv4网络目的是为了让虚机完成metadata功能我们容易ssh进去调试.

source ~/stsstack-bundles/openstack/novarc
neutron net-create --provider:network_type flat --provider:physical_network physnet1 --router:external=True ipv6_net
neutron subnet-create ipv6_net --name ipv6_subnet --ip_version 6 --ipv6_address_mode dhcpv6-stateless --allocation-pool start=2001:192:168:99::2,end=2001:192:168:99:ffff:ffff:ffff:ffff 2001:192:168:99::/64
openstack router create myrouter
openstack router add subnet myrouter ipv6_subnet
#openstack router set --external-gateway ext_net myrouter

# create another ipv4 network to make metadata feature pass
openstack network create ipv4-net
openstack subnet create --subnet-range 10.11.0.0/24 --network ipv4-net --allocation-pool start=10.11.0.50,end=10.11.0.100 --gateway 10.11.0.1 ipv4-subnet
openstack router add subnet myrouter ipv4-subnet

openstack image create --disk-format=raw --container-format=bare xenial --file /home/ubuntu/images/xenial-server-cloudimg-amd64-disk1.img
openstack security group rule create default --protocol tcp --remote-ip 0.0.0.0/0 --dst-port 22
openstack security group rule create default --protocol icmp --remote-ip 0.0.0.0/0
openstack security group rule list default
openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey
openstack server create --image xenial --flavor m1.small --key-name mykey --network=ipv4-net --network=ipv6_net ip6client

第六步, 虚机内部配置调试

juju scp ~/.ssh/id_rsa* neutron-gateway/0:/home/ubuntu/.ssh/
juju ssh neutron-gateway/0
sudo ip netns exec qrouter-f6d50237-add3-4bda-b848-861bfa68c7a3 ssh -i ~/.ssh/id_rsa ubuntu@10.11.0.51 -v

sudo vim /etc/network/interfaces.d/50-cloud-init.cfg
iface ens3 inet6 auto
    up echo 0 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6
    up sleep 5
    autoconf 1
    accept_ra 2
    dhcp 1
    
sudo dhclient -6 -d ens3
#sudo ifup --force --verbose ens3

# debug, the fe80 address of ipv6 client is fe80::f816:3eff:fe6f:121f
sudo tcpdump -ni ens3 ip6 host fe80::f816:3eff:fe6f:121f

注意, 在给br-data添加外部网卡及停用外部DHCPv6(这样使用openstack dnsmasq提供的IPv6)后, 所以除ovs网桥以外的网卡(ens3(VM) tap55f99960-ba(compute) -> qbr55f99960-ba -> qvb55f99960-ba(qbr) -> qvo55f99960-ba(br-int) -> br-data -> ens7 - > external)均可以成功抓到下列包(注: 在ovs bridge上抓到的包没有dhcp6 advertise, 但这似乎没有影响到整体功能, 可能就是对ovs bridge使用tcpdump看不到dhcp6 advertise吧):

07:44:01.209860 IP6 fe80::f816:3eff:fe6f:121f.546 > ff02::1:2.547: dhcp6 solicit
07:44:01.213087 IP6 fe80::f816:3eff:fe1f:95ba.547 > fe80::f816:3eff:fe6f:121f.546: dhcp6 advertise
07:44:02.294432 IP6 fe80::f816:3eff:fe6f:121f.546 > ff02::1:2.547: dhcp6 solicit
07:44:02.296687 IP6 fe80::f816:3eff:fe1f:95ba.547 > fe80::f816:3eff:fe6f:121f.546: dhcp6 advertise
07:44:04.399586 IP6 fe80::f816:3eff:fe6f:121f.546 > ff02::1:2.547: dhcp6 solicit
07:44:04.401203 IP6 fe80::f816:3eff:fe1f:95ba.547 > fe80::f816:3eff:fe6f:121f.546: dhcp6 advertise
07:44:05.035467 IP6 :: > ff02::1:ffe0:75b0: ICMP6, neighbor solicitation, who has 2001:192:168:99:f816:3eff:fee0:75b0, length 32
07:44:05.036112 IP6 2001:192:168:99:f816:3eff:fee0:75b0 > ff02::1: ICMP6, neighbor advertisement, tgt is 2001:192:168:99:f816:3eff:fee0:75b0, length 32
07:44:08.665018 IP6 fe80::f816:3eff:fe6f:121f.546 > ff02::1:2.547: dhcp6 solicit
07:44:08.666282 IP6 fe80::f816:3eff:fe1f:95ba.547 > fe80::f816:3eff:fe6f:121f.546: dhcp6 advertise
07:44:13.668128 IP6 fe80::f816:3eff:fe1f:95ba > fe80::f816:3eff:fe6f:121f: ICMP6, neighbor solicitation, who has fe80::f816:3eff:fe6f:121f, length 32
07:44:13.668187 IP6 fe80::f816:3eff:fe6f:121f > fe80::f816:3eff:fe1f:95ba: ICMP6, neighbor advertisement, tgt is fe80::f816:3eff:fe6f:121f, length 24

测试结果:

ubuntu@ip6client:~$ ip addr show ens3 |grep inet6
    inet6 2001:192:168:99:f816:3eff:fe6f:121f/64 scope global mngtmpaddr dynamic 
    inet6 fe80::f816:3eff:fe6f:121f/64 scope link 
ubuntu@ip6client:~$ ping6 2001:192:168:99:f816:3eff:fe1f:95ba
PING 2001:192:168:99:f816:3eff:fe1f:95ba(2001:192:168:99:f816:3eff:fe1f:95ba) 56 data bytes
64 bytes from 2001:192:168:99:f816:3eff:fe1f:95ba: icmp_seq=1 ttl=64 time=3.80 ms
ubuntu@juju-a09725-xenial-mitaka-5:~$ sudo ip netns exec qdhcp-0aed25cf-b578-460d-9f4c-10d2fbe40179 ping6 2001:192:168:99:f816:3eff:fe6f:121f
PING 2001:192:168:99:f816:3eff:fe6f:121f(2001:192:168:99:f816:3eff:fe6f:121f) 56 data bytes
64 bytes from 2001:192:168:99:f816:3eff:fe6f:121f: icmp_seq=1 ttl=64 time=2.97 ms

附件 - 防火墙及流表

实际上, security group的默认规则是不影响虚机使用外部IPv6路由器获取RA的.

root@juju-a09725-xenial-mitaka-7:~# sudo ovs-ofctl dump-flows br-data
NXST_FLOW reply (xid=0x4):
 cookie=0xa53ad21731e554e6, duration=10484.383s, table=0, n_packets=247, n_bytes=25966, idle_age=2619, priority=4,in_port=1,dl_vlan=2 actions=strip_vlan,NORMAL
 cookie=0xa53ad21731e554e6, duration=168851.241s, table=0, n_packets=39, n_bytes=4126, idle_age=7795, hard_age=65534, priority=2,in_port=1 actions=drop
 cookie=0xa53ad21731e554e6, duration=168852.108s, table=0, n_packets=1803, n_bytes=173620, idle_age=18, hard_age=65534, priority=0 actions=NORMAL

root@juju-a09725-xenial-mitaka-7:~# sudo ovs-ofctl dump-flows br-int
NXST_FLOW reply (xid=0x4):
 cookie=0x8a5790f2d6b535e6, duration=10493.212s, table=0, n_packets=0, n_bytes=0, idle_age=10493, priority=10,icmp6,in_port=3,icmp_type=136 actions=resubmit(,24)
 cookie=0x8a5790f2d6b535e6, duration=10487.634s, table=0, n_packets=39, n_bytes=3058, idle_age=2630, priority=10,icmp6,in_port=4,icmp_type=136 actions=resubmit(,24)
 cookie=0x8a5790f2d6b535e6, duration=10492.928s, table=0, n_packets=384, n_bytes=16128, idle_age=26, priority=10,arp,in_port=3 actions=resubmit(,24)
 cookie=0x8a5790f2d6b535e6, duration=10487.502s, table=0, n_packets=0, n_bytes=0, idle_age=10491, priority=10,arp,in_port=4 actions=resubmit(,24)
 cookie=0x8a5790f2d6b535e6, duration=168862.878s, table=0, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=2,in_port=1 actions=drop
 cookie=0x8a5790f2d6b535e6, duration=10493.495s, table=0, n_packets=31789, n_bytes=2827279, idle_age=0, priority=9,in_port=3 actions=resubmit(,25)
 cookie=0x8a5790f2d6b535e6, duration=10488.022s, table=0, n_packets=208, n_bytes=22908, idle_age=2632, priority=9,in_port=4 actions=resubmit(,25)
 cookie=0x8a5790f2d6b535e6, duration=10495.768s, table=0, n_packets=1803, n_bytes=173620, idle_age=30, priority=3,in_port=1,vlan_tci=0x0000 actions=mod_vlan_vid:2,NORMAL
 cookie=0x8a5790f2d6b535e6, duration=168864.656s, table=0, n_packets=32596, n_bytes=2637470, idle_age=0, hard_age=65534, priority=0 actions=NORMAL
 cookie=0x8a5790f2d6b535e6, duration=168864.535s, table=23, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=0 actions=drop
 cookie=0x8a5790f2d6b535e6, duration=10493.351s, table=24, n_packets=0, n_bytes=0, idle_age=10493, priority=2,icmp6,in_port=3,icmp_type=136,nd_target=fe80::f816:3eff:feb9:b96a actions=NORMAL
 cookie=0x8a5790f2d6b535e6, duration=10487.892s, table=24, n_packets=3, n_bytes=234, idle_age=2778, priority=2,icmp6,in_port=4,icmp_type=136,nd_target=2001:192:168:99:f816:3eff:fe6f:121f actions=NORMAL
 cookie=0x8a5790f2d6b535e6, duration=10487.762s, table=24, n_packets=36, n_bytes=2824, idle_age=2630, priority=2,icmp6,in_port=4,icmp_type=136,nd_target=fe80::f816:3eff:fe6f:121f actions=NORMAL
 cookie=0x8a5790f2d6b535e6, duration=10493.068s, table=24, n_packets=384, n_bytes=16128, idle_age=26, priority=2,arp,in_port=3,arp_spa=10.11.0.51 actions=resubmit(,25)
 cookie=0x8a5790f2d6b535e6, duration=168864.420s, table=24, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=0 actions=drop
 cookie=0x8a5790f2d6b535e6, duration=10493.780s, table=25, n_packets=32173, n_bytes=2843407, idle_age=0, priority=2,in_port=3,dl_src=fa:16:3e:b9:b9:6a actions=NORMAL
 cookie=0x8a5790f2d6b535e6, duration=10488.305s, table=25, n_packets=208, n_bytes=22908, idle_age=2632, priority=2,in_port=4,dl_src=fa:16:3e:6f:12:1f actions=NORMAL

root@juju-a09725-xenial-mitaka-7:~# ip6tables-save 
# Generated by ip6tables-save v1.6.0 on Fri Jan 25 09:47:42 2019
*raw
:PREROUTING ACCEPT [3823:312730]
:OUTPUT ACCEPT [47:3376]
:neutron-openvswi-OUTPUT - [0:0]
:neutron-openvswi-PREROUTING - [0:0]
-A PREROUTING -j neutron-openvswi-PREROUTING
-A OUTPUT -j neutron-openvswi-OUTPUT
-A neutron-openvswi-PREROUTING -m physdev --physdev-in qvb55f99960-ba -j CT --zone 2
-A neutron-openvswi-PREROUTING -m physdev --physdev-in tap55f99960-ba -j CT --zone 2
-A neutron-openvswi-PREROUTING -m physdev --physdev-in qvbb2d4fa47-0e -j CT --zone 1
-A neutron-openvswi-PREROUTING -m physdev --physdev-in tapb2d4fa47-0e -j CT --zone 1
COMMIT
# Completed on Fri Jan 25 09:47:42 2019
# Generated by ip6tables-save v1.6.0 on Fri Jan 25 09:47:42 2019
*mangle
:PREROUTING ACCEPT [3852:314932]
:INPUT ACCEPT [288:24080]
:FORWARD ACCEPT [2054:171782]
:OUTPUT ACCEPT [93:6580]
:POSTROUTING ACCEPT [1588:128442]
:neutron-openvswi-FORWARD - [0:0]
:neutron-openvswi-INPUT - [0:0]
:neutron-openvswi-OUTPUT - [0:0]
:neutron-openvswi-POSTROUTING - [0:0]
:neutron-openvswi-PREROUTING - [0:0]
:neutron-openvswi-scope - [0:0]
-A PREROUTING -j neutron-openvswi-PREROUTING
-A INPUT -j neutron-openvswi-INPUT
-A FORWARD -j neutron-openvswi-FORWARD
-A OUTPUT -j neutron-openvswi-OUTPUT
-A POSTROUTING -j neutron-openvswi-POSTROUTING
-A neutron-openvswi-PREROUTING -j neutron-openvswi-scope
-A neutron-openvswi-PREROUTING -m connmark ! --mark 0x0/0xffff0000 -j CONNMARK --restore-mark --nfmask 0xffff0000 --ctmask 0xffff0000
COMMIT
# Completed on Fri Jan 25 09:47:42 2019
# Generated by ip6tables-save v1.6.0 on Fri Jan 25 09:47:42 2019
*filter
:INPUT ACCEPT [186:15512]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [7:776]
:neutron-filter-top - [0:0]
:neutron-openvswi-FORWARD - [0:0]
:neutron-openvswi-INPUT - [0:0]
:neutron-openvswi-OUTPUT - [0:0]
:neutron-openvswi-i55f99960-b - [0:0]
:neutron-openvswi-ib2d4fa47-0 - [0:0]
:neutron-openvswi-local - [0:0]
:neutron-openvswi-o55f99960-b - [0:0]
:neutron-openvswi-ob2d4fa47-0 - [0:0]
:neutron-openvswi-s55f99960-b - [0:0]
:neutron-openvswi-scope - [0:0]
:neutron-openvswi-sg-chain - [0:0]
:neutron-openvswi-sg-fallback - [0:0]
-A INPUT -j neutron-openvswi-INPUT
-A FORWARD -j neutron-filter-top
-A FORWARD -j neutron-openvswi-FORWARD
-A OUTPUT -j neutron-filter-top
-A OUTPUT -j neutron-openvswi-OUTPUT
-A neutron-filter-top -j neutron-openvswi-local
-A neutron-openvswi-FORWARD -j neutron-openvswi-scope
-A neutron-openvswi-FORWARD -m physdev --physdev-out tap55f99960-ba --physdev-is-bridged -m comment --comment "Direct traffic from the VM interface to the security group chain." -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-in tap55f99960-ba --physdev-is-bridged -m comment --comment "Direct traffic from the VM interface to the security group chain." -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-out tapb2d4fa47-0e --physdev-is-bridged -m comment --comment "Direct traffic from the VM interface to the security group chain." -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-in tapb2d4fa47-0e --physdev-is-bridged -m comment --comment "Direct traffic from the VM interface to the security group chain." -j neutron-openvswi-sg-chain
-A neutron-openvswi-INPUT -m physdev --physdev-in tap55f99960-ba --physdev-is-bridged -m comment --comment "Direct incoming traffic from VM to the security group chain." -j neutron-openvswi-o55f99960-b
-A neutron-openvswi-INPUT -m physdev --physdev-in tapb2d4fa47-0e --physdev-is-bridged -m comment --comment "Direct incoming traffic from VM to the security group chain." -j neutron-openvswi-ob2d4fa47-0
-A neutron-openvswi-i55f99960-b -p ipv6-icmp -m icmp6 --icmpv6-type 130 -j RETURN
-A neutron-openvswi-i55f99960-b -p ipv6-icmp -m icmp6 --icmpv6-type 131 -j RETURN
-A neutron-openvswi-i55f99960-b -p ipv6-icmp -m icmp6 --icmpv6-type 132 -j RETURN
-A neutron-openvswi-i55f99960-b -p ipv6-icmp -m icmp6 --icmpv6-type 135 -j RETURN
-A neutron-openvswi-i55f99960-b -p ipv6-icmp -m icmp6 --icmpv6-type 136 -j RETURN
-A neutron-openvswi-i55f99960-b -m state --state RELATED,ESTABLISHED -m comment --comment "Direct packets associated with a known session to the RETURN chain." -j RETURN
-A neutron-openvswi-i55f99960-b -s fe80::f816:3eff:fe1f:95ba/128 -p udp -m udp --sport 547 -m udp --dport 546 -j RETURN
-A neutron-openvswi-i55f99960-b -m set --match-set NIPv6e50abcfd-5bdb-462d-aa21- src -j RETURN
-A neutron-openvswi-i55f99960-b -p ipv6-icmp -j RETURN
-A neutron-openvswi-i55f99960-b -m state --state INVALID -m comment --comment "Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack." -j DROP
-A neutron-openvswi-i55f99960-b -m comment --comment "Send unmatched traffic to the fallback chain." -j neutron-openvswi-sg-fallback
-A neutron-openvswi-ib2d4fa47-0 -p ipv6-icmp -m icmp6 --icmpv6-type 130 -j RETURN
-A neutron-openvswi-ib2d4fa47-0 -p ipv6-icmp -m icmp6 --icmpv6-type 131 -j RETURN
-A neutron-openvswi-ib2d4fa47-0 -p ipv6-icmp -m icmp6 --icmpv6-type 132 -j RETURN
-A neutron-openvswi-ib2d4fa47-0 -p ipv6-icmp -m icmp6 --icmpv6-type 135 -j RETURN
-A neutron-openvswi-ib2d4fa47-0 -p ipv6-icmp -m icmp6 --icmpv6-type 136 -j RETURN
-A neutron-openvswi-ib2d4fa47-0 -m state --state RELATED,ESTABLISHED -m comment --comment "Direct packets associated with a known session to the RETURN chain." -j RETURN
-A neutron-openvswi-ib2d4fa47-0 -m set --match-set NIPv6e50abcfd-5bdb-462d-aa21- src -j RETURN
-A neutron-openvswi-ib2d4fa47-0 -p ipv6-icmp -j RETURN
-A neutron-openvswi-ib2d4fa47-0 -m state --state INVALID -m comment --comment "Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack." -j DROP
-A neutron-openvswi-ib2d4fa47-0 -m comment --comment "Send unmatched traffic to the fallback chain." -j neutron-openvswi-sg-fallback
-A neutron-openvswi-o55f99960-b -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 131 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN
-A neutron-openvswi-o55f99960-b -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 135 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN
-A neutron-openvswi-o55f99960-b -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 143 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN
-A neutron-openvswi-o55f99960-b -j neutron-openvswi-s55f99960-b
-A neutron-openvswi-o55f99960-b -p ipv6-icmp -m icmp6 --icmpv6-type 134 -m comment --comment "Drop IPv6 Router Advts from VM Instance." -j DROP
-A neutron-openvswi-o55f99960-b -p ipv6-icmp -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN
-A neutron-openvswi-o55f99960-b -p udp -m udp --sport 546 -m udp --dport 547 -m comment --comment "Allow DHCP client traffic." -j RETURN
-A neutron-openvswi-o55f99960-b -p udp -m udp --sport 547 -m udp --dport 546 -m comment --comment "Prevent DHCP Spoofing by VM." -j DROP
-A neutron-openvswi-o55f99960-b -m state --state RELATED,ESTABLISHED -m comment --comment "Direct packets associated with a known session to the RETURN chain." -j RETURN
-A neutron-openvswi-o55f99960-b -j RETURN
-A neutron-openvswi-o55f99960-b -m state --state INVALID -m comment --comment "Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack." -j DROP
-A neutron-openvswi-o55f99960-b -m comment --comment "Send unmatched traffic to the fallback chain." -j neutron-openvswi-sg-fallback
-A neutron-openvswi-ob2d4fa47-0 -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 131 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN
-A neutron-openvswi-ob2d4fa47-0 -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 135 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN
-A neutron-openvswi-ob2d4fa47-0 -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 143 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN
-A neutron-openvswi-ob2d4fa47-0 -p ipv6-icmp -m icmp6 --icmpv6-type 134 -m comment --comment "Drop IPv6 Router Advts from VM Instance." -j DROP
-A neutron-openvswi-ob2d4fa47-0 -p ipv6-icmp -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN
-A neutron-openvswi-ob2d4fa47-0 -p udp -m udp --sport 546 -m udp --dport 547 -m comment --comment "Allow DHCP client traffic." -j RETURN
-A neutron-openvswi-ob2d4fa47-0 -p udp -m udp --sport 547 -m udp --dport 546 -m comment --comment "Prevent DHCP Spoofing by VM." -j DROP
-A neutron-openvswi-ob2d4fa47-0 -m state --state RELATED,ESTABLISHED -m comment --comment "Direct packets associated with a known session to the RETURN chain." -j RETURN
-A neutron-openvswi-ob2d4fa47-0 -j RETURN
-A neutron-openvswi-ob2d4fa47-0 -m state --state INVALID -m comment --comment "Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack." -j DROP
-A neutron-openvswi-ob2d4fa47-0 -m comment --comment "Send unmatched traffic to the fallback chain." -j neutron-openvswi-sg-fallback
-A neutron-openvswi-s55f99960-b -s 2001:192:168:99:f816:3eff:fe6f:121f/128 -m mac --mac-source FA:16:3E:6F:12:1F -m comment --comment "Allow traffic from defined IP/MAC pairs." -j RETURN
-A neutron-openvswi-s55f99960-b -s fe80::f816:3eff:fe6f:121f/128 -m mac --mac-source FA:16:3E:6F:12:1F -m comment --comment "Allow traffic from defined IP/MAC pairs." -j RETURN
-A neutron-openvswi-s55f99960-b -m comment --comment "Drop traffic without an IP/MAC allow rule." -j DROP
-A neutron-openvswi-sg-chain -m physdev --physdev-out tap55f99960-ba --physdev-is-bridged -m comment --comment "Jump to the VM specific chain." -j neutron-openvswi-i55f99960-b
-A neutron-openvswi-sg-chain -m physdev --physdev-in tap55f99960-ba --physdev-is-bridged -m comment --comment "Jump to the VM specific chain." -j neutron-openvswi-o55f99960-b
-A neutron-openvswi-sg-chain -m physdev --physdev-out tapb2d4fa47-0e --physdev-is-bridged -m comment --comment "Jump to the VM specific chain." -j neutron-openvswi-ib2d4fa47-0
-A neutron-openvswi-sg-chain -m physdev --physdev-in tapb2d4fa47-0e --physdev-is-bridged -m comment --comment "Jump to the VM specific chain." -j neutron-openvswi-ob2d4fa47-0
-A neutron-openvswi-sg-chain -j ACCEPT
-A neutron-openvswi-sg-fallback -m comment --comment "Default drop rule for unmatched traffic." -j DROP
COMMIT
# Completed on Fri Jan 25 09:47:42 2019


root@juju-a09725-xenial-mitaka-7:~# ovs-vsctl show
4c7ed97c-a113-48c5-840c-252bb0027f19
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
    Bridge br-int
        fail_mode: secure
        Port "qvob2d4fa47-0e"
            tag: 1
            Interface "qvob2d4fa47-0e"
        Port "qvo55f99960-ba"
            tag: 2
            Interface "qvo55f99960-ba"
        Port patch-tun
            Interface patch-tun

root@juju-a09725-xenial-mitaka-7:~# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8958 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:d5:28:84 brd ff:ff:ff:ff:ff:ff
    inet 10.5.0.4/16 brd 10.5.255.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fed5:2884/64 scope link 
       valid_lft forever preferred_lft forever
3: fan-252: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8908 qdisc noqueue state UP group default qlen 1000
    link/ether 62:59:80:9b:95:91 brd ff:ff:ff:ff:ff:ff
    inet 252.0.4.1/8 scope global fan-252
       valid_lft forever preferred_lft forever
    inet6 fe80::6059:80ff:fe9b:9591/64 scope link 
       valid_lft forever preferred_lft forever
4: ftun0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8908 qdisc noqueue master fan-252 state UNKNOWN group default qlen 1000
    link/ether 62:59:80:9b:95:91 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::6059:80ff:fe9b:9591/64 scope link 
       valid_lft forever preferred_lft forever
7: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1
    link/ether 32:5b:19:f0:ed:6d brd ff:ff:ff:ff:ff:ff
8: br-int: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1458 qdisc noqueue state UNKNOWN group default qlen 1
    link/ether 62:64:d4:23:dd:43 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::6064:d4ff:fe23:dd43/64 scope link 
       valid_lft forever preferred_lft forever
9: br-ex: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1
    link/ether 66:87:20:0d:38:4e brd ff:ff:ff:ff:ff:ff
10: br-data: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1
    link/ether a2:e0:9f:bb:06:46 brd ff:ff:ff:ff:ff:ff
    inet6 2001:192:168:99:a0e0:9fff:febb:646/64 scope global mngtmpaddr dynamic 
       valid_lft 82152sec preferred_lft 10152sec
    inet6 fe80::a0e0:9fff:febb:646/64 scope link 
       valid_lft forever preferred_lft forever
11: br-tun: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1
    link/ether 1e:29:d4:32:d7:43 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::1c29:d4ff:fe32:d743/64 scope link 
       valid_lft forever preferred_lft forever
12: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN group default qlen 1
    link/gre 0.0.0.0 brd 0.0.0.0
13: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN group default qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
14: gre_sys@NONE: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65490 qdisc pfifo_fast master ovs-system state UNKNOWN group default qlen 1000
    link/ether ae:90:21:f5:b5:8c brd ff:ff:ff:ff:ff:ff
    inet6 fe80::ac90:21ff:fef5:b58c/64 scope link 
       valid_lft forever preferred_lft forever
15: qbrb2d4fa47-0e: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1458 qdisc noqueue state UP group default qlen 1000
    link/ether ae:48:8e:e0:62:fc brd ff:ff:ff:ff:ff:ff
16: qvob2d4fa47-0e@qvbb2d4fa47-0e: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1458 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether 0a:4c:b6:39:72:ed brd ff:ff:ff:ff:ff:ff
    inet6 fe80::84c:b6ff:fe39:72ed/64 scope link 
       valid_lft forever preferred_lft forever
17: qvbb2d4fa47-0e@qvob2d4fa47-0e: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1458 qdisc noqueue master qbrb2d4fa47-0e state UP group default qlen 1000
    link/ether ae:48:8e:e0:62:fc brd ff:ff:ff:ff:ff:ff
    inet6 fe80::ac48:8eff:fee0:62fc/64 scope link 
       valid_lft forever preferred_lft forever
18: qbr55f99960-ba: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ce:fb:c8:4f:5f:cc brd ff:ff:ff:ff:ff:ff
19: qvo55f99960-ba@qvb55f99960-ba: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether 1a:a0:dc:19:6c:e2 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::18a0:dcff:fe19:6ce2/64 scope link 
       valid_lft forever preferred_lft forever
20: qvb55f99960-ba@qvo55f99960-ba: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue master qbr55f99960-ba state UP group default qlen 1000
    link/ether ce:fb:c8:4f:5f:cc brd ff:ff:ff:ff:ff:ff
    inet6 fe80::ccfb:c8ff:fe4f:5fcc/64 scope link 
       valid_lft forever preferred_lft forever
21: tapb2d4fa47-0e: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1458 qdisc pfifo_fast master qbrb2d4fa47-0e state UNKNOWN group default qlen 1000
    link/ether fe:16:3e:b9:b9:6a brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc16:3eff:feb9:b96a/64 scope link 
       valid_lft forever preferred_lft forever
22: tap55f99960-ba: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master qbr55f99960-ba state UNKNOWN group default qlen 1000
    link/ether fe:16:3e:6f:12:1f brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc16:3eff:fe6f:121f/64 scope link 
       valid_lft forever preferred_lft forever
23: ens7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovs-system state UP group default qlen 1000
    link/ether fa:16:3e:41:29:bf brd ff:ff:ff:ff:ff:ff
    inet6 fe80::f816:3eff:fe41:29bf/64 scope link 
       valid_lft forever preferred_lft forever

                type: patch
                options: {peer=patch-int}
        Port br-int
            Interface br-int
                type: internal
        Port int-br-data
            Interface int-br-data
                type: patch
                options: {peer=phy-br-data}
    Bridge br-tun
        fail_mode: secure
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port "gre-0a05000b"
            Interface "gre-0a05000b"
                type: gre
                options: {df_default="true", in_key=flow, local_ip="10.5.0.4", out_key=flow, remote_ip="10.5.0.11"}
        Port br-tun
            Interface br-tun
                type: internal
    Bridge br-data
        fail_mode: secure
        Port "ens7"
            Interface "ens7"
        Port phy-br-data
            Interface phy-br-data
                type: patch
                options: {peer=int-br-data}
        Port br-data
            Interface br-data
                type: internal
    ovs_version: "2.5.5"

20230407 - 一个分析

为一个问题的纯理论分析，但后来没继续做了，不确定下面的分析是否正确，先记录一下

In theory, if creating a tenant-network 2a0a:7b00:1:2:10::/80 with dhcpv6-stateful enabled based on the ext-net 2a0a:7b00:1:2::/64, we need to do the following things:

1, Use dnsmasq to create a dhcpv6-stateful server

2, Use radvd to create a IPv6 virtual router, configure radvd to dhcpv6-stateful mode, then dhcvpv6-stateful server will be used to set IPv6 address and DHCP for VMs instead of radvd

3, Configure the default route from the intranet to the internet so that the VMs can access the internet

   #inside virtual router
   ip -6 add default 2a0a:7b00:1:2:10::/80 dev qg-xxx
   #on the physical host which hosts the virutal router
   ip -6 route add 2a0a:7b00:1:2::/64 via 2a0a:7b00:1:2::1

4, Enable ipv6 forwarding in virutal router: sysctl -w net.ipv6.conf.all.forwarding=1

5, If we also want to access one VM(eg:2a0a:7b00:1:2:10::3/80) from the internet, we also need to do ND-proxy for every VMs

   #inside virtual router
   sysctl -w net.ipv6.conf.all.proxy_ndp=1
   ip -6 neigh add proxy 2a0a:7b00:1:2:10::3/80 dev qg-xxx

neutron will use dnsmasq to create a dhcpv6-stateful-server(ipv6_address_mode=dhcpv6-stateful) because ipv6_ra_mode is not empty(ipv6_ra_mode=dhcpv6-stateful), now the comment '2023-02-23 17:00 UTC' said the file /var/lib/neutron/dhcp/4403eda7-493a-4b0f-aa17-815aef05657d is valid and there is an entry for the 2a0a:7b00:1:2:10::7, so dhcpv6-stateful-server created by dnsmasq should have no problem.

Reference

[1] http://asdkda.github.io/2016/02/05/ipv6/
[2] http://tldp.org/HOWTO/html_single/Linux+IPv6-HOWTO/#idp57787920
[3] https://jochen.kirstaetter.name/dhcpv6-ipv6-in-your-local-network/
[4] https://www.slideshare.net/shixiongshang1/ipv6-case-study-v26

 类似资料：

相关阅读

相关文章

相关问答

快捷导航： 新手教程 算法原理 架构设计 Java进阶 数据库进阶 大厂专栏 面试经验 编程笔记 编程问答 所有专题 文档资料 工具软件 电子书籍 小牛导航
在线工具： 房贷计算器 个税计算器 Linux命令查询 Json格式化 正则表达式 颜色转换 AES加解密 SHA1加密 MD5加密 毒鸡汤 字数统计 随机密码生成 进制转换 Base64编解码 励志句子
Copyright © 2019-2024 小牛知识库@xnip.cn. All Rights Reserved.