关于nn.ReLU(inplace=True)和nn.ReLU(inplace=False)的区别

可以把inplace当作是一个进行relu运算后输入变化与否的开关，True代表输入改变，False输入保持原样。验证代码如下：

>>> input = torch.randn(7)
>>> input
tensor([ 0.1571, -0.6665, -0.5410, -0.6172,  0.1348,  0.4602,  0.0018])
>>> m = nn.ReLU(inplace=True)
>>> out_m = m(input)
>>> out_m
tensor([0.1571, 0.0000, 0.0000, 0.0000, 0.1348, 0.4602, 0.0018])
>>> input
tensor([0.1571, 0.0000, 0.0000, 0.0000, 0.1348, 0.4602, 0.0018])
>>> n = nn.ReLU(inplace=False)
>>> input = torch.randn(7)
>>> input
tensor([ 1.8938, -0.6148, -0.2590, -0.0688, -0.7195, -0.1793, -1.0633])
>>> out_n = n(input)
>>> out_n
tensor([1.8938, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000])
>>> input
tensor([ 1.8938, -0.6148, -0.2590, -0.0688, -0.7195, -0.1793, -1.0633])