I am trying to implement a custom dataset for my neural network. But got this error when running the forward function. The code is as follows.
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import Dataset, DataLoader
import numpy as np
class ParamData(Dataset):
def __init__(self,file_name):
self.data = torch.Tensor(np.loadtxt(file_name,delimiter = ',')) #first place
def __len__(self):
return self.data.size()[0]
def __getitem__(self,i):
return self.data[i]
class Net(nn.Module):
def __init__(self,in_size,out_size,layer_size=200):
super(Net,self).__init__()
self.layer = nn.Linear(in_size,layer_size)
self.out_layer = nn.Linear(layer_size,out_size)
def forward(self,x):
x = F.relu(self.layer(x))
x = self.out_layer(x)
return x
datafile = 'data1.txt'
net = Net(100,1)
dataset = ParamData(datafile)
n_samples = len(dataset)
#dataset = torch.Tensor(dataset,dtype=torch.double) #second place
#net.float() #thrid place
net.forward(dataset[0]) #fourth place
In the file data1.txt
is a csv formatted text file containing certain numbers, and each dataset[i]
is a size 100 by 1 torch.Tensor
object of dtype torch.float64
. The error message is as follows:
Traceback (most recent call last):
File "Z:\Wrong.py", line 33, in <module>
net.forward(dataset[0])
File "Z:\Wrong.py", line 23, in forward
x = F.relu(self.layer(x))
File "E:\Python38\lib\site-packages\torch\nn\modules\module.py", line 532, in __call__
result = self.forward(*input, **kwargs)
File "E:\Python38\lib\site-packages\torch\nn\modules\linear.py", line 87, in forward
return F.linear(input, self.weight, self.bias)
File "E:\Python38\lib\site-packages\torch\nn\functional.py", line 1372, in linear
output = input.matmul(weight.t())
RuntimeError: Expected object of scalar type Double but got scalar type Float for argument #2 'mat2' in call to _th_mm
It seems that I should change the dtype of the numbers in dataset
to torch.double
. I tried things like
changing the line at the first place to self.data = torch.tensor(np.loadtxt(file_name,delimiter = ','),dtype=torch.double)
changing the line at the fourth place to net.forward(dataset[0].double())
I think these are the solutions I have seen from similar questions, but they either give new errors or don't do anything. What should I do?
Update: So I got it working by changing the first place to
self.data = torch.from_numpy(np.loadtxt(file_name,delimiter = ',')).float()
which is weird because it is exactly the opposite of the error message. Is this a bug? I'd still like some explaining.
解释:
n short: your data has type double but your model has type float, this is not allowed in pytorch because only data with the same dtype can be fed into the model.
In long: This issue is related to the default dtype of PyTorch and Numpy. I will first explain why this error happens and then suggest some solutions(but I think you will not need my solution once you understand the principle.)
torch.float32
(aka torch.float
)torch.float64
(aka torch.double
)It's important to know the default dtype of PyTorch Tensors is torch.float32
(aka torch.float
). This means when you create a tensor, its default dtype is torch.float32
.try: torch.ones(1).dtype
. This will print torch.float32
in default case. And also the model's parameters are of this dtype by default.
In your case, net = Net(100,1)
will create a model whose dtype of parameters are torch.float32
Then we need to talk about Numpy:
The default dtype of Numpy ndarray is numpy.float64
. This means when you create a numpy array, its default dtype is numpy.float64
.try: np.ones(1).dtype
. This will print dtype('float64')
in default case.
In your case, your data come from a local file loaded by np.loadtxt
, so the data is first loaded as dtype('float64')
(as a numpy array) and then converted to a torch tensor of dtype torch.float64
(aka torch.double
). This is what happens when you convert a numpy array to torch tensor: they will have the corresponding dtype.
I think now the issue is pretty clear, you have a model whose parameters are of torch.float32
(aka torch.float
) but tries to run it on data of torch.float64
(aka torch.double
). This is also what the error message tries to say:Expected object of scalar type Double but got scalar type Float for argument
Solutions:
torch.float32
by calling tensor.float()