试图用pytorch实现多项式回归时的错误 - 渐变是损失之后的。

Question

我正在尝试使用Pytorch实施自定义的多项式回归，但是在培训过程中，我的实施未能计算梯度。即，即使在lose.backward（）命令之后，权重始终是none。下面我提供了所有必要的细节。

步骤1 我使用以下命令生成一些数据点：

import numpy as np
import torch
import matplotlib.pyplot as plt
from torch.autograd import Function
import torch.nn as nn
from torch.utils.data import Dataset
from torch.utils.data import DataLoader
seed_value = 42
np.random.seed(seed_value)
x = np.sort(np.random.rand(1000))
y = np.cos(1.2 * x * np.pi) + (0.1 * np.random.randn(1000))

然后，我使用从Sklearn的火车测试拆分将数据拆分为培训和测试集。

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(x,y,train_size = 0.7,
                                                   random_state = seed_value)

步骤2 i创建名为poly的自定义函数，该功能返回多项式p（x）= w0+w1x+... w5x^5，在x上评估给定的x权重w。

def poly(x,w,batch_size = 10,degree = 5):
    x = x.repeat(1,degree+1)
    w = w.repeat(batch_size,1)
    exp = torch.arange(0.,degree+1).repeat(batch_size,1)
    return torch.sum(w*torch.pow(x,exp),dim=1)

步骤3 我构造了类custom_dataset，该从Pytorch的数据集继承，将我的培训处理成批处理。

class custom_dataset(Dataset):
    def __init__(self,X,y):
        self.x = torch.from_numpy(X).type(torch.float32).reshape(len(X),1)
        self.y = torch.from_numpy(y).type(torch.float32)
    def __len__(self):
        return len(self.x)
    def __getitem__(self,idx):
        return self.x[idx], self.y[idx]

步骤4 我构造了处理训练程序的循环。

training_data = custom_dataset(X_train,y_train)
test_data = custom_dataset(X_test,y_test)
def training_loop(train_loader, w, epochs, lr, batch_size,
                  loss_fn = nn.MSELoss(), degree = 5):
    weights = torch.tensor(w,dtype = torch.float32, requires_grad = True)
    num_batches = len(train_loader)//batch_size
    for epoch in range(1,epochs+1):
        print(f"{5*'-'}>epoch:{epoch}<{5*'-'}")
        for i,sample in enumerate(train_loader):
            x,y = sample
            y_preds = poly(x,weights,batch_size = batch_size)
            loss = loss_fn(y,y_preds)
            loss.backward() # backpropagation
            weights = weights - lr*weights.grad # update - gradient descent
            
            if (i+1) % 100 == 0:
                print(f"- Batch:[{i+1}|{num_batches}]{5*' '}Samples:[{(i+1)*num_batches}|{len(train_loader)}]{5*' '}loss:{loss.item():.6f}")         
    return w

步骤5 我开始训练...

epochs = 10
lr = 1e-3
batch_size = 10
degree = 5
train_loader = DataLoader(training_data, batch_size = batch_size,
                         shuffle = True)
test_loader = DataLoader(test_data, batch_size = batch_size,
                        shuffle = True)
w = [0]*(degree+1)
w = training_loop(train_loader, w = w, epochs = 30, lr = lr,
                  batch_size = batch_size)

并获得以下错误

---------------------------------------------------------------------------  TypeError                                 Traceback (most recent call last) Input In [40], in <cell line: 10>()
      7 test_loader = DataLoader(test_data, batch_size = batch_size,
      8                         shuffle = True)
      9 w = [0]*(degree+1)
---> 10 w = training_loop(train_loader, w = w, epochs = 30, lr = lr,
     11                   batch_size = batch_size)

Input In [39], in training_loop(train_loader, w, epochs, lr, batch_size, loss_fn, degree)
     10 loss = loss_fn(y,y_preds)
     11 loss.backward() # backpropagation
---> 12 weights = weights - lr*weights.grad # update - gradient descent
     14 if (i+1) % 100 == 0:
     15     print(f"- Batch:[{i+1}|{num_batches}{5*' '}Samples:[{(i+1)*num_batches}|{len(train_loader)}]{5*' '}loss:{loss.item():.6f}")         

TypeError: unsupported operand type(s) for *: 'float' and 'NoneType'

，这意味着梯度的计算不会影响变量权重，因为它仍然设置为无。你知道怎么了吗？

Answer 1

您正在覆盖第一个循环迭代中的strights变量，该迭代将被strights 替换，而没有grad attribute 。可以使用以下最小代码复制此行为：

>>> weights.grad =  torch.ones_like(weights)
>>> for i in range(2):
...     print(weights.grad)
...     weights = weights - weights.grad

tensor([1., 1.])
None 

为了解决此问题，您可以使用现场操作更新参数：

        weights -= lr*weights.grad # update - gradient descent