梯度下降、残差平方和溢出的线性回归

Question

你好，我试图学习回归算法，并与此同时，我试图实现梯度下降的线性回归，并使用平方的残差和来确定收敛性。我注意到在迭代的某个时候，平方的残差和的求值，我认为这是有意义的，但我不知道如何解决这个问题。我做错了什么吗？

import math
import numpy as num

def get_regression_predictions(input_feature, intercept, slope):
   predicted_output = [intercept + xi*slope for xi in input_feature]
   return(predicted_output)

def get_residual_sum_of_squares(input_feature, output, intercept,slope):
   return num.sum( [(output.iloc[i] - (intercept + slope*input_feature.iloc[i]))**2 for i in range(0,len(output))] )

def train(input_feature,output,intercept,slope):
#the start value of intercept and slope are 0
    last = 0
    now = math.sqrt(get_residual_sum_of_squares(input_feature,output,intercept,slope))

    while abs(last - now) >= 0.01:
        last = now
        predictions = get_regression_predictions(input_feature,intercept,slope)
        errors = [output.iloc[i] - predictions[i] for i in range(0,len(predictions))]


        adjustements = (sum(errors)*0.05,sum([errors[i]*output.iloc[i] for i in range(0,len(errors))] ) *0.05)

        intercept ,slope = (intercept - adjustements[0],slope - adjustements[1] )
        now = math.sqrt(get_residual_sum_of_squares(input_feature,output,intercept,slope))

 return intercept,slope 

Answer 1

尝试随机梯度下降。您可能总结了许多导致溢出的实例错误。