Conv1D不更新权重。（全零）和测试输出始终相同，等于最后一层权重？

Question

我想用1D CNN来预测第二天的太阳能。时间序列数据分辨率为一小时，长度为一年。我正在用第一天的数据训练模型，以预测第二天。xtrain=第1天，ytrain=第2天，xtest=第3天，以预测第4天。

24小时数据输入-

我对数据进行了10天的培训（样本）；然后提前4天预测，问题是CNN总是给出相同的输出值，无论输入是什么。然后，我检查了权重，只有输出层权重为非零。此外，输出值不会随输入集的不同而变化。

数据链接：https://mega.nz/#哦！NpoTzIBJ！U5l8ToQgcJ6xif2tMjIrXuace3skhrtwLEdeoWe\U FkM

代码：

import pandas
from pandas import Series
from pandas import DataFrame
import keras
from keras.callbacks import ModelCheckpoint
from sklearn.metrics import r2_score
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation, regularizers, initializers
from keras.layers import Conv1D,  MaxPooling1D, Flatten, AveragePooling1D
from keras.activations import *
from keras.losses import *
from keras.optimizers import *
from keras.utils import plot_model
from keras.models import Sequential
from keras.utils import plot_model
import numpy as np
import matplotlib.pyplot as plt
from sklearn import preprocessing
from numpy import zeros, newaxis
from keras.callbacks import EarlyStopping
import sklearn.metrics
from sklearn.metrics import mean_squared_error
from math import sqrt

data = pandas.read_csv("meas.csv", header=0)
dataset = data.values[:,1]

all_pred_data = []
ytest_all = []

model = Sequential()
model.add(Conv1D(3,kernel_size=3,activation='relu', input_shape=xtrain.shape[1:3],kernel_initializer=initializers.RandomUniform(minval=-1, maxval=1),kernel_regularizer=regularizers.l2(0.1))) #input_shape=()
model.add(AveragePooling1D(pool_size=3))
model.add(Conv1D(3,kernel_size=3,activation='relu', input_shape=xtrain.shape[1:3],kernel_initializer=initializers.RandomUniform(minval=-1, maxval=1),kernel_regularizer=regularizers.l2(0.1))) #input_shape=()
model.add(AveragePooling1D(pool_size=3))
model.add(Flatten())
model.add(Dense(42,activation='tanh',kernel_regularizer=regularizers.l2(0.1),kernel_initializer=initializers.RandomUniform(minval=-1, maxval=1)))
model.add(Dense(24, activation='linear',kernel_regularizer=regularizers.l2(0.1),kernel_initializer=initializers.RandomUniform(minval=-1, maxval=1)))

model.compile(loss='mse',
          optimizer=keras.optimizers.Adam(),
          metrics=['mae','accuracy'])


for i in range(0,10-1,1): 

    xtrain = dataset[24*(i+1)-24:24*(i+1)]
    ytrain = dataset[24*(i+1):24*(i+2)]

    xtrain = xtrain.reshape(1,24,1)
    ytrain = ytrain.reshape(1,24)

    model.fit(xtrain,ytrain,epochs=500,verbose=2) 


# TEST
for i in range(20,25-1,1):
    xtest = dataset[24*(i+1):24*(i+2)] #(i+1):(i+6+1)
    ytest = dataset[24*(i+2):24*(i+2)+24]

    xtest = xtest.reshape(1,xtrain.shape[1], 1)

    pred_data = np.round(model.predict(xtest),3)

    pred_data_transpose = pred_data.transpose()

    all_pred_data.extend(pred_data_transpose)
    ytest_all.extend(np.round(ytest,3))

Answer 1

如果权重全部为0，它们将永远不会改变，因为梯度将始终为0，为了避免这种情况，请尝试规范化您的数据，从而更容易训练网络，如果规范化输入数据还不够，请尝试使用batch_normalization