预测函数的DeepLearning4j神经网络不收敛

Question

我试图在DL4j中做一个简单的预测（稍后将用于具有n个特性的大型数据集），但无论我做什么，我的网络都不想学习，行为非常奇怪。当然，我学习了所有的教程，并执行了dl4j repo中显示的相同步骤，但不知何故，它对我不起作用。

对于虚拟特性，我使用以下数据：

*双[val][x]特征；其中val=linspace(-10，10)...；和x=math.sqrt(math.abs(val))*val；

DataSetIterator dataset_train_iter = getTrainingData(x_features, y_outputs_train, batchSize, rnd);
    DataSetIterator dataset_test_iter = getTrainingData(x_features_test, y_outputs_test, batchSize, rnd);

    // Normalize data, including labels (fitLabel=true)
    NormalizerMinMaxScaler normalizer = new NormalizerMinMaxScaler(0, 1);
    normalizer.fitLabel(false);
    normalizer.fit(dataset_train_iter);              
    normalizer.fit(dataset_test_iter);

    // Use the .transform function only if you are working with a small dataset and no iterator
    normalizer.transform(dataset_train_iter.next());
    normalizer.transform(dataset_test_iter.next());

    dataset_train_iter.setPreProcessor(normalizer);
    dataset_test_iter.setPreProcessor(normalizer);

    //DataSet setNormal = dataset.next();

MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
                .seed(seed)
                .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
                .weightInit(WeightInit.XAVIER)
                //.miniBatch(true)
                //.l2(1e-4)
                //.activation(Activation.TANH)
                .updater(new Nesterovs(0.1,0.3))
                .list()
                .layer(new DenseLayer.Builder().nIn(numInputs).nOut(20).activation(Activation.TANH)
                        .build())
                .layer(new DenseLayer.Builder().nIn(20).nOut(10).activation(Activation.TANH)
                        .build())
                .layer( new DenseLayer.Builder().nIn(10).nOut(6).activation(Activation.TANH)
                        .build())
                .layer(new OutputLayer.Builder(LossFunctions.LossFunction.MSE)
                        .activation(Activation.IDENTITY)
                        .nIn(6).nOut(1).build())
                .build();

final MultiLayerNetwork net = new MultiLayerNetwork(conf);
net.init();
    net.setListeners(new ScoreIterationListener(100));
    //Train the network on the full data set, and evaluate in periodically
    final INDArray[] networkPredictions = new INDArray[nEpochs / plotFrequency];
    for (int i = 0; i < nEpochs; i++) {
        //in fit we have already Backpropagation. See Release deeplearning
        // https://deeplearning4j.konduit.ai/release-notes/1.0.0-beta3
        net.fit(dataset_train_iter);
        dataset_train_iter.reset();
        if((i+1) % plotFrequency == 0)  networkPredictions[i/ plotFrequency] = net.output(x_features, false);
    }

    dataset_test_iter.reset();
    dataset_train_iter.reset();

    INDArray predicted = net.output(dataset_test_iter, false);
    System.out.println("PREDICTED ARRAY                " + predicted);
    INDArray output_train = net.output(dataset_train_iter, false);

    //Revert data back to original values for plotting
    // normalizer.revertLabels(predicted);
    normalizer.revertLabels(output_train);
    normalizer.revertLabels(predicted);

    PlotUtil.plot(om, y_outputs_train, networkPredictions);

Answer 1

一个很常见的问题是人们在做玩具问题，比如DL4J的小批量假设（99%的问题都是这样）。您实际上并不是在进行minibatch学习（这实际上违背了实际使用迭代器的观点，迭代器的目的是遍历数据集的切片，而不是内存中的小型数据集）--一个小的建议是只使用普通的数据集api（它是从dataset.next（）返回的）

使用：.minibatch（false）确保关闭dl4j分配给所有损失的minibatch惩罚-您可以在以下位置看到该配置:https://github.com/eclipse/deeplearning4j/blob/master/deeplearning4j/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/conf/neuralnetconfiguration.java#l434

测试这种行为的单元测试可以在下面找到:https://github.com/eclipse/deeplearning4j/blob/b4047006ac8175df295c2f3c008e7601437ea4dc/deeplearning4j/deeplearning4j-core/src/test/java/org/deeplearning4j/gradientcheck/gradientchecktests.java#l94

        MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder().miniBatch(false)
                .dataType(DataType.DOUBLE)
                .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT).updater(new NoOp())
                .list()
                .layer(0,
                        new DenseLayer.Builder().nIn(4).nOut(3)
                                .dist(new NormalDistribution(0, 1))
                                .activation(Activation.TANH)
                                .build())
                .layer(1, new OutputLayer.Builder(LossFunction.MCXENT)
                        .activation(Activation.SOFTMAX).nIn(3).nOut(3).build())
                .build();