千层面，MLP零输出

Question

当我试图学习一些简单的MLP时，我得到了奇怪的结果，在把代码从所有东西中剥离出来之后，除了重要的东西，缩小了它，我仍然得到了奇怪的结果。在

代码：

import numpy as np
import theano
import theano.tensor as T
import lasagne


dtype = np.float32
states = np.eye(3, dtype=dtype).reshape(3,1,1,3)
values = np.array([[147, 148, 135,147], [147,147,149,148], [148,147,147,147]], dtype=dtype)
output_dim = values.shape[1]
hidden_units = 50

#Network setup
inputs = T.tensor4('inputs')
targets = T.matrix('targets')

network = lasagne.layers.InputLayer(shape=(None, 1, 1, 3), input_var=inputs)
network = lasagne.layers.DenseLayer(network, 50, nonlinearity=lasagne.nonlinearities.rectify)
network = lasagne.layers.DenseLayer(network, output_dim)

prediction = lasagne.layers.get_output(network)
loss = lasagne.objectives.squared_error(prediction, targets).mean()
params = lasagne.layers.get_all_params(network, trainable=True)
updates = lasagne.updates.sgd(loss, params, learning_rate=0.01)

f_learn = theano.function([inputs, targets],  loss, updates=updates)
f_test = theano.function([inputs], prediction)


#Training
it = 5000
for i in range(it):
    l = f_learn(states, values)
    print "Loss: " + str(l)
    print "Expected:"
    print values
    print "Learned:"
    print f_test(states)
    print "Last layer weights:"
    print lasagne.layers.get_all_param_values(network)[-1]

我希望网络能够学习“values”变量中给定的值，而且通常是这样，但同样经常会给一些输出节点留下零和巨大的损失。在

样本输出：

^{pr2}$

为什么会这样？在

Answer 1

我曾在谷歌千层面集团问过同样的问题，我在那里更幸运：https://groups.google.com/forum/#!topic/lasagne-users/ock-2RqTaFk 将背诵单元改为能容忍负输出的非线性单元有助于提高效率。在