权重在评估步骤十上衰减

Question

我的体重定义为

weights = {
        'W_conv1': tf.get_variable('W_conv1', shape=[...], dtype=tf.float32, initializer=tf.truncated_normal_initializer(stddev=0.01)),
        'W_conv2': tf.get_variable('W_conv2', shape=[...], dtype=tf.float32, initializer=tf.truncated_normal_initializer(stddev=0.01)),
        'W_conv3': tf.get_variable('W_conv3', shape=[...], dtype=tf.float32, initializer=tf.truncated_normal_initializer(stddev=0.01)),
        ...
}

# conv2d network
...

我想使用权重衰减，所以我添加，例如，参数

regularizer=tf.contrib.layers.l1_regularizer(0.0005)

到tf.get_variable。现在我想知道在求值阶段这是否仍然正确，或者我必须将正则化因子设置为0

还有另一个参数trainable。文件上写着If True also add the variable to the graph collection GraphKeys.TRAINABLE_VARIABLES.，我不清楚。我应该用它吗

有人能向我解释一下，如果权重以某种错误的方式衰减影响评估步骤吗？在那种情况下我怎么解决

Answer 1

在反向传播训练期间的权重更新步骤中使用权重衰减。在评估过程中没有这样的权重更新，因此在评估过程中没有任何影响。至于可训练的论点，我认为这段摘自官方文件的摘录相当简洁

When building a machine learning model it is often convenient to distinguish between variables holding the trainable model parameters and other variables such as a global step variable used to count training steps. To make this easier, the variable constructor supports a trainable= parameter. If True, the new variable is also added to the graph collection GraphKeys.TRAINABLE_VARIABLES. The convenience function trainable_variables() returns the contents of this collection. The various Optimizer classes use this collection as the default list of variables to optimize.

因此，您可以添加它以方便使用，但它不是强制性的