用Python实现回归/分类神经网络

Question

问题背景

我正在尝试用Python学习神经网络，我开发了一个基于Logistic回归的NN的实现。在

这是密码-

import numpy as np

# Input array
X = np.array([[1, 0, 1, 0], [1, 0, 1, 1], [0, 1, 0, 1]])

# Output
y = np.array([[1], [1], [0]])


# Sigmoid Function
def sigmoid(x):
    return 1 / (1 + np.exp(-x))


# Derivative of Sigmoid Function
def ddx_sigmoid(x):
    return x * (1 - x)


#####   Initialization - BEGIN  #####


# Setting training iterations
iterations_max = 500000

# Learning Rate
alpha = 0.5

# Number of Neruons in Input Layer = Number of Features in the data set
inputlayer_neurons = X.shape[1]

# Number of Neurons in the Hidden Layer
hiddenlayer_neurons = 3  # number of hidden layers neurons

# Number of Neurons at the Output Layer
output_neurons = 1  # number of neurons at output layer

# weight and bias initialization
wh = np.random.uniform(size=(inputlayer_neurons, hiddenlayer_neurons))
bh = np.random.uniform(size=(1, hiddenlayer_neurons))
wout = np.random.uniform(size=(hiddenlayer_neurons, output_neurons))
bout = np.random.uniform(size=(1, output_neurons))

#####   Initialization - END  #####

# Printing of shapes

print "\nShape X: ", X.shape, "\nShape Y: ", y.shape
print "\nShape WH: ", wh.shape, "\nShape BH: ", bh.shape, "\nShape Wout: ", wout.shape, "\nShape Bout: ", bout.shape

# Printing of Values
print "\nwh:\n", wh, "\n\nbh: ", bh, "\n\nwout:\n", wout, "\n\nbout: ", bout

#####   TRAINING - BEGIN  #####
for i in range(iterations_max):
    #####   Forward Propagation - BEGIN   #####

    # Input to Hidden Layer = (Dot Product of Input Layer and Weights) + Bias
    hidden_layer_input = (np.dot(X, wh)) + bh

    # Activation of input to Hidden Layer by using Sigmoid Function
    hiddenlayer_activations = sigmoid(hidden_layer_input)

    # Input to Output Layer = (Dot Product of Hidden Layer Activations and Weights) + Bias
    output_layer_input = np.dot(hiddenlayer_activations, wout) + bout

    # Activation of input to Output Layer by using Sigmoid Function
    output = sigmoid(output_layer_input)

    #####   Forward Propagation - END #####

    #####   Backward Propagation - BEGIN   #####

    E = y - output

    slope_output_layer = ddx_sigmoid(output)
    slope_hidden_layer = ddx_sigmoid(hiddenlayer_activations)

    d_output = E * slope_output_layer

    Error_at_hidden_layer = d_output.dot(wout.T)
    d_hiddenlayer = Error_at_hidden_layer * slope_hidden_layer

    wout += hiddenlayer_activations.T.dot(d_output) * alpha
    bout += np.sum(d_output, axis=0, keepdims=True) * alpha

    wh += X.T.dot(d_hiddenlayer) * alpha
    bh += np.sum(d_hiddenlayer, axis=0, keepdims=True) * alpha

    #####   Backward Propagation - END   #####
#####   TRAINING - END  #####

print "\nOutput is:\n", output

这段代码在输出是二进制（0,1）的情况下工作得很好。我想，这是因为我使用的sigmoid函数。在

问题

现在，我想缩放这段代码，以便它也能处理线性回归。在

众所周知，scikit库有一些预加载的数据集，可用于分类和回归。在

我想让NN训练和测试diabetes数据集。在

考虑到这一点，我修改了我的代码如下-

^{pr2}$

此代码的输出是-

Output is:
[[ 1.  1.  1.]
 [ 1.  1.  1.]
 [ 1.  1.  1.]
 ..., 
 [ 1.  1.  1.]
 [ 1.  1.  1.]
 [ 1.  1.  1.]]

很明显，我在做一些基本的事情。在

这是因为我在隐藏层和输出层使用sigmoid函数吗？在

我应该使用什么样的函数才能得到一个有效的输出，从而有效地训练我的神经网络？在

迄今为止的努力

我尝试过使用TANH函数，SOFTPLUS函数来激活这两个层，但没有成功。在

有人能帮忙吗？在

我试着用谷歌搜索这个，但其中的解释非常复杂。在

救命啊！在

Answer 1

您应该尝试删除输出上的sigmoid函数。在

对于线性回归，输出范围可能很大，而sigmoid或tanh函数的输出是[0,1]或[-1,1]，这使得误差函数的最小化成为可能。在

=======更新=====

我试图在tensorflow中完成它，它的核心部分是：

w = tf.Variable(tf.truncated_normal([features, FLAGS.hidden_unit], stddev=0.35))
b = tf.Variable(tf.zeros([FLAGS.hidden_unit]))

# right way
y = tf.reduce_sum(tf.matmul(x, w) + b, 1)

# wrong way: as sigmoid output in [-1, 1]
# y = tf.sigmoid(tf.matmul(x, w) + b)

mse_loss = tf.reduce_sum(tf.pow(y - y_, 2) / 2