Microsoft Cognitive Toolkit 简明教程

CNTK - Regression Model

在此处，我们将学习关于衡量回归模型的性能。

Here, we will study about measuring performance with regards to a regression model.

Basics of validating a regression model

众所周知，回归模型不同于分类模型，因为对于个体样本，没有正确或错误的二进制度量。在回归模型中，我们希望衡量预测值与实际值之间的接近程度。预测值与预期输出越接近，模型的性能就越好。

As we know that regression models are different than classification models, in the sense that, there is no binary measure of right or wrong for individuals’ samples. In regression models, we want to measure how close the prediction is to the actual value. The closer the prediction value is to the expected output, the better the model performs.

在此处，我们将使用不同的误差率函数来衡量用于回归的神经网络的性能。

Here, we are going to measure the performance of NN used for regression using different error-rate functions.

Calculating error margin

如前所述，在验证回归模型时，我们无法确定预测是正确还是错误。我们希望预测尽可能接近实际值。但是，在此处可以接受较小的误差范围。

As discussed earlier, while validating a regression model, we can’t say whether a prediction is right or wrong. We want our prediction to be as close as possible to the real value. But, a small error margin is acceptable here.

计算误差范围的公式如下 −

The formula for calculating the error margin is as follows −

error margin

在此，

Here,

Predicted value = 由帽指示的 y

Predicted value = indicated y by a hat

Real value = 由 y 预测

Real value = predicted by y

首先，我们需要计算预测值与实际值之间的距离。然后，为了获得总体误差率，我们需要对这些平方距离求和并计算平均值。这被称为 mean squared 误差函数。

First, we need to calculate the distance between the predicted and the real value. Then, to get an overall error rate, we need to sum these squared distances and calculate the average. This is called the mean squared error function.

但是，如果我们想要表示误差范围的性能数据，我们需要一个表示绝对误差的公式。 mean absolute 误差函数的公式如下 −

But, if we want performance figures that express an error margin, we need a formula that expresses the absolute error. The formula for mean absolute error function is as follows −

mean absolute

以上公式获取预测值与实际值之间的绝对距离。

The above formula takes the absolute distance between the predicted and the real value.

Using CNTK to measure regression performance

在此处，我们将了解如何与 CNTK 结合使用我们在讨论中提到的不同指标。我们将使用回归模型，根据以下步骤预测汽车每加仑英里的行程。

Here, we will look at how to use the different metrics, we discussed in combination with CNTK. We will use a regression model, that predicts miles per gallon for cars using the steps given below.

Implementation steps−

Step 1 − 首先，我们需要从 cntk 包导入所需组件，如下所示 −

Step 1 − First, we need to import the required components from cntk package as follows −

from cntk import default_option, input_variable
from cntk.layers import Dense, Sequential
from cntk.ops import relu

Step 2 − 接下来，我们需要使用 default_options 函数定义一个默认激活函数。然后，创建一个新的顺序层集并提供两个各具有 64 个神经元的稠密层。然后，我们将一个额外的稠密层（将充当输出层）添加到顺序层集并提供 1 个没有激活的神经元，如下所示 −

Step 2 − Next, we need to define a default activation function using the default_options functions. Then, create a new Sequential layer set and provide two Dense layers with 64 neurons each. Then, we add an additional Dense layer (which will act as the output layer) to the Sequential layer set and give 1 neuron without an activation as follows −

with default_options(activation=relu):
model = Sequential([Dense(64),Dense(64),Dense(1,activation=None)])

Step 3 − 创建网络后，我们需要创建一个输入特征。我们需要确保该特征与我们即将用于训练的特征具有相同的形状。

Step 3 − Once the network has been created, we need to create an input feature. We need to make sure that, it has the same shape as the features that we are going to be using for training.

features = input_variable(X.shape[1])

Step 4 − 现在，我们需要创建一个大小为 1 的另一个 input_variable 。它将用于存储神经网络的预期值。

Step 4 − Now, we need to create another input_variable with size 1. It will be used to store the expected value for NN.

target = input_variable(1)
z = model(features)

现在，我们需要训练模型，为此，我们将拆分数据集并使用以下实现步骤执行预处理 −

Now, we need to train the model and in order to do so, we are going to split the dataset and perform preprocessing using the following implementation steps −

Step 5 − 首先，从 sklearn.preprocessing 导入 StandardScaler 以获取 -1 到 +1 之间的值。这将有助于防止神经网络中梯度爆炸问题。

Step 5 −First, import StandardScaler from sklearn.preprocessing to get the values between -1 and +1. This will help us against exploding gradient problems in the NN.

from sklearn.preprocessing import StandardScalar

Step 6 − 接下来，从 sklearn.model_selection 中导入 train_test_split，如下所示 −

Step 6 − Next, import train_test_split from sklearn.model_selection as follows−

from sklearn.model_selection import train_test_split

Step 7 − 使用 drop*method. At last split the dataset into a training and validation set using the *train_test_split 函数按照如下方法在数据集中删除 mpg 列 −

Step 7 − Drop the mpg column from the dataset by using the drop*method. At last split the dataset into a training and validation set using the *train_test_split function as follows −

x = df_cars.drop(columns=[‘mpg’]).values.astype(np.float32)
y=df_cars.iloc[: , 0].values.reshape(-1, 1).astype(np.float32)
scaler = StandardScaler()
X = scaler.fit_transform(x)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

Step 8 − 现在，我们需要创建另一个大小为 1 的 input_variable。它将用于存储 NN 的预期值。

Step 8 − Now, we need to create another input_variable with size 1. It will be used to store the expected value for NN.

target = input_variable(1)
z = model(features)

我们在分割和预处理了数据后，现在我们需要训练 NN。与创建回归模型时的前几个部分一样，我们需要定义一个损失函数和 metric 函数的组合训练模型。

We have split as well as preprocessed the data, now we need to train the NN. As did in previous sections while creating regression model, we need to define a combination of a loss and metric function to train the model.

import cntk
def absolute_error(output, target):
   return cntk.ops.reduce_mean(cntk.ops.abs(output – target))
@ cntk.Function
def criterion_factory(output, target):
   loss = squared_error(output, target)
   metric = absolute_error(output, target)
   return loss, metric

现在，我们来看一下如何使用训练的模型。对于我们的模型，我们将 criterion_factory 用作损失函数和度量值组合。

Now, let’s have a look at how to use the trained model. For our model, we will use criterion_factory as the loss and metric combination.

from cntk.losses import squared_error
from cntk.learners import sgd
from cntk.logging import ProgressPrinter
progress_printer = ProgressPrinter(0)
loss = criterion_factory (z, target)
learner = sgd(z.parameters, 0.001)
training_summary=loss.train((x_train,y_train),parameter_learners=[learner],callbacks=[progress_printer],minibatch_size=16,max_epochs=10)

Complete implementation example

from cntk import default_option, input_variable
from cntk.layers import Dense, Sequential
from cntk.ops import relu
with default_options(activation=relu):
model = Sequential([Dense(64),Dense(64),Dense(1,activation=None)])
features = input_variable(X.shape[1])
target = input_variable(1)
z = model(features)
from sklearn.preprocessing import StandardScalar
from sklearn.model_selection import train_test_split
x = df_cars.drop(columns=[‘mpg’]).values.astype(np.float32)
y=df_cars.iloc[: , 0].values.reshape(-1, 1).astype(np.float32)
scaler = StandardScaler()
X = scaler.fit_transform(x)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
target = input_variable(1)
z = model(features)
import cntk
def absolute_error(output, target):
   return cntk.ops.reduce_mean(cntk.ops.abs(output – target))
@ cntk.Function
def criterion_factory(output, target):
loss = squared_error(output, target)
metric = absolute_error(output, target)
return loss, metric
from cntk.losses import squared_error
from cntk.learners import sgd
from cntk.logging import ProgressPrinter
progress_printer = ProgressPrinter(0)
loss = criterion_factory (z, target)
learner = sgd(z.parameters, 0.001)
training_summary=loss.train((x_train,y_train),parameter_learners=[learner],callbacks=[progress_printer],minibatch_size=16,max_epochs=10)

Output

-------------------------------------------------------------------
average  since   average   since  examples
loss     last    metric    last
------------------------------------------------------
Learning rate per minibatch: 0.001
690       690     24.9     24.9       16
654       636     24.1     23.7       48
[………]

为了验证我们的回归模型，我们需要确保模型处理新数据的效果与处理训练数据一样好。为此，我们需要使用测试数据在 loss 和 metric 组合上调用 test 方法，如下所示 −

In order to validate our regression model, we need to make sure that, the model handles new data just as well as it does with the training data. For this, we need to invoke the test method on loss and metric combination with test data as follows −

loss.test([X_test, y_test])

Output−

{'metric': 1.89679785619, 'samples': 79}