Artificial Neural Network 简明教程

Learning Vector Quantization

学习向量量化 (LVQ) 不同于矢量量化 (VQ) 和科霍宁自组织映射 (KSOM)，它本质上是一种竞争性网络，使用监督学习。我们可以将其定义为一个对模式进行分类的过程，其中每个输出单元代表一个类别。由于它使用监督学习，因此将会给网络提供一组具有已知分类的训练模式，以及输出类别的一个初始分布。在完成训练过程后，LVQ 将通过将输入向量分配到与输出单元相同的类别，对该向量进行分类。

Learning Vector Quantization (LVQ), different from Vector quantization (VQ) and Kohonen Self-Organizing Maps (KSOM), basically is a competitive network which uses supervised learning. We may define it as a process of classifying the patterns where each output unit represents a class. As it uses supervised learning, the network will be given a set of training patterns with known classification along with an initial distribution of the output class. After completing the training process, LVQ will classify an input vector by assigning it to the same class as that of the output unit.

Architecture

下图展示了 LVQ 的架构，该架构与 KSOM 的架构相当相似。正如我们所见，有 “n” 个输入单元和 “m” 个输出单元。这些层完全互联，并在其上设置权重。

Following figure shows the architecture of LVQ which is quite similar to the architecture of KSOM. As we can see, there are “n” number of input units and “m” number of output units. The layers are fully interconnected with having weights on them.

Parameters Used

以下是 LVQ 训练过程中以及流程图中使用的参数：

Following are the parameters used in LVQ training process as well as in the flowchart

x = training vector (x1,…,xi,…,xn)
T = class for training vector x
wj = weight vector for jth output unit
Cj = class associated with the jth output unit

Training Algorithm

Step 1 - 初始化参考向量，如下操作：

Step 1 − Initialize reference vectors, which can be done as follows −

Step 1(a) − From the given set of training vectors, take the first “m” (number of clusters) training vectors and use them as weight vectors. The remaining vectors can be used for training.
Step 1(b) − Assign the initial weight and classification randomly.
Step 1(c) − Apply K-means clustering method.

Step 2 - 初始化参考向量α

Step 2 − Initialize reference vector $\alpha$

Step 3 - 如果未达到该算法的停止条件，则继续步骤 4-9。

Step 3 − Continue with steps 4-9, if the condition for stopping this algorithm is not met.

Step 4 - 对每个训练输入向量 x 按照步骤 5-6 操作。

Step 4 − Follow steps 5-6 for every training input vector x.

Step 5 - 计算 j = 1 to m 和 i = 1 to n 的欧氏距离平方

Step 5 − Calculate Square of Euclidean Distance for j = 1 to m and i = 1 to n

D(j)\:=\:\displaystyle\sum\limits_{i=1} ^{n\displaystyle\sum\limits_{j=1}} m (x_{i}\:-\:w_{ij})^2

D(j)\:=\:\displaystyle\sum\limits_{i=1}^{n\displaystyle\sum\limits_{j=1}}m (x_{i}\:-\:w_{ij})^2

Step 6 - 获得 D(j) 最小的获胜单元 J 。

Step 6 − Obtain the winning unit J where D(j) is minimum.

Step 7 - 通过以下关系计算获胜单元的新权重：

Step 7 − Calculate the new weight of the winning unit by the following relation −

如果 T = Cj 则 $w_{j}(new)\:=\:w_{j}(old)\:+\:\alpha[x\:-\:w_{j}(old)]$

if T = Cj then $w_{j}(new)\:=\:w_{j}(old)\:+\:\alpha[x\:-\:w_{j}(old)]$

如果 T ≠ Cj 则 $w_{j}(new)\:=\:w_{j}(old)\:-\:\alpha[x\:-\:w_{j}(old)]$

if T ≠ Cj then $w_{j}(new)\:=\:w_{j}(old)\:-\:\alpha[x\:-\:w_{j}(old)]$

Step 8 - 降低学习率 α。

Step 8 − Reduce the learning rate $\alpha$.

Step 9 - 对停止条件进行测试。可能如下所示：

Step 9 − Test for the stopping condition. It may be as follows −

Maximum number of epochs reached.
Learning rate reduced to a negligible value.

Flowchart

Variants

科霍宁开发了另外三种变体，即 LVQ2、LVQ2.1 和 LVQ3。在这三种变体中，由于获胜单元和亚军单元都会学习，因此复杂性比 LVQ 更高。

Three other variants namely LVQ2, LVQ2.1 and LVQ3 have been developed by Kohonen. Complexity in all these three variants, due to the concept that the winner as well as the runner-up unit will learn, is more than in LVQ.

LVQ2

如上所述，LVQ 其他变体的概念，LVQ2 的条件由窗口形成。此窗口将基于以下参数：

As discussed, the concept of other variants of LVQ above, the condition of LVQ2 is formed by window. This window will be based on the following parameters −

x − the current input vector
yc − the reference vector closest to x
yr − the other reference vector, which is next closest to x
dc − the distance from x to yc
dr − the distance from x to yr

输入向量 x 落入窗口中，则

The input vector x falls in the window, if

\frac{d_{c}}{d_{r}}\:>\:1\:-\:\theta\:\:and\:\:\frac{d_{r}}{d_{c}}\:>\:1\:+\:\theta

此处，$\theta$ 是训练样本的数量。

Here, $\theta$ is the number of training samples.

使用以下公式可以进行更新：

Updating can be done with the following formula −

$y_{c}(t\:+\:1)\:=\:y_{c}(t)\:+\:\alpha(t)[x(t)\:-\:y_{c}(t)]$ (belongs to different class)

$y_{r}(t\:+\:1)\:=\:y_{r}(t)\:+\:\alpha(t)[x(t)\:-\:y_{r}(t)]$ (belongs to same class)

这里 $\alpha$ 是学习率。

Here $\alpha$ is the learning rate.

LVQ2.1

在 LVQ2.1 中，我们将取两个最接近的向量 yc1 和 yc2 ，窗口的条件如下：

In LVQ2.1, we will take the two closest vectors namely yc1 and yc2 and the condition for window is as follows −

Min\begin{bmatrix}\frac{d_{c1}}{d_{c2}},\frac{d_{c2}}{d_{c1}}\end{bmatrix}\:>\:(1\:-\:\theta)

Max\begin{bmatrix}\frac{d_{c1}}{d_{c2}},\frac{d_{c2}}{d_{c1}}\end{bmatrix}\:<\:(1\:+\:\theta)

使用以下公式可以进行更新：

Updating can be done with the following formula −

$y_{c1}(t\:+\:1)\:=\:y_{c1}(t)\:+\:\alpha(t)[x(t)\:-\:y_{c1}(t)]$ (belongs to different class)

$y_{c2}(t\:+\:1)\:=\:y_{c2}(t)\:+\:\alpha(t)[x(t)\:-\:y_{c2}(t)]$ (belongs to same class)

这里，$\alpha$ 是学习速率。

Here, $\alpha$ is the learning rate.

LVQ3

在 LVQ3 中，我们将采用两个最接近的向量，即 yc1 和 yc2 ，窗口的条件如下 −

In LVQ3, we will take the two closest vectors namely yc1 and yc2 and the condition for window is as follows −

Min\begin{bmatrix}\frac{d_{c1}}{d_{c2}},\frac{d_{c2}}{d_{c1}}\end{bmatrix}\:>\:(1\:-\:\theta)(1\:+\:\theta)

这里 $\theta\approx 0.2$

Here $\theta\approx 0.2$

使用以下公式可以进行更新：

Updating can be done with the following formula −

$y_{c1}(t\:+\:1)\:=\:y_{c1}(t)\:+\:\beta(t)[x(t)\:-\:y_{c1}(t)]$ (belongs to different class)

$y_{c2}(t\:+\:1)\:=\:y_{c2}(t)\:+\:\beta(t)[x(t)\:-\:y_{c2}(t)]$ (belongs to same class)

这里 $\beta$ 是学习速率 $\alpha$ 的倍数，对于每个 0.1 < m < 0.5

Here $\beta$ is the multiple of the learning rate $\alpha$ and $\beta\:=\:m \alpha(t)$ for every 0.1 < m < 0.5