Artificial Neural Network 简明教程

Adaptive Resonance Theory

此网络由 Stephen Grossberg 和 Gail Carpenter 于 1987 年开发。它基于竞争，并使用无监督学习模型。自适应共振理论 (ART) 网络，顾名思义，始终对新事物保持开放（适应）而不会丢失旧模式（共振）。基本上，ART 网络是一个向量分类器，它接受一个输入向量并根据它与哪个存储模式最相似将其分类到某个类别中。

This network was developed by Stephen Grossberg and Gail Carpenter in 1987. It is based on competition and uses unsupervised learning model. Adaptive Resonance Theory (ART) networks, as the name suggests, is always open to new learning (adaptive) without losing the old patterns (resonance). Basically, ART network is a vector classifier which accepts an input vector and classifies it into one of the categories depending upon which of the stored pattern it resembles the most.

Operating Principal

ART 分类的主要操作可分为以下阶段 −

The main operation of ART classification can be divided into the following phases −

  1. Recognition phase − The input vector is compared with the classification presented at every node in the output layer. The output of the neuron becomes “1” if it best matches with the classification applied, otherwise it becomes “0”.

  2. Comparison phase − In this phase, a comparison of the input vector to the comparison layer vector is done. The condition for reset is that the degree of similarity would be less than vigilance parameter.

  3. Search phase − In this phase, the network will search for reset as well as the match done in the above phases. Hence, if there would be no reset and the match is quite good, then the classification is over. Otherwise, the process would be repeated and the other stored pattern must be sent to find the correct match.

ART1

它是一种 ART，被设计为对二进制向量进行聚类。我们可以通过其架构来理解这一点。

It is a type of ART, which is designed to cluster binary vectors. We can understand about this with the architecture of it.

Architecture of ART1

它由以下两个单元组成 −

It consists of the following two units −

Computational Unit − 它由以下内容组成 −

Computational Unit − It is made up of the following −

  1. Input unit (F1 layer) − It further has the following two portions − F1(a) layer (Input portion) − In ART1, there would be no processing in this portion rather than having the input vectors only. It is connected to F1(b) layer (interface portion). F1(b) layer (Interface portion) − This portion combines the signal from the input portion with that of F2 layer. F1(b) layer is connected to F2 layer through bottom up weights bij and F2 layer is connected to F1(b) layer through top down weights tji.

  2. Cluster Unit (F2 layer) − This is a competitive layer. The unit having the largest net input is selected to learn the input pattern. The activation of all other cluster unit are set to 0.

  3. Reset Mechanism − The work of this mechanism is based upon the similarity between the top-down weight and the input vector. Now, if the degree of this similarity is less than the vigilance parameter, then the cluster is not allowed to learn the pattern and a rest would happen.

Supplement Unit − 重置机制的问题实际上在于，第 F2 层必须在某些条件下受到抑制，并且在发生某些学习时也必须可用。这就是加 gain control units 两个补充单元即 G1 和 G2 以及重置单元 R 的原因。这些单元接收并向网络中存在的其他单元发送信号。 ‘+’ 指示激励信号，而 ‘−’ 指示抑制信号。

Supplement Unit − Actually the issue with Reset mechanism is that the layer F2 must have to be inhibited under certain conditions and must also be available when some learning happens. That is why two supplemental units namely, G1 and G2 is added along with reset unit, R. They are called gain control units. These units receive and send signals to the other units present in the network. ‘+’ indicates an excitatory signal, while ‘−’ indicates an inhibitory signal.

supplement unit
units

Parameters Used

以下参数使用 −

Following parameters are used −

  1. n − Number of components in the input vector

  2. m − Maximum number of clusters that can be formed

  3. bij − Weight from F1(b) to F2 layer, i.e. bottom-up weights

  4. tji − Weight from F2 to F1(b) layer, i.e. top-down weights

  5. ρ − Vigilance parameter

  6. ||x|| − Norm of vector x

Algorithm

Step 1 − 初始化学习率、警觉性参数和权重如下 −

Step 1 − Initialize the learning rate, the vigilance parameter, and the weights as follows −

\alpha\:>\:1\:\:and\:\:0\:<\rho\:\leq\:1

0\:<\:b_{ij}(0)\:<\:\frac{\alpha}{\alpha\:-\:1\:+\:n}\:\:and\:\:t_{ij}(0)\:=\:1

Step 2 − 当停止条件为假时，继续执行步骤 3-9。

Step 2 − Continue step 3-9, when the stopping condition is not true.

Step 3 − 为每个训练输入继续执行步骤 4-6。

Step 3 − Continue step 4-6 for every training input.

Step 4 − 将所有 F1(a) 和 F1 单元的激活设置如下

Step 4 − Set activations of all F1(a) and F1 units as follows

F2 = 0 and F1(a) = input vectors

F2 = 0 and F1(a) = input vectors

Step 5 − 从 F1(a) 到 F1(b) 层的输入信号必须像

Step 5 − Input signal from F1(a) to F1(b) layer must be sent like

s_{i}\:=\:x_{i}

Step 6 − 对于每个抑制的 F2 节点

Step 6 − For every inhibited F2 node

$y_{j}\:=\:\sum_ib_{ij}x_{i}$ 条件为 yj ≠ -1

$y_{j}\:=\:\sum_i b_{ij}x_{i}$ the condition is yj ≠ -1

Step 7 − 当重置为真时，执行步骤 8-10。

Step 7 − Perform step 8-10, when the reset is true.

Step 8 - 找在所有节点 j 中 yJ ≥ yj 的 J

Step 8 − Find J for yJ ≥ yj for all nodes j

Step 9 - 再按如下方式计算F1(b)上的激活

Step 9 − Again calculate the activation on F1(b) as follows

x_{i}\:=\:sitJi

Step 10 - 在计算出矢量 x 的范数和矢量 s 的范数后，需要按如下方式检查重置条件 -

Step 10 − Now, after calculating the norm of vector x and vector s, we need to check the reset condition as follows −

如果 ||x||/ ||s|| < 警觉参数 ρ ，则抑制节点 J 并转到步骤7

If ||x||/ ||s|| < vigilance parameter ρ,⁡then⁡inhibit ⁡node J and go to step 7

否则，如果 ||x||/ ||s|| ≥ 警觉参数 ρ ，则继续。

Else If ||x||/ ||s|| ≥ vigilance parameter ρ, then proceed further.

Step 11 - 节点 J 的权重更新可按如下方式进行 -

Step 11 − Weight updating for node J can be done as follows −

b_{ij}(new)\:=\:\frac{\alpha x_{i}}{\alpha\:-\:1\:+\:||x||}

t_{ij}(new)\:=\:x_{i}

Step 12 - 必须检查算法的停止条件，它可能如下 -

Step 12 − The stopping condition for algorithm must be checked and it may be as follows −

  1. Do not have any change in weight.

  2. Reset is not performed for units.

  3. Maximum number of epochs reached.