Artificial Neural Network 简明教程

Boltzmann Machine

这些是具有递归结构的随机学习过程，是 ANN 中早期优化技术的基石。Boltzmann 机是由 Geoffrey Hinton 和 Terry Sejnowski 于 1985 年发明的。在 Hinton 对 Boltzmann 机的解读中可以看到更加清晰的解释。

These are stochastic learning processes having recurrent structure and are the basis of the early optimization techniques used in ANN. Boltzmann Machine was invented by Geoffrey Hinton and Terry Sejnowski in 1985. More clarity can be observed in the words of Hinton on Boltzmann Machine.

“此网络一个令人惊讶的特性是它仅使用局部可用的信息。权重的变化仅取决于它连接的两个单​​元的行为，即使该变化优化了全局度量值” - Ackley，Hinton 1985 年。

“A surprising feature of this network is that it uses only locally available information. The change of weight depends only on the behavior of the two units it connects, even though the change optimizes a global measure” - Ackley, Hinton 1985.

Boltzmann 机的一些重要要点 −

Some important points about Boltzmann Machine −

  1. They use recurrent structure.

  2. They consist of stochastic neurons, which have one of the two possible states, either 1 or 0.

  3. Some of the neurons in this are adaptive (free state) and some are clamped (frozen state).

  4. If we apply simulated annealing on discrete Hopfield network, then it would become Boltzmann Machine.

Objective of Boltzmann Machine

玻尔兹曼机的主要目的是优化问题的解决方案。玻尔兹曼机的任务就是优化与该特定问题相关的权重和数量。

The main purpose of Boltzmann Machine is to optimize the solution of a problem. It is the work of Boltzmann Machine to optimize the weights and quantity related to that particular problem.

Architecture

下图显示了玻尔兹曼机的结构。从图中可以清楚地看到，它是一个二维的单元阵列。这里，单位之间互联的权重为 –p ，其中 p > 0 。自连接的权重由 b 给出，其中 b > 0 。

The following diagram shows the architecture of Boltzmann machine. It is clear from the diagram, that it is a two-dimensional array of units. Here, weights on interconnections between units are –p where p > 0. The weights of self-connections are given by b where b > 0.

boltzmann

Training Algorithm

众所周知，玻尔兹曼机具有固定权重，因此不会有训练算法，因为我们不需要更新网络中的权重。但是，为了测试网络，我们必须设置权重以及找到一致函数 (CF)。

As we know that Boltzmann machines have fixed weights, hence there will be no training algorithm as we do not need to update the weights in the network. However, to test the network we have to set the weights as well as to find the consensus function (CF).

玻尔兹曼机具有一组单元 Ui 和 Uj ，并且在其上具有双向连接。

Boltzmann machine has a set of units Ui and Uj and has bi-directional connections on them.

  1. We are considering the fixed weight say wij.

  2. wij ≠ 0 if Ui and Uj are connected.

  3. There also exists a symmetry in weighted interconnection, i.e. wij = wji.

  4. wii also exists, i.e. there would be the self-connection between units.

  5. For any unit Ui, its state ui would be either 1 or 0.

玻尔兹曼机的主要目标是最大化一致函数 (CF)，其可以用以下关系给出

The main objective of Boltzmann Machine is to maximize the Consensus Function (CF) which can be given by the following relation

CF\:=\:\displaystyle\sum\limits_{i} \displaystyle\sum\limits_{j\leqslant i} w_{ij}u_{i}u_{j}

现在，当状态从 1 变为 0 或从 0 变为 1 时，一致性的变化可以用以下关系给出 −

Now, when the state changes from either 1 to 0 or from 0 to 1, then the change in consensus can be given by the following relation −

\Delta CF\:=\:(1\:-\:2u_{i})(w_{ij}\:+\:\displaystyle\sum\limits_{j\neq i} u_{i} w_{ij})

这里 ui 是 Ui 的当前状态。

Here ui is the current state of Ui.

系数 ( 1 - 2ui ) 的变化由以下关系给出 −

The variation in coefficient (1 - 2ui) is given by the following relation −

(1\:-\:2u_{i})\:=\:\begin{cases}+1, & U_{i}\:is\:currently\:off\\-1, & U_{i}\:is\:currently\:on\end{cases}

通常，单位 Ui 不会改变其状态，但如果改变，则信息将驻留在该单位的本地。通过这种改变，网络的一致性也会增加。

Generally, unit Ui does not change its state, but if it does then the information would be residing local to the unit. With that change, there would also be an increase in the consensus of the network.

网络接受单位状态变化的概率由以下关系给出 −

Probability of the network to accept the change in the state of the unit is given by the following relation −

AF(i,T)\:=\:\frac{1}{1\:+\:exp[-\frac{\Delta CF(i)}{T}]}

在此， T 是控制参数。当 CF 达到最高值时，它将减少。

Here, T is the controlling parameter. It will decrease as CF reaches the maximum value.

Testing Algorithm

Step 1 − 初始化以下内容以启动训练 −

Step 1 − Initialize the following to start the training −

  1. Weights representing the constraint of the problem

  2. Control Parameter T

Step 2 − 在停止条件不为真时，继续步骤 3-8。

Step 2 − Continue steps 3-8, when the stopping condition is not true.

Step 3 − 执行步骤 4-7。

Step 3 − Perform steps 4-7.

Step 4 − 假设某一状态已更改权重，并选择整数 I, J 作为 1 和 n 之间的随机值。

Step 4 − Assume that one of the state has changed the weight and choose the integer I, J as random values between 1 and n.

Step 5 − 如下计算共识度变化 −

Step 5 − Calculate the change in consensus as follows −

\Delta CF\:=\:(1\:-\:2u_{i})(w_{ij}\:+\:\displaystyle\sum\limits_{j\neq i} u_{i} w_{ij})

Step 6 − 计算此网络接受状态变化的概率

Step 6 − Calculate the probability that this network would accept the change in state

AF(i,T)\:=\:\frac{1}{1\:+\:exp[-\frac{\Delta CF(i)}{T}]}

Step 7 − 如下接受或拒绝此更改 −

Step 7 − Accept or reject this change as follows −

Case I − 如果 R < AF ，请接受更改。

Case I − if R < AF, accept the change.

Case II − 如果 R ≥ AF ，请拒绝更改。

Case II − if R ≥ AF, reject the change.

在此， R 是 0 到 1 之間の随机数。

Here, R is the random number between 0 and 1.

Step 8 − 如下减少控制参数（温度） −

Step 8 − Reduce the control parameter (temperature) as follows −

T(new) = ⁡0.95T(old)

T(new) = ⁡0.95T(old)

Step 9 − 测试可能如下所示的停止条件 −

Step 9 − Test for the stopping conditions which may be as follows −

  1. Temperature reaches a specified value

  2. There is no change in state for a specified number of iterations