The Network Equations

In general, let the network have N units, numbered from 1 to N. The weight assigned to the connection from the ith unit to the jth unit is w_ij for $1 \leq i,j \leq N$. Since no unit is connected to itself, w_ii = 0 for all $i = 1, \ldots , N$. Since the connection from the ith unit to the jth unit has the same weight as the connection from the jth unit to the ith unit, w_ij = w_ji.

Each unit has a threshold associated with it. The threshold determines the change in the state of the unit in response to inputs from the other units. We will denote the threshold of the ith unit by $\theta_i$.

Suppose we have M exemplar patterns, ${\bf p}_1, \ldots , {\bf p}_M$, each of which has N components, so ${\bf p}_i = (p_i^1, \ldots, p_i^N)$ for $i = 1, \ldots , M$. We compute the weights according to the formula $w_{ij} = \sum_{k=1}^M p_k^i p_k^j$ for $1 \leq i,j \leq N, i\neq j$.

Let the state of the ith unit at time t be $\mu_i(t), i = 1, \ldots , N$.Suppose we have an input pattern $(x^1, \ldots , x^N)$ that is to be classified. We impose the pattern on the network by setting $\mu_i(0) =x^i, i = 1, \ldots , N$.

To run the network, we compute the states of the units at successive instants using the formula

$$\mu_i(t+1) = f\left(\sum_{j=1,j \neq i}^N w_{ij} \mu_j(t) - \theta_i\right)$$

for $i = 1, \ldots , N$ and $t = 1, 2, \ldots$

In the case where the units have binary outputs, f is the step funtion: f(x) = -1 if x < 0 and f(x) = 1 if x > 0. (If $\sum_{j=1,j \neq i}^N w_{ij} \mu_j(t) - \theta_i = 0$ the state of the unit is not changed.) In the continuous case, f is a sigmoid function with range [a,b].