Working with Partial Recurrent Networks

In this subsection, the initialization, learning, and update functions for partial recurrent networks are described. These functions can not be applied to only the three network models described in the previous subsection. They can be applied to a broader class of partial recurrent networks. Every partial recurrent network, that has the following restrictions, can be used:

• If after the deletion of all context units and the links to and from them, the remaining network is a simple feedforward architecture with no cycles.

• Input units must not get input from other units.

• Output units may only have outgoing connections to context units, but not to other units.

• Every unit, except the input units, has to have at least one incoming link. For a context unit this restriction is already fulfilled when there exists only a self recurrent link. In this case the context unit receives its input only from itself.

In such networks all links leading to context units are considered as recurrent links.

Thereby the user has a lot of possibilities to experiment with a great variety of partial recurrent networks. E.g. it is allowed to connect context units with other context units.

Note: context units are realized as special hidden units. All units of type special hidden are assumed to be context units and are treated like this.

• The Initialization Function JE_Weights
• Learning Functions
• Update Functions