Neural Network Models and Functions

The following chapter introduces the models and learning functions implemented in SNNS. A strong emphasis is placed on the models that are less well known. They can not, however, be explained exhaustively here. We refer interested users to the literature.

• Backpropagation Networks
  • Vanilla Backpropagation
  • Enhanced Backpropagation
  • Batch Backpropagation
  • Backpropagation with Weight Decay
• Quickprop
• RPROP
  • Changes in Release 3.3
  • General Description
  • Parameters
• Backpercolation
• Counterpropagation
  • Fundamentals
  • Initializing Counterpropagation
  • Counterpropagation Implementation in SNNS
• Dynamic Learning Vector Quantization (DLVQ)
  • DLVQ Fundamentals
  • DLVQ in SNNS
  • Remarks
• Backpropagation Through Time (BPTT)
• The Cascade Correlation Algorithms
  • Cascade-Correlation (CC)
    • The Algorithm
    • Mathematical Background
  • Pruned-Cascade-Correlation (PCC)
    • The Algorithm
    • Mathematical Background
  • Recurrent Cascade-Correlation (RCC)
    • The Algorithm
    • Mathematical Background
  • Using the Cascade Algorithms in SNNS
• Time Delay Networks (TDNNs)
  • TDNN Fundamentals
  • TDNN Implementation in SNNS
    • Activation Function
    • Update Function
    • Learning Function
  • Building and Using a Time Delay Network
• Radial Basis Functions (RBFs)
  • RBF Fundamentals
  • RBF Implementation in SNNS
    • Activation Functions
    • Initialization Functions
      • RBF_Weights
      • RBF_Weights_Kohonen
      • RBF_Weights_Redo
    • Learning Functions
  • Building a Radial Basis Function Application
• Dynamic Decay Adjustment for RBFs (RBF--DDA)
  • The Dynamic Decay Adjustment Algorithm
  • Using RBF--DDA in SNNS
• ART Models in SNNS
  • ART1
    • Structure of an ART1 Network
    • Using ART1 Networks in SNNS
      • ART1 Initialization Function
      • ART1 Learning Function
      • ART1 Update Functions
  • ART2
    • Structure of an ART2 Network
    • Using ART2 Networks in SNNS
      • ART2 Initialization Function
      • ART2 Learning Function
      • ART2 Update Functions
  • ARTMAP
    • Structure of an ARTMAP Network
    • Using ARTMAP Networks in SNNS
      • ARTMAP Initialization Function
      • ARTMAP Learning Function
      • ARTMAP Update Functions
  • Topology of ART Networks in SNNS
• Self-Organizing Maps (SOMs)
  • SOM Fundamentals
  • SOM Implementation in SNNS
    • The KOHONEN Learning Function
    • The Kohonen Update Function
    • The Kohonen Init Function
    • The Kohonen Activation Functions
    • Building and Training Self-Organizing Maps
    • Evaluation Tools for SOMs
• Autoassociative Networks
  • General Characteristics
  • Layout of Autoassociative Networks
  • Hebbian Learning
  • McClelland & Rumelhart's Delta Rule
• Partial Recurrent Networks
  • Models of Partial Recurrent Networks
    • Jordan Networks
    • Elman Networks
    • Extended Hierarchical Elman Networks
  • Working with Partial Recurrent Networks
    • The Initialization Function JE_Weights
    • Learning Functions
    • Update Functions
• Stochastic Learning Functions
  • Monte-Carlo
  • Simulated Annealing
• Scaled Conjugate Gradient (SCG)
  • Conjugate Gradient Methods (CGMs)
  • Main features of SCG
  • Parameters of SCG
  • Complexity of SCG