The encoder-decoder attention mechanism allows the decoder to access and integrate contextual information from the entire input sequence that the encoder previously processed.

The Transformer architecture is made up of two core components: an encoder and a decoder. The encoder contains layers that process input data, like text and images, iteratively layer by layer.