Section 6.4
Sequence-to-sequence Encoder-decoder Model

A sequence-to-sequence model (sometimes abbreviated as “seq-to-seq”) is a model that consumes an arbitrary sequence as input and produces an arbitrary sequence as output. There are several variants where the model can either consume the entire input sequence before producing the output sequence, or the model can start producing elements of the output while still consuming input elements. The former can be regarded as an encoder-decoder model where we combined a many-to-one model with a one-to-many model (see Figure 86). Here an encoder recurrent neural network consumes the arbitrary length input sequence to produce a latent state vector that represents the input. This vector is then used to initialize the hidden state of a decoder recurrent neural network, which produces an arbitrary length output sequence. The only output of the encoder RNN is the latent state and the only input to the decoder RNN is the state being fed back in.