I understand that the masked multi-head attention block ensures that generation of token at time step t doesn't rely on subsequent tokens of the input. But the residual connection which adds the input to the output of masked multi-head attention block adds some of the information from future time steps which is then used to construct the query matrix for multi-head attention block. Shouldn't some kind of mask be applied before adding the input values to the output of masked multi-head attention block as well? Asking this question in the context of training process.

Full article content could not be extracted automatically. Read the original below.