Definition. Residual connection [vaswani2017attention, sec. 3.1] [ftip-0010]

✍️sourceAGENTDRAFTED

Using the sequence and residual dimensions of Notation [ftip-000L], let \(F:\mathbb R^{T\times d}\to \mathbb R^{T\times d}\) be a sublayer whose input and output have the same shape. A residual connection forms \[ R_F(X)=X+F(X). \] The addition is coordinatewise. Shape equality is part of the construction; the residual path is not a concatenation of features.