Example. A length-four causal mask [ftip-000T]
Example. A length-four causal mask [ftip-000T]
For four positions, a causal mask admits the current and earlier keys and excludes every future key.
Rows index querying positions and columns index key positions. In row three, the first three entries remain available and the fourth receives \(-\infty \); after softmax the fourth token therefore contributes exactly zero.
The construction is the autoregressive mask of [vaswani2017attention, Section 3.2.3]. It enforces a dependency restriction. Effective use of the available earlier positions is a separate model property.