Example. Decoder-only dataflow and a DeepSeek V4-Flash provenance overlay [ftip-0013]
✍️source

A decoder pipeline maps token identifiers through hidden states to logits; the released model instance and family report provide different evidence about that pipeline.

$\text{[math]}$

The pinned configuration [⧉] declares 43 layers, hidden width 4096, 64 attention heads, and one key--value head. Section 2 of the family report [⧉] describes the preview architecture. Neither source identifies a complete 0731-specific architecture or training delta. The diagram records the interface, not a small copy of the released model.

Example. Decoder-only dataflow and a DeepSeek V4-Flash provenance overlay [ftip-0013]
✍️source

Decoder-only Transformer architecture [ftip-000K]

Notation. Tensor dimensions, positions, heads, and layers [ftip-000L]

Definition. Token embedding [vaswani2017attention, sec. 3.4, Embeddings and Softmax] [ftip-000M]

Definition. Embedding-stage positional transformation [ftip-000N]

Remark. Where positional information enters [ftip-000O]

Definition. Query, key, and value projections [vaswani2017attention, sec. 3.2.1] [ftip-000P]

Definition. Scaled dot-product attention [vaswani2017attention, sec. 3.2.1, equation (1)] [ftip-000Q]

Example. A two-token attention calculation [ftip-000R]

Definition. Causal attention mask [vaswani2017attention, sec. 3.2.3] [ftip-000S]

Example. A length-four causal mask [ftip-000T]

Definition. Multi-head self-attention [vaswani2017attention, sec. 3.2.2] [ftip-000U]

Remark. Multi-head, grouped-query, and compressed-attention variants [ftip-000V]

Definition. Positionwise feed-forward sublayer [vaswani2017attention, sec. 3.3, equation (2)] [ftip-000W]

Definition. Sparse mixture-of-experts routing [ftip-000X]

Remark. Dense and routed feed-forward computation [ftip-000Y]

Example. Top-k expert routing and its DeepSeek counterpart [ftip-000Z]

Definition. Residual connection [vaswani2017attention, sec. 3.1] [ftip-0010]

Definition. Layer normalization [ba2016layer, sec. 3] [ftip-0011]

Definition. Normalization placement [ftip-006E]

Definition. Root mean square layer normalization [zhang2019root, Section 4, equation (4)] [ftip-006F]

Remark. Normalization operator and placement are separate choices [ftip-006G]

Definition. Decoder-only Transformer language model [phuong2022formal, sec. 6, Decoder-only transformers] [ftip-0012]

Example. Decoder-only dataflow and a DeepSeek V4-Flash provenance overlay [ftip-0013]
✍️source