Modules A07–A09 · concept player

Fusion, the ratio you never store

Unfused attention writes the N×N score matrix to HBM and reads it straight back. Fused attention never materializes it. Slide the sequence length against the head dimension and watch the traffic ratio move — then switch dtype and watch it not move at all.

Exact arithmetic on the module's formula, not a measurement. Unfused traffic is 8Nd + 8N² bytes; fused is 8Nd. The dtype multiplier scales both bars equally, which is why the ratio column never moves when you change it — A09 section 2 states this explicitly: "the dtype cancels out of that ratio." At the default d = 128 the three N presets reproduce the module's own worked points: 9× at N=1k, 33× at N=4k, 129× at N=16k.