Arithmetic Intensity
Arithmetic intensity is the ratio of arithmetic operations performed to bytes moved from memory, and it decides whether a kernel is memory- or compute-bound.
Operations per byte
Arithmetic intensity (AI) is defined as the number of floating-point operations a kernel executes divided by the number of bytes it must move to and from main memory. It is the single number that places a kernel on the roofline. High intensity means a byte fetched is reused for much arithmetic; low intensity means the kernel does little work per byte and will be starved by the memory system.
Worked examples
A vector operation like y = a*x + y (AXPY) reads x and y and writes y, moving about 24 bytes in double precision to perform 2 operations, giving an intensity near 0.08 ops per byte: firmly memory-bound. Dense matrix multiply of n-by-n matrices performs about 2n^3 operations while moving on the order of n^2 data, so intensity grows with n and the kernel becomes compute-bound for large matrices. This contrast is why GEMM can approach peak while AXPY cannot.
- AI = floating-point operations / bytes moved from memory.
- Low AI (streaming, stencils, sparse): memory-bound.
- High AI (dense GEMM, N-body): compute-bound.
- Data reuse and operation fusion raise AI; recomputation can too.
Raising it
Because AI determines the ceiling, raising it is a core optimization. Techniques include cache and register blocking (reuse a tile many times before evicting it), loop fusion (do several operations while data is in cache instead of re-reading it), and sometimes recomputation (recompute a cheap value instead of storing and reloading it). Each cuts bytes moved per operation, sliding the kernel rightward on the roofline toward the compute roof.
A subtlety
Measured AI depends on where in the hierarchy you count bytes. Counting traffic to DRAM gives the machine-level intensity that matches the roofline; counting traffic to L1 gives a different, usually higher number. Consistent accounting matters when comparing kernels. For a Hyperion sparse solver, the DRAM-level AI is low, confirming that data layout and reuse, not faster arithmetic, are the levers that matter.