Tokens per Second
The headline speed number — and two very different numbers get called by this name
Performance
In plain English
How fast the AI types its answer. About 7 is slower than you read, 20 feels comfortable, and past 50 it arrives faster than you can follow.
Technically
Throughput of token generation, but the figure is ambiguous unless the phase is stated. Prefill (prompt processing) is compute-bound and runs in the hundreds or thousands of tok/s; decode (generation) is memory-bandwidth-bound and is the number users actually feel. Batched serving throughput is higher still, because one weight read is amortized across several sequences, so vendor benchmarks often quote it. Human reading is roughly 5-8 tok/s, so decode above about 15 tok/s feels responsive for chat, while coding agents that generate long files want far more.