ailocalinference

← Glossary

TTFT

Time To First Token - How long until the AI starts speaking

Performance

In plain English

The pause after you ask a question before the person starts speaking.

Technically

The latency between submitting an input prompt and receiving the very first output token. Determined by prompt length, arithmetic throughput (TFLOPS), and kernel launch overhead during the prefill phase.

Related terms

PrefillKV Cache

Lessons that use it

More in Performance

Continuous BatchingDFlash & DFlash-2FlashAttentionKV CacheMemory BandwidthPagedAttentionPrompt CachingSpeculative Decoding