Perplexity (PPL)
The standard metric for measuring language model accuracy and compression loss
Fundamentals
In plain English
A confusion score. The lower the score, the less confused the AI is when reading text. When you compress a model, you want perplexity to barely change.
Technically
The exponentiated cross-entropy loss PPL = exp(-1/N ∑ log p(x_i)) evaluating how well a probability model predicts a test dataset (e.g. WikiText-2). Used as the ground-truth benchmark for assessing quantization quality loss.