Language model evaluation

Choose the unit that matters: a question, conversation, tool workflow, document, or user session. Write criteria for a successful result, including abstention when required information is absent. Multiple-choice accuracy is convenient but may not reflect whether a system can produce a supported explanation or execute a workflow safely. HELM argues for evaluation across several scenarios and metrics, motivating a measurement portfolio rather than one universal quality number. Include latency, cost, robustness, and important task-specific failure categories.

Construct representative slices such as language, context length, rare entity type, difficult tool state, and no-answer questions. Set target population weights before viewing the final comparison; otherwise an apparent aggregate improvement can result from changing the mix. Preserve dataset versions and separate examples used for prompt tuning, model selection, and final testing. A public benchmark can be useful while still being vulnerable to training contamination or repeated indirect tuning, so supplement it with private or newly constructed task evidence where appropriate.

Evaluate both systems on the same frozen cases so disagreement counts become visible. If both succeed on easy cases and differ on a few hard ones, an unpaired summary throws away useful structure. A bootstrap should resample the independent evaluation unit, such as whole conversations, not arbitrary turns from the same conversation. For stochastic systems, repeat samples or define the exact single-run policy being estimated. A small difference can have substantial uncertainty despite impressive total accuracies.

Automatic metrics need a validity argument. Exact match can penalize equivalent answers; an embedding score can reward a fluent contradiction. Model-based judges are useful but can prefer their own style, longer answers, or a particular answer position. Randomize order, hide system identity, check against human labels, and measure judge disagreement. TruthfulQA supplies a concrete test of falsehood-related behavior, but success on its question distribution cannot prove universal truthfulness. Match each claim to the evidence actually collected.

Record model and tokenizer revisions, full prompts, retrieved corpus versions, tool state, decoding parameters, random seeds, timeouts, and grader configuration. Score incomplete, invalid, and timed-out outputs according to a declared policy rather than silently dropping them. Keep resource budgets comparable: one system allowed ten attempts is not directly comparable to another allowed one unless the difference is part of the question. Report both quality and resources so a gain from extra computation is visible.

After testing, inspect error mechanisms and create a new development set. Do not continuously tune against the same final test while still calling it untouched. Distinguish lack of knowledge, missing retrieval, evidence misuse, tool failure, and grader ambiguity. Regression tests should preserve important discovered failures without replacing broader sampling. A credible release decision combines effect size, uncertainty, operational benefit, and unacceptable failures; no single benchmark average resolves every tradeoff.