How to measure translation quality at scale

Quality is more than one score

Translation quality includes accuracy, fluency, terminology consistency, format integrity and cultural fit. Define weights by content type: product UI prioritizes terms and length, legal copy prioritizes accuracy, and marketing pages also need tone and conversion context.

Use automation for hard rules

Automation is best at deterministic failures: missing placeholders, changed numbers, broken URLs or HTML tags, strings over length limits, forbidden terms and glossary misses. Run these checks for every job and again before release.

checks:
  placeholders: pass
  links: pass
  glossary_accuracy: 98.7%
  max_ui_length: pass

Calibrate with sampled human review

Stratify samples from new content, high-traffic pages, languages with higher error rates and user complaints. Have two reviewers score independently and record issue types, then reconcile differences to keep the rubric consistent. Adjust the sample rate as historical risk changes.

Connect quality to product outcomes

Alongside reviewer scores, watch search CTR, conversion, support tickets, fallback rate and engagement. A higher score with lower conversion may mean the copy is accurate but not locally persuasive. Fewer translation errors with more fallbacks points to loading or language-detection issues.

A reusable scorecard

• Accuracy: meaning, facts and numbers are correct.
• Fluency: the target language reads naturally.
• Terminology: product terms and keywords are consistent.
• Format: placeholders, tags, links and length are valid.
• Context: tone, culture and user intent are appropriate.