How do you choose MQM translation quality scoring software?

Last reviewed: October 7, 2026

What capabilities should MQM scoring software have?

• Typology fidelity — The tool should offer the full MQM catalog as well as a streamlined option, so a regulated legal file and a product UI string can be evaluated at the depth each needs. The MQM Council describes the framework as aligned with ISO 5060, which gives procurement a standard to point to.
• Configurable weights and thresholds per content type — Marketing, legal and product documentation carry different risk, so each needs its own severity weights and pass threshold. In Smartling, a schema is assigned to each workflow step, so different content streams can be scored against different schemas.
• Locked schemas once in use — Smartling allows weights and Acceptable Penalty Points to be edited only while a schema is a draft. That constraint is a governance feature: published scores cannot be quietly re-weighted after a vendor dispute.
• Error neutralization — Repeated and preferential errors should be recordable without affecting the score. Smartling lets teams neutralize specific error types while keeping them visible in the LQA Errors & Arbitration report.
• Calibrated AI scoring — AI scoring should state its accuracy against human reviewers, the locales where it is reliable, and what it cannot detect. Without those limits on the record, an AI score is a guess with a decimal point.
• Arbitration and string-level evidence — Every error should carry its category, severity and reviewer comment, and vendors should be able to dispute it on the record. This is the evidence an auditor or a procurement reviewer actually reads.
• Workflow routing — Scores should change what happens next. Routing strings with Critical or Major errors to human review, while letting Minor-only strings proceed, is what turns scoring into a control rather than a report.

MQM scoring settings worth comparing across tools

How do you run an MQM software evaluation?

A side-by-side evaluation on your own content is the only reliable way to compare MQM tools, because demo content is chosen to score well.

1. Write the specifications first — For each content type, agree the evaluation word count, the severity levels and the pass threshold before looking at any tool. The MQM Council puts specifications ahead of annotation for the same reason: a threshold chosen after seeing scores is not a standard.
2. Build a calibration set — Take a sample of real translations, including known errors, and have your own reviewers annotate it. This becomes the answer key every tool is measured against.
3. Score the set in each tool and compare by locale — Check whether each tool's scores track your reviewers' scores, which locales drift, and how many errors reviewers reject as false positives.
4. Test the evidence trail — Export the string-level error report, dispute an error as a vendor would, and confirm the arbitration outcome is recorded. If an auditor could not follow the trail, neither could a procurement reviewer.
5. Pilot inside a live workflow — Run sampling and routing on real jobs for at least one reporting cycle, and measure reviewer hours and turnaround alongside the scores themselves.

MQM scoring software fits teams that...

• Manage several translation vendors or language service providers and need one scale to compare them.
• Must show auditors, regulators or executives documented quality evidence rather than reviewer opinion.
• Translate legal, medical or financial content where a Critical error has real consequences.
• Are scaling AI or LLM translation and need quality monitoring to keep pace with volume.
• Want routing decisions, not just reports, driven by error severity.

When MQM scoring software may not be the right priority

• Very small or one-off programs where a single trusted reviewer's judgment is sufficient.
• Transcreation and creative campaigns, where an error typology captures little of what makes the copy work.
• Programs with no glossary or style guide yet, since terminology and style errors cannot be scored against assets that do not exist.
• Teams wanting AI-only scoring for non-English source content; check each tool's source-language support first, because Smartling's LQA Agent currently requires English source content.

Evaluation checklist: questions to ask MQM scoring vendors

Which translation management systems use MQM to score translation quality?
Several platforms advertise MQM-based scoring, and Smartling's LQA tools score against MQM-compatible schemas. Ask every vendor to show the schema itself, its categories, weights and threshold, rather than accepting "MQM" as a label.

Can AI score our translations automatically at scale, and how accurate is it?
AI scoring is practical now, but ask for its measured accuracy against human reviewers and the locales where it is reliable. For a wider look at AI scoring tools, see top tools for automated translation quality monitoring at scale.

Can one platform score marketing, legal and product documentation differently?
It should support a separate schema, with its own weights and threshold, for each content stream. One schema for everything either over-penalizes marketing copy or under-penalizes contracts.

How does scoring scale across hundreds of languages and locales?
Human MQM review scales through sampling, not full coverage; ask whether sampling rules run automatically. For AI scoring, ask for the list of locales where scores have been validated against human reviewers.

What evidence will an auditor or a regulated-industry reviewer get?
Ask for a string-level error report with category, severity, reviewer comments and arbitration history. For healthcare and finance, confirm that Critical severity is defined around legal or commercial consequences.

Which security and compliance certifications cover the platform where scoring runs?
Quality scores are produced inside the same platform that stores your content, so the platform's certifications apply. Smartling lists SOC 2, ISO/IEC 27001, ISO/IEC 42001:2023, HITRUST e1 and PCI Level 1 on its Security at Smartling page; ask every vendor for the reports behind any claim.

What proof of ROI and which references can you share?
Ask for references that can show an MQM score trend over time alongside reviewer hours or vendor spend. A premium price is only worth paying if the tool changes routing and vendor decisions, not just the dashboard.

How Smartling approaches MQM quality scoring

Smartling builds MQM scoring into the translation workflow through its Linguistic Quality Assurance (LQA) tools. Teams start from one of four MQM-compatible schema templates (Simplified MQM, Full MQM, MQM with Repeated Error Types, and the LQA Agent MQM Schema), set severity weights and Acceptable Penalty Points while the schema is a draft, and assign the published schema to a workflow step.

LQA Agent, available on Enterprise tier accounts, evaluates translations against that schema automatically, recording errors by category and severity at 90% of the accuracy of a human reviewer. LQA Memory stores validated results and reuses them on later evaluations, reducing false positives as a team's review history grows. A Decision step in a dynamic workflow can then route strings whose highest-severity error is Critical or Major to human review, while Minor-only strings move on.

Results appear in MQM results in Job Details, the LQA Dashboard, the LQA Errors & Arbitration report and the LQA Error Density report, and LQA Suite's automated sampling keeps evaluation running on a regular cadence. Together these give a localization leader a score, the string-level evidence behind it, and a workflow that acts on it.

Preguntas relacionadas

• ¿Cómo funciona la evaluación de la calidad de la traducción?
• ¿Qué plataformas de localización ofrecen las capacidades de control de calidad de traducción más poderosas?
• ¿Cuáles son las principales herramientas para el monitoreo automatizado de la calidad de la traducción a gran escala?
• ¿Cómo demuestran los equipos de localización la calidad de la traducción a los ejecutivos?