How do you go about assessing the quality of a translation?

Surprisingly, it’s hard to find the practical information you need for this. Like:
– what specifically to look for, and
– how you do that in practice.

This article fills that gap.

We outline our remarkably effective 3-step process and tell you exactly what you need to do to assess translation quality.

It’s a model that works a treat for almost all business translation projects – no wonder translators and reviewers love it!

But before we can assess translation quality, we first need to be clear what that looks like.

Defining translation quality – what is a high-quality translation?

Here’s our definition:

A high-quality translation is one that captures the meaning of the original text and is appropriately worded, free from error and internally consistent

This definition identifies the 4 parameters that determine translation quality:

• Meaning
• Wording / expression
• Errors
• Consistency

And the standards needed in each for a translation to be high-quality:

• Equivalence of meaning
• Appropriate wording / expression
• No mistakes
• Fully consistent

In summary:

So we can assess a translation’s quality by seeing if it meets these four criteria.

Which we can do by asking these four questions:

• Does it capture the meaning of the original?
• Does it read like it should?
• Are there any grammatical or spelling mistakes?
• Has it handled everything consistently?

The 3-step translation assessment model that follows is a systematic process for answering these 4 questions. It’s the process we use in our second translator review in our quality-assured translations, and a crucial component to our broader quality assurance measures.

Our simple but highly effective 3-Step Model for Assessing Translation Quality

Why three steps?

Because it’s too demanding to keep all 4 translation evaluation criteria and questions in mind and check for them at the same time.

The 3 separate phases break the task up into more manageable chunks, so each criterion receives the attention it needs.

This ensures the process is effective – that everything that has to be assessed is covered and nothing is missed.

Video, slide show or article?
Choose your preferred version.

Step 1. Assess the translation for accuracy of meaning

What to look for

The focus here is solely on meaning – with no other distractions.

The task is to assess whether all meaning in the source text has been correctly carried over into the translation, with nothing extra added and nothing omitted.

How to do it

You need to systematically compare the translation and original, phrase-by-phrase or sentence-by-sentence.

You should work from hard copies – because things are much more easily missed if it’s done on screen.

We recommend placing the two texts side-by-side, covered over, and uncovering matching sections only as you’re working on them on. This keeps your focus solely on the text you’re comparing and minimises potential distractions.

Important:

It’s essential to work with short chunks of text – no more than 7 to 10 words at a time. Anything longer makes it hard to retain all the information in short-term memory, and therefore much more likely you’ll miss a difference in meaning.

Each chunk of text should ideally be a discrete and complete unit of meaning. That’s because complete ideas are easier to retain in memory than incomplete ones. In practice this means we often work at phrase rather than sentence level.

For example, it will be more effective to break this earlier sentence into two chunks:

Each chunk of text should ideally be  /  a discrete and complete unit of meaning.

This will be easier than working with the whole sentence, where for example you might easily not notice if the concepts of “ideally” or “discrete” were missing in the translation.

And you wouldn’t want to divide the sentence up like this:

Each chunk of text should ideally be a discrete  /  and complete unit of meaning.

That’s because the first part is not a complete idea, so is more difficult for our minds to handle and retain in memory.

What this step achieves

You’ll either confirm all meaning has been captured in the translation, or identify places where it hasn’t and so needs changing.

Because you’re focusing so intensely at phrase and word level, you should also automatically pick up any typos and spelling or grammar mistakes.

And you should notice any inconsistency in vocabulary – the same word or term translated in different ways.

So:

Step 2: Assess the translation’s quality of expression

What to look for

There are three aspects to a translation’s quality of wording – naturalness, ambiguity and register/style.

In this step we want to assess whether the translation:

• uses natural sounding language, flows nicely, is easy to understand, and has no awkward sounding parts;
• has any text that’s ambiguous and so open to misinterpretation;
• has similar tone and formality/informality to the original and an appropriate style for the intended purpose or function of the text.

Note:

You need to be a native speaker of the language to judge these things.

How to do it

Simply read through the translation carefully, looking out for any wording that seems unnatural, is hard to follow, or could be misinterpreted.

Anything that makes you pause should be revisited.

At the same time, you need to take in the tone and style of the text and be happy this is appropriate.

If you’re struggling to handle all this at the same time, split the step into two phases. Assess the naturalness of the translation wording in the first phase, and tone and style in the second pass through.

What this step achieves

You’ll confirm the translation is appropriately worded, or identify parts needing amending.

You may also automatically pick up any remaining inconsistency of vocab or style during this read through. With experience you develop a sort of sixth sense, an inbuilt radar, for anything that isn’t consistent.

So:

Step 3: Technical/formal check

Here we run an eye over formatting and layout to ensure they are appropriate and consistent throughout.

Optionally, you may also check there are no spelling or grammar errors, generally using the spell check function available in most programs.

We also recommend you double check proper name spellings, dates and numbers as these are a common source of error.

How the 3 steps in our model have assessed the 4 translation evaluation criteria:

Note:

Our method doesn’t include a separate check for consistency – we assume you’ll pick up any issues in Steps 1 and 2.

If you’re not confident of that, add an additional step.

Identify the key vocabulary in the source text and review how each occurrence has been handled in the translation.

The two great translation assessment challenges: meaning and wording

Of the 4 translation quality criteria, identifying errors and inconsistencies is straightforward.

But assessing if the translation is accurate and appropriately worded is not so cut and dried.

Here are our thoughts on the main issues.

The role of meaning in assessing translation quality

Meaning is a notoriously slippery concept.

As soon as we ask what something really means, things can get murky and unexpectedly complex. Academics have spent decades trying to nail it all down.

And yet, we all know what things mean, don’t we? We couldn’t communicate or understand each other if we didn’t.

It’s this practical, intuitive understanding of meaning that anyone assessing translation quality needs to call on.

Equivalence of meaning – what is it?

A high-quality translation will express all the meaning contained in the source text. And only that meaning.

In other words, it won’t omit, alter or add any meaning. This is what we typically mean when we say a translation is accurate.

Simple right? Fortunately, mostly yes.

Firstly, it’s generally easy to identify when some meaning has been missed in a translation (omissions). And so too if it adds something that’s not in the original (additions).

That just leaves altered meaning – when the translation says something different to the original. This is often referred to as a mistranslation or translation mistake.

Most mistranslations will also be obvious and easy to identify.

So linguists will generally agree when meanings match or don’t match.

But not always.

How different is different? – it’s not all black and white

There are grey areas, and what one reviewer might assess as a difference, another may not.

Take these two sentences:

Place the device on the skin and check the readout on the screen

Attach the device to the skin and check the readout on the screen

Someone focusing on the precise or literal meaning of the verbs “place” and “attach” might identify a difference in meaning.

But someone else might conclude the sentence is all about the device working and giving a readout, so this distinction isn’t important.

Anyone involved in evaluating translation quality will have many cases like this. Times when they need to decide if something is sufficiently different to no longer reflect the meaning of the original.

There’ll be times where they’re in two minds.

One approach to resolving these uncertainties is to focus on the key, underlying or essential meaning or message. As opposed to the superficial or literal. So you’d ask “Does the translation capture the essential meaning/message of the original?”

Another approach is to assess how relevant or significant a potential difference in meaning is – is it a difference that matters?

Both approaches accept that some non-equivalence of meaning is acceptable, providing it’s not significant.

But is this OK?

Some people aren’t comfortable with this. They feel a translation should carry over all meaning, both essential and superficial, wherever possible.

And they don’t see it as the translator’s (or reviewer’s) role to decide what matters and doesn’t matter, or what’s important and what isn’t, in the original text.

But this can create a problem.

If we insist on all nuances of meaning being precisely and directly conveyed, we’re forcing the translator into a pretty literal translation.

And literal translations frequently don’t read well. They tend to use vocabulary and wording that isn’t natural in that language. So they can seem clunky and a bit awkward – which as we’ll see below won’t meet our definition of a high-quality translation.

So, there’s a dilemma:
If we want every nuance of meaning to be conveyed, we’ll tick the “Equivalence of Meaning” box but may well fail at the “Quality of Expression” hurdle.

So what to do?

People won’t always agree

Judging translation accuracy and meaning equivalence is inevitably subjective. It’s often a value judgement.

So people will sometimes see things differently. They might disagree on both the extent of any differences and their importance.

A reviewer might identify a difference in meaning that the translator doesn’t see. This is best seen as different viewpoints or opinions, rather than black and white, right and wrong.

They’ll likely both have sound reasons for seeing things as they do.

Putting all this into perspective

This may be sounding rather complicated and ill-defined, but here’s the good news:

• mostly it’ll be clear cut where a translation has captured the meaning of the original text and where it differs;
• any differences of opinion on this should be few and far between – just the odd phrase or clause here and there, and
• these will typically involve more subtle shades of meaning rather than the essential or main message of that section.

Evaluating a translation’s quality of expression

Only native speakers can really assess quality of expression. It requires their native speaker “feel” for the language.

Generally native speakers will agree on what is and isn’t good wording.

But again, not always. Phrasing that is well worded to one linguist, may seem a little clunky to another.

This shouldn’t be a surprise. As we go through life we all develop connotations and emotional attachments around certain words and expressions – both positive and negative.

This leads to preferred ways of saying things, and other words and phrasing we don’t like as much. Because everyone’s life experiences are different, so too are our preferences.

In practice this means one linguist may occasionally consider some phrasing unnatural that another thinks is perfectly fine.

Again it’s not a matter of black and white, right and wrong, but rather reflections of individual wording preferences. Occasional differences in viewpoint on wording are natural and to be expected.

About Ambiguity

A sentence is ambiguous if it can be interpreted in different ways. An oft-quoted example is:

“I saw a man on a hill with a telescope”.

There are six reasonable interpretations here. They depend on who is on the hill (the writer or the man) and whether the telescope is being used by either or is installed on the hill.

Another example is President Bush’s response when asked about finding weapons of mass destruction in Iraq:

“But for those who say we haven’t found the banned manufacturing devices or banned weapons, they’re wrong, we found them”.

What does “them” refer to – the devices, the weapons or both?

Some potential interpretations will be unrealistic in context – like “saw a man” above meaning cutting a man with a saw.

So how does this relate to translations?

Generally, we don’t want a translation to be ambiguous and so open to misinterpretation. Especially not where the original wasn’t ambiguous and had a single clear meaning.

However, there are exceptions. Sometimes an ambiguity will be intentional, in which case the translation needs to reflect that. Perhaps as in President Bush’s response?

And in certain genres, writers may encourage reader interpretation – in poetry for example.

But for most business texts requiring translation, clients want clarity of meaning. Translations should reflect that and not introduce ambiguities.

Three key tips for clients to achieve better translation quality assessment outcomes

1. Use experienced personnel for both your translation and any review process.
An experienced translator should automatically capture all meaning. And an experienced reviewer should make sound judgements on any grey areas and be less likely to make or request changes that others may not see as necessary.

2. Avoid getting embroiled in discussions over grey area issues.
These seldom reach a definitive conclusion – they’re a matter of opinion after all. And they can take up a lot of time – for you, the translator and reviewer.

3. Be crystal clear on who has final say on wording.
If the reviewer, we suggest they simply make any changes they think necessary. If the translator, the reviewer should flag any queries for the translator to consider.

Conclusion

Our method is a remarkably effective way of assessing translation quality across most business needs.

It works so well because it:
– covers all the necessary bases
– breaks the task down into manageable steps

The key is that each step focuses on a different criterion – so doesn’t overload the brain by trying to check too many things at once. That simply doesn’t work and is the perfect recipe for missing things.

It’s something of a tragedy that sound translation assessment methodologies aren’t widely known or used.

They’re seldom taught, either in translation degree courses or in the workplace.

We think everyone should follow this model, and recommend clients ensure any reviewers they engage adopt it.

The following summary pdf can be downloaded to help keep your translation assessment process on track.

Like your own summary of this article?