Linguistic Testing: What Is It And Why Do You Ne...
Learn about this crucial step in maximizing your globalization efforts.
10.01.2023
Google Translate has come a long way since its #googlefail days of yore. But whoever uses the automated translation tool can tell you that it still has its glitches and faults. So, how accurate is Google Translate today?
And who better to assess its accuracy than a leading translation services provider?
We gave Google Translate a spin with four languages and assessed the accuracy of their translations into English: Spanish, French, Italian, and Russian. The results are rather interesting.
But let’s start with some background first.
Google launched its automated translation tool back in 2006. At the time, it was based on zillions of human translations made by the European Parliament and the United Nations. But the translation algorithm was built on a statistical model. Simply put, it meant that Google translated meaning on a word level.
The result?
This is just an analogy for all the #googlefails memes that swarmed the internet.
Truth be told, the most significant problems came up when translating languages with utterly different word order. With a statistical model, Google had no way of understanding how to turn the sentence structure around. (Remember, german students: the verb always goes at the end). And so a German-to-English translation went something like:
Roses and violets will you go pluck.
We wonder who that might resemble…
Fast-forward ten years and Google decides to uproot its computing entirely, introducing Google Neural Machine Translation (GNMT). The new artillery operates on a wholly different level. It doesn’t translate individual words anymore. Instead, it uses an AI-powered neural machine translation algorithm that fetches the meaning from a broader context.
And instead of mirroring the source text’s word sequence, it tries to mimic the target language’s grammar and syntax rules. Which is basically how a human brain translates. Translating meaning, not individual words. And putting it into appropriate grammar structures and sequences.
With so great a postulate, the days of human translations were to be counted, surely. After all, with access to so much multilingual content, Google Translate could surely “learn” fast enough to outdo any human translator.
Or could it?
It’s 2023, so we decided to check how well Google Translate has “learned” over the years.
We selected four prominent language combinations:
And asked our professional human translators to give their assessment.
Fail, near-miss, or stellar pass?
Spanish
Las vacaciones para este verano están a la vuelta de la esquina, y si deseas salir con tus amistades o familiares para disfrutar algún destino hondureño, no te olvides tomar todas las medidas de bioseguridad para tengas unos días relajantes y seguros.
English
Vacations for this summer are just around the corner, and if you want to go out with your friends or family to enjoy a Honduran destination, don’t forget to take all the biosecurity measures to have a few relaxing and safe days.
At first glance, Google's translation quality seems to have done a decent job. No epic translation fails or odd style choices.
But let’s look at a few thorny details.
‘Vacations for this summer’ is overly wordy and not precisely the expression of choice in English. And vacation? It works in US English, but Brits tend to call them summer holidays. So mind the gap here.
The most salient point is this one: ‘biosecurity measures.’ Really? The text is referring to safety measures to be taken during this pandemic era. Biosecurity seems just a tad over the top. (Is that still Yoda speaking from the future?).
Summer vacations are just around the corner. If you wish to come to enjoy one of our Honduran spots with your family or friends, make sure to take all the necessary health precautions for a relaxing and safe stay.
French
Balade dans le Paris enjazzé de Boris Vian
Écrivain, ingénieur, trompettiste, chanteur, poète… À Paris, Boris Vian a exercé tous ses talents.
English
A stroll through Boris Vian’s invaded Paris
Writer, engineer, trumpeter, singer, poet… In Paris, Boris Vian has exercised all his talents.
Truth be told, when the author of a source text “invents” words, it isn’t easy for a human translator, either. Enjazzé doesn’t exist in French, but any human translator would understand what the author made by adding the prefix en -a suffix –é to jazz.
But one wonders what was going on inside Google Translate’s robot head when it decided to translate a word containing “jazz” into “invasion.” Probably it isn’t a big jazz fan.
A stroll through Boris Vian’s Jazzy (or Jazzed-Up) Paris
Writer, engineer, trumpeter, singer, poet… In Paris, Boris Vian has done it all.
Italian
Nuova stagione condotta da Giancarlo Magalli con accanto Samanta Togni, Umberto Broccoli, Paolo Fox e il gruppo musicale di Stefano Palatresi. Quotidiano di attualità, costume, spettacolo, musica e curiosità.
English
New season conducted by Giancarlo Magalli alongside Samanta Togni, Umberto Broccoli, Paolo Fox and the Stefano Palatresi musical group. Newspaper of current affairs, costumes, entertainment, music and curiosities.
It’s interesting to see how even a contemporary writing piece can cause significant math-ache for Google Translate. In the case of Italian, the translation quality is already showing significant misses. We have a full array of errors, from grammar errors in English (starting the sentence with “New season…” without an article) to many style choices which sound so unnatural in English:
Conducted — We usually host a show
The Stefano Palatresi musical group — It’s not the name of the musical group. It’s a person’s group, and the person’s name is Stefano Palatresi. And while we’re at it: musical group? Why not simply band?
Newspaper — Here is a hashtag-worthy fail. A quotidiano can either be a newspaper or a daily show. And the beginning of the sentence already alludes it’s not a newspaper because you can’t conduct one.
Costumes — In this context, it’s fashion or outfits, not costumes.
Curiosities — Would anyone ever use it in English? We bet interesting facts sounds more like it, doesn’t it?
A new season hosted by Giancarlo Magalli, together with Samanta Togni, Umberto Broccoli, Paolo Fox, and music by Stefano Palatresi and his band. A show covering news, fashion, entertainment, music, and other topics of interest.
Let’s see how Google Translate manages a more distant language combination now.
Russian
Москва была одним из любимых городов Александра Пушкина. Поэт родился здесь, провел детство, часто бывал, повзрослев, встретил главную любовь своей жизни и женился. Не обошел он вниманием Москву и в «Евгении Онегине».
English
Moscow was one of the favorite cities of Alexander Pushkin. The poet was born here, spent his childhood, often visited, having matured, met the main love of his life and got married. He also paid attention to Moscow in Eugene Onegin.
Now, you can see how things start getting murkier the moment you translate from a more distant language family.
The syntax is not clear. It looks like a string of clauses piled up. But in Russian, that finite-type gerund повзрослев actually functions as an action breaker. In fact, the English translation is not only weird. It’s grammatically inaccurate. No human translator would make such a mistake.
And then, we have a locution that got Google Translate’s math going berserk.
Не обошел он вниманием Москву и From an accuracy point of view, one could say everything is correct. The word-for-word translation would be: “He did not ignore Moscow in Eugene Onegin.
But what is missing is the salience. The last sentence means Pushkin liked Moscow so much he even included the city in his masterpiece. But putting it as Google Translate did makes it read as “just one more thing” Pushkin did.
So the meaning is there. But the tone and the heightened relation in respect to what was said before has gone astray.
Google Translate has a cousin called Yandex Translate. This Eastern brother algorithm claims it can do a better job with translating Slavic languages.
Let’s take a look:
Russian
Москва была одним из любимых городов Александра Пушкина. Поэт родился здесь, провел детство, часто бывал, повзрослев, встретил главную любовь своей жизни и женился. Не обошел он вниманием Москву и в «Евгении Онегине».
English – Google Translate
Moscow was one of the favorite cities of Alexander Pushkin. The poet was born here, spent his childhood, often visited, having matured, met the main love of his life and got married. He also paid attention to Moscow in Eugene Onegin.
English – Yandex Translate
Moscow was one of Alexander Pushkin’s favorite cities. The poet was born here, spent his childhood, often visited, grew up, met the main love of his life and got married. He did not ignore Moscow in “Eugene Onegin”.
Yandex does a better job at using possessives, which shortens the English translation. But as far as including Moscow, it falls flat the same way Google Translate did.
Moscow was one of Alexander Pushkin’s favorite cities. The poet was born here, spent his childhood, often visited—and once grown up, the place where he found the love of his life and got married. He didn’t even fail to give a worthwhile mention of the city in his Eugene Onegin.
Here are the most prominent mistakes you will find:
Google Translate struggles less with it today. But you can still sense that something’s off. Usually, it has to do with clauses being piled up unnaturally.
Unless there are pronouns determining the gender of the subject, Google can’t figure it out on its own.
Many languages, like French, have two forms of pronouns: Tu (informal) and Vous (formal). However, when translating from English, Google Translate still can’t figure out the tone because the tone is signaled by other elements such as word choice in English.
Many words are spelled the same but have different functions, especially nouns and verbs. It’s getting better at it, but Google Translate can still pull a laugh from words like squash.
Google Translate will automatically propose the US spelling variant. So if you’re expecting British spelling but get a bunch of favorite neighbors, you know something’s off.
Yes. It’s called LingPerfect, your go-to professional translation services provider.
But jokes aside, there are many other machine translation tools out there. Unfortunately, neither of them is an absolute winner. A lot depends on the type of text and the language combination. So you have to try them out and see which one produces a more accurate translation.
Let’s be honest. People who claim you should never use it are a tad harsh. Google's neural machine translation system has made enormous progress since its early #fail days. Most of the time, the automatic translation can figure out the meaning and transfer it into a grammatically acceptable form in the target language. And the level of "natural language" applied can be quite astonishing. The fact that an algorithm can do such heavy lifting is a significant achievement in its own right.
But what Google Translate can’t and probably won’t be able to do for quite some time is understanding how to translate texts containing rule-breaking sentences or locutions, the non-conforming with the "grammatical rules" which are actually the would of the way a real person will approach writing. And by extension, only a human translator and a native speaker can understand how to apply this across the language barrier gap into a new language.
So it’s perfectly fine to use Google Translate to get the gist of the text. But you should never use it to translate official documents or content that has any business value.
For those, there is no artificial intelligence that can beat the human translation of a professional translation services provider.
Learn about this crucial step in maximizing your globalization efforts.
Patent translation is no stroll in the park. Here are a few tips from on how to tackle it like a pro.
And how can recognizing and accepting diversity help you foster creativity and innovation?
Learn about their difference and how localization can impact your bottom line.
Have you ever wondered why they sort of look the same… but sound completely different? Read on, and you’ll find out.
Ever wondered if they are the same? Not at all! Discover their differences here.
Do you need an interpreter or a translator for your next business meeting? Here’s a breakdown of the key differences between these two types of professionals.
When it comes time to market your products internationally, you’ll need to start looking for the…
Everyone knows what translation is. But what about Transcreation? Click here to learn about this valuable language service.
How accurate is Google translate in 2021? We asked 4 expert translators to assess whether it’s a fail or a pass.
Want to make sure your content is easy to translate? Here are 15 easy tips from the experts at LingPerfect.
Accurate and timely translation services are integral in today’s increasingly globalized legal…
Learning a new language is hard. Make sure it’s worth the buck.
When it comes time to market your products internationally, you’ll need to start looking for the…
Accurate and timely translation services are integral in today’s increasingly globalized legal…
When it comes time to market your products internationally, you’ll need to start looking for the…
Accurate and timely translation services are integral in today’s increasingly globalized legal…