Study comparing human and machine translation in health promotion Thread poster: Kaspars Melkis
|
|---|
Kaspars Melkis 
United Kingdom
Local time: 18:46
English to Latvian
+ ...
I was intrigued by the IJC report mentioned on Proz and looked at their sources. What studies are really telling us? I have just read the first study that seemed very interesting.
The study: A Comparison of Human and Machine Translation of Health Promotion Materials for Public Health Practice: Time, Costs, and Quality
URL: https://www.ncbi.nlm.nih.gov/pubmed/24084391... See more I was intrigued by the IJC report mentioned on Proz and looked at their sources. What studies are really telling us? I have just read the first study that seemed very interesting.
The study: A Comparison of Human and Machine Translation of Health Promotion Materials for Public Health Practice: Time, Costs, and Quality
URL: https://www.ncbi.nlm.nih.gov/pubmed/24084391
This conclusion of this study that human translation (HT) and machine translation (MT) of health promotional materials in English-Spanish language pair was equally preferred but MT is much cheaper and can be done considerably faster. This concerns to simple texts targeted to general public, so there is little complicated terminology involved and even a translator without specialization should have been able to do them satisfactory.
This is a very small study which obviously may be biased but the reasons why this study was done may show a regular picture how translation buyers see our industry.
The first thing that doesn't seem right is that the quality assessors are bilingual staff without mentioning any qualifications. They could be people who speak Spanish at home but really have no formal education in Spanish. It is mentioned that their internal cost is based on the average hourly wage of $18 which seems quite low. When discounting employer's contributions and payments into sick pay and pension funds, it may be close to the minimum wage. Maybe someone from the US can comment on this because I am not very familiar with US wage structure. In any case, I don't really think we can trust self claimed bilinguals without formal qualifications and probably not very educated in general, to evaluate text quality. That especially applies to Spanish which have many regional variations in Latin America and speakers in the US may have different opinions of what is considered better style.
The next howler is the footnote e: the quality of one HT was so bad that it was not paid for. How bad it could be? I don't think that assessors had a reason to be very strict considering that they were using MTs as well. More likely it was much worse than MT or alternatively it was just a wrong version of Spanish that clearly put them off. It is not clear whether this translation was still included in the final evaluation of HT vs MT.
Were other HTs of similarly poor quality and just a little bit better to just pass the quality filter? We don't know but I very much suspect so. In most cases they had to spend a longer time working on HTs than MTs, sometimes even considerably so. Could it be just because the bilingual staff were very picky and didn't like the original style? Maybe but I am not very convinced. More logical explanation is that there were real quality problems with HTs.
In any case, this is very damning for the translation industry overall. I don't what's going on but it appears that most human translations despite agency involvement are opportunistic and translation quality is very low. How do the a public health department select their translation service suppliers? Do they visit Proz.com or similar sites and chose those that seem ok or offer the lowest price? Whatever is their method, it seems that what they are getting is never of publishing quality. In such circumstances it makes all sense to use MT exclusively and avoid dealing with suppliers at all.
Their target readers are immigrants, most likely poor people who do not speak English and will be happy for any communication into their native language, so they are not going to criticize those translations. This would not work in the target country where poor translations would be very unacceptable. The translation quality standards in the US might be very low. I have seen a lot of validated clinical instruments coming from US institutions that are full with typos, grammar mistakes and mistranslations that I wonder how did they get validated in the first place?
Now my worry is that the lax standards have really created low quality translation market. And now the gig is up because it is being replaced by MT. This study also shows is that MT users can use google translate and do post-editing themselves, so any opportunistic agencies will not be saved by providing post editing services. Their clients will eventually get wiser and will do it in-house.
For me the biggest takeaway is that there is a reputation problem in the translation industry. If you provide high quality specialized translations how do you exactly explain potential clients that you are worth your rate? If the health department suddenly needs a translation of a specialist text (for example, a medical report) that cannot be translated by MT, wouldn't they assume that they just need something a little bit better than MT. So, they are going to pay you, not 8 or 10 cents per word but maybe 12 or 14 cents (just an example).
Another conclusion is that it is a real problem of connecting potential buyers with the qualified translators. It is extremely difficult for a potential buyer to get high quality service. In part, it may be due to the difficulty of letting the translator to know what the client expects, what are the specifications, expected style etc. These issues were supposed to be solved by translation agencies but in practice they only seem to exacerbate the problem. The industry seems to be very broken. ▲ Collapse
| | | |
When all the authors of this study have been writing on machine translation for a long while it’s no wonder their results are biased and tend to prove their hypotheses…
| | | | Daryo
United Kingdom
Local time: 18:46
Serbian to English
+ ...
| Have you noticed | Jul 20, 2019 |
the "post-editing" part?
As long as you can find enough mugs to trick them to practically do the translation at "editing" rates, of course it looks as a good alternative.
Imagine you find someone you can pay to "just polish a bit" you car after you have been racing through muddy fields for days, you would certainly like the price ...
[Edited at 2019-07-20 05:57 GMT]
| | | | Kay Denney
France
Local time: 19:46
French to English
Working at an agency, I once came back from my holiday to find an email from a PM asking me to assess and defend a translation produced during my holiday. All the good translators were off like me, so they found a new translator. After all it was just a little letter along the lines of "please find attached our new catalogue, we hope you have a happy new year browsing and ordering", what could go wrong?
The client was disgusted with the delivered translation, and had an in-house native sp... See more Working at an agency, I once came back from my holiday to find an email from a PM asking me to assess and defend a translation produced during my holiday. All the good translators were off like me, so they found a new translator. After all it was just a little letter along the lines of "please find attached our new catalogue, we hope you have a happy new year browsing and ordering", what could go wrong?
The client was disgusted with the delivered translation, and had an in-house native speaker re-do it. They accused the agency of using MT, so my task was to prove that MT was not used.
I took one look at the delivered translation and wrote back that of course MT was not used, the computer would never make that many spelling mistakes. Because MT may be awful, using the wrong words, but those words are spelled properly at least. ▲ Collapse
| | |
|
|
|
| Interesting article, especially in the light of your excellent comments Kaspars | Jul 20, 2019 |
as it points to one of the one of the reasons why some companies use machine translation - as there are quite a few very poor translators who are quite out of their depth in some specialist areas, and many agencies, judging from some of the texts I receive for review, seem unable to weed them out, the average quality of machine translations may approach the average quality of human translations. The test should obviously compare work done by acknowledged expert translators and machine tra... See more as it points to one of the one of the reasons why some companies use machine translation - as there are quite a few very poor translators who are quite out of their depth in some specialist areas, and many agencies, judging from some of the texts I receive for review, seem unable to weed them out, the average quality of machine translations may approach the average quality of human translations. The test should obviously compare work done by acknowledged expert translators and machine translations with an expert review panel to judge the results and in this case human "assisted" translation would of course win outright. No machine translation that I have seen can be trusted for any medical translation and it is irresponsible to pretend otherwise
However, I say "assisted" as there is a lot of rather ignorant criticism of MT on Proz. Even though I have been a specialist medical translator for 30 years, I have used MT to help me translate medical websites or texts/articles that are already published on the web and are therefore not confidential, and this has helped me to considerably improve the quality of my work - MT does speed things up a bit allowing you to concentrate on parts requiring more care and attention, but mainly all translators develop habits - as we are always translating in a hurry we tend to use the same words all the time. MT continually throws up new alternatives, sometimes crazily wrong but quite often stimulating as they come from a giant corpus of texts that no translator has time to read.
No one is going to compete with MT by ignoring it - even for translating Poetry - and machine assistance enlarges the mind.
[Edited at 2019-07-20 08:49 GMT] ▲ Collapse
| | | | To report site rules violations or get help, contact a site moderator: You can also contact site staff by submitting a support request » Study comparing human and machine translation in health promotion | Protemos translation business management system | Create your account in minutes, and start working! 3-month trial for agencies, and free for freelancers!
The system lets you keep client/vendor database, with contacts and rates, manage projects and assign jobs to vendors, issue invoices, track payments, store and manage project files, generate business reports on turnover profit per client/manager etc.
More info » |
| | Trados Studio 2022 Freelance | The leading translation software used by over 270,000 translators.
Designed with your feedback in mind, Trados Studio 2022 delivers an unrivalled, powerful desktop
and cloud solution, empowering you to work in the most efficient and cost-effective way.
More info » |
|
| | | | X Sign in to your ProZ.com account... | | | | | |
Login to reply/comment 
Translation news
