|
 
 |
| ORIGINAL ARTICLE |
|
| Year : 2019 | Volume
: 35
| Issue : 2 | Page : 119-124 |
|
Feasibility of using “Google Translate” in adaptation of survey questionnaire from English to Bengali: A pilot study
Himel Mondal1, Shaikat Mondal2, Sarika Mondal3
1 Department of Physiology, Fakir Mohan Medical College and Hospital, Balasore, Odisha, India
2 Department of Physiology, Kalna SD Hospital, Purba Bardhaman, India
3 Freelance Medical Writer, ORCID: 0000-0001-5039-9919, Kolkata, West Bengal, India
| Date of Submission | 29-May-2018 |
| Date of Decision | 16-Jul-2018 |
| Date of Acceptance | 08-Oct-2018 |
| Date of Web Publication | 26-Jun-2019 |
Correspondence Address:
Dr. Himel Mondal
Department of Physiology, Fakir Mohan Medical College and Hospital, Balasore - 756 019, Odisha
India
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/ijsp.ijsp_39_18
Background: Questionnaires are forward-translated and then back-translated by human translators during adaptation of a survey questionnaire in the local language. The machine translation method is now available free from “Google Translate” website and it can be used for translation from English to Bengali and vice versa. Using machine translation method may save researchers from investing money and time in translation by human experts. Aim: The aim of this pilot study was to test the feasibility of “Google Translate” in forward-translation and back-translation during adaptation of a survey questionnaire from English to Bengali. Materials and Methods: A survey questionnaire, originally developed in English, was forward-translated to Bengali and then back-translated to English by machine translation method and manual translation method. Both the versions were checked for errors expressed in subjective sentence error rate (SSER). Back-translated English questionnaire obtained from machine translation and manual translation was checked for equivalence with the original questionnaire. Results: SSER for forward-translated Bengali questionnaire by machine translation method was 66.60 ± 4.04 and manual translation method was 117.20 ± 1.30 (P < 0.001). SSER for back-translated English questionnaire by machine translation method was 61.20 ± 5.02 and manual translation method was 116.20 ± 2.77 (P < 0.001). Equivalence between back-translated questionnaire and original questionnaire by machine translation method was 48.40 ± 6.02 and by manual translation method was 112.40 ± 3.29 (P < 0.001). Conclusion: The machine-translated questionnaire showed higher error than manually translated questionnaire. Machine-translated questionnaire also showed lower equivalence with the original questionnaire. Hence, the use of machine translation (from English to Bengali) offered by “Google Translate” should be used with caution.
Keywords: Back-translation, forward-translation, machine translation, medical survey, questionnaire
How to cite this article:
Mondal H, Mondal S, Mondal S. Feasibility of using “Google Translate” in adaptation of survey questionnaire from English to Bengali: A pilot study. Indian J Soc Psychiatry 2019;35:119-24 |
How to cite this URL:
Mondal H, Mondal S, Mondal S. Feasibility of using “Google Translate” in adaptation of survey questionnaire from English to Bengali: A pilot study. Indian J Soc Psychiatry [serial online] 2019 [cited 2021 Apr 21];35:119-24. Available from: https://www.indjsp.org/text.asp?2019/35/2/119/261479 |
| Introduction | |  |
A carefully designed survey questionnaire is an effective tool for the collection of epidemiological data.[1] Previously validated questionnaire from a published study is used in many medical surveys.[2] However, if the original questionnaire is of different language, then it is to be adapted for the local language before administration.[3] Adaptation of survey questionnaire requires the involvement of time, workforce, and financial support. A typical adaptation method involves forward-translation, back-translation, and cognitive interview among target population. Before the cognitive interview, conceptual equivalence check is another important step which determines the equivalence between original and back-translated questionnaire.[4] Among these three major steps, the first two steps (i.e., forward-translation and back-translation) are usually carried out by the expert translators.[5]
Google Translate is a machine translation service provided by Google Inc. (https://translate.google.com). The software, run in the website, helps the user to translate sentences between two languages. For example, if the user types a sentence in English in the left-hand side text box, the language is auto detected and then the user can select any language of choice to translate. The translated sentence is shown on the right-hand side box. For better understanding, a paragraph of text was copied from the “About us” of this journal and translated in Bengali and the screenshot is shown in [Figure 1]. This translation service can be accessed by any user from majority of the region of the world free of cost. The website helps in translation in 103 languages both ways and it is continuously enriched by native language speakers.[6] Hence, the forward-translation and back-translation can be carried out by the help of this machine translation method. | Figure 1: Screenshot of “Google Translate” website after conduction of a translation from English to Bengali
Click here to view |
Previous studies have established the utility of “Google Translate” in medical communication with different level of accuracy.[7],[8] However, the adaptation of survey questionnaire requires perfection. Even a minor level of deficiency in linguistic or cultural issue may decrease the effectiveness of the questionnaire. The applicability of “Google Translate” has been tested for the Chinese language with diverse result.[9],[10] To the best of our knowledge, no study has been conducted to ascertain the feasibility of “Google Translate” in the adaptation of English survey questionnaire in Bengali.
With this context, this study was designed to check the feasibility of machine translation service provided by “Google Translate” in forward-translation and back-translation during the adaptation of survey questionnaire from English to Bengali. In addition, conceptual equivalence was checked between the original and the back-translated version as the equivalence may be affected by the quality of translation.
| Materials and Methods | | |
This study did not involve any human subject as a sample for the study. The expert translators, reviewers, and experts for equivalence checking were adults (i.e., age >18 years) who provided their written consent for their voluntary contribution in the study.
The language used in forward-translation
Bengali is the 6th language in the world with 242 million speaker strength.[11] Several questionnaires, originally developed in English, have been adapted in Bengali for conducting survey among native Bengali speakers.[12],[13],[14],[15] We aimed to translate a survey questionnaire available in English to Bengali.
The questionnaire used in translation
Kessler Psychological Distress Scale (K10) is a tool to screen global distress in the preceding 1-month period.[16],[17] This questionnaire was originally developed in English. It contains 10 questions and each question has 5-point Likert-type response options. The Likert-type options are same for each question. These are comprised of incomplete sentences (i.e., words/phrases). Hence, those were not included in the analysis. There are two sentences for instruction to the respondents. Hence, a total of 12 sentences were taken as final for using those in translation. The steps carried out during this study are depicted in [Figure 2]. | Figure 2: Steps involved in machine translation and manual translation methods and its quality checkpoint ([1] Forward-translation from original English [V 1.0] questionnaire to Bengali by machine [V 1.1e] and by manual method [V 1.1m],[2] subjective sentence error rating of the forward-translated questionnaire,[3] back-translation from Bengali [V 1.1e and V 1.1m] to English [V 1.2e and V 1.2m],[4] subjective sentence error rating of the back-translated questionnaire,[5] equivalence check between original questionnaire [V 1.0] and back-translated questionnaire [V 1.2e and V 1.2m])
Click here to view |
Manual translation method
The questionnaire was forward-translated from English (i.e., the language in which the questionnaire was developed originally) to Bengali by an expert translator who had previous experience in the adaptation of survey questionnaire. Then, it was back-translated to English by another expert translator who had previous experience in translation of health survey questionnaire. Both the experts were having doctoral degree in preventive and social medicine whose mother tongue was Bengali. Both the translated questionnaires were preserved for linguistic error analysis.
Machine translation method
For the task, a Windows 7-based personal computer with Intel Pentium Processor T3400 (2.16 GHz, 2GB RAM) with Google Chrome web browser was used. The 12 sentences from the questionnaire were copied individually and were translated to Bengali on the website https://translate.google.com. Then, the translated individual sentence (i.e., question) was copied from the website and pasted on a Microsoft word document word to word. After that, these sentences were copied verbatim from the word document and back-translated to English on the website. Thus, a Bengali and English questionnaires were formed. Both were kept for linguistic error analysis.
Linguistic error analysis of forward-translated Bengali questionnaire
Five Bengali language experts (having master's degree in Bengali and mother tongue was Bengali) were randomly selected from nonmedical faculty for scoring errors in the sentences. These experts had previous experience in subjective sentences error rate (SSER). SSER is a method to evaluate the quality of translation by quantitative method where 0 denotes “nonsense” and 10 denotes “perfect.”[18],[19] Hence, a lower score indicates higher errors in the sentence. Forward-translated Bengali questionnaires obtained from machine translation method and manual translation method were given to experts for SSER. They were instructed to rate the machine-translated version first and then manually translated version. The ratings obtained from the experts were entered in spreadsheet for statistical analysis.
Linguistic error analysis of back-translated English questionnaire
A set of English language experts (having a master's degree in English and mother tongue was Bengali) (n = 5) who had experience in SSER was consulted for the English version. They were also selected randomly form nonmedical faculty. The machine-translated English questionnaire and manually translated English questionnaire both were given to them for SSER. The ratings obtained from the experts were entered in a spreadsheet for statistical analysis.
Equivalence check by experts
We approached another five experts who had previous experience in the adaptation of survey questionnaire in Bengali. They were selected from medical faculty. Two were from the department of preventive and social medicine, two were from the department of psychiatry, and one was from the department of physiology. They rated the conceptual equivalence between the original questionnaire and back-translated questionnaire in 10-point scale where 10 denotes “perfectly equivalent” and 0 denotes “unequal.”First, they were given the machine-translated questionnaire then the manually translated questionnaire. Score of both questionnaires was stored and coded in a spreadsheet for statistical analysis.
Statistical analysis
Data were expressed in mean and standard deviation. SSER and equivalence scores obtained for machine-translated and manually translated questionnaires were compared by paired t-test. Two-tailed P < 0.05 was considered statistically significant. The analyses were carried out in Microsoft Excel and GraphPad Prism 6.01 for Windows (GraphPad Software, Inc., CA, USA).
| Results | | |
The result of this study is shown in three tables. Linguistic errors in forward-translated questionnaire in Bengali, as reported in terms of SSER score, are shown in [Table 1]. Except for two sentences (7th and 9th), all the sentences showed statistically significant higher error in machine-translated questionnaire. | Table 1: Score of subjective sentence error rate for the Bengali survey questionnaire obtained by forward-translation carried out by machine and expert translator
Click here to view |
SSER score obtained for back-translated questionnaire in English is shown in [Table 2]. Except the 4th and 8th question, all other sentences showed statistically significant higher level of error in machine-translated questionnaire. | Table 2: Score of subjective sentence error rate for the English survey questionnaire obtained by back-translation carried out by machine and expert translator
Click here to view |
Equivalence check scores between original questionnaire and back-translated questionnaire are shown in [Table 3]. Manually translated questionnaire showed higher level of equivalence than machine-translated questionnaire. | Table 3: Score of equivalence check for the English questionnaire obtained by back-translation carried out by machine and expert translator
Click here to view |
| Discussion | | |
The aim of this study was to check the feasibility of a machine translation method in steps of adaptation of a questionnaire in Bengali. For the study, a questionnaire was chosen which was originally developed in English. This questionnaire was forward-translated (from English to Bengali) manually by human experts and by machine on “Google Translate” [Figure 2]. Then, the level of error was ascertained by SSER for both manual- and machine-translated questionnaire.
The SSER is a method of subjective evaluation of errors in sentences with quantitative output (score numbers). There are several other methods to detect errors in sentences. One of the methods is Word Error Rate. In this method, the percentages of words which need to be added, deleted, or modified are checked with the reference. Sentence Error Rate is another method where percentages of sentences which are different from reference are checked.[18] Both of these methods could not be applied in this study due to two reasons. First, there was no culturally validated K10 scale for the area from where the assessors were selected. Hence, reference was unavailable. For this reason, the comparison might not be valid for analysis of Bengali and English questionnaire. Second, we were unable to find experts who had previous experience of these methods. In addition, there is another subjective method, which is called information item error rate (IER). This method is commonly used for long sentences.[18] In this method, the long sentences are segmented into information items and then analyzed. As the questionnaire chosen for this study was comprised of simple and short questions, IER was not considered for the study.
Level of error in machine-translated questionnaire was higher (overall score: 66.60 ± 4.04) than the manually translated questionnaire (overall score: 117.20 ± 1.30). The difference was huge and statistically significant (P < 0.001). Sentences used for the instruction section of the questionnaire showed highest level of errors [Table 1]. A proper instruction on the beginning of a questionnaire is of paramount importance in self-administered questionnaire. It helps the respondents to get a fair idea about the aim of the questionnaire. It also helps to understand the response options and how to mark those responses. If the instruction is not conveyed effectively, it may confuse the respondents. Hence, chances of unanswered questions increase.
Among the 10 questions of the questionnaire, the 2nd question showed the lowest score (0.80 ± 0.84). Totally 8 questions among 10 questions showed statistically significant higher error in machine-translated version of the questionnaire [Table 1]. In previous studies, it has been found that “Google Translate” may help in medical communication (e.g., doctor taking history from the patient) between health-care providers and patients or patient parties.[7],[8] In verbal communication, a minor grammatical and syntax error can be overcome by human perception during understanding. However, a survey questionnaire should have correct sentences to the highest level for its successful application. In this study, we found high level of error in machine-translated questionnaire. Hence, it may be the major constraining factor for questionnaire adaptation by machine translation method.
The Bengali questionnaire obtained by forward-translation was then back-translated to English by machine translation method and by expert human translator. This English questionnaire was rated by SSER by experts. A surprising result was found for the machine-translated questionnaire. At first, the English questionnaire was translated to Bengali by “Google Translate,” and then, it was back-translated to English verbatim. Hence, it was assumed that the English questionnaire would be the same as the original questionnaire. However, that English questionnaire showed high level of error in SSER (overall score: 61.20 ± 5.02), whereas manually translated questionnaire showed minor level of error score (overall score: 116.20 ± 2.77) (P < 0.001) [Table 2].
In the process of adaptation of the survey questionnaire, after forward- and back-translation, the next step is equivalence check.[4] This is an important part for any biomedical survey questionnaire as the conceptual equivalence is more important than the lingual equivalence. In this step, conceptual equivalence is checked between the back-translated questionnaire and the original questionnaire. If these two questionnaires show a satisfactory level of equivalence, it is considered that the forward-translated questionnaire can be adapted for further process. Back-translated questionnaire by machine translation method was first checked for equivalence with the original questionnaire, and the score was not convincing (overall score: 48.40 ± 6.02). In contrast, manually back-translated questionnaire showed statistically significant (P < 0.001) higher score in equivalence (overall score: 112.40 ± 3.29) [Table 3].
It is evident from the result that at this point of time, the “Google Translate” has limitations for its application in forward- and back-translations from English to Bengali. However, translation suggestions are being continuously provided to “Google Translate” by the contributors worldwide [Figure 1].[6] Hence, chances of improvements are there.
Limitations
This study has several limitations. A single questionnaire with limited number of questions was used for translation according to our limited experts and their contributing time for this study. Higher number of questions and increased number of questionnaires could provide further insight into the capability of “Google Translate” as a translating platform. SSER is a subjective analysis method for scoring errors in sentences. Hence, the level of perfection in rating may not be uniform across different evaluator.
| Conclusion | | |
At this point of time, for translation of survey questionnaire from English to Bengali by “Google Translate” has questionable feasibility with high level of lingual errors. Machine-translated questionnaire showed higher sentence errors rate and weak conceptual equivalence when compared to original questionnaire. Hence, Google Translate has limitations in translation of questionnaires when it is used alone. However, considering the easy accessibility and ease of use, it can serve as an adjunct during the adaptation of questionnaire. Further studies are needed to explore the potential of “Google Translate” in effective translation of questionnaire with higher number of questionnaire.
Acknowledgment
We are grateful to the expert translators, assessors of the linguistic errors by SSER, and the experts who checked the conceptual equivalence.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | |
| 2. | Boynton PM, Greenhalgh T. Selecting, designing, and developing your questionnaire. BMJ 2004;328:1312-5.  |
| 3. | |
| 4. | Mondal H, Mondal S. Pretesting and cognitive interviewing are integral parts in translation of survey questionnaire. Indian J Psychiatry 2017;59:251-2. [ PUBMED] [Full text] |
| 5. | Squires A, Aiken LH, van den Heede K, Sermeus W, Bruyneel L, Lindqvist R, et al. Asystematic survey instrument translation process for multi-country, comparative health workforce studies. Int J Nurs Stud 2013;50:264-73. |
| 6. | |
| 7. | Kaliyadan F, Pillai SG. The use of Google language tools as an interpretation aid in cross-cultural doctor-patient interaction: A pilot study. Inform Prim Care 2010;18:141-3. |
| 8. | Sciarra AM, Batigália F, Oliveira MA. Technological devices improving system of translating languages: What about their usefulness on the applicability in medicine and health sciences? Braz J Cardiovasc Surg 2015;30:664-7. |
| 9. | Beh TH, Canty DJ. English and mandarin translation using google translate software for pre-anaesthetic consultation. Anaesth Intensive Care 2015;43:792-3. |
| 10. | Turner AM, Dew KN, Desai L, Martin N, Kirchhoff K. Machine translation of public health materials from English to Chinese: A Feasibility study. JMIR Public Health Surveill 2015;1:e17. |
| 11. | Simons GF, Fennig CD. Summary by language size. In: Ethnologue: Languages of the World. 12 th ed. Dallas, Texas: SIL International; 2017. Available from: http://www.ethnologue.com/statistics/size. [Last accessed on 2018 May 02]. |
| 12. | Arafat SM. Psychometric validation of the Bangla version of the patient-doctor relationship questionnaire. Psychiatry J 2016;2016:9385364. |
| 13. | Muquith MA, Islam MN, Haq SA, Ten Klooster PM, Rasker JJ, Yunus MB, et al. Cross-cultural adaptation and validation of a Bengali version of the modified fibromyalgia impact questionnaire. BMC Musculoskelet Disord 2012;13:157. |
| 14. | Mondal H, Mondal S, Baidya C. Comparison of perceived sleep quality among urban and rural adult population by Bengali Pittsburgh sleep quality index. Adv Hum Biol 2018;8:36-40. [Full text] |
| 15. | Islam N, Basak TB, Voshaar MA, Ferdous N, Rasker JJ, Haq SA, et al. Cross-cultural adaptation and validation of a Bengali health assessment questionnaire for use in rheumatoid arthritis patients. Int J Rheum Dis 2013;16:413-7. |
| 16. | Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek DK, Normand SL, et al. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002;32:959-76. |
| 17. | Andrews G, Slade T. Interpreting scores on the kessler psychological distress scale (K10). Aust N Z J Public Health 2001;25:494-7. |
| 18. | Tomás J, Mas JA, Casacuberta F. Quantitative method for machine translation evaluation. Budapest, Hungary: Association for Computational Linguistics; 2003. p. 27-34. Available from: http://dl.acm.org/ft_gateway.cfm?id=1641401&type=pdf [Last accessed on 2018 May 11]. |
| 19. | |
[Figure 1], [Figure 2]
[Table 1], [Table 2], [Table 3]
|
Search Pubmed for