• Users Online: 540
  • Home
  • Print this page
  • Email this page
Home About us Editorial board Ahead of print Current issue Search Archives Submit article Instructions Subscribe Contacts Login 


 
 Table of Contents  
ORIGINAL ARTICLE
Year : 2019  |  Volume : 35  |  Issue : 2  |  Page : 119-124

Feasibility of using “Google Translate” in adaptation of survey questionnaire from English to Bengali: A pilot study


1 Department of Physiology, Fakir Mohan Medical College and Hospital, Balasore, Odisha, India
2 Department of Physiology, Kalna SD Hospital, Purba Bardhaman, India
3 Freelance Medical Writer, ORCID: 0000-0001-5039-9919, Kolkata, West Bengal, India

Date of Submission29-May-2018
Date of Decision16-Jul-2018
Date of Acceptance08-Oct-2018
Date of Web Publication26-Jun-2019

Correspondence Address:
Dr. Himel Mondal
Department of Physiology, Fakir Mohan Medical College and Hospital, Balasore - 756 019, Odisha
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/ijsp.ijsp_39_18

Rights and Permissions
  Abstract 


Background: Questionnaires are forward-translated and then back-translated by human translators during adaptation of a survey questionnaire in the local language. The machine translation method is now available free from “Google Translate” website and it can be used for translation from English to Bengali and vice versa. Using machine translation method may save researchers from investing money and time in translation by human experts. Aim: The aim of this pilot study was to test the feasibility of “Google Translate” in forward-translation and back-translation during adaptation of a survey questionnaire from English to Bengali. Materials and Methods: A survey questionnaire, originally developed in English, was forward-translated to Bengali and then back-translated to English by machine translation method and manual translation method. Both the versions were checked for errors expressed in subjective sentence error rate (SSER). Back-translated English questionnaire obtained from machine translation and manual translation was checked for equivalence with the original questionnaire. Results: SSER for forward-translated Bengali questionnaire by machine translation method was 66.60 ± 4.04 and manual translation method was 117.20 ± 1.30 (P < 0.001). SSER for back-translated English questionnaire by machine translation method was 61.20 ± 5.02 and manual translation method was 116.20 ± 2.77 (P < 0.001). Equivalence between back-translated questionnaire and original questionnaire by machine translation method was 48.40 ± 6.02 and by manual translation method was 112.40 ± 3.29 (P < 0.001). Conclusion: The machine-translated questionnaire showed higher error than manually translated questionnaire. Machine-translated questionnaire also showed lower equivalence with the original questionnaire. Hence, the use of machine translation (from English to Bengali) offered by “Google Translate” should be used with caution.

Keywords: Back-translation, forward-translation, machine translation, medical survey, questionnaire


How to cite this article:
Mondal H, Mondal S, Mondal S. Feasibility of using “Google Translate” in adaptation of survey questionnaire from English to Bengali: A pilot study. Indian J Soc Psychiatry 2019;35:119-24

How to cite this URL:
Mondal H, Mondal S, Mondal S. Feasibility of using “Google Translate” in adaptation of survey questionnaire from English to Bengali: A pilot study. Indian J Soc Psychiatry [serial online] 2019 [cited 2019 Jul 20];35:119-24. Available from: http://www.indjsp.org/text.asp?2019/35/2/119/261479




  Introduction Top


A carefully designed survey questionnaire is an effective tool for the collection of epidemiological data.[1] Previously validated questionnaire from a published study is used in many medical surveys.[2] However, if the original questionnaire is of different language, then it is to be adapted for the local language before administration.[3] Adaptation of survey questionnaire requires the involvement of time, workforce, and financial support. A typical adaptation method involves forward-translation, back-translation, and cognitive interview among target population. Before the cognitive interview, conceptual equivalence check is another important step which determines the equivalence between original and back-translated questionnaire.[4] Among these three major steps, the first two steps (i.e., forward-translation and back-translation) are usually carried out by the expert translators.[5]

Google Translate is a machine translation service provided by Google Inc. (https://translate.google.com). The software, run in the website, helps the user to translate sentences between two languages. For example, if the user types a sentence in English in the left-hand side text box, the language is auto detected and then the user can select any language of choice to translate. The translated sentence is shown on the right-hand side box. For better understanding, a paragraph of text was copied from the “About us” of this journal and translated in Bengali and the screenshot is shown in [Figure 1]. This translation service can be accessed by any user from majority of the region of the world free of cost. The website helps in translation in 103 languages both ways and it is continuously enriched by native language speakers.[6] Hence, the forward-translation and back-translation can be carried out by the help of this machine translation method.
Figure 1: Screenshot of “Google Translate” website after conduction of a translation from English to Bengali

Click here to view


Previous studies have established the utility of “Google Translate” in medical communication with different level of accuracy.[7],[8] However, the adaptation of survey questionnaire requires perfection. Even a minor level of deficiency in linguistic or cultural issue may decrease the effectiveness of the questionnaire. The applicability of “Google Translate” has been tested for the Chinese language with diverse result.[9],[10] To the best of our knowledge, no study has been conducted to ascertain the feasibility of “Google Translate” in the adaptation of English survey questionnaire in Bengali.

With this context, this study was designed to check the feasibility of machine translation service provided by “Google Translate” in forward-translation and back-translation during the adaptation of survey questionnaire from English to Bengali. In addition, conceptual equivalence was checked between the original and the back-translated version as the equivalence may be affected by the quality of translation.


  Materials and Methods Top


This study did not involve any human subject as a sample for the study. The expert translators, reviewers, and experts for equivalence checking were adults (i.e., age >18 years) who provided their written consent for their voluntary contribution in the study.

The language used in forward-translation

Bengali is the 6th language in the world with 242 million speaker strength.[11] Several questionnaires, originally developed in English, have been adapted in Bengali for conducting survey among native Bengali speakers.[12],[13],[14],[15] We aimed to translate a survey questionnaire available in English to Bengali.

The questionnaire used in translation

Kessler Psychological Distress Scale (K10) is a tool to screen global distress in the preceding 1-month period.[16],[17] This questionnaire was originally developed in English. It contains 10 questions and each question has 5-point Likert-type response options. The Likert-type options are same for each question. These are comprised of incomplete sentences (i.e., words/phrases). Hence, those were not included in the analysis. There are two sentences for instruction to the respondents. Hence, a total of 12 sentences were taken as final for using those in translation. The steps carried out during this study are depicted in [Figure 2].
Figure 2: Steps involved in machine translation and manual translation methods and its quality checkpoint ([1] Forward-translation from original English [V 1.0] questionnaire to Bengali by machine [V 1.1e] and by manual method [V 1.1m],[2] subjective sentence error rating of the forward-translated questionnaire,[3] back-translation from Bengali [V 1.1e and V 1.1m] to English [V 1.2e and V 1.2m],[4] subjective sentence error rating of the back-translated questionnaire,[5] equivalence check between original questionnaire [V 1.0] and back-translated questionnaire [V 1.2e and V 1.2m])

Click here to view


Manual translation method

The questionnaire was forward-translated from English (i.e., the language in which the questionnaire was developed originally) to Bengali by an expert translator who had previous experience in the adaptation of survey questionnaire. Then, it was back-translated to English by another expert translator who had previous experience in translation of health survey questionnaire. Both the experts were having doctoral degree in preventive and social medicine whose mother tongue was Bengali. Both the translated questionnaires were preserved for linguistic error analysis.

Machine translation method

For the task, a Windows 7-based personal computer with Intel Pentium Processor T3400 (2.16 GHz, 2GB RAM) with Google Chrome web browser was used. The 12 sentences from the questionnaire were copied individually and were translated to Bengali on the website https://translate.google.com. Then, the translated individual sentence (i.e., question) was copied from the website and pasted on a Microsoft word document word to word. After that, these sentences were copied verbatim from the word document and back-translated to English on the website. Thus, a Bengali and English questionnaires were formed. Both were kept for linguistic error analysis.

Linguistic error analysis of forward-translated Bengali questionnaire

Five Bengali language experts (having master's degree in Bengali and mother tongue was Bengali) were randomly selected from nonmedical faculty for scoring errors in the sentences. These experts had previous experience in subjective sentences error rate (SSER). SSER is a method to evaluate the quality of translation by quantitative method where 0 denotes “nonsense” and 10 denotes “perfect.”[18],[19] Hence, a lower score indicates higher errors in the sentence. Forward-translated Bengali questionnaires obtained from machine translation method and manual translation method were given to experts for SSER. They were instructed to rate the machine-translated version first and then manually translated version. The ratings obtained from the experts were entered in spreadsheet for statistical analysis.

Linguistic error analysis of back-translated English questionnaire

A set of English language experts (having a master's degree in English and mother tongue was Bengali) (n = 5) who had experience in SSER was consulted for the English version. They were also selected randomly form nonmedical faculty. The machine-translated English questionnaire and manually translated English questionnaire both were given to them for SSER. The ratings obtained from the experts were entered in a spreadsheet for statistical analysis.

Equivalence check by experts

We approached another five experts who had previous experience in the adaptation of survey questionnaire in Bengali. They were selected from medical faculty. Two were from the department of preventive and social medicine, two were from the department of psychiatry, and one was from the department of physiology. They rated the conceptual equivalence between the original questionnaire and back-translated questionnaire in 10-point scale where 10 denotes “perfectly equivalent” and 0 denotes “unequal.”First, they were given the machine-translated questionnaire then the manually translated questionnaire. Score of both questionnaires was stored and coded in a spreadsheet for statistical analysis.

Statistical analysis

Data were expressed in mean and standard deviation. SSER and equivalence scores obtained for machine-translated and manually translated questionnaires were compared by paired t-test. Two-tailed P < 0.05 was considered statistically significant. The analyses were carried out in Microsoft Excel and GraphPad Prism 6.01 for Windows (GraphPad Software, Inc., CA, USA).


  Results Top


The result of this study is shown in three tables. Linguistic errors in forward-translated questionnaire in Bengali, as reported in terms of SSER score, are shown in [Table 1]. Except for two sentences (7th and 9th), all the sentences showed statistically significant higher error in machine-translated questionnaire.
Table 1: Score of subjective sentence error rate for the Bengali survey questionnaire obtained by forward-translation carried out by machine and expert translator

Click here to view


SSER score obtained for back-translated questionnaire in English is shown in [Table 2]. Except the 4th and 8th question, all other sentences showed statistically significant higher level of error in machine-translated questionnaire.
Table 2: Score of subjective sentence error rate for the English survey questionnaire obtained by back-translation carried out by machine and expert translator

Click here to view


Equivalence check scores between original questionnaire and back-translated questionnaire are shown in [Table 3]. Manually translated questionnaire showed higher level of equivalence than machine-translated questionnaire.
Table 3: Score of equivalence check for the English questionnaire obtained by back-translation carried out by machine and expert translator

Click here to view



  Discussion Top


The aim of this study was to check the feasibility of a machine translation method in steps of adaptation of a questionnaire in Bengali. For the study, a questionnaire was chosen which was originally developed in English. This questionnaire was forward-translated (from English to Bengali) manually by human experts and by machine on “Google Translate” [Figure 2]. Then, the level of error was ascertained by SSER for both manual- and machine-translated questionnaire.

The SSER is a method of subjective evaluation of errors in sentences with quantitative output (score numbers). There are several other methods to detect errors in sentences. One of the methods is Word Error Rate. In this method, the percentages of words which need to be added, deleted, or modified are checked with the reference. Sentence Error Rate is another method where percentages of sentences which are different from reference are checked.[18] Both of these methods could not be applied in this study due to two reasons. First, there was no culturally validated K10 scale for the area from where the assessors were selected. Hence, reference was unavailable. For this reason, the comparison might not be valid for analysis of Bengali and English questionnaire. Second, we were unable to find experts who had previous experience of these methods. In addition, there is another subjective method, which is called information item error rate (IER). This method is commonly used for long sentences.[18] In this method, the long sentences are segmented into information items and then analyzed. As the questionnaire chosen for this study was comprised of simple and short questions, IER was not considered for the study.

Level of error in machine-translated questionnaire was higher (overall score: 66.60 ± 4.04) than the manually translated questionnaire (overall score: 117.20 ± 1.30). The difference was huge and statistically significant (P < 0.001). Sentences used for the instruction section of the questionnaire showed highest level of errors [Table 1]. A proper instruction on the beginning of a questionnaire is of paramount importance in self-administered questionnaire. It helps the respondents to get a fair idea about the aim of the questionnaire. It also helps to understand the response options and how to mark those responses. If the instruction is not conveyed effectively, it may confuse the respondents. Hence, chances of unanswered questions increase.

Among the 10 questions of the questionnaire, the 2nd question showed the lowest score (0.80 ± 0.84). Totally 8 questions among 10 questions showed statistically significant higher error in machine-translated version of the questionnaire [Table 1]. In previous studies, it has been found that “Google Translate” may help in medical communication (e.g., doctor taking history from the patient) between health-care providers and patients or patient parties.[7],[8] In verbal communication, a minor grammatical and syntax error can be overcome by human perception during understanding. However, a survey questionnaire should have correct sentences to the highest level for its successful application. In this study, we found high level of error in machine-translated questionnaire. Hence, it may be the major constraining factor for questionnaire adaptation by machine translation method.

The Bengali questionnaire obtained by forward-translation was then back-translated to English by machine translation method and by expert human translator. This English questionnaire was rated by SSER by experts. A surprising result was found for the machine-translated questionnaire. At first, the English questionnaire was translated to Bengali by “Google Translate,” and then, it was back-translated to English verbatim. Hence, it was assumed that the English questionnaire would be the same as the original questionnaire. However, that English questionnaire showed high level of error in SSER (overall score: 61.20 ± 5.02), whereas manually translated questionnaire showed minor level of error score (overall score: 116.20 ± 2.77) (P < 0.001) [Table 2].

In the process of adaptation of the survey questionnaire, after forward- and back-translation, the next step is equivalence check.[4] This is an important part for any biomedical survey questionnaire as the conceptual equivalence is more important than the lingual equivalence. In this step, conceptual equivalence is checked between the back-translated questionnaire and the original questionnaire. If these two questionnaires show a satisfactory level of equivalence, it is considered that the forward-translated questionnaire can be adapted for further process. Back-translated questionnaire by machine translation method was first checked for equivalence with the original questionnaire, and the score was not convincing (overall score: 48.40 ± 6.02). In contrast, manually back-translated questionnaire showed statistically significant (P < 0.001) higher score in equivalence (overall score: 112.40 ± 3.29) [Table 3].

It is evident from the result that at this point of time, the “Google Translate” has limitations for its application in forward- and back-translations from English to Bengali. However, translation suggestions are being continuously provided to “Google Translate” by the contributors worldwide [Figure 1].[6] Hence, chances of improvements are there.

Limitations

This study has several limitations. A single questionnaire with limited number of questions was used for translation according to our limited experts and their contributing time for this study. Higher number of questions and increased number of questionnaires could provide further insight into the capability of “Google Translate” as a translating platform. SSER is a subjective analysis method for scoring errors in sentences. Hence, the level of perfection in rating may not be uniform across different evaluator.


  Conclusion Top


At this point of time, for translation of survey questionnaire from English to Bengali by “Google Translate” has questionable feasibility with high level of lingual errors. Machine-translated questionnaire showed higher sentence errors rate and weak conceptual equivalence when compared to original questionnaire. Hence, Google Translate has limitations in translation of questionnaires when it is used alone. However, considering the easy accessibility and ease of use, it can serve as an adjunct during the adaptation of questionnaire. Further studies are needed to explore the potential of “Google Translate” in effective translation of questionnaire with higher number of questionnaire.

Acknowledgment

We are grateful to the expert translators, assessors of the linguistic errors by SSER, and the experts who checked the conceptual equivalence.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.



 
  References Top

1.
Coggon D, Rose G, Barker DJ. Planning and conducting a survey. In: Epidemiology for the Uninitiated. 5th ed. USA: Wiley-Blackwell; 2003. Available from: http://www.bmj.com/about-bmj/resources-readers/publications/epidemiology-uninitiated/5-planning-and-conducting-survey. [Last accessed on 2018 Apr 10].  Back to cited text no. 1
    
2.
Boynton PM, Greenhalgh T. Selecting, designing, and developing your questionnaire. BMJ 2004;328:1312-5.  Back to cited text no. 2
    
3.
World Health Organization. Process of Translation and Adaptation of Questionnaires. Geneva: World Health Organization. Available from: http://www.who.int/substance_abuse/research_tools/translation/en. [Last accessed on 2018 Apr 19].  Back to cited text no. 3
    
4.
Mondal H, Mondal S. Pretesting and cognitive interviewing are integral parts in translation of survey questionnaire. Indian J Psychiatry 2017;59:251-2.  Back to cited text no. 4
[PUBMED]  [Full text]  
5.
Squires A, Aiken LH, van den Heede K, Sermeus W, Bruyneel L, Lindqvist R, et al. Asystematic survey instrument translation process for multi-country, comparative health workforce studies. Int J Nurs Stud 2013;50:264-73.  Back to cited text no. 5
    
6.
Google Translate. Available from: http://www.translate.google.com/intl/en/about/languages. [Last accessed on 2018 Apr 25].  Back to cited text no. 6
    
7.
Kaliyadan F, Pillai SG. The use of Google language tools as an interpretation aid in cross-cultural doctor-patient interaction: A pilot study. Inform Prim Care 2010;18:141-3.  Back to cited text no. 7
    
8.
Sciarra AM, Batigália F, Oliveira MA. Technological devices improving system of translating languages: What about their usefulness on the applicability in medicine and health sciences? Braz J Cardiovasc Surg 2015;30:664-7.  Back to cited text no. 8
    
9.
Beh TH, Canty DJ. English and mandarin translation using google translate software for pre-anaesthetic consultation. Anaesth Intensive Care 2015;43:792-3.  Back to cited text no. 9
    
10.
Turner AM, Dew KN, Desai L, Martin N, Kirchhoff K. Machine translation of public health materials from English to Chinese: A Feasibility study. JMIR Public Health Surveill 2015;1:e17.  Back to cited text no. 10
    
11.
Simons GF, Fennig CD. Summary by language size. In: Ethnologue: Languages of the World. 12th ed. Dallas, Texas: SIL International; 2017. Available from: http://www.ethnologue.com/statistics/size. [Last accessed on 2018 May 02].  Back to cited text no. 11
    
12.
Arafat SM. Psychometric validation of the Bangla version of the patient-doctor relationship questionnaire. Psychiatry J 2016;2016:9385364.  Back to cited text no. 12
    
13.
Muquith MA, Islam MN, Haq SA, Ten Klooster PM, Rasker JJ, Yunus MB, et al. Cross-cultural adaptation and validation of a Bengali version of the modified fibromyalgia impact questionnaire. BMC Musculoskelet Disord 2012;13:157.  Back to cited text no. 13
    
14.
Mondal H, Mondal S, Baidya C. Comparison of perceived sleep quality among urban and rural adult population by Bengali Pittsburgh sleep quality index. Adv Hum Biol 2018;8:36-40.  Back to cited text no. 14
  [Full text]  
15.
Islam N, Basak TB, Voshaar MA, Ferdous N, Rasker JJ, Haq SA, et al. Cross-cultural adaptation and validation of a Bengali health assessment questionnaire for use in rheumatoid arthritis patients. Int J Rheum Dis 2013;16:413-7.  Back to cited text no. 15
    
16.
Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek DK, Normand SL, et al. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002;32:959-76.  Back to cited text no. 16
    
17.
Andrews G, Slade T. Interpreting scores on the kessler psychological distress scale (K10). Aust N Z J Public Health 2001;25:494-7.  Back to cited text no. 17
    
18.
Tomás J, Mas JA, Casacuberta F. Quantitative method for machine translation evaluation. Budapest, Hungary: Association for Computational Linguistics; 2003. p. 27-34. Available from: http://dl.acm.org/ft_gateway.cfm?id=1641401&type=pdf [Last accessed on 2018 May 11].   Back to cited text no. 18
    
19.
Nießen S, Och FJ, Leusch G, Ney H. An evaluation tool for machine translation: Fast evaluation for MT research. Second International Conference on Language Resources and Evaluation - LREC 2000. Athens, Greece; 2000. Available from: http://www.citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.298.1706&rep=rep1&type=pdf. [Last accessed on 2018 May 19].  Back to cited text no. 19
    


    Figures

  [Figure 1], [Figure 2]
 
 
    Tables

  [Table 1], [Table 2], [Table 3]



 

Top
 
 
  Search
 
Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

 
  In this article
Abstract
Introduction
Materials and Me...
Results
Discussion
Conclusion
References
Article Figures
Article Tables

 Article Access Statistics
    Viewed132    
    Printed9    
    Emailed0    
    PDF Downloaded25    
    Comments [Add]    

Recommend this journal


[TAG2]
[TAG3]
[TAG4]