Teaching Translation of Text Types

with MT Error Analysis and Post-MT Editing

ext types in translation are rarely explored as a special pedagogical subject in translation studies. This MT-based project probes the effectiveness of teaching text types in translation. More specifically, we investigate the effectiveness of teaching students the concept of text types and their relevance to translation quality and translation performance with comparative analysis of machine translation (MT) errors across text types in the process of post-MT editing.

I. Introduction

Text types can be taught in different ways. Hőnig (1986), Kussmaul (1995), and Nord (1997) have proposed several models of source text analysis for the research of text types within translation studies. Colina modified Nord's example to provide a theoretical model of parallel text analysis and raised two significant steps for the teaching procedure. One step is to "identify which features are indicators of text type and whether the same features are used in the target culture to make the same text types" (Colina, 2003, p. 16). The other step is to "decide, in combination with consideration of the translation brief and the norms for the [target text] and culture, which units are relevant to a translation purpose, which aspects need to be changed, whether the function/purpose of the translation can be the same, and what strategies will be used to accomplish the translation goal" (Colina, 2003, p. 16). The purpose of this teaching is to help students to obtain generalization of language features and pragmatic functions with respect to a certain text type and to consider how to retain the same function in the target language text.

Another model of teaching text-types in translation, retrieved on the Internet, was provided by the University of the West of England (see the website of UWE in UK.). The teaching and learning procedures start with the reading of various text types, followed by the instructor's highlighting the problems areas and linguistic hallmarks of those text types. After that, students are asked to collect different text types with their translations. In addition, an online discussion forum is provided for students to analyze and discuss points of interest or translation difficulties. Finally, students are asked to submit a brief commentary on the translations of text types.

In this translation project, a different model of teaching text types will be conducted in the elective MT class at Kaohsiung First University of Science and Technology in Taiwan. Neither games nor quizzes are used, nor will there be a formal lecture on the knowledge of text types at the beginning of this project. The translation instructor will simply ask students to analyze lexical-specific and syntactic-level problems in the MT outputs of informative, evocative and expressive text types. After that, the instructor will encourage students to think, judge and infer the distinctive linguistic features of the three text types while editing the MT errors and noting down their reflections. This teaching helps students to realize that the difficulties of post-MT editing vary according to the linguistic features of the three text types, and different diction categories and syntactic methods in the three text types have close bearing on the MT errors and the translator's decisions in the translation process.

This teaching approach emphasizes students' active mental involvement in the process of analyzing and editing the MT errors, not the instructor's informational input and the students' passive acquisition. In addition, we teach the concept of text types rather than practice of some translation skills. Out of the need of investigating how students develop the concept of text types using MT error analysis and post-MT editing as the means, we seek answers to the following questions.

1. Can MT error analysis and post-MT editing help students to differentiate the distinctive linguistic features of informative, evocative and expressive text types at the lexical, pragmatic, and syntactic levels?
2. Can students identify the different functions of text types in the process of editing the different types of MT errors?
3. Can students realize the impact of text types on the translation process and the translation quality with MT error analysis and post-MT editing?
4. Can MT error analysis and post-MT editing help students acquire an impressive knowledge of text types and learn its relevance to translation performance without undue stress?

Finding answers to these questions will achieve a twofold goal. One is to justify the effectiveness of teaching the concept of text types through the cognitive method of MT error analysis, not as a means of improving the performance of translating text types. The other objective is to allow students to shift their attention, during the process of translation, from local problems such as lexical and syntactic equivalence between SL and TL, to global problems at the textual level. We are more concerned with how the translation quality is determined by certain textual issues such as the specific functions of text types, rather than pure code-switching and linguistic issues.

Since text types are the main subject of this research, we will provide their definitions in the next section. After that, we will introduce MT errors and illustrate how post-MT editing can help students to identify different linguistic features of the informative, evocative and expressive text types in Section Three. In Section Four, we will discuss the methods, subjects, and teaching procedures of this project. Section Five will report the findings of the statistical figures of MT errors through students' comparative analysis and the result of the questionnaire. In Section Six, we will discuss the pedagogical implications based on the result of the questionnaire and students' reflections. Finally, we will provide a summary of this study with some suggestions.

The term "text type" refers to a specific "mode of discourse" or "mode of presentation" that aims to fulfill a certain rhetorical and communicative purpose (Trosborg, 1997). Neubert recognizes text types as "socially effective, efficient, and appropriate molds into which the linguistic material available in the system of a language is recast" (Neubert, 1985, p. 125). Hatim and Mason look at text types as "a conceptual framework which enables us to classify texts in terms of communicative intentions serving an overall rhetorical purpose" (1990, p. 140). These statements clearly define text types as the functional benchmark against which we may classify or categorize various texts into a certain type for achieving particular functions.

Classification of text types is controversial because a text type tends to be multifunctional and overlaps with certain textual elements of other text types. Nevertheless, for the convenience of translation studies, a number of ways of distinguishing text types have been suggested. Crystal and Davy (1969) classified texts according to field of discourse or subject matter, "giving rise to types such as journalistic texts, religious text, scientific texts and so on" (qtd. in Hatim, 1998, p. 263). Beaugrande and Dressler (1981) raised a "different classification of texts into types such as literary, poetic and didactic" based on domain (ibid.). Hatim and Mason (1990) classified text types according to their rhetorical purposes: argumentative, expositive and instruction-based. Argumentation is subdivided into counter-argumentation "in which a thesis is cited, then opposed" and through-argumentation, "in which a thesis is cited, then extensively defended" (Hatim, 2001, p. 179). Exposition is subdivided into descriptive, narrative and conceptual types. In addition, Bühler's theory of functional typology proposed a three-way distinction depending on the foci of "the producer (emotive), the subject-mater (referential) or the receiver (conative)" (qtd. in Bell, 1991, p. 204). This functional typology labels three text types as expressive, informative and vocative (ibid.). Similar to this functional typology is Reiss's three-way division of texts into "informative texts which convey information, expressive text which communicate thoughts in a creative way, and operative text which persuade" (qtd. in Hatim, 2001, p. 77).

In this project, we adopt Reiss's and Bühler's text typology that distinguishes one text type from the other two in terms of intention, function, and rhetorical purpose. We hope that students learn that each text type has predominant linguistic features that perform a particular function in the translated text. The linguistic characteristics for our check and classification can be borrowed from Hatim's and Munday's description (2004). In the informative text type, the dominant form of language is functional and the text is "structured on the semantic-syntactic level" (Hatim and Munday, 2004, p. 183). In contrast, the expressive text type is "doubly structured; first on the syntactic-semantic level and second on the level of artistic organization (qtd. in Hatim and Munday, 2004, p. 183). The language used in this text type is artistic and creative. Finally, the operative text type is "doubly or even triply structured on the syntactic-semantic level (in some circumstances, but not necessarily) on the level of artistic organization and on the level of persuasion" (Hatim and Munday, 2004, p. 184). Its language tends to be psychological or persuasive. These descriptions serve as linguistic norms to bring about some constraints that govern the translator's lexical choice and syntactic processing. These constraints also affect the success or failure of MT performance and the ease or difficulty of post-MT editing.

MT errors refer to the inappropriate translations at the lexical, pragmatic and syntactic levels in the MT output. Since the current English-to-Chinese MT system cannot produce a satisfactory MT output, human editing is required to improve the quality, and this is known as post-MT editing. Post-MT editing, as defined by Juan C. Sager, is "the adaptation and revision of output of a machine translation system either to eliminate errors which impede comprehension or to make the output read like a natural-language text" (1994, p. 327). Melby defined this term as "the process of revising a translation after the draft translation has been completed" (1987, p.146). In short, post-MT editing is mainly undertaken to improve the quality of the MT output for publication purposes.

MT errors can be classified into lexical, syntactic and pragmatic errors due to the lexicon-specific, syntax-specific and pragmatics-specific limitations of the MT system in operation. For the convenience of calculating MT errors in this project, the instructor asked students to group the lexical and pragmatic errors together into a single category. Table 1 shows some examples of three types of MT errors and certain hidden reasons. In the column of examples, the source language sentence, its MT output and post-MT editing are provided.

MT errors can be classified into lexical, syntactic and pragmatic errors due to the lexicon-specific, syntax-specific and pragmatics-specific limitations of the MT system in operation. For the convenience of calculating MT errors in this project, the instructor asked students to group the lexical and pragmatic errors together under one category. Table 1 shows some examples of three types of MT errors and certain hidden reasons. In the column of examples, the source language sentence, its MT output and post-MT editing are provided.

Examples of Three Types of MT Errors with Hidden Reasons.

Table 1 shows that the number of MT errors varies with text types. It has been observed that the MT system produces the highest-quality automatic translation from texts that feature simple sentence structures and single-meaning words or domain-specific terms. This rule leads us to hypothesize that the text type that produces the fewest MT errors is the informative text type because this text type tends to use domain-specific terms with referential meanings to transmit facts or information. In contrast, evocative/operative and expressive text types produce more MT errors, compared to the informative text-type because they tend to use the figurative speech and metaphorical expressions with multiple meanings and emotive connotations. They usually employ more complicated structures.

IV. Methodology

Since we conduct an MT-based project with students to test the effectiveness of the cognitive learning of text types in translation, this section needs to introduce the method we have adopted, the students participating in this project and the teaching procedures.

This project integrates quantitative measurement and qualitative analysis. Students' statistical results of MT errors and post-MT editing were used as a quantitative means to see whether students could identify MT errors and use appropriate strategies to correct the errors. At the end of this project, a questionnaire was administered and the responses were statistically assessed to determine the students' understanding of the concept of text types with a comparative analysis of MT errors across text types. In addition, the students' reflections as recorded in the weekly assignment were exclusively used for a qualitative analysis.

To obtain the statistical figures, students were asked to edit MT errors and then count the frequency of the occurrence of particular MT errors at the lexical, syntactical and, if necessary, pragmatic levels. Students eventually calculated the average number of MT errors for each text type and then used the statistics to rank the syntactic complexity and the degree of lexical ambiguity or pragmatic clarity. This ranking allowed students to infer and observe the distinctive syntactic, lexical or pragmatic features of the three text types because these features were crucial in affecting and governing MT performance across text types.

The questionnaire consisted of twenty multiple-choice questions in the four areas of "Effectiveness of MT error analysis," "Relevance of text types to translation," "Learning distinctive linguistic features of text types," and "Affective and cognitive contribution." The multiple-choice questions were easy for students to answer and students were asked to answer honestly because the result of the questionnaire would be used only for the instructor's research. In addition, students were asked to write down their reflections at the end of their post-MT editing. Students had to examine the hallmarks of the three given text types and then identify their similarities and differences. Doing so could help students to understand the concept of text types.

Twenty students participated. All were students from the Department of English at NKFUST. Their median age was 22. Three were males and seventeen were females. They had not been trained in MT technology before taking this course; only two of them had learned about text types in other English classes, but not in the translation class. These students' English level was intermediate and they had studied translation for nearly one year. All had Chinese as their mother language.

This MT-based project was completed in a weekly three-hour machine translation class that ran for nine hours over three consecutive weeks. The project tasks moved from data analysis to concept building. During the first week, students were asked to use the MT system (TransWhiz, developed and released by Otek Technological Company in Taipei, Taiwan) to translate an excerpt from instructions for buying an Olympus lithium-polymer battery (the informative text type) and an excerpt from one advertisement for the Gap Incorporation (the evocative/operative text type). Students had to edit and count the MT errors, and then they compared MT performance across two text types in the areas of lexical choice and syntactic structures.

During the remaining two weeks, students were asked to use the same MT system to translate an excerpt from a recipe for instant noodles and a user's manual on the installation of a refrigerator (representative of the informative text type), an excerpt from a short speech and one advertisement of the tour package for newlyweds (representative of the evocative/operative text type) and an excerpt from Shakespeare's play, Hamlet, and from Lord of the Rings (representative of the expressive text type). After that, students had to edit and count the MT errors, and then to identify the distinctive linguistic features of the three text types through analysis of the limitations of the MT system. In addition, students were asked to write an assessment of the MT performance and then to discriminate the dominant linguistic features of the three text types that constitute certain constraints which govern and control MT processing and MT performance. The process of writing reflections helps students to be aware of the similarities and differences among the three text types.

V. Findings

The findings in this MT-based project will be discussed in two areas: 1) statistical MT errors and the result of questionnaire responses for a quantitative analysis, and 2) a summary of students' reflections for a qualitative analysis.

In the students' first week assignments, the average number of lexical/pragmatic MT errors in the informative text type was 4 and the average number of syntactic MT errors was 1. In contrast, lexical/pragmatic MT errors in the evocative/operative text type averaged 5 and syntactic MT errors averaged 3. In students' second- and third-week assignments, the average number of the lexical/pragmatic MT errors in the informative text type was 7, and that in the evocative/operative text type was 10, and that in the expressive text type was 11. However, their average numbers of syntactic errors was 1 in the informative text type, 3.5 in the evocative/operative text type. and 1.5 in the expressive text type. The average numbers of MT errors in the students' three-week assignments along with some examples of post-MT editing are tabulated below.

Statistics of MT Errors Across Three Text Types

We calculated the total number of responses in each category and the total percentages. Tables 3-6 show the results of student responses to these "yes-no" questions.

Statistics on Student Responses to the Questions Concerning the Effectiveness of MT Error Analysis (Resp= Respondents)

Table 3 indicated that student responses to Q4 and Q5 were more positive than the responses to Q1, Q2 and Q3.

Student Responses to the Questions Concerning Relevance of Text Types to Translation (Resp= Respondents)

Table 4 showed that student responses to Q9 presented the highest percentage while student response to Q8 showed the lowest percentage.

Student Responses to the Questions Concerning Learning Distinctive Linguistic Features of Text Types (Resp= Respondents)

Table 5 showed that student responses to Q11, Q14 and Q15 were more positive than the responses to Q12 and Q13.

Student Responses to the Questions Concerning Affective and Cognitive Contributor (Resp= Respondents)

Table 6 reveals that student responses to Q20 have the highest percentage, but one student responded "disagree" to the same question. The entire statistics of this survey showed that the percentage of "agree" responses was 73.75%; of the "no attitude" responses was 26% and of the "disagree" responses was 0.25%. These figures prove that a majority of students agrees with the learning of text types in translation using the methods of MT error analysis and post-MT editing.

We summarize students' reflections as follows. Students generally found that the average number of lexical, pragmatic and syntactic errors in the MT output of the informative text type was lower than in the evocative/operative and expressive text types. They observed that the excerpts from the drama and fiction or passages extracted from the advertisement and speech contained many metaphorical expressions that required translators to use esthetic or artistic language to modify the literal translations produced by the MT system. In the meantime, they found that most of the lexical items in the expressive text type were used to describe facts and embody referential meanings, so that these items could be more satisfactorily processed by the MT system. In addition, the rule-based computing parser of the MT system was incapable of analyzing the complicated sentence structures in advertisements and literary works. In contrast, the user's manual or product instructions tended to use simple, imperative sentence structures, so that the MT system could process these sentences more accurately. Finally, students realized that comparative analysis of MT errors in the three text types helped them to realize that each text type had its special functions and dominant linguistic features, and these linguistic factors affected the MT performance and led to different numbers of MT errors.

This section discussed the insights yielded by the result of the questionnaire and by the reflections in students' assignments. These insights may be discussed in the areas of the effectiveness of MT error analysis, identification of dominant linguistic features of the three text types, awareness of relevance of text type to translation and affective, cognitive learning.

The result of the questionnaire shows that 85% of the students, have favorable responses to the second part of the questionnaire (Questions 1 to 5). This indicates that a majority of students agrees with the use of MT error analysis and post-MT editing to learn text types in translation. MT errors analysis facilitates students' active mental involvement because students need to make cross-references between SL and TL, and then assess what linguistic features in a text type prevent satisfactory processing by the MT system. Students' approval of the effectiveness of MT error analysis changes our assumption that MT errors mislead students and interfere with their development of translation competence. Actually, MT errors can benefit students if used in the right way.

6.2. Learning dominant linguistic features of the three text types

With regard to the learning of different lexical-pragmatic categories and syntactic structures across text types, more than half of the students agreed that error analysis in the process of post-MT editing helped them to learn the different and similar linguistic features between the informative, evocative/operative and expressive text types. In addition, the data collected from their reflections revealed that students were aware of different types of languages that different text types used to achieve different purposes. For example, students found that the informative text type tended to use verb actions, e.g., "put," "turn" and "leave" and the nouns with straightforward meanings, e.g., "kitchen sink," "soft case," "boards," and "leveling screws." As a contrast to this phenomenon, the expressive text type preferably used metaphorical expressions, e.g., "pouring out" (implying "huge volumes of mails sent out like snow"), "a constant stream of postmen" (referring to "many postmen"), and "a sea of trouble" (indicating "much trouble"). The evocative/operative text type which exists in between these two types was dotted with the lexical items that had straightforward and connotative meanings, e.g., "brides' land package" (meaning "the package trip designed for newlyweds").

Students also found that the three text types had distinctive sentence structures. For example, the user's manual (the informative text type) tended to use imperative sentences, passive voices, and fewer complicated sentences while Shakespeare's drama (the expressive text type) used complicated sentences with a personal creative style. However, the evocative/operative text type combined simple and complicated sentences. Students' reflections in general revealed that students could identify the differences among the three text types at the lexical and syntactic levels after they had received training in post-MT editing coupled with MT error analysis over the three consecutive weeks.

Compared to their responses to questions in the other three parts, only 62% of students agreed that they had acquired the knowledge of the relevance of text types to translation. This indicates that it is more difficult for students to infer or judge how text types are relevant to the translation process. Due to this difficulty, the instructor's explicit instructions are required. The instructor in this project did suggest that awareness of text types was beneficial to translation because only a translator who understands the function of a text-type can use the proper linguistic elements. This suggestion was not adequate for students to realize the close and crucial relationship between text types and translation. Thus, the instructor needs to deliberately discuss, based on students' post-MT editing, how different linguistic factors of text types determine the presentation of different functions of text types and how the different functions of text types determine the overall translation performance. This discussion will help students to understand text-in-translation interaction, relating textual functions or rhetorical purposes (e.g., for aesthetic appreciation or for information acquisition) to the decisions on lexical choice and organizational structures in the translation.

As noted in the questionnaire, an overwhelming majority of students (73.75%) agreed with the effectiveness of cognitively learning the concept of text types in this project. Actually, MT error analysis gave students the opportunity to explore the hidden reasons for the MT system's failure to process the different linguistic features of the three text types. The knowledge of the limitations of MT systems made them aware of the concept of text types in translation when they tried to find solutions in correcting different MT errors. Such an empirical experience in a learning-by-doing environment helped students to develop the concept of text types in a non-stressful way. In addition, noting reflections actively involved students in assessment and analysis, and this contributed to their cognitive learning of text types in translation.

However, we noticed that one student disagreed with Q20, showing that he did not find it helpful and useful to learn by actual participation. We have to investigate whether this response is attributable to his personal learning style or to other factors. This respondent's age was 25, making him the oldest in the class. This revealed that the older student had more difficulty adapting to a new way of teaching and learning. As a result, a mixture of the teacher's lecture and students' practice could be a better way to work out some problems arising from the implementation of this innovative MT-based teaching approach.

The entire MT-based project started with students' actual practice of MT error analysis and post-MT editing to arrive at a conceptual acquisition of text types in translation. Students learned not only the relevance of text types to the translator's decision but also had a clear concept of three text types in translation. More importantly, students stated that they had non-stressfully mastered the different contextual components of the three text types, including organizational patterns, syntactic modes and lexical categories using the method of post-MT editing and MT error analysis. This finding confirmed the effectiveness of teaching text type in translation with MT error analysis and post-MT editing.

We admitted that the sample size in this project is very small (20 undergraduate students), a total of eight texts for post-MT editing (three informative text types, three evocative/operative text types and two expressive text types) and a period of three weeks. Because of these limitations, the conclusion of this project cannot be fully representative. Further research is thus required, using more texts to test more students over a longer period of time to seek more genuine and convincing results. However, since 99% of students, as noted in the student profile of the questionnaire, admitted that they had never learned the concept of text types in translation in other translation or language classes, this study of the pedagogical subject of text types in translation is worth a further try at universities. This study is contributive to the teaching of theory through practice since it leads students from empirical practice to cognitive, conceptual acquisition of text types in translation.