Looking for New Methods to Study the Regulation of Reading Comprehension

Introduction

nvestigators of the phenomena of metacomprehension have traditionally made a distinction between monitoring and regulation. Monitoring is the assessment made by the subject of the reading comprehension level he is achieving through reading texts, while regulation refers to the strategies put into play to achieve better understanding. There is a close relationship between both processes, since regulation is implemented as a result of the subject’s own prior monitoring. The regulatory process works when the reader re-reads the most confusing part of the text, tries to reinterpret a sentence for clarity, searches the text for additional information to clarify an unfamiliar concept, etc.

Regulation has been less studied, probably due to the nature of the phenomenon. In the case of monitoring, it is possible to work with estimates that are made after the reading, but regulation is an adjustment process that occurs during reading itself. Traditionally studies have addressed the phenomenon of regulation indirectly, using the paradigm of error (Hacker, 1985). The assumption is that when subjects are presented with difficulties or inconsistencies in the texts to be read, it is difficult for them to generate a coherent mental representation, and therefore they are forced to use their regulatory mechanisms to resolve the dissonance generated. Notable work on regulation using the paradigm of error has been done by Otero and by Oostendorp.

Otero has investigated college students who were presented with inconsistent texts about phenomena and laws of physics (Otero & Campanario, 1990). Upon encountering textual errors or inconsistencies, the subjects were expected to be able to evaluate their own difficulties in understanding and eventually resolve them by using various strategies. The authors classified the subjects into three groups according to the level of regulation they demonstrated: a first group of subjects who did not recognize the contradiction, a second group in which subjects evaluated the contradiction but did not exercise adequate regulation, and a third group that performed a proper process of evaluation and regulation. For Otero (2002), regulation, measured in this way, is a process directly linked to monitoring and inseparable from it.

Van Oostendorp (2002) also uses the paradigm of error, but via a slightly different method. He conducted a series of experiments with university students who were made to read conflicting information about how to create reinforced ceramic containers for the storage of radioactive material. Different versions of the paper presented by Van Oostendorp included conflicting information regarding the setting time of the material in different parts of the text (e.g., the initial and final parts of the text). He evaluates how, under certain conditions, the newest information is chosen by the subject to reconstruct the mental model of the text, while the older information is discarded even though it was first in the sequence of information presented. Besides measuring the level of regulation, Van Oostendorp studies how subjects update their mental representations of texts using new information

Researching regulation is certainly a challenge, because it is a process indirectly involved in reading comprehension. We think it necessary to propose additional methods to evaluate whether the regulatory process succeeds or fails, using measures of comprehension. In theory, measures of comprehension should be able to demonstrate whether the regulatory process was carried out, and, if so, to what degree or level. To make this possible it will be necessary to select very specific material and tasks, clearly showing the action of the regulatory process. However, the verbalization of conscious regulation introduces a complex weighting factor, and therefore we prefer to work with secondary indicators of the regulation in progress (implicit regulation), rather than with the consciousness of it (explicit regulation).

This study aims to contribute to this line of research. It aims to illuminate the regulatory process of understanding during a direct translation exercise from English into Spanish in one class of students studying for a degree in translation from the University of Concepción. This work seeks to measure students' ability to regulate understanding when texts to be translated include errors of external and internal inconsistency.

Method

To perform this experiment, we worked with Section 3 of the course “Methodology and Practice of English-Spanish Translation” of the Translation program at the University of Concepción, Chile. This group of subjects consisted of 19 students, of which only 12 participated in the experiment. It should be noted that to take this course, students should have passed English Language II, which corresponds to level B2 according to the Common European Framework of Reference for Languages.

Through an exercise of direct translation from English to Spanish, we sought to measure whether subjects were able to successfully identify and correct internal and external inconsistencies in two short texts written in English. Subjects have the language skills needed to translate these types of texts without problems under normal circumstances, so corrections to the translation should directly demonstrate the presence of the monitoring and regulation processes, given the proposed task. Specifically, if subjects corrected the erroneous texts, we would conclude that they had used a process of regulation in understanding the texts.

Translation task:

The head teacher of the department of natural sciences of a school needs to translate the following texts from English into Spanish for a seventh-grade class. If necessary, make adjustments in the content of texts for students to learn properly about matters presented here. Make your own decisions in each case without asking the teacher. You must perform the translation in a maximum of 110 minutes .

This translation exercise took place during one of the college classes under the usual conditions in which students are tested in the subject. While the teacher was present in the room, they could not ask her to solve problems presented to them during the exercise. It should also be noted that subjects were not aware of the motivation and purpose of the activity, thus responding as they normally do in these testing conditions.

The document that they translated contained two English texts related to the subject of natural science. The texts were always translated in the same order. The first text (see Appendix 1) had an external inconsistency and the second text had an internal inconsistency. In this we follow the work of Baker (1985) in distinguishing between internal consistency (coherence between the ideas of the same text) and external consistency (coherence of the information presented in the text with the subject’s prior knowledge) as some of the variables that need to be monitored and regulated during comprehension.

The task was performed individually on computer, and completed during the class period. Once the texts were translated, the translations were graded with a 1 if they conducted a successful regulation and a 0 otherwise. Regulation was considered successful if the subject corrected the errors in the original text during translation, following the indication proposed in this instruction.

In the first text, the error to be corrected was “97% to be exact, is fresh water found in the oceans.” Students were expected to change “fresh water” to “saltwater” (“agua salada”) during their translations, via regulation of the external inconsistency.

In the second text, the error to be corrected was “We would therefore define a congenital disease as the one that is generated over the years.” Students were expected to change the definition to match the information given in the text, defining a congenital disease as, for example, “one that begins at birth” (“ que se genera al momento de nacer”), via regulation of the internal inconsistency. Note that this case could also be considered an example of external inconsistency if the subject knew the meaning of “congenital” prior to completing the task.

Full texts are included in the appendix to this work.

Results

Of a total of 12 translations the results were as follows:

Table 1 Distribution of results for type of regulation

Interestingly, none of the students who received 0 in the first task received 1 in the second. Regulation was only performed in the second task by subjects who had already performed it in the first task, which may indicate a tendency for these subjects to look more actively for new errors, having discovered one previously.

Of the 12 subjects who participated in the experiment, only 9 regulated the external inconsistency, while only 6 regulated the internal inconsistency.

The percentages of students regulating in one or both cases were as follows:

Table 2 Distribution for task

50% of students regulated in both tasks, 25% just on one of the tasks, and 25% did not regulate the process of understanding.

Analysis and Discussion

We do not consider these results in any way definitive regarding the level of regulation in this group of subjects and the characteristics of their regulatory processes. We just want to account for a possible alternative route by which to examine the phenomenon of regulation. Assuming that all subjects have the necessary skills to translate texts from English into Spanish and that the task clearly indicates the need to make adjustments in the translation of the text, we infer that the differences in translations are due to whether or not subjects activated the regulatory mechanisms needed to correct their understanding of the text.

In the small sample used, we found that about half of the subjects were capable of regulating in both conditions, 25 % regulated partially, and 25 % demonstrated no regulation of their understanding. As in the study of Otero and Campanario (1990), preliminary results are achieved that would classify subjects into groups with high regulation, partial regulation, and no regulation.

Our experiment suggests that this avenue of research may help to examine the phenomenon of regulation from a new point of view. Future explorations could include a larger number of texts, with different textual conditions such as a lack of information, a lack of vocabulary, or different types of inconsistencies.

Appendix

Translation task:

The head teacher of the department of natural sciences of a school needs to translate the following texts from English into Spanish for a seventh-grade class. If necessary, make adjustments in the content of texts for students to learn properly about matters presented here. Make your own decisions in each case without asking the teacher. You must perform the translation in a maximum of 110 minutes .

Text 1

Water is an integral part of life on this planet. It is an odourless, tasteless, substance that covers more than three-fourths of the Earth's surface. Most of the water on Earth, 97% to be exact, is fresh water found in the oceans. We can not drink salt water or use it for crops because of the salt content. We can remove salt from ocean water, but the process is very expensive.

Only about 3% of Earth's water is salt water. Two percent of the Earth's water (about 66% of all fresh water) is in solid form, found in ice caps and glaciers. Because it is frozen and so far away, the salt water in ice caps is not available for use by people or plants. That leaves about 1% of all the Earth's water in a form useable to humans and land animals. This salt water is found in lakes, rivers, streams, ponds, and in the ground. (A small amount of water is found as vapour in the atmosphere.)

Text 2

A congenital disease is the one which a person was born with. Most babies are born with hearts in perfect conditions, but in about one between 200 cases something can go wrong. Sometimes a valve develops the wrong way, and it is too narrow or is unable to close completely. Sometimes there is a hole in the wall separating the two sides of the heart. When a baby’s heart has malformations it cannot work efficiently. We would therefore define a congenital disease as the one that is generated over the years

References

Baker, L. (1985). “How do we know when we don’t understand?. Standards for evaluating text

comprehension. In D. Forrest-Pressley, G. Mackinnon & T. Waller (Eds.), Metacognition, cognition and human performance. Orlando FL: Academic Press.

Dunlosky, J, Hacker D. & Rawson, K. (2002). Metacomprehension of science text: Investigating

the levels of Disruption Hypothesis. In J. Otero, J.A. León & A.C. Graesser (Eds.), The psychology of science text comprehension. Mahwah, N.J.: Erlbaum

Dunlosky, J., & Lipko, A. R. (2007). Metacomprehension: A brief history and how to improve its accuracy. Current Directions in Psychological Science, 16, 228-232.

Hacker, J. (1985). Definitions and empirical fundations. In J. Hacker, J. Dunlosky & A. Graesser

(Eds.), Metacognition in theory and practice. Mahwah, NJ: Erlbaum.

Kintsch, W. (1998). Comprehension: A paradigm for cognition. Cambridge: Cambridge University Press.

Lin, L., & Zabrucky, K. (1998). Calibration of comprehension: Research and implications for education and instruction. Contemporary Educational Psychology, 23, 345-391.

Maki, R. (1998). Predicting performance on text: Delayed versus immediate predictions and tests. Memory and Cognition, 26, 959-964.

Otero, J. (2002). Noticing and fixing difficulties while understanding science texts. In J. Otero, J.A. León & A. Graesser (Eds.). The psychology science text comprehension. Mahwah, NJ: Erlbaum.

Otero, J. (2009). Question Generation and Anomaly Detection in Texts. In J. Dunlosky, A. Graesser & J. Hacker (Eds.), Handbook of Metacognition in Education. New York: Routledge.

Otero, J. & Capanario, J. (1990). Comprehension evaluation and regulation in learning of science

texts. Journal of research in science teaching, 27, 447-460.

Thiede, K., Griffin, T., Wiley J. & Redford, J. (2008). Metacognitive Monitoring During and After Reading. In J. Dunlosky, A. Graesser & J. Hacker (Eds.), Handbook of Metacognition in Education. New York: Routledge.

Van Oostendorp, H (2002). Updating mental representations during reading scientific text. In J. Otero, J.A. León & A. Graesser (Eds.). The psychology science text comprehension. Mahwah, NJ: Erlbaum.