This volumepresents a collection of papers that cover different issues concerning assessment in translation and interpreting. This main theme is studied from different perspectives throughout the book, thus providing a really updated overview of how testing and assessment could be addressed in the different domains of translation, interpreting and localisation, while also considering the different purposes it may have: from diagnostic, formative and summative evaluation in educational settings, to professional certification or quality assessment of translation as a product.
The introduction, by the editors of the book, offers a brief overview of the previous discussions around testing and assessment in both translation and interpreting, with references to widely known theories, such as Nida's formal and dynamic equivalence, Newmark's semantic and communicative translation or Toury's adequacy and acceptability, just to mention three of them. This serves as an excellent point of departure for the book, which is also briefly reviewed in the introduction.
The volume is divided into three sections. The first section delves into theoretical aspects of assessment, with a special focus on construct definition and on the use of rubrics. Both concepts are carefully explained in Claudia V. Angelelli's contribution: constructs, as "clearly spelled out definitions of exactly what a designer understands to be involved in a given skill or ability" (p.13), and rubrics, which contain "all the sub-components that constitute the construct" and provide "descriptive statements of behaviours that candidates may exhibit in a particular sub-component" (p.38). Angelelli also reviews the debate around the definition of translation competence, which should be the basis of the constructs and rubrics used to assess translation competence, as can be seen from the rubric the author proposes in the article. Angelelli's paper is thus a perfect introduction to the use of constructs and rubrics in translation assessment, helping readers to fully understand the importance of these concepts in the design of tests.
In the second chapter, Holly E. Jacobson examines the topic of community interpreters' performance in healthcare settings, while claiming for the need to analyse it beyond the lexico-semantic level. Therefore, the author exposes a rubric based on the theoretical frameworks of interactional sociolinguistics (IS) and conversation analysis (CA), in order to assess community interpreters' transfer of contextualisation cues and discourse management during the interpreted interaction respectively. Jacobson's contribution may surely be of great interest for those working in the training and certification of community interpreters, two key steps in the professionalisation of this wide-spreading practice. Perhaps some readers would find it useful to have more detailed information on some of the theories covered through the paper, since some of them are just briefly presented as part of the theoretical framework for the rubric; nevertheless, the article is well argued and reasoned, and certainly sheds light onto a new and interesting aspect worth considering in community interpreters' testing and assessment.
In the second section of the book, five empirical examples of the development of assessment instruments are presented. The methodological approach is explained in detail as a key element in these contributions, as well as the limitations of the studies and their implications for further research in the future. Therefore, this section, by bringing new ideas into play, may be really enlightening for those starting research projects.
The first paper describes the potential of the Calibration of Dichotomous Items (CDI) as a method for evaluating translation competence. After describing this method, June Eyckmans, Philippe Anckaert and Winibert Segers compare it to two traditional evaluation methods: the holistic (intuitivie-impressionistic) method and the analytical method (based on the use of assessment grids), while advocating the usefulness of the CDI. The paper is clearly structured and organised and the results are complemented by several graphics. The discussion relies heavily on statistical concepts which may be difficult to follow for those not familiar with this kind of approach; however, this is not an obstacle to the understanding of how results are achieved and, therefore, how the hypotheses are confirmed, i.e. the CDI method proved more reliable and less subjective than the other two methods included in the study conducted by the authors, a fact that reinforces its potential as assessment method for a more reliable and valid certification of translation competence.
The second paper in this section is Elisabet Tiselius' pilot study on the adaptation and application of Carroll's (1966) scales to grade interpreted renditions in simultaneous mode. Tiselius selected two groups of graders —non-interpreter graders without experience in grading vs. interpreter graders with experience— and asked them to grade interpreters' performance using her adaptation of Carroll's scales. The results support the validity of the scales, even though the author claims for further research to be developed before generalising the conclusions. However, one weak point in Tiselius' study is that the interpreted renditions were transcribed before being graded, which is not the normal practice when grading interpreting and, therefore, perhaps reduces the feasibility of this method.
In the next contribution, Mira Kim examines the possibilities of meaning-oriented criteria to assess students' translations. These criteria are based on the systemic functional linguistics (SFL) theory and intend to redress the habitual lack of systematic criteria in translation assessment. SFL underpinnings are thus carefully drawn in Kim's contribution, conforming the basis for the meaning-oriented criteria then developed in the article and supported by examples of English-Korean translations. Kim's own experience in the application of the method in a pedagogical context is exposed, clearly showing the benefits of these criteria in helping students become autonomous learners and their own quality controllers. Even though the theoretical part requires attentive reading, the part where the criteria are developed and applied is easy to follow; moreover, the positive results presented should really encourage translation trainers to try this new assessment approach.
Brian James Baer and Tatyana Bystrova-McIntyre's contribution offers a case study where textual cohesion is assessed based on the use of corpora. The authors first explain the design of the corpora and the collection of comparable data from Russian and English texts and, through their results, they confirm the differences between Russian and English in specific features such as the use of commas, colons, em-dashes and parentheses, as well as in the length of paragraphs and sentences. The long discussion they provide is one of the strengths of this article, as statistical findings are contrasted by rules from American English and Russian style guides, while the possibilities of emphatic punctuation or "creative approach" are also considered. As a whole, their arduous work can be regarded as a potential tool for feedback in both formative and summative evaluation.
In the next contribution, Keiran Dunne introduces software localisation and its assessment and starts a critical dialogue suggesting a more customer-oriented approach in the quality measurement of localised software. Some of the concepts explained in this paper may already be common knowledge to translators working in localisation projects, but being a relatively new field of work and research, the definitions included by the author are really useful for those without experience in this field. Therefore, not only is this paper appropriate to understand quality assessment in localisation, but also to understand how this assessment is usually conducted and the kind of factors and potential problems that must be taken into account. Several graphics are included in order to complement the explanations and, despite its technicalities, the author's discussion is easy to follow and understand, all in all, being a very didactic article.
The third section of the book includes case studies of a broader scope, all of them devoted to the interpreting field, such as Sarka Timarova and Harry Ungoed-Thomas' revision of admissions test for conference interpreting courses in Europe. While discussing the effectiveness and efficiency of this kind of tests, the authors provide an overview of the basic concepts of aptitude testing discussed in previous studies, tracing the theoretical framework for their own study based on data from a European university. Even though the data collection and the results of their research can be a bit confusing, because a lot of variables are explained through different graphics and statistics, the discussion and conclusion are clear and confirm their hypotheses: admissions tests very often do not predict the final outcome, evidencing that further research still needs to be undertaken.
Karen Bontempo and Jemina Napier's paper also focuses on admission testing, but in the case of signed language interpreters. Their contribution also starts with a review of previous literature on the topic of admission testing, which is followed by an outline on the current situation of signed language interpreter education in Australia. Two studies are then presented: the first was based on a survey sent to Australian signed language (Auslan) interpreters, where they were asked about their profile and their perceptions of the effectiveness of interpreter education programs; the second was the development and application of a pilot program admission test based on the comments by respondents of the first survey. Both studies are explained accurately, especially the second, where much attention is paid to the results and discussion parts. Even though one of the conclusions shows that the authors' own admission test was not predictive of final examination performance, their work is certainly a solid basis for future research on this topic.
In the next contribution, Hildegard Vermeiren, Jan Van Gucht and Leentje De Bontridder expound how certification for social interpreters has been developed in Flanders, Belgium. After the introduction, where the overall sociological framework for their study is presented, the different parts of the certification exam are described —language proficiency, reproduction, transfer and role play— as well as its summary evaluation grid and the measures to ensure a fair assessment. The authors then provide a critical overview of the implications and limitations the certification may have, pinpointing especially those aspects that should be improved in order to grant validity and reliability. One of these aspects is graders training, which is expanded in the next section, where the authors develop a professional competency profile for graders. In this regard, this is one of the assets of the paper, shedding light onto issues such as graders' code of ethics or planning and organisation.
In the last contribution, Debra Russell and Karen Malcolm overview the certification system for American Signed Language–English interpreters in Canada, implemented by the Association of Visual Language Interpreters of Canada (AVLIC). The initial two-phase testing system —based on a written test of knowledge and a test of interpreting— is thoroughly described and compared to those of the National Accreditation Authority for Translators and Interpreters (NAATI) in Australia and of the Registry of Interpreters for the Deaf (RID) in the United States. The present four-phase AVLIC testing model is then presented, including the two new phases: two compulsory preparation workshops and a certificate maintenance program. The part of limitations and future research especially reflects the constant effort undertaken by the authors to ensure appropriateness of testing processes.
In summary, the variety of the topics and approaches chosen in the volume faithfully reflects the current trends of research in translation and interpreting studies, all of them with a common focus on assessment. Tests reliability and validity and the need for more objective assessment tools are the leitmotiv of the book, as they are explored and discussed from different perspectives in most papers. The length of the contributions —an average of 33 pages— enables authors to develop methodological aspects in detail and even include tables and graphics where needed, which is one of the strengths of the book. In addition, most contributions also state their limitations, implications and future research potential, three aspects which may be highly appreciated by those readers currently doing research in the field of assessment of translation and interpreting. The quality and originality of the selected papers is perceived throughout the volume, which may certainly be of special interest to trainers, graders and researchers, especially to those working in professional certification projects. Last but not least, the extensive references provided at the end of each paper will undoubtedly be of great value to those lay to the topic of testing and assessment and wishing to continue reading on it.
Mireia Vargas Urpi, Inter-Asia; Miras research groups; Translation and Interpreting Department, Universitat Autònoma de Barcelona
mireia.vargas@uab.cat