Testing textbook theories and tests: The case of suprasegmentals in a pronunciation textbook

This report deals with dual challenges teachers often face: (1) testing the theoretical assumptions made by the authors of the textbooks; and (2) evaluating achievement tests that have been developed for use with those textbooks. The purpose of this report is to address these issues in the context of the use of an English as a Foreign Language (EFL)/Second Language pronunciation textbook and achievement test, 'Clear Speech' (Gilbert, 1993, Clear speech student's book, 2nd Edition. Cambridge University Press, New York), with a group of 24 second-year Japanese EFL students attending a private four-year college near Tokyo. After 32 h of production-focused instruction, students' perception of suprasegmentals seemed to improve, while their production did not. However, a content validity and generalizability analysis of the textbook-based speaking test used to estimate students' achievement revealed that the results of the test could not be trusted. Thus the test may have not allowed students adequate opportunity to demonstrate their abilities in producing suprasegmentals. This raises the question of whether tests and tasks used in classroom and research settings are allowing students to demonstrate their learning achievement.