Using Instruments in Education Abroad Outcomes Assessment
By R. Michael Paige and Elizabeth M. Stallman
University of Minnesota
This chapter focuses on the use of instruments for outcomes assessment in education abroad. The rapid growth in international educational exchange has been accompanied by an increasing demand for accountability, i.e., for evidence that the investments in education abroad produces the desired individual and institutional outcomes. In this chapter we examine a set of paper and pencil instruments and inventories that are designed to measure specific traits, qualities, developmental outcomes, and other characteristics relevant to education abroad. The selected instruments primarily focus on the individual education abroad participant as the unit of analysis rather than the organization (such as the university or academic department), although we will comment on the latter. Further, the "participant" definition has grown in recent years to include faculty, staff, and alumni in addition to students. Depending upon the program and degree of outreach by education abroad administrators, the instruments that follow may possibly be used in training and assessment for any of these groups. The central purpose of the chapter is to identify and describe instruments which are appropriate for education abroad outcomes assessment, are valid and reliable, and, preferably, have been utilized in education abroad research.
It was a daunting task to identify an appropriate set of instruments from the plethora of inventories that exist. There is an extensive reference literature, some of the best known publications being Tests in Print and the Mental Measurements Yearbook, both of which are updated periodically. In addition, there are additional texts in a variety of areas such as multicultural assessment and psychology, education, and business. Important contributions from the intercultural literature include the substantive descriptions of seven different instruments that appear in Fowler and Mumford's Intercultural Sourcebook, vol. II along with commentaries about instruments that can be found in works by Gudykunst, Brislin and Yoshida, Reddin, Bhawuk and Triandis, Cushner and Brislin, and Singelis. The most elaborate account to date is Paige's review of 35 "intercultural instruments" that appeared in the Handbook of Intercultural Training. This was of particular value to us in helping narrow the field of instruments, identify relevant instruments, and establish categories of instruments. Finally, we examined the Fall 2004 special research issue of Frontiers: The Interdisciplinary Journal of Study Abroad to see which instruments were being used in the latest, state of the art study abroad research. These literature reviews were significant because we could then identify the most commonly used instruments, look at their properties, and determine if they met our criteria for inclusion. The result of our analysis was the identification of 15 assessment instruments that we determined could be valuable for education abroad outcomes assessment.
The organization of this chapter is as follows. We first discuss in greater detail the purposes for using instruments in education abroad research, assessment, and evaluation. We then discuss the selection criteria we used for selecting these particular assessment instruments. The core of the chapter is the description of the instruments themselves. In this section, we provide an overview of the instrument along with information regarding the key concepts being measured, the item format of the instrument, and reliability and validity data. We conclude the chapter with some closing observations about the limitations, challenges, and opportunities of using these instruments in education abroad assessment.
The rest of this assessment will be available in March 2007 at the Forum's annual conference. You can visit the Forum website for more information.
|