An Historical Perspective and a Model for Evaluation of Intelligent Tutoring Systems
- 1 March 1994
- journal article
- research article
- Published by SAGE Publications in Journal of Educational Computing Research
- Vol. 10 (2), 103-128
- https://doi.org/10.2190/xuwl-g11v-9lfx-67xp
Abstract
The purposes of this article are to 1) examine the changes which have occurred in the development and evaluation of ITS systems for the last twenty years, 2) speculate on future directions, and 3) propose a conceptual model to evaluate and institutionalize this technological innovation into training and educational settings. First, we review theoretical and technical dimensions that could be considered relevant to internal evaluation of ITS technology. Secondly, we discuss evaluation dimensions which, while external to ITS per se, are relevant for situational evaluation of any technological innovation. After theoretical and methodological discussions of intelligent features and effectiveness of ITS, its research and development activities are examined from a historical perspective. The development of ITS is divided into three stages and the development focus for each stage is discussed to show the shifts. It is observed that theoretical and methodological criticisms of ITS, needs unique to practical applications, and development of computer technology have influenced the shifts of ITS development focuses: purpose, staffing, hardware and software technology used, and contributions. In stage III, most recent development focuses and future prospects are discussed, including investigation of specific instructional strategies, creation of flexible environments for instructional strategy research, modeling of the human tutor, simulation of human learning and cognition, and creation of multimedia environments. After discussing problems associated with moving ITS from the laboratory to an institutionalized part of the practical world, a conceptual model for evaluating technological innovation in training and educational environments is proposed. This model considers the technology purpose, processes, outcome measures, and the maturity of the technological innovation (the latter especially relevant for ITS). It has three major stages: adoption, implementation, and institutionalization and two major processes: assimilation and accommodation of the innovation. These stages and processes occur within a context of multiple levels of users/stakeholders or decision-makers. Taken together they make up the dimensions of the evaluation model for any technological innovation, including ITS.Keywords
This publication has 24 references indexed in Scilit:
- The Function of Conceptual Understanding in the Learning of Arithmetic ProceduresCognition and Instruction, 1991
- Educational Psychology and Pedagogical Expertise: New Findings and New Opportunities for Thinking About TrainingEducational Psychologist, 1991
- A Large‐Scale Evaluation of an Intelligent Discovery World: SmithtownInteractive Learning Environments, 1990
- Cognition and Improvisation: Differences in Mathematics Instruction by Expert and Novice TeachersAmerican Educational Research Journal, 1989
- Structuring and Adjusting Content for Students: A Study of Live and Simulated Tutoring of AdditionAmerican Educational Research Journal, 1987
- Intelligent Tutoring SystemsScience, 1985
- Learning to program in LISPCognitive Science, 1984
- Reconsidering Research on Learning from MediaReview of Educational Research, 1983
- Non-monotonic logic IArtificial Intelligence, 1980
- Teaching MachinesScience, 1958