Research Monograph
The Modeling of Systems and Macro-Systemic Change: Lessons for Evaluation from Epidemiology and Ecology
James Ridgway
Abstract
The paper begins with a discussion of the roles of science in the political world and,
borrowing from Shakley and Wynne (1996), describes some different relationships which can
exist between science and policy. It argues that education systemic reform (ESR)
constitutes a novel approach to educational reform, about which little is known, and about
which much is yet to be discovered. ESR requires "abracadabra" science, in the
language of Shakley and Wynne. To be consistent with the philosophy of ESR, the evaluation
of ESR must be an evaluation of systems undergoing change, and so evaluation itself also
requires a good deal of abracadabra science.
The paper describes three styles of modeling in science: analytic, exemplified by
eighteenth century physics; systemic, exemplified by biology; and macro-systemic,
exemplified by studies of ecologies undergoing change. Each modeling style depends on and
incorporates its predecessor. The dominant intellectual traditions in education have been
analytic, rather than systemic. The emergence of systemic reform as a paradigm for
educational change has created a need for approaches to educational evaluation that set
out to judge the functioning of systems; this will require attention to major system
phases--the evaluation of plans about some new system, the evaluation of the
implementation of the plans, and the provision of summative feedback about its success or
otherwise. However, systemic reform requires more than just an understanding of systems;
rather it requires an understanding of systems undergoing change. It follows that the
evaluation of education systemic reform then is more a kind of macro-systemic model than a
systemic model as drawn from science. Several disciplines outside education have systemic
and macro-systemic approaches as their dominant intellectual traditions. This monograph
considers the approaches taken to evaluation and inquiry in some of these disciplines,
notably epidemiology and ecology, and the central roles that evaluation plays in planning
and monitoring change.
From a description of the methods used in other disciplines, a specification of the
evidence base needed to conduct evaluations of ESR is derived. Attention is given to some
of the research styles from a number of different academic disciplines (including physics
and earth sciences) that face the same problems as those faced by the education in terms
of handling complexity. Some ideas on data gathering, modeling, and strategies for further
research into educational evaluation are presented.
The monograph points out the importance of making full use of existing knowledge and the
knowledge that the evaluation community is rapidly creating. It endorses arguments made by
Wilson (1994) and Scriven (1993) that there is a pressing need for an intellectual
community to emerge that addresses the issues of the management and evaluation of systems
undergoing change.