|
II. The Importance of System Alignment for Student Attainment of MPS Standards Alignment is an important
principle of a standards-based system, and systemic reform. The underlying theory of change for standards-based education
is that through having explicitly stated expectations for what students are to
know and do and holding students accountable for achieving high expectations,
student achievement will improve. In a standards-based system, teachers,
students, administrators, staff, and all others in the system work toward one
goal—developing student capacity to meet challenging academic standards. What
each person in the system does is supported by rigorous curricula and
assessments that reflect those standards. Through a confluence of effort, all of
the policies, mandates, teacher professional development initiatives,
instruction, resource allocations, and other system components support students
in achieving high standards. Because what is expected of
students will be more explicit, students’ progress in meeting expectations
will be more effectively measured and reported. Teachers will have a clear
understanding of what students are to learn. Teachers will receive the
additional training they need to become more effective, thus increasing their
capacity to provide the necessary instructional activities for students to
attain the standards. Teachers will reduce redundancy in classroom instruction
(teaching students what they already know) and will know better what emphasis to
place on one topic compared to another. Because what students are to know is
clearly defined, teachers will be more effective in coordinating the instruction
students receive as they move from grade to grade. As students advance through
the grades in a standards-based system, teachers should be able to monitor more
effectively the progress students make towards attaining the standards. Through
the timely use of this information, teachers should be able to provide
instructional experiences to their students that build on students’ prior
learning, thus using instruction and resources more effectively to increase
student achievement. An aligned education system will be more focused and
efficient. Studying the alignment of a system
has value for educational purposes. Alignment studies are also important for
legal reasons. Because Milwaukee Public Schools and the State of Wisconsin
(118.30 Wis. Stats.) require students by 2003 to pass a high school graduation
test, MPS is subject to legal challenges based on the issue of whether students
have had the opportunity to learn what they are required to demonstrate on the
test. Undergoing formal alignment studies is one means for providing evidence
that the district has met its obligations for giving students the educational
experiences they need to perform satisfactorily on the graduation test. Most commonly, alignment of
standards, assessment, and instruction is assumed if these components all
address the same content topics. If, for example, the standards state students
are to be proficient in computing with whole numbers, the curriculum includes a
chapter on computation of whole numbers, and computation items are included on
the assessment instrument, then these components are judged to be aligned. At a
very general level, the set of such standards, assessment, and instruction is
aligned because the same topics are incorporated in all three. They all include
computation as a main category. But this level of alignment is insufficient to
assure that standards-based education will produce the desired results. A more
refined analysis is required that includes the depth of knowledge at which a
topic is being addressed, the range of subtopics included, the balance among the
subtopics, as well as how the topic is advanced from grade to grade, and other
criteria. Are students taught only to recall computation facts rather than apply
computation facts to solve problems, as expressed in the standards? Are students
only exposed to whole number computation rather than rational number
computation? If students are exposed to rational number computation, is 90% of
the instructional time spent only on adding whole numbers? Grade 8 MPS proficiency guidelines
in mathematics and science give students the alternative of demonstrating their
knowledge on the MPS performance assessment or by taking the Wisconsin Student
Assessment System (WSAS) test (TerraNova), among other criteria. Tests can vary
with respect to the topics that are emphasized and the level at which students
are required to demonstrate their knowledge. Alternative measures create a
critical alignment issue because their criteria can vary greatly from each
other. It is possible for students to demonstrate proficiency on one and not the
other because neither is fully aligned with the standards and both vary in the
emphasis placed on topics. Even
standardized norm-referenced tests can vary in what topics are included and what
topics are emphasized. The difference can be even greater between a
norm-referenced test (WSAS) and a criterion-referenced test (MPS performance
assessment). In a recent analysis, two grade 7 mathematics tests (TerraNova and
the Stanford Achievement Test) varied noticeably in the proportions in which
each test addressed different topics (Figure 1).
Figure 1.
Analysis of the distribution of items by content topics on two
standardized norm-referenced tests. Although both tests include some
items measuring knowledge for nearly all of the topics (with one exception), the
two tests varied clearly in the degree of emphasis in number/computation,
geometry, statistics/probability, patterns/relationships, and discrete
mathematics. Alternative test alignment or misalignment can potentially become
an even greater problem. If eighth graders can demonstrate their proficiencies
by passing alternative tests with low alignment, they may be prepared unequally
for advancing towards the graduation requirement where all are to meet the same
qualifications.
* Wisconsin Reading Comprehension Test Figure 2. Analysis of distribution of items on the Wisconsin Grade 3 Reading Test compared to a draft of four standards from the MPS 3rd Grade Reading Academic Standards. A variation in emphasis and depth
of knowledge among standards, tests, and instruction can explain lower scores on
tests. Standards generally imply
that all components are given equal weight.
If teachers give equal attention to each standard in their instruction,
they may be putting their students at a disadvantage in taking a test that
places a greater emphasis on some topics over others.
An analysis of the alignment of the Wisconsin Grade 3 Reading Test with a
draft of MPS standards indicates that the 93-item test varies in emphasis from
the draft of the standards (Figure 2). We also did a preliminary analysis
of the comparison of the depth of knowledge required by the draft of the MPS 3rd
Grade Reading Standards and the depth of knowledge required to correctly respond
to the WRCT (Wisconsin Reading Comprehension Test) items (Figure 3). Over half
of the test items on the 1998 WRCT were rated at the lowest level (Level 1) of
reading knowledge/skills, recall of information, or direct reproduction of
information from the text. The highest depth-of-knowledge levels required by the
draft MPS Reading Standards were Level 2, conceptual understanding, and Level 3,
synthesizing information. A more detailed description of depth-of-knowledge
levels is given in Appendix A. This analysis indicates that on depth of
knowledge there is, in general, good agreement between the four standards and
the test with one exception. Of the 36 items that measure knowledge related to
Standard A.3.2 (read, interpret, and critically analyze literature), only 9
(25%) were rated at a Level 3. This percentage of items on the test is lower
than is necessary for students to fully demonstrate their attainment of standard
A.3.2. This is based on the criterion that a student could be judged proficient
on the test without answering any of the nine Level 3 items related to A.3.2
correctly. Thus, a satisfactory test score may misrepresent a student’s full
attainment of the four standards.
Figure 3.
Analysis of the depth of knowledge expected on the draft of four MPS
3rd Grade Reading Standards and required by items on the Wisconsin
Grade 3 Reading Test. These three examples illustrate
the variation possible between the way in which standards and assessments can
differ on more than one criterion. We describe in the next section a set of
criteria that can be applied to the alignment of standards and assessments and
of these documents with other system components, such as classroom practices and
professional development.
Figure 4.
Alignment criteria. Judging Alignment Multiple
criteria are needed to adequately judge the alignment within a system and
whether all parts of the system are working toward the same goals for student
achievement. III. Current Status of Milwaukee Public Schools’ Standards and
Expectations Our work through October, 1998, in
understanding the District’s alignment clearly reveals that Milwaukee Public
Schools has made advances towards becoming a standards-based system. On February
28, 1996, the Milwaukee Board of School Directors adopted new graduation
requirements, as well as a series of high stakes middle school proficiencies.
These proficiencies have been specified and efforts have been made to inform
teachers, parents, and students what grade 8 students in the school year
1999-2000, and beyond, will be expected to do. Staff members have been appointed
to coordinate tasks related to informing parents and students about the
requirements. Committees of teachers in four content areas--language
arts/reading, mathematics, science, and social studies-- have drafted new K-12
academic standards for the district. This is in compliance with State of
Wisconsin Statute 118.30 and in support of district efforts to set high
standards for student attainment. In October, drafts of the standards were
presented to the Board for its review, as a step toward formal approval. The
standards were approved by the Board in November. These standards will define
what Milwaukee Public School students are to know and be able to do at each
grade level. They are to be a driving force behind curricular as well as
assessment change in the district. In October, 1997, the Wisconsin
Legislature passed State Statute 118.30, which required all districts to either
adopt the state standards in language arts, mathematics, science, and social
studies, or to develop their own, by August, 1998. MPS chose to develop its own
academic standards. Work leading to the specification of the MPS standards began
as early as 1994 and was derived from the district’s K-12 teaching and
learning reform initiative. This initiative set broad educational goals for the
district. Thus, the district’s standards development process coincided with
the state’s efforts. The district established guidelines in the form of
standards. Writing committees considered standards developed by national groups
such as the National Council of Teachers of Mathematics, the National Research
Council, the National Council of Teachers of English, and other professional
groups. Along with standards documents from these groups, the MPS development
teams consulted drafts of the state standards. In the summer and fall of 1998,
committees of 20 to 30 teachers prepared the version of the MPS Standards
presented to the Board for adoption. Over
the course of four or five years, up to 90 teachers and district staff for each
content area have actively engaged in developing grade-level expectations and
standards. These groups had some autonomy in how they created their respective
subject/grade level and course standards and/or expectations. Language
Arts and Reading The organizational form of the
October 6, 1998 draft of the MPS Standards varies by content area. The MPS
language arts/reading standards, K-12, are organized by
headings that, in general, correspond to those used by the
state—reading/literature, writing, oral language, language, and research and
inquiry (grades 6-12 only). Grade-level expectations are listed under each
heading that provide more specific statements of what students are to know and
do. For example, in language arts for grades 6-12 “grade-level expectations”
are described for grade ranges of two grades. Grades 11-12 have expectations
that are different from grades 9-10, which are different from those for grades
6-8. The elementary (K-6) language arts/reading standards use the terms
“standards and grade-level expectations” as the general headings. These
“standards and grade-level expectations” are specified for each grade level,
rather than for a grade range of two to four grades. Even though most headings
in the MPS standards correspond to headings in the 1998 Wisconsin Model Academic
Standards, the draft of the MPS language arts/reading standards do not include a
section on media and technology. On the other hand, MPS specified multicultural standards are
only generally referenced in the state standards. The MPS standards provide more
detail than the state standards for all four content areas and specify
benchmarks that students are expected to achieve at three points during their
schooling career—at the end of grades 4, 8, and 12—not by grade level. These
state standards are prefaced by the proviso, “By the end of grade four,
students will . . .” Social Studies For kindergarten through grade 8,
the MPS social studies standards are organized by “standards and grade-level
expectations” for each grade (“By the end of ___ grade students will . .
.”). However, the elementary
teachers’ writing team differed from the middle school writing team in
approach. The K-5 social studies standards specified what students are to
achieve by the end of each grade. The grades 6-8 standards describe expectations
for the progress students are to make in order to meet the grade 8 standards.
The high school writing team specified expectations for each grade, 9-12, in
social studies courses. They decided to do this because there could be no
assurance that all of the social studies courses would be available to
all of the students. The high school social studies standards are
course-specific—e.g., “By the end of the study of world geography, students
will . . .” There are, however, some common expectations of all students by
the end of grade 12. Science Both grade-range standards and
grade-level expectations form the “Science Curriculum Framework” for grades
K-12. The standards for grade ranges K-4, 5-8, and 9-12 are specified for broad
categories of physical, earth and space, life and environmental, science and
society, and science process studies. For
grades K-8, learning expectations are specified for each grade. For grades 9-12,
learning expectations are specified by course—integrated science (grade 9) and
biology (grade 10)—and high school science processes. The science framework
expresses standards as “concepts” students will engage in studying (e.g.,
properties of earth materials, physical events, changes in substances/objects
etc.). Then the grade-level learning expectations for each grade or course
identify factual information (e.g., water is found in many different contexts,
rocks and sand have weight, there are basic colors that can be identified,
etc.); processes (e.g., design and perform a controlled experiment); and
understandings (e.g., understand the way in which scientists of diverse
cultures, ethnicities, and genders have contributed to science) for which
students are held responsible. The organization of the state science standards
differs from that of the MPS science standards. They specify standards for eight
broad headings: science connections, nature of science, science inquiry,
physical science, earth and space science, life and environmental science,
science applications, and science in personal and social perspectives. The evolution of the MPS Science
Curriculum Framework illustrates how standards have developed from efforts
expended over a number of years. In 1993, a science committee was formed that
included about 60 K-8 teachers, 26 high school teachers, and others. Prior to
this, the Milwaukee Science Materials Center, established in the late 1980s, had
provided science kits to elementary teachers for each activity in the
grade-level textbooks, a support teacher who was on-call to work with teachers
in their classrooms, and a supervisor who provided inservices by grade level. In
1992-93, an evaluation under the coordination of the science specialist was
conducted of the district’s science program prior to a K-5 textbook adoption
in 1993-94. The Addison-Wesley
series was adopted, in part, because it provided an inquiry-based approach to
teaching science. At the time, professional development programs emphasized such
teaching strategies as appropriate. However, the dominant approach used by MPS
elementary teachers was activity-based, where students would carry out a
sequence of steps. Through inservices, teachers learned to use one or two
general kits, but teachers failed to become effective users of the kits because
they did not organize their teaching around big ideas that could be developed
through use of a range of kits. The science curriculum specialist
interviewed teachers, observed classrooms, and administered surveys to
better understand how elementary teachers were teaching science, how their
teaching supported students actively engaged in learning science, and what
teachers’ professional development needs were. The science committee prepared
grade-level expectations and then aligned the science kits with the expectations
for each grade, K-5. Together, the grade level expectations and science
kits—balanced among life, physical, and earth science—represent the core
content knowledge for students. The district was provided funds by the National
Science Foundation to create the Milwaukee Urban Systemic Initiative (MUSI) in
1996-97. The goals of this initiative were compatible with the ongoing MPS work
in science at the time, improvement in science and mathematics achievement. The
Board’s adoption of proficiency requirements for grade 8 students and the
implementation of performance assessments increased the attention that more
resistant teachers gave to implementing the science grade level expectations and
kits. The MPS science committee prepared curriculum modules aligned with the
grade 8 science proficiencies. These modules were piloted for the first time in
the summer of 1998. The current school year, 1998-99, is the first year for the
full implementation of the modules. Middle schools have been grouped into seven
clusters at ten schools. Those teaching science in all of the middle schools are
to meet once a month, beginning in October, with those from the other schools in
their clusters. At these meetings, teachers are expected to discuss the
implementation of the prepared modules. These discussions are to be facilitated
by three trained teacher-facilitators for each cluster, one for each grade 6, 7,
and 8. The mathematics teachers are engaging in the same process, with monthly
meetings of school clusters. The cluster design is being employed to reach all
middle school science and mathematics teachers in the district. The facilitators
and mathematics and science resource teachers (MSRT—funded through MUSI)
provide a communication channel between teachers and the mathematics and science
curriculum specialists. For the high school science
programs, the department chair is the conduit of information for the science
specialist. Recent changes in the high school science curriculum includes the
development of a grade 9 integrated science course that exposes students to
life, physical, and earth sciences. Ongoing issues in the MPS high school
science program include establishing the balance between presenting science as a
fixed body of knowledge, where students do prescribed activities to reveal
specific results, and science presented as a means for understanding the world,
where students design their own experiments to understand better the process of
science. Mathematics The structure of the mathematics standards and grade-level expectations is similar in form to the structure used in science. The same development and implementation process was employed for mathematics as for science. Standards are specified for grade ranges K-4, 5-8, and 9-12 for broad categories of processes, algebraic reasoning, geometry and measurement, numeracy, and probability and statistics. Standards indicate what context and what concepts should be engaged in learning for each grade range. The grade level expectations specified for each grade specify processes and skills students are to perform and what understandings they are to develop. The organization of the MPS mathematics standards is very comparable to the organization of the state mathematics standards with two notable exceptions. Instead of one standard for geometry and measurement, the state separates these two areas into two separate standards. The state also labels numeracy and algebraic reasoning differently from the labels used in the MPS Standards. Instead of numeracy, the state standard is labeled “Number Operations and Relationships,” and instead of algebraic reasoning, the state standard is labeled “Algebraic Relationships.” |
|