|
Norman L. Webb Abstract Our
main focus during the first six months of 1999 in the Milwaukee Public Schools
(MPS) study has been to study the alignment among important MPS components. We
did an analysis of the alignment of the MPS Standards and Grade Level
Expectations with the Wisconsin Model Academic Standards for four content areas.
In general, we found that the MPS Standards and Grade Level Expectations were
comparable in content and depth-of-knowledge in language arts, mathematics, and
social studies. We had more difficulty in determining the relation of the two
documents in science. In the science content area, there appeared to be less
agreement in content coverage common to both, (with only about 60% agreement as
compared to 85%-100% agreement in other content areas), and a higher percentage
of the MPS standards were of a lower depth-of-knowledge level. MPS curriculum
specialists and teachers are currently reformatting the standards in a more
consumable form for teachers’ use. Further analysis of the standards documents
and how these documents are used by teachers and principals is needed to fully
understand how these are linked with other system components and how they
contribute to high achievement among students. I.
Introduction One premise of systemic reform is that the major components of an education system must work together to guide the process of helping students achieve higher levels of understanding (Smith & O’Day, 1991, Zucker et al., 1998). Educators recognize that if system components are not aligned, the system will be fragmented, will send mixed messages, and will be less effective (CPRE, 1991; Newmann, 1993). For example, the systemic initiatives program of the National Science Foundation (NSF) is directed toward states, districts, and regions setting ambitious goals for student learning that are based upon a coherent policy system. The Improving America's Schools Act explicated how assessments are to relate to standards: ". . . such assessments (high quality, yearly student assessments) shall . . . be aligned with the State's challenging content and student performance standards and provide coherent information about student attainment of such standards . . ." (U.S. Congress, 1994, p. 8). Similarly, the U.S. Department of Education's explanation of the Goals 2000: Educate America Act and the Elementary and Secondary Education Act (which includes Title I) indicated alignment of curriculum, instruction, professional development, and assessments as a key performance indicator for states, districts, and schools striving to meet challenging standards.
Milwaukee Public Schools has
important pieces in place for an aligned system that is capable of concentrating
its efforts toward improved student achievement. The district has written
content standards and grade level expectations; developed assessment systems and
student proficiencies for grade 8 to grade 9 promotion (in 1999-2000) and for
graduation from high school (in 2003-2004); and established a standards-based
assessment system, with related intervention programs and professional
development. The district also operates with school-based decision-making that
allows principals and their staffs some autonomy. The district is aligned to the
degree that all of these components are working toward the same ends. The main
goal of the research for this study is to determine whether all of the
components are aligned. In the pursuit of this goal, we seek to answer specific
questions about the district: ·
Is there a clear vision among teachers, administrators, and
students of what students are to know and do? ·
Do teachers and schools provide students with the opportunity to
learn what they are to know and do? ·
Do MPS assessments give accurate information on students’
progress and attainment of what they are to know and do? (See also, Robert
Meyer’s report on analyses of MPS assessment data, pp. 29-34). II.
Progress to Date In the first six months of the grant, May through November, 1998, our
major effort was to become acquainted with the district. We identified the
district’s assessment system; tracked the historical developments in
mathematics, science, language arts, and social studies; identified the
expectations for student learning; and began identifying the range of
professional development experiences available to teachers (Clune & Webb,
1998). After the Board of School Directors adopted the MPS standards in November
1998, we concentrated our efforts on two activities. We sought to verify that
the adopted standards were aligned with the Wisconsin Model of Academic
Standards, one criterion of the validity of the MPS standards. We tried to
understand how useful the standards documents are in providing district staff
the information they need to guide student learning. Over the period from
January through May 1999, we conducted a content analysis of the alignment
between the state curriculum standards in language arts, mathematics, science
and social studies with the MPS curriculum standards.
We engaged in other activities as requested by
district staff and used these activities as opportunities to understand MPS
reform efforts. We reviewed the proposed mathematics performance assessment
activities prior to administration in March and sent Kathy Swope, Office of
Research and Assessment, our comments on February 19, 1999. We sent her our
comments on the proposed science performance assessments on March 29, 1999. On
February 24, 1999, the alignment research team met with Jocklyn Smith and
curriculum specialists to present our approach to thinking about and studying
alignment in the district and to seek their cooperation. By March 4, 1999, we
had completed interviewing five curriculum specialists, including Carmen Baxter,
science; Karen Villcock, mathematics; Sharon Durkta, social studies; Sandra
Dickerson, language arts; and, Frankie Johnson, reading. In April, we identified eight elementary and middle
school sites for gathering more data on the use of the standards in schools, the
alignment between the instructional experiences students received in schools and
the standards, and how the degree of alignment relates to student learning. A
team of three to five researchers conducted one- and two-day visits at three
schools by the end of the school year. Other school visits are planned for the
fall. As a new initiative, related to our objective of
developing district capacity for more effective use of information as a basis
for instructional decisions, we prepared and submitted a proposal to the Joyce
Foundation. The proposed work, in cooperation with the National Center for
Research on Evaluation, Standards, and Student Testing (CRESST) at the
University of California, Los Angeles, is to implement the Quality School
Portfolio (QSP) in MPS middle schools and study how teachers and principals are
able to use this tool to make more effective decisions. QSP is a school-based
electronic information system designed to help school staff make more effective
use of data. Development of a Standards-Based System at MPS Over the past six years, the district has been on a trajectory leading
towards a standards-based system and increased student achievement. Such
large-scale reform takes time to reach coherence among all of the district's
components. There is evidence in all of the four content
areas—language arts, mathematics, science, and social studies—that progress
has been made in aligning the important components of standards, curriculum,
assessment, and professional development. Important steps towards a
standards-based system include: ·
Adoption
of K-12 Teaching and Learning Goals (about 1994); ·
Development
of grade-level expectations in mathematics and science (from 1994); ·
Mandated
district graduation requirement and middle school proficiencies (1996); ·
Implementation
of writing and mathematics performance assessments; ·
Milwaukee
Urban Systemic Initiative (initiated 1996-97); ·
MPS
Standards and Grade Level Expectations (1998); ·
District-wide
adoption of mathematics curriculum, grades 6-8 (1998-99); and, ·
Professional
development on new mathematics curriculum. ·
Mandated
high school graduation requirements. In November 1998, the Milwaukee Board of School
Directors approved the Milwaukee Public
Schools K-12 Academic Standards and Grade Level Expectations for language
arts, mathematics, science, and social studies. Responding to demands of the
state and building on work that had already been done, the MPS Division of
Curriculum and Instruction developed these content standards. The standards and
grade level expectations in each content area were developed under the
leadership of the content area curriculum specialist and with the help of
teacher committees. The work in each content area took a different approach,
based in part on what was already in existence. As a consequence, the format of
standards and grade-level expectations among the four content areas are
different. In language arts, the curriculum specialist generally
took the Wisconsin Model of Academic Standards and then nested grade-level
expectations under these by detailing what students had to learn at each grade
level in order to attain the standards at grades 4, 8, and 12. Grade-level
expectations were already available for mathematics and science. The curriculum
specialists in mathematics and science worked with teachers to specify general
standards of what students were to study. Most of these standards stated only
the grade
range, the grade-level expectations were appended to the standards without
describing the relation between the standards and the grade-level expectations.
Over the 1998-99 school year, teachers and the curriculum specialists worked on
reformatting the standards in a column format that would explicate the
relationship between standards, grade level expectations, performance standards,
and assessments. Three committees of teachers for social studies, one for each
grade range, built standards for their grade range based on the Wisconsin
standards. The high school group adopted the state standards organized by major
topic (world geography, history, etc.). The elementary group essentially took
the state standards and wrote grade-level expectations for each intervening
grade. The middle school group wrote their own standards and then modified them
to include a local transition from elementary to high school and a similar
format. III. Findings If standards are to drive the system and contribute to improving student
achievement, it is essential that the standards represent important content,
that they are succinct, that they can be understood by diverse populations, and
that they provide direction for curriculum, professional development,
assessments, and classroom practices. During the spring semester, we conducted
an analysis of the MPS standards in the four content areas, using Wisconsin’s
Model Academic Standards as a model of high quality standards. Our preliminary
analysis indicates that there is strong alignment within language arts,
mathematics, and social studies in the content topics students are expected to
have knowledge about and in the depth at which students are expected to know
these topics. From 85% to 100% of the topics in the Wisconsin Model Academic
Standards were at least partially incorporated in the MPS Standards and Grade
Level Expectations, and from 79% to 94% of the topics from the MPS standards
were at least partially incorporated in the Wisconsin standards. There were a
number of reasons for the difference in content coverage, including: 1. MPS Standards and Grade Level Expectations are more detailed statements
than those in the state standards; 2. Some differences exist in the grade level at which students are expected
to acquire specific knowledge; and 3. A vagueness in language makes it difficult to judge the relationship
between documents. We also considered how demanding the MPS standards are in relation to the
Wisconsin Model Academic Standards. We used four levels of depth-of-knowledge to
compare what students were expected to do for MPS and the state:
Level 1
Recall of information
Level 2
Skills and conceptual knowledge
Level 3
Strategic thinking
Level 4
Extended thinking In
language arts and social studies, the depth of knowledge students were expected
to attain by the MPS Standards and Grade Level Expectations and the Wisconsin
Model Academic Standards were nearly identical. In general, the percentage of
expectations with a higher depth-of-knowledge level increased in the higher
grades. In language arts, about 65% of the grade 4, 53% of the grade 8, and
about 27% of the grade 12 expectations and standards for both MPS and Wisconsin
were judged to be a Level 2 (Skill/Concept). The percentage of language arts
standards at Level 3 (Strategic Thinking) increased by grade from about 22% for
grade 4 and 32% for grade 8 to 47% for grade 12. In social studies, for grades 4, 8, and 12, about 20
% of the standards were rated at Level 1 (Recall). However, for grades 4 and 8,
the highest percentage of standards were rated as Level 2, from 50 to 70%. As
could be expected, for grade 12 the highest percentage of standards were rated
as level 3, about 55% for both MPS and the state. MPS standards and the state standards had a similar
pattern in the depth-of-knowledge levels for mathematics. Both sets of standards
have a high degree of Level 2 standards. Level 2 standards dominate grade 4,
comprising about 75% of the standards. For grades 4 and 12, about 50% of the
standards were rated as Level 2 and about 40% of the standards were rated as
Level 3 for both MPS and the state. A noticeably higher percentage of the MPS standards
for science was rated as Level 1 compared to the state standards, from 26% to
32% for MPS compared to 2 to 18% for the state. For grades 8 and 12, the state
had a higher percentage of standards rated as Level 3 than did MPS, about 20%
compared to about 30%. This indicates that a greater number of the state
standards required students to evaluate, predict, or analyze, whereas more of
the MPS standards required students to describe, identify, observe, apply, or
illustrate. IV. Next Steps: Policy Recommendations and Future Research
·
teachers, administrators, and students ·
students
are provided the opportunity to learn what they are to know and do; ·
assessments
provide accurate information on students’ progress and attainment of what they
are to know and do; ·
assessment
activities ·
knowledge
in certain content areas is being adequately assessed. We also will identify the
articulation (vertical alignment) of standards and assessments across the
grades, which is so important if students' learning experiences are to
effectively accumulate over time. 7.
We will continue to gather data through reviewing documents and
interviewing people. Also, we will continue to forge a relationship with the MPS
Office of Research and Assessment staff to contribute to developing more valid
assessments. We will prepare a written report in August on our analysis of the
three schools. This will be shared with the staff at each school and with
district staff. Other reports will be prepared as work is completed. References Clune, W. H., & Webb, N. L.
(1999). Technical reports on Milwaukee Public Schools policy, information
systems, and instructional alignment. Madison: University of Wisconsin, Wisconsin Center for
Education Research. Consortium
for Policy Research in Education. (1991).
Putting the pieces together: Systemic school reform (CPRE Policy Briefs).
New Brunswick: Rutgers, The State University of New Jersey, Eagleton Institute
of Politics. Newmann,
F. M. (1993). Beyond common sense in educational restructuring: The issues of
content and linkage. Educational
Researcher, 22 (2), 4-13, 22. Smith,
M. S., & O’Day, J. (1991). Systemic
school reform. In S. H. Fuhrman
& B. Malen (Eds.), The politics of
curriculum and testing. Politics of Education association Yearbook. (1990,
pp. 233-267). U.S.
Congress, House of Representatives. (1994, September 28). Improving
America’s Schools Act. Conference
report to accompany H.R. 6 Report 103-761. Washington, DC: U.S. Government
Printing Office. Zucker,
A. A., Shields, P. M., Adelman, N. E., Corcoran, T. B., & Goertz, M. E.
(1998) The National Science Foundation’s
Statewide Systemic Initiatives Program: Evaluation Report. Menlo Park, CA:
SRI International. Footnote [1] Identifiers designate a specific MPS or Wisconsin standard:
e.g., this reference is to MPS Life and Environmental Sciences Standard C,
Grade 8, Grade Level Expectation 4.
|
|