Center for the Study of Systemic Reform
   in Milwaukee Public Schools

 

II. The Importance of System Alignment for Student Attainment of MPS Standards 

Alignment is an important principle of a standards-based system, and systemic reform.  The underlying theory of change for standards-based education is that through having explicitly stated expectations for what students are to know and do and holding students accountable for achieving high expectations, student achievement will improve. In a standards-based system, teachers, students, administrators, staff, and all others in the system work toward one goal—developing student capacity to meet challenging academic standards. What each person in the system does is supported by rigorous curricula and assessments that reflect those standards. Through a confluence of effort, all of the policies, mandates, teacher professional development initiatives, instruction, resource allocations, and other system components support students in achieving high standards. 

Because what is expected of students will be more explicit, students’ progress in meeting expectations will be more effectively measured and reported. Teachers will have a clear understanding of what students are to learn. Teachers will receive the additional training they need to become more effective, thus increasing their capacity to provide the necessary instructional activities for students to attain the standards. Teachers will reduce redundancy in classroom instruction (teaching students what they already know) and will know better what emphasis to place on one topic compared to another. Because what students are to know is clearly defined, teachers will be more effective in coordinating the instruction students receive as they move from grade to grade. As students advance through the grades in a standards-based system, teachers should be able to monitor more effectively the progress students make towards attaining the standards. Through the timely use of this information, teachers should be able to provide instructional experiences to their students that build on students’ prior learning, thus using instruction and resources more effectively to increase student achievement. An aligned education system will be more focused and efficient. 

Studying the alignment of a system has value for educational purposes. Alignment studies are also important for legal reasons. Because Milwaukee Public Schools and the State of Wisconsin (118.30 Wis. Stats.) require students by 2003 to pass a high school graduation test, MPS is subject to legal challenges based on the issue of whether students have had the opportunity to learn what they are required to demonstrate on the test. Undergoing formal alignment studies is one means for providing evidence that the district has met its obligations for giving students the educational experiences they need to perform satisfactorily on the graduation test. 

Most commonly, alignment of standards, assessment, and instruction is assumed if these components all address the same content topics. If, for example, the standards state students are to be proficient in computing with whole numbers, the curriculum includes a chapter on computation of whole numbers, and computation items are included on the assessment instrument, then these components are judged to be aligned. At a very general level, the set of such standards, assessment, and instruction is aligned because the same topics are incorporated in all three. They all include computation as a main category. But this level of alignment is insufficient to assure that standards-based education will produce the desired results. A more refined analysis is required that includes the depth of knowledge at which a topic is being addressed, the range of subtopics included, the balance among the subtopics, as well as how the topic is advanced from grade to grade, and other criteria. Are students taught only to recall computation facts rather than apply computation facts to solve problems, as expressed in the standards? Are students only exposed to whole number computation rather than rational number computation? If students are exposed to rational number computation, is 90% of the instructional time spent only on adding whole numbers? 

Grade 8 MPS proficiency guidelines in mathematics and science give students the alternative of demonstrating their knowledge on the MPS performance assessment or by taking the Wisconsin Student Assessment System (WSAS) test (TerraNova), among other criteria. Tests can vary with respect to the topics that are emphasized and the level at which students are required to demonstrate their knowledge. Alternative measures create a critical alignment issue because their criteria can vary greatly from each other. It is possible for students to demonstrate proficiency on one and not the other because neither is fully aligned with the standards and both vary in the emphasis placed on topics.  Even standardized norm-referenced tests can vary in what topics are included and what topics are emphasized. The difference can be even greater between a norm-referenced test (WSAS) and a criterion-referenced test (MPS performance assessment). In a recent analysis, two grade 7 mathematics tests (TerraNova and the Stanford Achievement Test) varied noticeably in the proportions in which each test addressed different topics (Figure 1).           

TOPIC STANFORD ACHIEVEMENT TerraNova
Number/computation 60% 42%
Estimation 11% 8%
Measurement 1% 4%
Geometry 7% 12%
Statistics/probability 9% 18%
Patterns/relationships 3% 10%
Algebra 5% 6%
Discrete mathematics 4% 0%

Figure 1.   Analysis of the distribution of items by content topics on two standardized norm-referenced tests. 

Although both tests include some items measuring knowledge for nearly all of the topics (with one exception), the two tests varied clearly in the degree of emphasis in number/computation, geometry, statistics/probability, patterns/relationships, and discrete mathematics. Alternative test alignment or misalignment can potentially become an even greater problem. If eighth graders can demonstrate their proficiencies by passing alternative tests with low alignment, they may be prepared unequally for advancing towards the graduation requirement where all are to meet the same qualifications. 

MPS 3rd Grade Reading Academic Standards

1998 WRCT* Test items

Total

A.3.1

Use effective reading strategies to achieve their purposes in reading

12, 14-18, 22-24, 26, 32, 33, 36, 37-44, 46-50, 52-65, 68, 92, 93, 95, 103, 104

46

49%

A.3.2

Read, interpret, and critically analyze literature

13, 20, 21, 25, 27, 30, 31, 34, 35, 45, 51, 66, 67, 69-81, 83-89, 96-98, 100

36

39%

A.3.3

Read and discuss the literary and nonliterary texts in order to understand human experiences

19, 28, 29, 82, 90, 91, 94, 99, 101, 102

11

12%

A.3.4

Read to acquire information

 

0

* Wisconsin Reading Comprehension Test  

Figure 2.   Analysis of distribution of items on the Wisconsin Grade 3 Reading Test compared to a draft of four standards from the MPS 3rd Grade Reading Academic Standards.

A variation in emphasis and depth of knowledge among standards, tests, and instruction can explain lower scores on tests.  Standards generally imply that all components are given equal weight.  If teachers give equal attention to each standard in their instruction, they may be putting their students at a disadvantage in taking a test that places a greater emphasis on some topics over others.  An analysis of the alignment of the Wisconsin Grade 3 Reading Test with a draft of MPS standards indicates that the 93-item test varies in emphasis from the draft of the standards (Figure 2). 

We also did a preliminary analysis of the comparison of the depth of knowledge required by the draft of the MPS 3rd Grade Reading Standards and the depth of knowledge required to correctly respond to the WRCT (Wisconsin Reading Comprehension Test) items (Figure 3). Over half of the test items on the 1998 WRCT were rated at the lowest level (Level 1) of reading knowledge/skills, recall of information, or direct reproduction of information from the text. The highest depth-of-knowledge levels required by the draft MPS Reading Standards were Level 2, conceptual understanding, and Level 3, synthesizing information. A more detailed description of depth-of-knowledge levels is given in Appendix A. This analysis indicates that on depth of knowledge there is, in general, good agreement between the four standards and the test with one exception. Of the 36 items that measure knowledge related to Standard A.3.2 (read, interpret, and critically analyze literature), only 9 (25%) were rated at a Level 3. This percentage of items on the test is lower than is necessary for students to fully demonstrate their attainment of standard A.3.2. This is based on the criterion that a student could be judged proficient on the test without answering any of the nine Level 3 items related to A.3.2 correctly. Thus, a satisfactory test score may misrepresent a student’s full attainment of the four standards. 

Level of Complexity

1998 MPS 3rd Grade Reading Standards

Total

Matches

1998 WRCT Test items

Total

Matches

Level 1

 

 

14-18, 21-24, 26, 32, 37-44, 47, 49, 50, 54-65, 68, 73, 75-79, 81, 92-95

46

Level 2

A.3.1, A.3.3, A.3.4

3

12, 13, 19, 20, 25, 27, 30, 33-35, 45, 46, 48, 51-53, 66, 72, 74, 80, 82, 83, 86, 87, 89, 91, 96-98, 101, 103, 104

33

Level 3

A.3.2

1

29, 31, 36, 67, 69-71, 84, 85, 88, 90, 99, 100, 102

14

Level 4

 

 

 

 

Figure 3.   Analysis of the depth of knowledge expected on the draft of four MPS 3rd Grade Reading Standards and required by items on the Wisconsin Grade 3 Reading Test. 

These three examples illustrate the variation possible between the way in which standards and assessments can differ on more than one criterion. We describe in the next section a set of criteria that can be applied to the alignment of standards and assessments and of these documents with other system components, such as classroom practices and professional development.

Text Box: 1 – Content Focus. System components should focus consistently on developing students’ knowledge of subject matter.  Consistency will be present to the extent components’ logic of action and the ends achieved share the following attributes:
A.	Categorical concurrence. Agreement in content topics addressed.
B.	Depth-of-Knowledge Consistency.  Agreement in level of cognitive complexity of information required. 
C.	Range-of-Knowledge Correspondence. Agreement in the span of topics.	
D.	Structure-of-Knowledge Comparability. Agreement in what it means to know concepts.
E.	Balance of Representation. Agreement in emphasis given to different content topics.
F.	Dispositional Consonance. Agreement in attention to students’ attitudes and beliefs. 

2 – Articulation Across Grades and Ages. Students’ knowledge of subject matter grows over time. All system components must be rooted in a common view of how students develop, and how best to help them learn at different developmental stages. This common view is based on: 
A.	Cognitive Soundness Determined by Superior Research and Understanding. All components build on principles for sound learning programs.
B.	Cumulative Growth in Knowledge During Students’ Schooling. All components are based on a common rationale regarding progress in student learning. 

3 – Equity and Fairness. When expectations are that all students can meet high standards, aligned instruction, assessments, and resources must give every student a reasonable opportunity to demonstrate attainment of what is expected. System components that are aligned will serve the full diversity in the education system through demanding equally high learning standards for all students while fairly providing means for students to achieve and demonstrate the expected level of learning. To be equitable and fair, time is required for patterns to form in order to decipher how system components are working in concert with each other.  Judging a system on the criterion of equity and fairness will require analysis over a period of time.

4 – Pedagogical Implications. Classroom practice greatly influences what students learn. Other system components, including expectations and assessments, can and should have a strong impact on these practices, and should send clear and consistent messages to teachers about appropriate pedagogy. Critical elements to be considered in judging alignment related to pedagogy include:
A.	Engagement of Students and Effective Classroom Practices. Agreement among components in a range of learning activities and in what they are to attain. 
B.	Use of Technology, Materials, and Tools. Agreement among components in how and to what ends applications of technology, materials, and tools are to be included.

5 – System Applicability. Although system components should seek to encourage high expectations for student performance, they also need to form the basis for a program that is realistic and manageable in the real world. The policy elements must be in a form that can be used by teachers and administrators in a day-to-day setting. Also, the public must feel that these elements are credible, and that they are aimed at getting students to learn the mathematics and science that are important and useful in society. 



   

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Figure 4.   Alignment criteria.

Judging Alignment 

            Multiple criteria are needed to adequately judge the alignment within a system and whether all parts of the system are working toward the same goals for student achievement. A review of national and state standards and different alignment studies suggests the following categories for critiquing the alignment of system components: content focus, articulation across grades and ages, equity and fairness, pedagogical implications, and system applicability (Webb, 1997).  These same criteria were analyzed and judged to be useful also for studying the alignment of professional development and expectations (Webb, 1998).  With a few additional criteria, they appear to be appropriate as well for judging alignment among other components that  included policies, school organization, textbooks, higher education requirements, work expectations, and student outcomes.  The criteria and subcriteria are given in Figure 4 above.  

III. Current Status of Milwaukee Public Schools’ Standards and Expectations  

Our work through October, 1998, in understanding the District’s alignment clearly reveals that Milwaukee Public Schools has made advances towards becoming a standards-based system. On February 28, 1996, the Milwaukee Board of School Directors adopted new graduation requirements, as well as a series of high stakes middle school proficiencies. These proficiencies have been specified and efforts have been made to inform teachers, parents, and students what grade 8 students in the school year 1999-2000, and beyond, will be expected to do. Staff members have been appointed to coordinate tasks related to informing parents and students about the requirements. Committees of teachers in four content areas--language arts/reading, mathematics, science, and social studies-- have drafted new K-12 academic standards for the district. This is in compliance with State of Wisconsin Statute 118.30 and in support of district efforts to set high standards for student attainment. In October, drafts of the standards were presented to the Board for its review, as a step toward formal approval. The standards were approved by the Board in November. These standards will define what Milwaukee Public School students are to know and be able to do at each grade level. They are to be a driving force behind curricular as well as assessment change in the district.  

In October, 1997, the Wisconsin Legislature passed State Statute 118.30, which required all districts to either adopt the state standards in language arts, mathematics, science, and social studies, or to develop their own, by August, 1998. MPS chose to develop its own academic standards. Work leading to the specification of the MPS standards began as early as 1994 and was derived from the district’s K-12 teaching and learning reform initiative. This initiative set broad educational goals for the district. Thus, the district’s standards development process coincided with the state’s efforts. The district established guidelines in the form of standards. Writing committees considered standards developed by national groups such as the National Council of Teachers of Mathematics, the National Research Council, the National Council of Teachers of English, and other professional groups. Along with standards documents from these groups, the MPS development teams consulted drafts of the state standards. In the summer and fall of 1998, committees of 20 to 30 teachers prepared the version of the MPS Standards presented to the Board for adoption.  Over the course of four or five years, up to 90 teachers and district staff for each content area have actively engaged in developing grade-level expectations and standards. These groups had some autonomy in how they created their respective subject/grade level and course standards and/or expectations. 

Language Arts and Reading  

The organizational form of the October 6, 1998 draft of the MPS Standards varies by content area. The MPS language arts/reading standards, K-12, are organized by  headings that, in general, correspond to those used by the state—reading/literature, writing, oral language, language, and research and inquiry (grades 6-12 only). Grade-level expectations are listed under each heading that provide more specific statements of what students are to know and do. For example, in language arts for grades 6-12 “grade-level expectations” are described for grade ranges of two grades. Grades 11-12 have expectations that are different from grades 9-10, which are different from those for grades 6-8. The elementary (K-6) language arts/reading standards use the terms “standards and grade-level expectations” as the general headings. These “standards and grade-level expectations” are specified for each grade level, rather than for a grade range of two to four grades. Even though most headings in the MPS standards correspond to headings in the 1998 Wisconsin Model Academic Standards, the draft of the MPS language arts/reading standards do not include a section on media and technology.  On the other hand, MPS specified multicultural standards are only generally referenced in the state standards. The MPS standards provide more detail than the state standards for all four content areas and specify benchmarks that students are expected to achieve at three points during their schooling career—at the end of grades 4, 8, and 12—not by grade level. These state standards are prefaced by the proviso, “By the end of grade four, students will . . .”  

Social Studies 

For kindergarten through grade 8, the MPS social studies standards are organized by “standards and grade-level expectations” for each grade (“By the end of ___ grade students will . . .”).  However, the elementary teachers’ writing team differed from the middle school writing team in approach. The K-5 social studies standards specified what students are to achieve by the end of each grade. The grades 6-8 standards describe expectations for the progress students are to make in order to meet the grade 8 standards. The high school writing team specified expectations for each grade, 9-12, in social studies courses. They decided to do this because there could be no  assurance that all of the social studies courses would be available to all of the students. The high school social studies standards are course-specific—e.g., “By the end of the study of world geography, students will . . .” There are, however, some common expectations of all students by the end of grade 12. 

Science 

Both grade-range standards and grade-level expectations form the “Science Curriculum Framework” for grades K-12. The standards for grade ranges K-4, 5-8, and 9-12 are specified for broad categories of physical, earth and space, life and environmental, science and society, and science process studies.  For grades K-8, learning expectations are specified for each grade. For grades 9-12, learning expectations are specified by course—integrated science (grade 9) and biology (grade 10)—and high school science processes. The science framework expresses standards as “concepts” students will engage in studying (e.g., properties of earth materials, physical events, changes in substances/objects etc.). Then the grade-level learning expectations for each grade or course identify factual information (e.g., water is found in many different contexts, rocks and sand have weight, there are basic colors that can be identified, etc.); processes (e.g., design and perform a controlled experiment); and understandings (e.g., understand the way in which scientists of diverse cultures, ethnicities, and genders have contributed to science) for which students are held responsible. The organization of the state science standards differs from that of the MPS science standards. They specify standards for eight broad headings: science connections, nature of science, science inquiry, physical science, earth and space science, life and environmental science, science applications, and science in personal and social perspectives. 

The evolution of the MPS Science Curriculum Framework illustrates how standards have developed from efforts expended over a number of years. In 1993, a science committee was formed that included about 60 K-8 teachers, 26 high school teachers, and others. Prior to this, the Milwaukee Science Materials Center, established in the late 1980s, had provided science kits to elementary teachers for each activity in the grade-level textbooks, a support teacher who was on-call to work with teachers in their classrooms, and a supervisor who provided inservices by grade level. In 1992-93, an evaluation under the coordination of the science specialist was conducted of the district’s science program prior to a K-5 textbook adoption in 1993-94.  The Addison-Wesley series was adopted, in part, because it provided an inquiry-based approach to teaching science. At the time, professional development programs emphasized such teaching strategies as appropriate. However, the dominant approach used by MPS elementary teachers was activity-based, where students would carry out a sequence of steps. Through inservices, teachers learned to use one or two general kits, but teachers failed to become effective users of the kits because they did not organize their teaching around big ideas that could be developed through use of a range of kits. The science curriculum specialist  interviewed teachers, observed classrooms, and administered surveys to better understand how elementary teachers were teaching science, how their teaching supported students actively engaged in learning science, and what teachers’ professional development needs were. 

The science committee prepared grade-level expectations and then aligned the science kits with the expectations for each grade, K-5. Together, the grade level expectations and science kits—balanced among life, physical, and earth science—represent the core content knowledge for students. The district was provided funds by the National Science Foundation to create the Milwaukee Urban Systemic Initiative (MUSI) in 1996-97. The goals of this initiative were compatible with the ongoing MPS work in science at the time, improvement in science and mathematics achievement. The Board’s adoption of proficiency requirements for grade 8 students and the implementation of performance assessments increased the attention that more resistant teachers gave to implementing the science grade level expectations and kits. The MPS science committee prepared curriculum modules aligned with the grade 8 science proficiencies. These modules were piloted for the first time in the summer of 1998. The current school year, 1998-99, is the first year for the full implementation of the modules. Middle schools have been grouped into seven clusters at ten schools. Those teaching science in all of the middle schools are to meet once a month, beginning in October, with those from the other schools in their clusters. At these meetings, teachers are expected to discuss the implementation of the prepared modules. These discussions are to be facilitated by three trained teacher-facilitators for each cluster, one for each grade 6, 7, and 8. The mathematics teachers are engaging in the same process, with monthly meetings of school clusters. The cluster design is being employed to reach all middle school science and mathematics teachers in the district. The facilitators and mathematics and science resource teachers (MSRT—funded through MUSI) provide a communication channel between teachers and the mathematics and science curriculum specialists. 

For the high school science programs, the department chair is the conduit of information for the science specialist. Recent changes in the high school science curriculum includes the development of a grade 9 integrated science course that exposes students to life, physical, and earth sciences. Ongoing issues in the MPS high school science program include establishing the balance between presenting science as a fixed body of knowledge, where students do prescribed activities to reveal specific results, and science presented as a means for understanding the world, where students design their own experiments to understand better the process of science. 

Mathematics 

The structure of the mathematics standards and grade-level expectations is similar in form to the structure used in science. The same development and implementation process was employed for mathematics as for science. Standards are specified for grade ranges K-4, 5-8, and 9-12 for broad categories of processes, algebraic reasoning, geometry and measurement, numeracy, and probability and statistics. Standards indicate what context and what concepts should be engaged in learning for each grade range. The grade level expectations specified for each grade specify processes and skills students are to perform and what understandings they are to develop. The organization of the MPS mathematics standards is very comparable to the organization of the state mathematics standards with two notable exceptions. Instead of one standard for geometry and measurement, the state separates these two areas into two separate standards. The state also labels numeracy and algebraic reasoning differently from the labels used in the MPS Standards. Instead of numeracy, the state standard is labeled “Number Operations and Relationships,” and instead of algebraic reasoning, the state standard is labeled “Algebraic Relationships.”

 

Information Staff Reports Links