Center for the Study of Systemic Reform
   in Milwaukee Public Schools

 

III. Current Status of Milwaukee Public Schools’ Standards and Expectations 

Our work through October, 1998, in understanding the District’s alignment clearly reveals that Milwaukee Public Schools has made advances towards becoming a standards-based system. On February 28, 1996, the Milwaukee Board of School Directors adopted new graduation requirements, as well as a series of high stakes middle school proficiencies. These proficiencies have been specified and efforts have been made to inform teachers, parents, and students what grade 8 students in the school year 1999-2000, and beyond, will be expected to do. Staff members have been appointed to coordinate tasks related to informing parents and students about the requirements. Committees of teachers in four content areas--language arts/reading, mathematics, science, and social studies-- have drafted new K-12 academic standards for the district. This is in compliance with State of Wisconsin Statute 118.30 and in support of district efforts to set high standards for student attainment. In October, drafts of the standards were presented to the Board for its review, as a step toward formal approval. The standards were approved by the Board in November. These standards will define what Milwaukee Public School students are to know and be able to do at each grade level. They are to be a driving force behind curricular as well as assessment change in the district.  

In October, 1997, the Wisconsin Legislature passed State Statute 118.30, which required all districts to either adopt the state standards in language arts, mathematics, science, and social studies, or to develop their own, by August, 1998. MPS chose to develop its own academic standards. Work leading to the specification of the MPS standards began as early as 1994 and was derived from the district’s K-12 teaching and learning reform initiative. This initiative set broad educational goals for the district. Thus, the district’s standards development process coincided with the state’s efforts. The district established guidelines in the form of standards. Writing committees considered standards developed by national groups such as the National Council of Teachers of Mathematics, the National Research Council, the National Council of Teachers of English, and other professional groups. Along with standards documents from these groups, the MPS development teams consulted drafts of the state standards. In the summer and fall of 1998, committees of 20 to 30 teachers prepared the version of the MPS Standards presented to the Board for adoption.  Over the course of four or five years, up to 90 teachers and district staff for each content area have actively engaged in developing grade-level expectations and standards. These groups had some autonomy in how they created their respective subject/grade level and course standards and/or expectations. 

Language Arts and Reading 

The organizational form of the October 6, 1998 draft of the MPS Standards varies by content area. The MPS language arts/reading standards, K-12, are organized by  headings that, in general, correspond to those used by the state—reading/literature, writing, oral language, language, and research and inquiry (grades 6-12 only). Grade-level expectations are listed under each heading that provide more specific statements of what students are to know and do. For example, in language arts for grades 6-12 “grade-level expectations” are described for grade ranges of two grades. Grades 11-12 have expectations that are different from grades 9-10, which are different from those for grades 6-8. The elementary (K-6) language arts/reading standards use the terms “standards and grade-level expectations” as the general headings. These “standards and grade-level expectations” are specified for each grade level, rather than for a grade range of two to four grades. Even though most headings in the MPS standards correspond to headings in the 1998 Wisconsin Model Academic Standards, the draft of the MPS language arts/reading standards do not include a section on media and technology.  On the other hand, MPS specified multicultural standards are only generally referenced in the state standards. The MPS standards provide more detail than the state standards for all four content areas and specify benchmarks that students are expected to achieve at three points during their schooling career—at the end of grades 4, 8, and 12—not by grade level. These state standards are prefaced by the proviso, “By the end of grade four, students will . . .”  

Social Studies 

For kindergarten through grade 8, the MPS social studies standards are organized by “standards and grade-level expectations” for each grade (“By the end of ___ grade students will . . .”).  However, the elementary teachers’ writing team differed from the middle school writing team in approach. The K-5 social studies standards specified what students are to achieve by the end of each grade. The grades 6-8 standards describe expectations for the progress students are to make in order to meet the grade 8 standards. The high school writing team specified expectations for each grade, 9-12, in social studies courses. They decided to do this because there could be no  assurance that all of the social studies courses would be available to all of the students. The high school social studies standards are course-specific—e.g., “By the end of the study of world geography, students will . . .” There are, however, some common expectations of all students by the end of grade 12.  

Science 

Both grade-range standards and grade-level expectations form the “Science Curriculum Framework” for grades K-12. The standards for grade ranges K-4, 5-8, and 9-12 are specified for broad categories of physical, earth and space, life and environmental, science and society, and science process studies.  For grades K-8, learning expectations are specified for each grade. For grades 9-12, learning expectations are specified by course—integrated science (grade 9) and biology (grade 10)—and high school science processes. The science framework expresses standards as “concepts” students will engage in studying (e.g., properties of earth materials, physical events, changes in substances/objects etc.). Then the grade-level learning expectations for each grade or course identify factual information (e.g., water is found in many different contexts, rocks and sand have weight, there are basic colors that can be identified, etc.); processes (e.g., design and perform a controlled experiment); and understandings (e.g., understand the way in which scientists of diverse cultures, ethnicities, and genders have contributed to science) for which students are held responsible. The organization of the state science standards differs from that of the MPS science standards. They specify standards for eight broad headings: science connections, nature of science, science inquiry, physical science, earth and space science, life and environmental science, science applications, and science in personal and social perspectives. 

The evolution of the MPS Science Curriculum Framework illustrates how standards have developed from efforts expended over a number of years. In 1993, a science committee was formed that included about 60 K-8 teachers, 26 high school teachers, and others. Prior to this, the Milwaukee Science Materials Center, established in the late 1980s, had provided science kits to elementary teachers for each activity in the grade-level textbooks, a support teacher who was on-call to work with teachers in their classrooms, and a supervisor who provided inservices by grade level. In 1992-93, an evaluation under the coordination of the science specialist was conducted of the district’s science program prior to a K-5 textbook adoption in 1993-94.  The Addison-Wesley series was adopted, in part, because it provided an inquiry-based approach to teaching science. At the time, professional development programs emphasized such teaching strategies as appropriate. However, the dominant approach used by MPS elementary teachers was activity-based, where students would carry out a sequence of steps. Through inservices, teachers learned to use one or two general kits, but teachers failed to become effective users of the kits because they did not organize their teaching around big ideas that could be developed through use of a range of kits. The science curriculum specialist  interviewed teachers, observed classrooms, and administered surveys to better understand how elementary teachers were teaching science, how their teaching supported students actively engaged in learning science, and what teachers’ professional development needs were. 

The science committee prepared grade-level expectations and then aligned the science kits with the expectations for each grade, K-5. Together, the grade level expectations and science kits—balanced among life, physical, and earth science—represent the core content knowledge for students. The district was provided funds by the National Science Foundation to create the Milwaukee Urban Systemic Initiative (MUSI) in 1996-97. The goals of this initiative were compatible with the ongoing MPS work in science at the time, improvement in science and mathematics achievement. The Board’s adoption of proficiency requirements for grade 8 students and the implementation of performance assessments increased the attention that more resistant teachers gave to implementing the science grade level expectations and kits. The MPS science committee prepared curriculum modules aligned with the grade 8 science proficiencies. These modules were piloted for the first time in the summer of 1998. The current school year, 1998-99, is the first year for the full implementation of the modules. Middle schools have been grouped into seven clusters at ten schools. Those teaching science in all of the middle schools are to meet once a month, beginning in October, with those from the other schools in their clusters. At these meetings, teachers are expected to discuss the implementation of the prepared modules. These discussions are to be facilitated by three trained teacher-facilitators for each cluster, one for each grade 6, 7, and 8. The mathematics teachers are engaging in the same process, with monthly meetings of school clusters. The cluster design is being employed to reach all middle school science and mathematics teachers in the district. The facilitators and mathematics and science resource teachers (MSRT—funded through MUSI) provide a communication channel between teachers and the mathematics and science curriculum specialists. 

For the high school science programs, the department chair is the conduit of information for the science specialist. Recent changes in the high school science curriculum includes the development of a grade 9 integrated science course that exposes students to life, physical, and earth sciences. Ongoing issues in the MPS high school science program include establishing the balance between presenting science as a fixed body of knowledge, where students do prescribed activities to reveal specific results, and science presented as a means for understanding the world, where students design their own experiments to understand better the process of science. 

Mathematics 

The structure of the mathematics standards and grade-level expectations is similar in form to the structure used in science. The same development and implementation process was employed for mathematics as for science. Standards are specified for grade ranges K-4, 5-8, and 9-12 for broad categories of processes, algebraic reasoning, geometry and measurement, numeracy, and probability and statistics. Standards indicate what context and what concepts should be engaged in learning for each grade range. The grade level expectations specified for each grade specify processes and skills students are to perform and what understandings they are to develop. The organization of the MPS mathematics standards is very comparable to the organization of the state mathematics standards with two notable exceptions. Instead of one standard for geometry and measurement, the state separates these two areas into two separate standards. The state also labels numeracy and algebraic reasoning differently from the labels used in the MPS Standards. Instead of numeracy, the state standard is labeled “Number Operations and Relationships,” and instead of algebraic reasoning, the state standard is labeled “Algebraic Relationships.” 

IV. Current Status of the MPS Assessment System 

Milwaukee Public Schools, since the February 28, 1996 adoption by the Milwaukee Board of School Directors of new, higher graduation requirements, has developed an assessment system for the district that incorporates multiple measures of student learning. In addition to requiring students in the high school graduation class of 2004 to pass assessments in writing and mathematics, Middle School Proficiencies have been set that this same cohort of students is required to pass for promotion to grade 9.  These proficiencies, effective for the first time with grade 8 students in the 1999-2000 school year, will require students to demonstrate an acceptable level of accomplishment in communication, mathematics, science, and research. In addition to the district’s requirement, state law requires that, beginning September 1, 2002, students pass a high school graduation test, or that students who have been excused by the Board meet alternative criteria, in order to receive a high school diploma. 

The current district assessment program consists of state-mandated tests, proficiency assessments, performance assessments, and portfolios. Students in grades 4, 8, and 10 are required by the state to take the Wisconsin Student Assessment System  (WSAS) Knowledge and Concept Examinations. Grade 3 students are required to take the Wisconsin Reading Comprehension Test (WRCT). In addition to these assessments, an MPS mathematics proficiency assessment and a writing proficiency assessment are administered to students in grades 11 and 12 as a high school graduation requirement. MPS performance assessments are currently given in writing, science, fine arts, and oral communications. In the spring, students in grades 4 and 5 are required to write an essay to a specific prompt. Science performance assessments are administered to students in grades 10 to 12, grade 9, and grade 5. Each high school has a plan that assesses about one-third of students in that grade range each year. Each school is required to administer either a fine arts assessment or an oral communication assessment. High schools and middle schools determine when the fine arts or oral assessments are administered. Elementary schools are to administer these assessments to students in grade 4 or 5. Portfolio assessments can be completed in writing and mathematics as an alternative means for meeting the District graduation requirement by students who do not pass the proficiency assessments in these two content areas. 

State-Mandated Testing

Since the 1993-94 school year, the Milwaukee Public Schools, and all other school districts in the state, are required to administer the Wisconsin State Assessment System’s Knowledge and Concepts Examinations (TerraNova, developed by CTB/McGraw Hill, has been used since 1996-97) at grades 4, 8, and 10. These tests measure achievement in the areas of reading, language arts, mathematics, social studies, science, and writing.  All students in grades 4, 8, and 10, except those exempt under guidelines for Exceptional Educational Needs and limited-English speaking, are required to take these tests which are administered during the spring semester of each academic year.[1]

Student scores for the WSAS Knowledge and Concepts Examinations are reported by four proficiency categories:

            Advanced: Distinguished in the content area. Academic achievement is beyond mastery. Test score provides evidence of in-depth understanding in the academic content area.

            Proficient:  Competent in the content area. Academic achievement includes mastery of the important knowledge and skills. Test score shows evidence of skills necessary for progress in the academic content area tested. 

            Basic:  Somewhat competent in the content area. Academic achievement includes mastery of most of the important knowledge and skills. Test score shows evidence of at least one major flaw in understanding the academic content area tested. 

            Minimal Performance: Limited in the content area. Test score shows evidence of major misconceptions or gaps in knowledge and skills basic to progress in the academic content area tested. 

A recent alignment study, conducted by the Wisconsin Department of Public Instruction under the guidance of CTB/McGraw Hill staff, found the Knowledge and Concepts Examinations in all four subject areas (reading/language arts, mathematics, science, and social studies) to be “aligned” (at or above 97% average agreement for each grade level test) with the 1998 Wisconsin Model Academic Standards. Alignment in this study was determined if each test item corresponded to a standard from the Wisconsin Model Academic Standards. What constituted a “match” was not described in the report. Multiple matches (i.e., when a TerraNova question matched multiple standards) were not indicated. The percent agreement did not take into consideration standards not assessed by any test item. For example, the study found that about half of the Wisconsin Model Academic Standards for English Language Arts could not be assessed with a large-scale, paper-and-pencil test. 

MPS is required by the state to administer the Wisconsin Reading Comprehension Test (Office of Educational Accountability, Wisconsin Department of Public Instruction) to all grade 3 students. This state mandated standardized achievement test is used to determine the level of reading proficiency of third grade students.  The state requires districts to provide remedial service for pupils who do not score above the performance standard on the Wisconsin Reading Comprehension Test.[2]  No reports have been made available on the alignment between the WRCT and the Wisconsin Model Academic Standards.  

MPS Performance Assessments  

The mathematics proficiency test that students take in grades 11 and 12, and is required for graduation, has two sections. Six problems, worth 2 points each, assess knowledge and skills. Two problems, worth 12 points each, assess problem solving and mathematical thinking. A minimum score for passing is 5 of 12 possible points on the knowledge and skills questions and 11 of 24 possible points on the problem-solving questions. 

The writing proficiency test that students take in grades 11 and 12 and is required for graduation has two sections. On one section, worth 8 points, students are required to write a letter applying for employment; on the second section, worth 16 points, students are required to write an essay to a specific prompt in a specific genre. Students are required to achieve a total of 15 points on both parts of the assessment to pass. 

The science performance assessment that high schools administer to selected students in grades 10-12 consists of three problems, worth 8 points each. Students need to achieve a minimum of 16 points to pass. Students do not need to pass this test in order to graduate. 

Each elementary, middle, and high school administers either an oral communication or fine arts performance assessment to at least one grade level as designated by their building staff and principal. The assessment(s) may be administered at any time during the school year and all oral communication assessments are video taped and scored by the student’s classroom teacher.  There are no stated consequences associated with these assessments.   

Currently, MPS Middle Schools administer a district-created MPS Writing Performance Assessment, MPS Science Performance Assessment, and MPS Mathematics Assessment.  Each of these tests is administered in grades 6 (spring) and 7 (fall and spring).  They are scored by the district, using MPS teachers as scorers.  There are no stated consequences associated with these assessments. 

V. The New MPS Assessment System

MPS is preparing to implement middle school proficiencies in four areas, Communications, Mathematics, Science, and Research. Grade 8 students in 1999-2000 will be required to attained a passing score on these requirements in order to be promoted to grade 9. These proficiencies will be in addition to students in grades 4, 8, and 10 taking the WSAS Knowledge and Concepts Examinations and students in grade 3 taking the WRCT. The middle school proficiencies allow students some choice in the assessment tool used and the date the proficiency is demonstrated (see attached tables).  

Communications  

In communications, middle school students must demonstrate proficiency in three areas: writing, reading, and oral communication.  In writing, all eighth grade students must produce four different samples of their writing (imaginative, expository, persuasive, and narrative). They may select these samples from their work in grades 6, 7, or 8.[3] 

In addition, middle school students will be required to take a formal writing assessment from among three alternatives. They will be able to demonstrate their writing proficiency on the MPS Writing Performance Test. This test will be administered in the fall and spring and will be available to students in grades 7 and 8. Or, students use the results from the WSAS that is administered in the spring for students in grades 7 and 8. Or, if students have not yet demonstrated proficiency in writing by the spring of grade 8, they may submit a portfolio of their writing to the MPS Curriculum and Instruction office for review. Under district guidelines students must be given at least three opportunities to meet each proficiency, whether administered by classroom teacher, school, or district. 

For reading, each student by the end of grade 8 must demonstrate proficiency on a formal reading assessment and receive an acceptable rating by a teacher on the MPS Reading Assessment/Instruction Card.[4] To meet proficiency, a student must attain a seventh grade reading level as measured by the Gates MacGinite, Metropolitan, Woodcock, S.T.A.R., or Jerry Johns tests. 

Each student must also prepare and present a 3- to 5-minute video-tapped speech for evaluation. Classroom teachers will judge whether students have met this requirement using a five-point rubric.[5] 

Mathematics  

For mathematics, by the end of grade 8 students will have developed a portfolio with certain specifications. They must demonstrate their understanding of a range of algebra topics by including in their portfolio five examples of their work. What the five examples should be and how the quality of work will be scored, as of October, 1998, was under development.  

Also, students will need to include in their mathematics portfolio one of two alternatives to demonstrate their proficiency in passing an on-demand mathematics assessment. They either need to satisfactorily pass the MPS mathematics performance assessment or the WSAS Knowledge and Concepts Examination on mathematics.[6]  

As a third requirement for demonstrating their proficiency in mathematics, by the end of grade 8 students will have to satisfactorily complete a project such as a scale model or package design that demonstrates understanding of measurement and geometric relationships.[7] 

Science  

In science, by the end of grade 8 students are to satisfactorily complete three requirements. They are to develop a portfolio that contains selections of their work that demonstrate their understanding of content-rich, hands-on, inquiry-based science content. Their work must provide evidence of growth in their knowledge and understanding from grades 6 through 8. They must successfully communicate their reasons for including the selections they chose and how these selections demonstrate the understanding of science they have gained. Types of selections include embedded assessments, presentations, investigations, reflective writing, conceptual framework module tests, and self-evaluations.[8] 

For science, students also will be required to achieve a proficient rating on one of two alternative on-demand science assessments.  One alternative is the MPS science performance assessment. The other alternative is the WSAS Knowledge and Concepts science test for grade 8. Students who do not have a proficient score on either of these assessments can fulfill this requirement by either successfully writing an essay, given a prompt in an experimental context, or answering specific questions from a panel of science personnel regarding a prompt from an experimental context.[9] 

The third science requirement is for students to create a science project that shows the student understands content, can use and apply the scientific method, can use technology and scientific equipment, and is able to explain how the project applies to the real world.[10]

Research  

By the end of grade 8, students must write a research paper and orally present the paper in communications, mathematics, and science. In the paper, the student must focus on a project that uses statistical data to analyze a problem, interpret and report the results, document the sources used, and follow a standard format. Students can also meet multiple proficiency requirements with one project.  For example, one research investigation could count for the communications research paper, the science research presentation, and the mathematics investigation, if it meets the criteria in all areas. 

New High School Graduation Assessments  

The 1997 Wisconsin Act 27 (sections 118.30 and 118.33) mandates that “each school board operating high school grades shall adopt a high school graduation examination that is designed to measure whether pupils meet the pupil academic standards adopted by the school board . . . beginning in the 2000-01 school year.”  Beginning on September 1, 2002, a school board may not grant a high school diploma to any student who has not passed the high school graduation test.  If a student fails the test, DPI has stated that the student must be provided with at least three opportunities to re-take the test.  It is state-mandated policy that a school district may not use the state graduation test unless their board has adopted the 1998 State Standards; therefore, MPS is required to develop its own graduation test or secure a test from an outside vendor.   

In addition to the above high school graduation test, the Milwaukee Board of School Directors, at its February 28, 1996 meeting, adopted requirements for the graduating class of 2004.  Those students will be required to demonstrate mastery in the following areas before graduation: 

Mathematical Reasoning

Students will demonstrate mastery of mathematical proficiency equivalent to three years of study beyond Algebra 1.  Students will be expected to show proficiency in first-year algebra by the end of eighth grade. 

Scientific Reasoning

Students will demonstrate a high level of proficiency in science, equivalent to three years of high school study, to include the physical, biological, and chemical sciences.  Students will demonstrate an understanding of scientific inquiry and application to real-life situations. 

Communication

Students will demonstrate mastery of written and spoken expression by writing, presenting, and defending a clearly reasoned, persuasively argued research paper. 

Community Membership

Through participation in a group project that benefits the community, students will demonstrate the capacity to analyze a social issue from multiple points of view and to interact as a constructive member of a team. 

Year

MPS Middle School Proficiencies

State Mandated Graduation Test

(WSAS or local/district developed test) 4 opportunities during H.S.

WSAS

1998-1999

 

 

 

1999-2000

All 8th Grade students must have demonstrated proficiencies for promotion to 9th Grade

 

 

2000-2001

 

All 12th grade students must pass a graduation test

10th Grade testing terminated after the academic year

  Figure 5.  Schedule of state- and District-mandated tests for MPS.    

Exceptional Education Needs  

            The new middle school proficiencies and the high school graduation requirements apply to all MPS students. Special accommodations will be allowed for exceptional education students to take the test. However, at the present time, there is significant discrepancy between the goal of having students with disabilities fully participating in the regular curriculum and their participation in the assessments.  Some of the barriers to preventing full participation include the historical paradigms of individualized education (in which special educators generally created entirely separate curricular goals and objectives, and often entirely separate teaching materials, for students with disabilities), lack of knowledge among special educators regarding general education curriculum and assessments, and some significant institutional barriers with respect to inclusion of students with disabilities in regular classrooms. 

            With respect to the “paradigm shift” needed to include students with disabilities in the regular education curriculum and in general education assessments, one intended strategy is for the special education staff to attend inservice presentations for general educators regarding curriculum and assessment. However, there is a question whether this initiative carries sufficient resources and incentives to: 1) reach a significant proportion of  special education staff; and 2) provide that staff with adequate information. In particular, this is a problem because participation in such inservices is not required, nor are additional resources, such as substitute teachers, made available to encourage participation. 

            With respect to institutional barriers to inclusion, the primary barrier identified is the potential difficulty with change in teacher roles, particularly as mediated by unions. That is, the Milwaukee Teachers’ Education Association (MTEA) generally resists inclusion of students with disabilities in general education classrooms, since they see it as an additional burden on general education teachers.  Furthermore, some special educators resist the change to a decentralized model, which requires that they provide services in the regular classroom (this also has been grieved as a change in role by special educators). The discussion on this issue is characterized by open dialogue with union officials, but thus far, there has been little progress. 

One of the strongest institutional supports for inclusion of students with disabilities is the flexibility of the current assessment structure.  That is, all students (including those with disabilities) are provided a variety of methods and multiple opportunities for demonstrating competence.  Furthermore, the assessment staff’s knowledge of assessment accommodations/modifications is strong, suggesting that Milwaukee is relatively further along than many other districts in its efforts to accommodate students with disabilities within district (and state) assessment systems.  However, it is not clear that in Milwaukee students with disabilities are being exposed accordingly to the general education curriculum. It is clear that the assessment systems in place have a good or sound structural base within them for accommodating individual differences in students with disabilities, but all children with special needs are not yet   fully included in the general education classroom and curriculum. 

VI. Design of Proposed Alignment Study  

We propose to continue our study of alignment within Milwaukee Public Schools by doing a series of alignment studies. The intended purpose of these studies is to reveal how Milwaukee Public Schools is functioning as an aligned standards-based system. The higher the degree of alignment that can be reached with the standards to be adopted, the proficiencies, the assessments, the curriculum, and classroom practices, the greater is the likelihood that students in the system will meet the challenging knowledge criteria required by the standards. 

The series of coordinated alignment studies will answer the following questions: 

1.      Are MPS curriculum standards in language arts/reading, mathematics, science, and social studies aligned with the district’s assessment system and do they reflect the level of expectations advanced by state and national groups?

2.      Do students have the opportunity to learn from grade to grade what is required by the MPS standards and proficiencies so they are able to satisfactorily achieve these standards and proficiencies?

3.      Do students have the opportunity to receive the instructional experiences from grade to grade necessary for them to achieve the proficiencies and standards?

4.      Do teachers have the pedagogical content knowledge, time, resources, and information on students’ progress necessary if they are to provide students with the instructional experiences needed for students to achieve the proficiencies and standards?

5.      Are teachers given the support, professional development opportunities, and incentives from principals and district administration that will enable them to provide their students with the instructional progression from one grade or class to the next that is necessary if students are to achieve the standards and proficiencies? 

We propose that parts of the alignment studies be conducted concurrently and not in sequence. Even though the below studies are numbered, the numbers do not signify the order in which the study will be initiated or completed. An important pre-condition of these studies, of course, is the availability of standards in the four content areas. 

Study 1: Alignment of the Standards, Assessments, and Proficiencies 

Study 1 will be a content analysis of the MPS Academic Standards in language arts/reading, mathematics, science, and social studies compared with state assessment instruments administered through the district, or to meet district requirements and other proficiencies requirements. A panel of four reviewers for each grade level and content area will first analyze the standards by rating the required depth of knowledge of each. Then the panel will match and code assessment items and proficiencies requirements with the appropriate standards. The data from the review panels will be analyzed and reports will be written that will describe the degree of agreement among documents by category, depth of knowledge, range of knowledge, balance, articulation across grades, equity, and pedagogical implications. Each panel will include two raters from within MPS and two external raters. 

Study 2: Alignment of Curriculum with Standards, Assessments, and Proficiencies 

Study 2 will analyze curricula from a random sample of schools. Four schools at each of the two levels—elementary and middle schools—will be randomly selected to analyze the curricula in the four content areas at those schools. The curriculum materials will be blocked by topic and compared to the MPS Academic Standards, assessments and proficiencies. The progression of curriculum materials in two grade ranges—2, 3, and 4 for elementary and 6, 7, and 8 for middle—will be analyzed. The curriculum topic blocks will be analyzed by content coverage and depth of knowledge and compared to MPS standards, assessments, and proficiencies. Reports will be prepared that analyze the curriculum to determine the opportunity students have to develop the knowledge required by the MPS Academic Standards, which will be measured on assessments and through proficiency requirements at critical points in their schooling career—grades 4 and 8. 

Study 3: Alignment of Classroom Practices and Teacher Capacity with Standards, Assessments, and Proficiencies 

Study 3 will analyze classroom practices, teachers’ knowledge of the standards and proficiencies, and teachers’ knowledge of pedagogy, as well as students’ development of the content knowledge required for them to successfully meet the standards and proficiencies. Data will be collected in eight schools randomly selected for Study 3. In each school, the instructional experiences students have in two or three grades will be analyzed in relation to the proficiencies/graduation requirements, assessments, and standards. Data will be collected within a school from a cluster of teachers, including at least one or two who are teaching the subject matter to students at each grade level taught in the school. The purpose of the clustering of teachers across grades is twofold: to study the way in which students’ knowledge builds from one grade to the next and to avoid putting a burden on any one teacher. Data will be collected to determine the teachers’ awareness of what their students are to know in order to achieve stated proficiencies and graduation requirements; what their students know when they enter their classrooms; and what instructional activities they have to provide their students in order for them to build on their current knowledge of subject matter and to determine what they are learning.  In a sample of classrooms, observers will be present to describe classroom practices, the coverage of content, the pace of instruction, and the monitoring of students’ learning by teachers as this correlates with students’ achievement of grade-level proficiencies and requirements. Teachers will be interviewed and asked to complete a questionnaire to determine how their planning and pedagogical or content knowledge relates to students’ attainment of the proficiencies and requirements. One area to be studied will be the communication among teachers and other linkages across grade levels that are factors in enabling students to reach the knowledge level necessary for them to demonstrate the proficiencies specified. 

Study 4: Alignment of School and District Support of Classroom Practices with Standards, Assessments, and Proficiencies 

Study 4 will gather and analyze data on school and District variables that provide teachers with the support in developing instructional practices that enable their students to successfully attain the targeted proficiencies and requirements. School principals and other staff will be interviewed and asked to complete questionnaires on their initiatives in support of improved learning. Information will be sought on specific key questions: 

1.      What leadership is provided by the principal and others for having students achieve the proficiencies?

2.      What support is provided for teachers to work collaboratively, or in concert with each other, to provide students with a seamless learning experience?

3.      How are needs for special groups of students met, including transfer students, special education students, students with a high number of absences, and limited-English- speaking students? 

Information will be sought from appropriate District and school staff to determine what District support is given to schools in a number of areas: 

1.      What support is given schools by the District in the curriculum areas to assure that teachers fully understand what students are required to achieve?

2.      What support does the District give the schools that reinforces the efforts of the teachers to provide the instructional experiences their students need to attain these requirements?

What efforts are made by the District to provide teachers with the professional development they need if they are to continue to improve in their content  knowledge and in their ability to work with students in attaining the proficiencies and meeting requirements? 

VII. Conclusions

We have made some observations about standards, assessments, and alignment. At this time we do not have the information to judge the full alignment of the MPS system. The studies proposed are designed to produce such an analysis for K-8. Our work to date has revealed findings of some potential issues and problems: 

1.                  Creating and developing a consistent format for state of standards across content areas: Grade-level expectations are used consistently across the content areas, but standards are not. The standards listed for science and mathematics state what students should be engaged in learning over a grade range, K-4, 5-8, 9-12. Grade expectations are used to express what students are to know and be able to do.  In social studies, grade-level expectations are grouped under headings called standards. Thus, standards represent an organizing label for expectations. The lack of consistency in format may cause difficulty in teachers’ clear understanding of what they are to do. The inconsistency in format has the greatest potential of being a problem in the elementary grades, where teachers are required to teach and attend to standards for more than one subject area.

2.                  Creating and developing a consistent format for rubrics: Nearly all rubrics have five points, including 0 for not scorable: Many are labeled as Advanced, Proficient, Basic, and Minimal Performance and correspond to the rubric used by the state in rating the results from the WSAS examinations. Assuring that these rubrics are consistent across the content areas will help to assure they are reliability applied. A generic rubric  that defines each of these levels would be helpful. Then content-specific rubrics that identify the detailed criteria for each content area and instrument could be developed based on appropriate language drawn from the generic rubric. The principle is for MPS teachers, students, administrators, and parents to become very familiar with what proficient academic work means.

3.                  Equating alternative forms of proficiencies and verifying reliable scoring by different raters: The Middle Schools Proficiencies provide for students to use different forms of assessment to meet specific requirements. There is a need to show that alternative forms for meeting the same requirement are equivalent in value and that they provide fair and equitable means for students to demonstrate their proficiencies. At a minimum, a content analysis should be made of alternative forms to determine whether they are comparable. Ideally, a study needs to be conducted in which a group of students takes all alternative forms to determine whether in fact students get comparable scores on both forms. Classroom teachers will be responsible for scoring some of the proficiencies. The consistency among teachers for doing this needs to be examined. A moderation or back-up process may be important to assure that if two or more teachers score a student’s work, there would be agreement in the scoring.

 4.                  Assuring balance among multiple proficiency criteria and articulation of standards across grades: When three or more criteria are required for students to meet the proficiencies, each criterion is given equal weight. In mathematics, for example, students are to show understanding of algebra, pass an on-demand assessment, and produce a scale model. The weighing of these should be aligned with the standards and the curriculum. These criteria also should be shown as important for achieving the expected level of mathematical understanding in high school.  

References

Webb, N. (1998).  Criteria for alignment of expectations and professional development in mathematics and science education.  Paper presented at the American Educational Research Association annual meeting, San Diego, CA. 

Webb, N. (1997). Criteria for alignment of expectations and assessments in mathematics and science education. Council of Chief State School Officers and National Institute for Science Education Research Monograph No. 6. Madison, WI: University of Wisconsin.

 

FOOTNOTES

[1]  Roughly 75 percent of each test contains multiple-choice items and 25 percent constructed-response, or short-answer, items.  Each student also completes an on-demand writing essay after reading a short passage about the assigned topic.  The multiple-choice items are machine-scored. Trained scorers hired by the testing company score the short-answer items. Each item is rated by one professional reader and is assigned a specific performance level.  Responses on the short-answer items are scored using a two-point scale (0-1) up to a five-point scale (0-4). The written essays are scored holistically.  Each essay is rated by two professional readers each who each independently assigns a rating using a six-point scale (1 to 6). The ratings of the two readers are averaged to produce a single score.  If the readers’ ratings differ by more than one point, a third reader assigns an independent rating.  The reported holistic score is then the average of the two closest scores.  The total WSAS test time for all five content area tests is approximately six hours.  

[2]  The test is administered in three sessions, each session lasting approximately one hour, and consists of three reading passages, two narrative stories of about 1,200 words each, and one expository report of about 700 words.  In addition to 60 comprehension questions, students are asked a number of questions that measure reading strategies and prior knowledge.  There are a total of 100 questions on the test, though only scores on the 60 comprehension questions are used to establish a performance standard.  Results from the remaining 40 questions are used locally to interpret comprehension scores.  All answers to questions are in a multiple-choice format.  However, the 1998-99 test will include one question that requires a short answer.

[3]  The quality of the student’s writing will be judged by the classroom teachers using a five- point rubric (4—well organized, imprinted personality, effective word choice, sentence fluency, and correct mechanics to 1—unclear, unorganized, no personal relationship, incorrect language usage, lack of sentence sense, and incorrect mechanics, and 0 for blank, illegible, or off the point). 

[4]  The Assessment/Instruction Card is a compilation of 11 good reading behaviors such as preview text, build background for new concepts, and set a specific purpose for reading. A student will be evaluated on the Assessment/Instruction Card requirement by the student’s teachers on whether he or she has met the behavior standards, using a five-point rubric (advanced/exceeding (4), proficient (3), approaching (2), beginning (1), and not scorable (0)). 

[5]  At the highest level 4, students must have eye contact and facial expressions that enhance and support the purpose; poised gestures; skillfully used speech patterns; a well-established intent; and effective use of audio/visual aids. At a level 3, students must have eye contact and facial expressions that support the purpose; natural and appropriate gestures; appropriate speech patterns; an established intent; and appropriately used audio/visual aids. Level 2 represents behaviors that are inappropriate or ineffectual. Level 1 represents behaviors that are limited, inappropriate, distracting, and/or not clearly established. The level 0 is used to indicate the student refused to do the oral presentation. 

[6]  A five-point rubric (4-0), with 0 indicating there was no attempt or the response was off the point or illegible, will be used to score student work on the performance assessment. Student work is judged on five attributes to determine the level at which the student is performing: strategy, organization, communication, solution, and mathematics. For the highest level, level 4, a student must effectively use an appropriate strategy; have a highly organized and well-documented response; give effective and appropriate responses to all parts of the question; provide a correct solution; and use efficient and accurate mathematics principles throughout. For a level 3, the students are to meet similar criteria but can have a minor flaw in applying the mathematics. A level 2 represents the student’s strategy as incomplete; the response is poorly organized and vague; the solution is only partially correct; and inappropriate or frequently flawed mathematics is used. For a level 1, students demonstrate an unclear, or lack of, strategy; disorganized and random work, or only a restatement of the problem; and little or no use of the appropriate mathematics. 

[7]  To fulfill this requirement, students are to generate a question, problem, or situation that can be represented by a scale model, or that requires a package design. They need to show a capacity to make consistent scale adjustments throughout the project; be able to explain their reasoning for using selected scales; and demonstrate understanding of measurement, proportional reasoning, and geometric relationships. They are to either write or videotape their conclusion or resolution showing their understanding of the mathematics. Work on students’ projects will be rated by their classroom teachers using a five-point rubric similar to the one used on the WSAS examinations: advanced, proficient, basic, minimal performance, and not scorable. 

[8]  A five-point rubric similar to that used to score students’ work on the mathematics performance assessment will be used to score the students’ portfolio. Students’ work will be judged on five attributes: critical thinking, science process, science content, transfer of knowledge, and communications. Advanced level (4) requires students to show evidence of original and insightful thinking, unique insights, in-depth understanding, transference of knowledge to other situations, and clear communications. Proficient level (3) requires evidence of appropriate thinking, understanding, application of concepts, accurate transfer of knowledge to real-life situations, and clear communications. For a basic level (2), a student only provides some evidence of his/her thinking, demonstrates partial understanding, and produces an incomplete response. A minimal performance level (1) includes very little evidence of appropriate thinking and understanding, unsuccessful use of process skills, no evidence of transfer of knowledge, and unclear or ambiguous communications. Not scorable is level 0. 

[9]  The five-point rubric used by the WSAS will be used to score work for this requirement. 

[10]  A five-point rubric similar to one used by the WSAS will be used to score students’ work on their projects

Information Staff Reports Links