|
III. Current Status of Milwaukee Public Schools’ Standards and
Expectations Our work through October, 1998,
in understanding the District’s alignment clearly reveals that Milwaukee
Public Schools has made advances towards becoming a standards-based system. On
February 28, 1996, the Milwaukee Board of School Directors adopted new
graduation requirements, as well as a series of high stakes middle school
proficiencies. These proficiencies have been specified and efforts have been
made to inform teachers, parents, and students what grade 8 students in the
school year 1999-2000, and beyond, will be expected to do. Staff members have
been appointed to coordinate tasks related to informing parents and students
about the requirements. Committees of teachers in four content areas--language
arts/reading, mathematics, science, and social studies-- have drafted new K-12
academic standards for the district. This is in compliance with State of
Wisconsin Statute 118.30 and in support of district efforts to set high
standards for student attainment. In October, drafts of the standards were
presented to the Board for its review, as a step toward formal approval. The
standards were approved by the Board in November. These standards will define
what Milwaukee Public School students are to know and be able to do at each
grade level. They are to be a driving force behind curricular as well as
assessment change in the district. In October, 1997, the Wisconsin
Legislature passed State Statute 118.30, which required all districts to
either adopt the state standards in language arts, mathematics, science, and
social studies, or to develop their own, by August, 1998. MPS chose to develop
its own academic standards. Work leading to the specification of the MPS
standards began as early as 1994 and was derived from the district’s K-12
teaching and learning reform initiative. This initiative set broad educational
goals for the district. Thus, the district’s standards development process
coincided with the state’s efforts. The district established guidelines in
the form of standards. Writing committees considered standards developed by
national groups such as the National Council of Teachers of Mathematics, the
National Research Council, the National Council of Teachers of English, and
other professional groups. Along with standards documents from these groups,
the MPS development teams consulted drafts of the state standards. In the
summer and fall of 1998, committees of 20 to 30 teachers prepared the version
of the MPS Standards presented to the Board for adoption.
Over the course of four or five years, up to 90 teachers and district
staff for each content area have actively engaged in developing grade-level
expectations and standards. These groups had some autonomy in how they created
their respective subject/grade level and course standards and/or expectations. Language
Arts and Reading The organizational form of
the October 6, 1998 draft of the MPS Standards varies by content area. The MPS
language arts/reading standards, K-12, are organized by
headings that, in general, correspond to those used by the
state—reading/literature, writing, oral language, language, and research and
inquiry (grades 6-12 only). Grade-level expectations are listed under each
heading that provide more specific statements of what students are to know and
do. For example, in language arts for grades 6-12 “grade-level
expectations” are described for grade ranges of two grades. Grades 11-12
have expectations that are different from grades 9-10, which are different
from those for grades 6-8. The elementary (K-6) language arts/reading
standards use the terms “standards and grade-level expectations” as the
general headings. These “standards and grade-level expectations” are
specified for each grade level, rather than for a grade range of two to four
grades. Even though most headings in the MPS standards correspond to headings
in the 1998 Wisconsin Model Academic Standards, the draft of the MPS language
arts/reading standards do not include a section on media and technology. On the other hand, MPS specified multicultural standards are
only generally referenced in the state standards. The MPS standards provide
more detail than the state standards for all four content areas and specify
benchmarks that students are expected to achieve at three points during their
schooling career—at the end of grades 4, 8, and 12—not by grade level.
These state standards are prefaced by the proviso, “By the end of grade
four, students will . . .” Social Studies For kindergarten through grade
8, the MPS social studies standards are organized by “standards and
grade-level expectations” for each grade (“By the end of ___ grade
students will . . .”). However,
the elementary teachers’ writing team differed from the middle school
writing team in approach. The K-5 social studies standards specified what
students are to achieve by the end of each grade. The grades 6-8 standards
describe expectations for the progress students are to make in order to meet
the grade 8 standards. The high school writing team specified expectations for
each grade, 9-12, in social studies courses. They decided to do this because
there could be no assurance that
all of the social studies courses would be available to all of the students.
The high school social studies standards are course-specific—e.g., “By the
end of the study of world geography, students will . . .” There are,
however, some common expectations of all students by the end of grade 12. Science Both grade-range standards and
grade-level expectations form the “Science Curriculum Framework” for
grades K-12. The standards for grade ranges K-4, 5-8, and 9-12 are specified
for broad categories of physical, earth and space, life and environmental,
science and society, and science process studies.
For grades K-8, learning expectations are specified for each grade. For
grades 9-12, learning expectations are specified by course—integrated
science (grade 9) and biology (grade 10)—and high school science processes.
The science framework expresses standards as “concepts” students will
engage in studying (e.g., properties of earth materials, physical events,
changes in substances/objects etc.). Then the grade-level learning
expectations for each grade or course identify factual information (e.g.,
water is found in many different contexts, rocks and sand have weight, there
are basic colors that can be identified, etc.); processes (e.g., design and
perform a controlled experiment); and understandings (e.g., understand the way
in which scientists of diverse cultures, ethnicities, and genders have
contributed to science) for which students are held responsible. The
organization of the state science standards differs from that of the MPS
science standards. They specify standards for eight broad headings: science
connections, nature of science, science inquiry, physical science, earth and
space science, life and environmental science, science applications, and
science in personal and social perspectives. The evolution of the MPS Science
Curriculum Framework illustrates how standards have developed from efforts
expended over a number of years. In 1993, a science committee was formed that
included about 60 K-8 teachers, 26 high school teachers, and others. Prior to
this, the Milwaukee Science Materials Center, established in the late 1980s,
had provided science kits to elementary teachers for each activity in the
grade-level textbooks, a support teacher who was on-call to work with teachers
in their classrooms, and a supervisor who provided inservices by grade level.
In 1992-93, an evaluation under the coordination of the science specialist was
conducted of the district’s science program prior to a K-5 textbook adoption
in 1993-94. The Addison-Wesley
series was adopted, in part, because it provided an inquiry-based approach to
teaching science. At the time, professional development programs emphasized
such teaching strategies as appropriate. However, the dominant approach used
by MPS elementary teachers was activity-based, where students would carry out
a sequence of steps. Through inservices, teachers learned to use one or two
general kits, but teachers failed to become effective users of the kits
because they did not organize their teaching around big ideas that could be
developed through use of a range of kits. The science curriculum specialist
interviewed teachers, observed classrooms, and administered surveys to
better understand how elementary teachers were teaching science, how their
teaching supported students actively engaged in learning science, and what
teachers’ professional development needs were. The science committee prepared
grade-level expectations and then aligned the science kits with the
expectations for each grade, K-5. Together, the grade level expectations and
science kits—balanced among life, physical, and earth science—represent
the core content knowledge for students. The district was provided funds by
the National Science Foundation to create the Milwaukee Urban Systemic
Initiative (MUSI) in 1996-97. The goals of this initiative were compatible
with the ongoing MPS work in science at the time, improvement in science and
mathematics achievement. The Board’s adoption of proficiency requirements
for grade 8 students and the implementation of performance assessments
increased the attention that more resistant teachers gave to implementing the
science grade level expectations and kits. The MPS science committee prepared
curriculum modules aligned with the grade 8 science proficiencies. These
modules were piloted for the first time in the summer of 1998. The current
school year, 1998-99, is the first year for the full implementation of the
modules. Middle schools have been grouped into seven clusters at ten schools.
Those teaching science in all of the middle schools are to meet once a month,
beginning in October, with those from the other schools in their clusters. At
these meetings, teachers are expected to discuss the implementation of the
prepared modules. These discussions are to be facilitated by three trained
teacher-facilitators for each cluster, one for each grade 6, 7, and 8. The
mathematics teachers are engaging in the same process, with monthly meetings
of school clusters. The cluster design is being employed to reach all middle
school science and mathematics teachers in the district. The facilitators and
mathematics and science resource teachers (MSRT—funded through MUSI) provide
a communication channel between teachers and the mathematics and science
curriculum specialists. For the high school science
programs, the department chair is the conduit of information for the science
specialist. Recent changes in the high school science curriculum includes the
development of a grade 9 integrated science course that exposes students to
life, physical, and earth sciences. Ongoing issues in the MPS high school
science program include establishing the balance between presenting science as
a fixed body of knowledge, where students do prescribed activities to reveal
specific results, and science presented as a means for understanding the
world, where students design their own experiments to understand better the
process of science. Mathematics The structure of the mathematics
standards and grade-level expectations is similar in form to the structure
used in science. The same development and implementation process was employed
for mathematics as for science. Standards are specified for grade ranges K-4,
5-8, and 9-12 for broad categories of processes, algebraic reasoning, geometry
and measurement, numeracy, and probability and statistics. Standards indicate
what context and what concepts should be engaged in learning for each grade
range. The grade level expectations specified for each grade specify processes
and skills students are to perform and what understandings they are to
develop. The organization of the MPS mathematics standards is very comparable
to the organization of the state mathematics standards with two notable
exceptions. Instead of one standard for geometry and measurement, the state
separates these two areas into two separate standards. The state also labels
numeracy and algebraic reasoning differently from the labels used in the MPS
Standards. Instead of numeracy, the state standard is labeled “Number
Operations and Relationships,” and instead of algebraic reasoning, the state
standard is labeled “Algebraic Relationships.” IV. Current Status of the MPS Assessment System Milwaukee Public Schools, since
the February 28, 1996 adoption by the Milwaukee Board of School Directors of
new, higher graduation requirements, has developed an assessment system for
the district that incorporates multiple measures of student learning. In
addition to requiring students in the high school graduation class of 2004 to
pass assessments in writing and mathematics, Middle School Proficiencies have
been set that this same cohort of students is required to pass for promotion
to grade 9. These proficiencies,
effective for the first time with grade 8 students in the 1999-2000 school
year, will require students to demonstrate an acceptable level of
accomplishment in communication, mathematics, science, and research. In
addition to the district’s requirement, state law requires that, beginning
September 1, 2002, students pass a high school graduation test, or that
students who have been excused by the Board meet alternative criteria, in
order to receive a high school diploma. The current district assessment program consists of state-mandated tests, proficiency assessments, performance assessments, and portfolios. Students in grades 4, 8, and 10 are required by the state to take the Wisconsin Student Assessment System (WSAS) Knowledge and Concept Examinations. Grade 3 students are required to take the Wisconsin Reading Comprehension Test (WRCT). In addition to these assessments, an MPS mathematics proficiency assessment and a writing proficiency assessment are administered to students in grades 11 and 12 as a high school graduation requirement. MPS performance assessments are currently given in writing, science, fine arts, and oral communications. In the spring, students in grades 4 and 5 are required to write an essay to a specific prompt. Science performance assessments are administered to students in grades 10 to 12, grade 9, and grade 5. Each high school has a plan that assesses about one-third of students in that grade range each year. Each school is required to administer either a fine arts assessment or an oral communication assessment. High schools and middle schools determine when the fine arts or oral assessments are administered. Elementary schools are to administer these assessments to students in grade 4 or 5. Portfolio assessments can be completed in writing and mathematics as an alternative means for meeting the District graduation requirement by students who do not pass the proficiency assessments in these two content areas. State-Mandated Testing Since the 1993-94 school year, the Milwaukee Public Schools, and all other school districts in the state, are required to administer the Wisconsin State Assessment System’s Knowledge and Concepts Examinations (TerraNova, developed by CTB/McGraw Hill, has been used since 1996-97) at grades 4, 8, and 10. These tests measure achievement in the areas of reading, language arts, mathematics, social studies, science, and writing. All students in grades 4, 8, and 10, except those exempt under guidelines for Exceptional Educational Needs and limited-English speaking, are required to take these tests which are administered during the spring semester of each academic year.[1] Student scores for the WSAS Knowledge and Concepts Examinations are reported by four proficiency categories: Advanced: Distinguished in the content area. Academic achievement is beyond mastery. Test score provides evidence of in-depth understanding in the academic content area. Proficient: Competent in the content area. Academic achievement includes mastery of the important knowledge and skills. Test score shows evidence of skills necessary for progress in the academic content area tested.
Basic: Somewhat competent in the content area. Academic achievement includes
mastery of most of the important knowledge and skills. Test score shows
evidence of at least one major flaw in understanding the academic content area
tested.
Minimal Performance: Limited
in the content area. Test score shows evidence of major misconceptions or gaps
in knowledge and skills basic to progress in the academic content area tested. A recent alignment study,
conducted by the Wisconsin Department of Public Instruction under the guidance
of CTB/McGraw Hill staff, found the Knowledge and Concepts Examinations in all
four subject areas (reading/language arts, mathematics, science, and social
studies) to be “aligned” (at or above 97% average agreement for each grade
level test) with the 1998 Wisconsin Model Academic Standards. Alignment in
this study was determined if each test item corresponded to a standard from
the Wisconsin Model Academic Standards. What constituted a “match” was not
described in the report. Multiple matches (i.e., when a TerraNova question
matched multiple standards) were not indicated. The percent agreement did not
take into consideration standards not assessed by any test item. For example,
the study found that about half of the Wisconsin Model Academic Standards for
English Language Arts could not be assessed with a large-scale,
paper-and-pencil test. MPS is required by the state to
administer the Wisconsin Reading Comprehension Test (Office of Educational
Accountability, Wisconsin Department of Public Instruction) to all grade 3
students. This state mandated standardized achievement test is used to
determine the level of reading proficiency of third grade students.
The state requires districts to provide remedial service for pupils who
do not score above the performance standard on the Wisconsin Reading
Comprehension Test.[2]
No reports have been made available on the alignment between the WRCT
and the Wisconsin Model Academic Standards. MPS Performance
Assessments The mathematics proficiency test
that students take in grades 11 and 12, and is required for graduation, has
two sections. Six problems, worth 2 points each, assess knowledge and skills.
Two problems, worth 12 points each, assess problem solving and mathematical
thinking. A minimum score for passing is 5 of 12 possible points on the
knowledge and skills questions and 11 of 24 possible points on the
problem-solving questions. The writing proficiency test
that students take in grades 11 and 12 and is required for graduation has two
sections. On one section, worth 8 points, students are required to write a
letter applying for employment; on the second section, worth 16 points,
students are required to write an essay to a specific prompt in a specific
genre. Students are required to achieve a total of 15 points on both parts of
the assessment to pass. The science performance
assessment that high schools administer to selected students in grades 10-12
consists of three problems, worth 8 points each. Students need to achieve a
minimum of 16 points to pass. Students do not need to pass this test in order
to graduate. Each elementary, middle, and high school administers
either an oral communication or fine arts performance assessment to at least
one grade level as designated by their building staff and principal. The
assessment(s) may be administered at any time during the school year and all
oral communication assessments are video taped and scored by the student’s
classroom teacher. There are no
stated consequences associated with these assessments. Currently, MPS Middle Schools
administer a district-created MPS Writing Performance Assessment, MPS Science
Performance Assessment, and MPS Mathematics Assessment.
Each of these tests is administered in grades 6 (spring) and 7 (fall
and spring). They are scored by
the district, using MPS teachers as scorers.
There are no stated consequences associated with these assessments. V. The New MPS
Assessment System MPS is preparing to implement
middle school proficiencies in four areas, Communications, Mathematics,
Science, and Research. Grade 8 students in 1999-2000 will be required to
attained a passing score on these requirements in order to be promoted to
grade 9. These proficiencies will be in addition to students in grades 4, 8,
and 10 taking the WSAS Knowledge and Concepts Examinations and students in
grade 3 taking the WRCT. The middle school proficiencies allow students some
choice in the assessment tool used and the date the proficiency is
demonstrated (see attached tables). Communications In communications, middle school
students must demonstrate proficiency in three areas: writing, reading, and
oral communication. In writing,
all eighth grade students must produce four different samples of their writing
(imaginative, expository, persuasive, and narrative). They may select these
samples from their work in grades 6, 7, or 8.[3] In addition, middle school
students will be required to take a formal writing assessment from among three
alternatives. They will be able to demonstrate their writing proficiency on
the MPS Writing Performance Test. This test will be administered in the fall
and spring and will be available to students in grades 7 and 8. Or, students
use the results from the WSAS that is administered in the spring for students
in grades 7 and 8. Or, if students have not yet demonstrated proficiency in
writing by the spring of grade 8, they may submit a portfolio of their writing
to the MPS Curriculum and Instruction office for review. Under district
guidelines students must be given at least three opportunities to meet each
proficiency, whether administered by classroom teacher, school, or district. For reading, each student by the
end of grade 8 must demonstrate proficiency on a formal reading assessment and
receive an acceptable rating by a teacher on the MPS Reading
Assessment/Instruction Card.[4]
To meet proficiency, a student must attain a seventh grade reading level as
measured by the Gates MacGinite, Metropolitan, Woodcock, S.T.A.R., or Jerry
Johns tests. Each student must also prepare
and present a 3- to 5-minute video-tapped speech for evaluation. Classroom
teachers will judge whether students have met this requirement using a
five-point rubric.[5] Mathematics For mathematics, by the end of
grade 8 students will have developed a portfolio with certain specifications.
They must demonstrate their understanding of a range of algebra topics by
including in their portfolio five examples of their work. What the five
examples should be and how the quality of work will be scored, as of October,
1998, was under development. Also, students will need to
include in their mathematics portfolio one of two alternatives to demonstrate
their proficiency in passing an on-demand mathematics assessment. They either
need to satisfactorily pass the MPS mathematics performance assessment or the
WSAS Knowledge and Concepts Examination on mathematics.[6] As a third requirement for
demonstrating their proficiency in mathematics, by the end of grade 8 students
will have to satisfactorily complete a project such as a scale model or
package design that demonstrates understanding of measurement and geometric
relationships.[7] Science In science, by the end of grade
8 students are to satisfactorily complete three requirements. They are to
develop a portfolio that contains selections of their work that demonstrate
their understanding of content-rich, hands-on, inquiry-based science content.
Their work must provide evidence of growth in their knowledge and
understanding from grades 6 through 8. They must successfully communicate
their reasons for including the selections they chose and how these selections
demonstrate the understanding of science they have gained. Types of selections
include embedded assessments, presentations, investigations, reflective
writing, conceptual framework module tests, and self-evaluations.[8] For science, students also will
be required to achieve a proficient rating on one of two alternative on-demand
science assessments. One
alternative is the MPS science performance assessment. The other alternative
is the WSAS Knowledge and Concepts science test for grade 8. Students who do
not have a proficient score on either of these assessments can fulfill this
requirement by either successfully writing an essay, given a prompt in an
experimental context, or answering specific questions from a panel of science
personnel regarding a prompt from an experimental context.[9] The third science requirement is
for students to create a science project that shows the student understands
content, can use and apply the scientific method, can use technology and
scientific equipment, and is able to explain how the project applies to the
real world.[10]. Research By the end of grade 8, students
must write a research paper and orally present the paper in communications,
mathematics, and science. In the paper, the student must focus on a project
that uses statistical data to analyze a problem, interpret and report the
results, document the sources used, and follow a standard format. Students can
also meet multiple proficiency requirements with one project.
For example, one research investigation could count for the
communications research paper, the science research presentation, and the
mathematics investigation, if it meets the criteria in all areas. New High School
Graduation Assessments The 1997 Wisconsin Act 27
(sections 118.30 and 118.33) mandates that “each school board operating high
school grades shall adopt a high school graduation examination that is
designed to measure whether pupils meet the pupil academic standards adopted
by the school board . . . beginning in the 2000-01 school year.” Beginning on September 1, 2002, a school board may not grant
a high school diploma to any student who has not passed the high school
graduation test. If a student
fails the test, DPI has stated that the student must be provided with at least
three opportunities to re-take the test.
It is state-mandated policy that a school district may not use the
state graduation test unless their board has adopted the 1998 State Standards;
therefore, MPS is required to develop its own graduation test or secure a test
from an outside vendor. In addition to the above high
school graduation test, the Milwaukee Board of School Directors, at its
February 28, 1996 meeting, adopted requirements for the graduating class of
2004. Those students will be
required to demonstrate mastery in the following areas before graduation: Mathematical Reasoning Students will demonstrate
mastery of mathematical proficiency equivalent to three years of study beyond
Algebra 1. Students will be
expected to show proficiency in first-year algebra by the end of eighth grade. Scientific Reasoning Students will demonstrate a high
level of proficiency in science, equivalent to three years of high school
study, to include the physical, biological, and chemical sciences.
Students will demonstrate an understanding of scientific inquiry and
application to real-life situations. Communication Students will demonstrate
mastery of written and spoken expression by writing, presenting, and defending
a clearly reasoned, persuasively argued research paper. Community Membership Through participation in a group
project that benefits the community, students will demonstrate the capacity to
analyze a social issue from multiple points of view and to interact as a
constructive member of a team. Year MPS Middle School Proficiencies State Mandated Graduation Test (WSAS
or local/district developed test) 4 opportunities during H.S. WSAS 1998-1999 1999-2000 All
8th Grade students must have demonstrated proficiencies for
promotion to 9th Grade 2000-2001 All
12th grade students must pass a graduation test 10th
Grade testing terminated after the academic year Exceptional
Education Needs
The new middle school proficiencies and the high school graduation
requirements apply to all MPS students. Special accommodations will be allowed
for exceptional education students to take the test. However, at the present
time, there is significant discrepancy between the goal of having students
with disabilities fully participating in the regular curriculum and their
participation in the assessments. Some
of the barriers to preventing full participation include the historical
paradigms of individualized education (in which special educators generally
created entirely separate curricular goals and objectives, and often entirely
separate teaching materials, for students with disabilities), lack of
knowledge among special educators regarding general education curriculum and
assessments, and some significant institutional barriers with respect to
inclusion of students with disabilities in regular classrooms.
With respect to the “paradigm shift” needed to include students
with disabilities in the regular education curriculum and in general education
assessments, one intended strategy is for the special education staff to
attend inservice presentations for general educators regarding curriculum and
assessment. However, there is a question whether this initiative carries
sufficient resources and incentives to: 1) reach a significant proportion of
special education staff; and 2) provide that staff with adequate
information. In particular, this is a problem because participation in such
inservices is not required, nor are additional resources, such as substitute
teachers, made available to encourage participation.
With respect to institutional barriers to inclusion, the primary
barrier identified is the potential difficulty with change in teacher roles,
particularly as mediated by unions. That is, the Milwaukee Teachers’
Education Association (MTEA) generally resists inclusion of students with
disabilities in general education classrooms, since they see it as an
additional burden on general education teachers.
Furthermore, some special educators resist the change to a
decentralized model, which requires that they provide services in the regular
classroom (this also has been grieved as a change in role by special
educators). The discussion on this issue is characterized by open dialogue
with union officials, but thus far, there has been little progress. One of the strongest
institutional supports for inclusion of students with disabilities is the
flexibility of the current assessment structure.
That is, all students (including those with disabilities) are provided
a variety of methods and multiple opportunities for demonstrating competence.
Furthermore, the assessment staff’s knowledge of assessment
accommodations/modifications is strong, suggesting that Milwaukee is
relatively further along than many other districts in its efforts to
accommodate students with disabilities within district (and state) assessment
systems. However, it is not clear
that in Milwaukee students with disabilities are being exposed accordingly to
the general education curriculum. It is clear that the assessment systems in
place have a good or sound structural base within them for accommodating
individual differences in students with disabilities, but all children with
special needs are not yet fully
included in the general education classroom and curriculum. VI.
Design of Proposed Alignment Study We propose to continue our study
of alignment within Milwaukee Public Schools by doing a series of alignment
studies. The intended purpose of these studies is to reveal how Milwaukee
Public Schools is functioning as an aligned standards-based system. The higher
the degree of alignment that can be reached with the standards to be adopted,
the proficiencies, the assessments, the curriculum, and classroom practices,
the greater is the likelihood that students in the system will meet the
challenging knowledge criteria required by the standards. The series of coordinated alignment studies will answer
the following questions: 1.
Are MPS curriculum standards in language arts/reading, mathematics,
science, and social studies aligned with the district’s assessment system
and do they reflect the level of expectations advanced by state and national
groups? 2.
Do students have the opportunity to learn from grade to grade what is
required by the MPS standards and proficiencies so they are able to
satisfactorily achieve these standards and proficiencies? 3.
Do students have the opportunity to receive the instructional
experiences from grade to grade necessary for them to achieve the
proficiencies and standards? 4.
Do teachers have the pedagogical content knowledge, time, resources,
and information on students’ progress necessary if they are to provide
students with the instructional experiences needed for students to achieve the
proficiencies and standards? 5.
Are teachers given the support, professional development opportunities,
and incentives from principals and district administration that will enable
them to provide their students with the instructional progression from one
grade or class to the next that is necessary if students are to achieve the
standards and proficiencies? We propose that parts of the alignment studies be
conducted concurrently and not in sequence. Even though the below studies are
numbered, the numbers do not signify the order in which the study will be
initiated or completed. An important pre-condition of these studies, of
course, is the availability of standards in the four content areas. Study 1: Alignment of the Standards, Assessments, and
Proficiencies Study 1 will be a content
analysis of the MPS Academic Standards in language arts/reading, mathematics,
science, and social studies compared with state assessment instruments
administered through the district, or to meet district requirements and other
proficiencies requirements. A panel of four reviewers for each grade level and
content area will first analyze the standards by rating the required depth of
knowledge of each. Then the panel will match and code assessment items and
proficiencies requirements with the appropriate standards. The data from the
review panels will be analyzed and reports will be written that will describe
the degree of agreement among documents by category, depth of knowledge, range
of knowledge, balance, articulation across grades, equity, and pedagogical
implications. Each panel will include two raters from within MPS and two
external raters. Study 2: Alignment of Curriculum with Standards,
Assessments, and Proficiencies Study 2 will analyze curricula
from a random sample of schools. Four schools at each of the two
levels—elementary and middle schools—will be randomly selected to analyze
the curricula in the four content areas at those schools. The curriculum
materials will be blocked by topic and compared to the MPS Academic Standards,
assessments and proficiencies. The progression of curriculum materials in two
grade ranges—2, 3, and 4 for elementary and 6, 7, and 8 for middle—will be
analyzed. The curriculum topic blocks will be analyzed by content coverage and
depth of knowledge and compared to MPS standards, assessments, and
proficiencies. Reports will be prepared that analyze the curriculum to
determine the opportunity students have to develop the knowledge required by
the MPS Academic Standards, which will be measured on assessments and through
proficiency requirements at critical points in their schooling career—grades
4 and 8. Study 3: Alignment of Classroom Practices and Teacher
Capacity with Standards, Assessments, and Proficiencies Study 3 will analyze classroom
practices, teachers’ knowledge of the standards and proficiencies, and
teachers’ knowledge of pedagogy, as well as students’ development of the
content knowledge required for them to successfully meet the standards and
proficiencies. Data will be collected in eight schools randomly selected for
Study 3. In each school, the instructional experiences students have in two or
three grades will be analyzed in relation to the proficiencies/graduation
requirements, assessments, and standards. Data will be collected within a
school from a cluster of teachers, including at least one or two who are
teaching the subject matter to students at each grade level taught in the
school. The purpose of the clustering of teachers across grades is twofold: to
study the way in which students’ knowledge builds from one grade to the next
and to avoid putting a burden on any one teacher. Data will be collected to
determine the teachers’ awareness of what their students are to know in
order to achieve stated proficiencies and graduation requirements; what their
students know when they enter their classrooms; and what instructional
activities they have to provide their students in order for them to build on
their current knowledge of subject matter and to determine what they are
learning. In a sample of
classrooms, observers will be present to describe classroom practices, the
coverage of content, the pace of instruction, and the monitoring of
students’ learning by teachers as this correlates with students’
achievement of grade-level proficiencies and requirements. Teachers will be
interviewed and asked to complete a questionnaire to determine how their
planning and pedagogical or content knowledge relates to students’
attainment of the proficiencies and requirements. One area to be studied will
be the communication among teachers and other linkages across grade levels
that are factors in enabling students to reach the knowledge level necessary
for them to demonstrate the proficiencies specified. Study 4: Alignment of School and District Support of
Classroom Practices with Standards, Assessments, and Proficiencies Study 4 will gather and analyze
data on school and District variables that provide teachers with the support
in developing instructional practices that enable their students to
successfully attain the targeted proficiencies and requirements. School
principals and other staff will be interviewed and asked to complete
questionnaires on their initiatives in support of improved learning.
Information will be sought on specific key questions: 1.
What leadership is provided by the principal and others for having
students achieve the proficiencies? 2.
What support is provided for teachers to work collaboratively, or in
concert with each other, to provide students with a seamless learning
experience? 3.
How are needs for special groups of students met, including transfer
students, special education students, students with a high number of absences,
and limited-English- speaking students? Information will be sought from
appropriate District and school staff to determine what District support is
given to schools in a number of areas: 1.
What support is given schools by the District in the curriculum areas
to assure that teachers fully understand what students are required to
achieve? 2.
What support does the District give the schools that reinforces the
efforts of the teachers to provide the instructional experiences their
students need to attain these requirements? What efforts are made by the
District to provide teachers with the professional development they need if
they are to continue to improve in their content
knowledge and in their ability to work with students in attaining the
proficiencies and meeting requirements? We have made some observations
about standards, assessments, and alignment. At this time we do not have the
information to judge the full alignment of the MPS system. The studies
proposed are designed to produce such an analysis for K-8. Our work to date
has revealed findings of some potential issues and problems: 1. Creating and developing a consistent format for state of standards across content areas: Grade-level expectations are used consistently across the content areas, but standards are not. The standards listed for science and mathematics state what students should be engaged in learning over a grade range, K-4, 5-8, 9-12. Grade expectations are used to express what students are to know and be able to do. In social studies, grade-level expectations are grouped under headings called standards. Thus, standards represent an organizing label for expectations. The lack of consistency in format may cause difficulty in teachers’ clear understanding of what they are to do. The inconsistency in format has the greatest potential of being a problem in the elementary grades, where teachers are required to teach and attend to standards for more than one subject area. 2.
Creating
and developing a consistent format for rubrics:
Nearly all rubrics have five points, including 0 for not scorable: Many are
labeled as Advanced, Proficient, Basic, and Minimal Performance and correspond
to the rubric used by the state in rating the results from the WSAS
examinations. Assuring that these rubrics are consistent across the content
areas will help to assure they are reliability applied. A generic rubric
that defines each of these levels would be helpful. Then
content-specific rubrics that identify the detailed criteria for each content
area and instrument could be developed based on appropriate language drawn
from the generic rubric. The principle is for MPS teachers, students,
administrators, and parents to become very familiar with what proficient
academic work means. 3.
Equating
alternative forms of proficiencies and verifying reliable scoring by
different raters: The Middle Schools
Proficiencies provide for students to use different forms of assessment to
meet specific requirements. There is a need to show that alternative forms for
meeting the same requirement are equivalent in value and that they provide
fair and equitable means for students to demonstrate their proficiencies. At a
minimum, a content analysis should be made of alternative forms to determine
whether they are comparable. Ideally, a study needs to be conducted in which a
group of students takes all alternative forms to determine whether in fact
students get comparable scores on both forms. Classroom teachers will be
responsible for scoring some of the proficiencies. The consistency among
teachers for doing this needs to be examined. A moderation or back-up process
may be important to assure that if two or more teachers score a student’s
work, there would be agreement in the scoring. 4.
Assuring
balance among multiple proficiency criteria and articulation of standards
across grades: When three or
more criteria are required for students to meet the proficiencies, each
criterion is given equal weight. In mathematics, for example, students are to
show understanding of algebra, pass an on-demand assessment, and produce a
scale model. The weighing of these should be aligned with the standards and
the curriculum. These criteria also should be shown as important for achieving
the expected level of mathematical understanding in high school. Webb, N. (1998).
Criteria for alignment of
expectations and professional development in mathematics and science education.
Paper presented at the American Educational Research Association annual
meeting, San Diego, CA. Webb, N. (1997). Criteria for alignment of expectations and assessments in mathematics and science education. Council of Chief State School Officers and National Institute for Science Education Research Monograph No. 6. Madison, WI: University of Wisconsin.
FOOTNOTES [1]
Roughly 75 percent of each test contains multiple-choice items and 25
percent constructed-response, or short-answer, items.
Each student also completes an on-demand writing essay after reading
a short passage about the assigned topic.
The multiple-choice items are machine-scored. Trained scorers hired
by the testing company score the short-answer items. Each item is rated by
one professional reader and is assigned a specific performance level.
Responses on the short-answer items are scored using a two-point
scale (0-1) up to a five-point scale (0-4). The written essays are scored
holistically. Each essay is
rated by two professional readers each who each independently assigns a
rating using a six-point scale (1 to 6). The ratings of the two readers are
averaged to produce a single score. If
the readers’ ratings differ by more than one point, a third reader assigns
an independent rating. The
reported holistic score is then the average of the two closest scores.
The total WSAS test time for all five content area tests is
approximately six hours. [2]
The test is administered in three sessions, each session lasting
approximately one hour, and consists of three reading passages, two
narrative stories of about 1,200 words each, and one expository report of
about 700 words. In addition to
60 comprehension questions, students are asked a number of questions that
measure reading strategies and prior knowledge.
There are a total of 100 questions on the test, though only scores on
the 60 comprehension questions are used to establish a performance standard.
Results from the remaining 40 questions are used locally to interpret
comprehension scores. All answers to questions are in a multiple-choice format.
However, the 1998-99 test will include one question that requires a
short answer. [3]
The quality of the student’s writing will be judged by the
classroom teachers using a five- point rubric (4—well organized, imprinted
personality, effective word choice, sentence fluency, and correct mechanics
to 1—unclear, unorganized, no personal relationship, incorrect language
usage, lack of sentence sense, and incorrect mechanics, and 0 for blank,
illegible, or off the point). [4]
The Assessment/Instruction Card is a compilation of 11 good reading
behaviors such as preview text, build background for new concepts, and set a
specific purpose for reading. A student will be evaluated on the
Assessment/Instruction Card requirement by the student’s teachers on
whether he or she has met the behavior standards, using a five-point rubric
(advanced/exceeding (4), proficient (3), approaching (2), beginning (1), and
not scorable (0)). [5]
At the highest level 4, students must have eye contact and facial
expressions that enhance and support the purpose; poised gestures;
skillfully used speech patterns; a well-established intent; and effective
use of audio/visual aids. At a level 3, students must have eye contact and
facial expressions that support the purpose; natural and appropriate
gestures; appropriate speech patterns; an established intent; and
appropriately used audio/visual aids. Level 2 represents behaviors that are
inappropriate or ineffectual. Level 1 represents behaviors that are limited,
inappropriate, distracting, and/or not clearly established. The level 0 is
used to indicate the student refused to do the oral presentation. [6] A five-point rubric (4-0), with 0 indicating there was no attempt or the response was off the point or illegible, will be used to score student work on the performance assessment. Student work is judged on five attributes to determine the level at which the student is performing: strategy, organization, communication, solution, and mathematics. For the highest level, level 4, a student must effectively use an appropriate strategy; have a highly organized and well-documented response; give effective and appropriate responses to all parts of the question; provide a correct solution; and use efficient and accurate mathematics principles throughout. For a level 3, the students are to meet similar criteria but can have a minor flaw in applying the mathematics. A level 2 represents the student’s strategy as incomplete; the response is poorly organized and vague; the solution is only partially correct; and inappropriate or frequently flawed mathematics is used. For a level 1, students demonstrate an unclear, or lack of, strategy; disorganized and random work, or only a restatement of the problem; and little or no use of the appropriate mathematics. [7]
To fulfill this requirement, students are to generate a question,
problem, or situation that can be represented by a scale model, or that
requires a package design. They need to show a capacity to make consistent
scale adjustments throughout the project; be able to explain their reasoning
for using selected scales; and demonstrate understanding of measurement,
proportional reasoning, and geometric relationships. They are to either
write or videotape their conclusion or resolution showing their
understanding of the mathematics. Work on students’ projects will be rated
by their classroom teachers using a five-point rubric similar to the one
used on the WSAS examinations: advanced, proficient, basic, minimal
performance, and not scorable. [8]
A five-point rubric similar to that used to score students’ work on
the mathematics performance assessment will be used to score the students’
portfolio. Students’ work will be judged on five attributes: critical
thinking, science process, science content, transfer of knowledge, and
communications. Advanced level (4) requires students to show evidence of
original and insightful thinking, unique insights, in-depth understanding,
transference of knowledge to other situations, and clear communications.
Proficient level (3) requires evidence of appropriate thinking,
understanding, application of concepts, accurate transfer of knowledge to
real-life situations, and clear communications. For a basic level (2), a
student only provides some evidence of his/her thinking, demonstrates
partial understanding, and produces an incomplete response. A minimal
performance level (1) includes very little evidence of appropriate thinking
and understanding, unsuccessful use of process skills, no evidence of
transfer of knowledge, and unclear or ambiguous communications. Not scorable
is level 0. [9]
The five-point rubric used by the WSAS will be used to score work for
this requirement. [10] A five-point rubric similar to one used by the WSAS will be
used to score students’ work on their projects |
|