Chapter 3

The following short papers from each of the nine NISE Forum panelists are provided in the order in which they were presented at the Forum. These papers are based upon thirty- to fifty-page papers written prior to the conference, which are available at http://www.nise.org. The short papers include an emphasis on the needs for future research, often contextualized by first highlighting some important points established through existing research. Each panel’s papers are followed by a transcription of remarks made at the Forum by a discussant. The discussant remarks range from specific comments on their panel’s papers to broader discussions about directions for future research.

The three panels had assigned themes¾content and instructional practices, programs, and achievement and course-taking patterns, respectively. After analyzing the panelists’ papers following the Forum, some slightly different themes emerged and became the organizing topics of the highlights presented earlier and the synthesis in chapter 2. Following are brief descriptions of the panelists’ papers, also organized by these themes.

· Inequitable course availability and assignments, and teacher assignments. Jeannie Oakes, Kate Muir, and Rebecca Joseph cover a wide range of inequities in the educational system, including inequities in the assignment of students to mathematics and science courses and the resulting influence of those decisions on students’ access to higher education.

· Content, instruction, and assessment. Based on pilot work he has done to investigate the phenomenon of students from underrepresented groups who have good grades in mathematics courses yet low test scores, Lloyd Bond describes the need for future research on how teachers move students from solving well-structured problems to being able to solve word problems. Elizabeth Fennema articulates the kinds of gender research that would continue to advance goals for gender equity in mathematics education. Maria Klawe and colleagues describe the effects of their programs in addressing gender issues regarding the use of computers for science education and then suggest future research needs in this area. Okhee Lee states that research involving diverse languages and cultures in the curriculum is in an early stage and is heavily influenced by goals; she goes on to identify next stages for this research area.

· Understanding and scaling up effective programs. Manuel Gómez and Norma Dávila discuss lessons learned and questions still pending in their extensive work on NSF-funded systemic reform efforts in Puerto Rico. Vinetta Jones and Anne Bouie emphasize research needed on three aspects of programs that are designed to meet the needs of students from underrepresented groups: the important roles of families and communities, the institutionalization of programs, and the scaling up of effective programs.

· Better research methods and dissemination. Patricia Campbell and Lesli Hoey discuss the need for more and better evaluation of programs. Richard Tapia and Cynthia Lanius call for a serious redesign of mathematics and science education to address diverse students’ needs; they believe explicit attention to minority subgroups is important.

“Is mathematics really necessary for a life of value in the twenty-first century? This is a heretical question coming from a mathematics educator, but one that needs to be addressed… Are we just making the chosen roles of females in society (that often don’t involve mathematics) less important, less adequate, or of less value than the chosen roles of males (that often include mathematics)?”

Research over the past three decades has made significant contributions to defining and understanding the complexity of all issues dealing with gender and mathematics. It is clear that females, less than do males, participate in post–high school mathematics study and mathematics-related careers. That differences exist in the learning of mathematics seems clear also, although many scholars believe either that learning differences are diminishing or that, if any differences do exist, they are unimportant. My best analysis of the data and literature has led me to conclude that the more the test measures true mathematical problem solving, the more apt one is to find gender differences in mathematical learning that favors males at almost any age. Females appear to hold more negative values about mathematics and their relationship to mathematics than do males, but there is some evidence that these differences are also decreasing. These simplistic statements, however, hide more than they reveal. What mathematics was being measured in tests where gender differences have been studied? How was the information about values obtained? Were females’ voices a part of the data-gathering procedures? Too often the research that has reported gender differences has provided an incomplete picture at best and has only helped to perpetuate the belief that females are somehow inadequate in relation to mathematics.

Even if there was consensus on the truth about gender differences in mathematics, we would not have clear direction on what to do in order to achieve educational equity. Consider the current reform recommendations for organizing classroom instruction that are assumed will result in educational equity for all students. One major reform recommendation has to do with encouraging students to communicate their mathematical thinking by presenting their ideas and convincing peers of their correctness by arguing and questioning. It is widely believed that those who enter into this kind of debate about mathematics will learn better. But will girls enter into this kind of communication as willingly as do boys? Many teachers have reported informally that girls will not do so for a variety of reasons.

Another reform recommendation has to do with the use of technology in the classroom. It is clear that currently boys have more experiences with technological toys than do girls. Does this reflect interest and/or ability with technology? How should teachers take this into consideration as they plan their instruction?

Another recommendation is that mathematics should be situated in problem-solving contexts that are socially relevant. Unfortunately, many textbooks and teachers are more aware of contexts that are from male-dominated fields such as parabolic equations for projectiles or sports statistics. Can mathematics be situated equally in female-dominated contexts, and, if so, will boys willingly participate in such problems? Should classrooms be competitively organized or organized around cooperative activities? Some studies have suggested that boys learn better in a competitive situation, while girls learn better in a cooperative situation. Is this always true? Is the solution to have single-sex classrooms? And would the experiences we have had with black/white schools be repeated, and females’ classrooms become inevitably less adequate?

Research into gender and mathematics must continue in order to monitor learning, attitudes, and participation. In addition, we need to apply new paradigms of research that will provide insight into why gender differences occur. In other words, gender as a critical variable must enter the mainstream of mathematics education research. It is insufficient to say and to believe that the study of gender differences can be left to those who are specifically interested in gender. That is just not fair. Aren’t we all interested in how all students learn mathematics? And doesn’t that “all” include that 50 percent of the student body that happens to be female? Fairness and justice demand that all researchers be concerned with all the students even when results are obtained that cannot be easily interpreted and understood.

Specifically, we need to continue the study of gender in relation to mental processing of both students and teachers. As research on teachers continues to mature and improve, we must include gender as a variable. We probably cannot study how the gender of the teacher influences instruction because of the limitations imposed by the relatively low number of male teachers. However, we can study teachers’ beliefs and knowledge about girls and boys and the impact that teachers’ cognition has on instructional decisions for both girls and boys.

Classrooms that reflect the various demands for reform are becoming more prevalent. But are they equally effective for boys and girls? One study provided some evidence that just reforming classrooms without paying specific attention to traditionally under-achieving groups is insufficient to achieve equity. The learning that results from these reformed classrooms needs to be monitored carefully. Perhaps as we do this, we will begin to develop an image of what equitable mathematics education is.

When I became an educational researcher, I believed that I would discover truth. That has not happened, and it probably will not happen in the area of gender and mathematics. But research has deepened our knowledge about gender and mathematics, and the many, many studies about gender have provided some insight into the inequities that have existed, leading to heightened awareness of things that need to be changed. There is one question about gender and mathematics for which research cannot provide the answers, however. Is mathematics really necessary for a life of value in the twenty-first century? This is a heretical question coming from a mathematics educator, but one that needs to be addressed. There is no right answer to this question, but perhaps we should consider the following. Is it possible that I, and others who have been doing work related to gender and mathematics, have been doing a major injustice to females by pursuing issues related to gender and mathematics? Are we just making the chosen roles of females in society (which often don’t involve mathematics) less important, less adequate, or of less value than the chosen roles of males (which often do include mathematics)? Is it critical for everyone to learn mathematics? Are those who learn mathematics at lower levels of less value than those who learn at higher levels?

Research on gender and mathematics has provided a powerful scientific discourse during the past three decades. The entire educational community—practitioners, researchers, and policymakers—needs to continue to engage in this discourse and explore ways to deepen our understanding of what equity is and how it can be achieved. It is through discourse about philosophical questions as well as research questions that our understanding of gender and mathematics will grow.

Mainstreaming Underserved Students in the K–16+ Continuum to Achieve Academic Excellence

“What are the most important implementation issues faced by systemic educational reforms in addressing the needs of underserved students? How have the reforms faced these issues?”

Over the last twenty years, the University of Puerto Rico’s (UPR’s) Resource Center for Science and Engineering (RCSE) has pioneered a series of reform strategies that have been successful in transforming teaching and learning in Puerto Rico throughout the K–16+ continuum following an educational pipeline model. The main features of this model, which are critical to any systemic reform project if it is going to be successful, are: (1) identification of the major weak points in the pipeline, (2) systemic thinking in the design and implementation stages, (3) development of strategic alliances, (4) strategic application of limited reform resources, and (5) development of a virtual organization to orchestrate the reform and harness the system’s resources to transform the system. Even though UPR has applied these principles to the entire K–16+ continuum, this fact should not dissuade other researchers from working with smaller subsets of the continuum such as the K–12 system.

To reform complex educational systems, a catalytic agent is needed to orchestrate the reform, forge the necessary alliances, analyze the system, and identify the weak or missing connections among the system’s elements. Because of the magnitude and complexity of the task, a virtual organization that is not part of the existing administrative structure and that is capable of thinking and acting across systemic boundaries is needed. RCSE, an alliance of all the major higher education institutions in Puerto Rico administered from the UPR president’s office, has played this role in designing and implementing the K–16+ reform model to improve the participation and mainstream underserved (i.e., low-income) students in the educational pipeline.

RCSE interprets the education pipeline to encompass all levels from kindergarten to Ph.D. and postdoctorate. RCSE carries the reform further to include the development of UPR into a Research 1 institution developing research and development (R&D) and contributing to Puerto Rico’s economic development. At the K–12 level, the Puerto Rico Statewide Systemic Initiative (PR-SSI) is the central project; at level 13–16, the Puerto Rico Louis Stokes Alliance for Minority Participation (PR-LSAMP) and the Puerto Rico Collaboratives for Excellence in Teacher Preparation (PR-CETP) lead the way; at the 16+level, EPSCoR (R&D) and EPSCoT (technology transfer) projects are leading the reform. In addition, the K–12 reform model is being disseminated and adapted to New York City realities through the Puerto Rico/New York City Educational Linkages Demonstration Project sponsored by the U.S. Department of Education.

RCSE served as the virtual organization that orchestrated the K–12 reform and harnessed the alliance’s members (the Puerto Rico Department of Education, RCSE, and the community at large) into an effective operational coalition. The Puerto Rico Department of Education exercised its leadership by establishing rigorous nationally accepted standards for the teaching of science and mathematics, pioneered policies to decentralize school academic and administrative management by creating the community school concept, and established systemwide assessments for student performance in science and mathematics. At present, 50 percent of all public K–12 schools have been or are being developed into reformed schools using a whole-school-based approach to transform their teaching and learning culture and using the assessment, attribution, and accountability feedback (A³⁾strategy to help them become self-correcting systems.

PR-LSAMP followed a two-prong approach to systemic reform . A cadre of reform-inclined faculty willing to experiment with change was identified and nurtured to pioneer the educational reform of university science, math, engineering and technology (SMET) courses, curricula, and teaching methodologies. At the same time, using the A³ approach, chief executive officers, presidents, chancellors, vice presidents, and some deans were persuaded with the evidence of successful reform efforts to support the institutional cultural transformation needed to make the undergraduate educational pipeline more effective and efficient.

PR-CETP has brought together the faculties from the Schools of Education and Natural Sciences to revise their curricula in order to prepare future teachers to perform effectively in the PR-SSI reformed schools. The PR-CETP staff is using the A³ strategy to assess the pilot-tested reformed courses and redesigned curricula and to persuade higher education leaders to support the overall reform effort.

In our paper, we show evidence of the success of these strategies in achieving the goals of the reform, which is primarily directed toward mainstreaming traditionally underserved students (low-income students) using the A³ cycle. For instance, in 1998–99, at the K–12 level, students who attended PR-SSI schools for six years outperformed their private school counterparts by 58 points in the mathematics reasoning part of the college admissions tests administered by The College Entrance Examination Board. At the same time, students who attended PR-SSI schools for three years outperformed students from non-PR-SSI public schools on the same test by 32 points. Similar trends were sustained for 1999–2000. PR-LSAMP results have also been impressive for the 13–16 level. Against a backdrop of declining absolute enrollment of SMET students at UPR, the number of SMET B.S. degrees conferred by the university rose ¾from 1,200 to 2,100 in a seven-year period (displaced by five years to compensate for the average time to complete the degree). Moreover, graduation rates for science and mathematics have gone from 46 percent to 60 percent in the same time period; for engineering, these rates have risen from 58 to 75 percent. The number of students who go on to complete their Ph.D.s in SMET disciplines in UPR’s Río Piedras campus is one in ten, a number that places UPR among the top performers in the nation.

We would now like to highlight some of the lessons that we have learned in this process as well as some of the questions that are arising along the way. We end this paper with suggestions regarding some of the issues that remain unanswered.

One of the main lessons learned is that reform strategies cannot be developed and implemented in a vacuum. Without a solid theoretical and philosophical foundation to guide the development of these models and strategies, a reform is likely to be just a project instead of a true change in institutional culture.

Another lesson that we learned is to start small and then scale up. This lesson enabled us to monitor our efforts carefully using the A³ cycle, so that we could make the necessary corrections in our design to ensure quality control. As we learned about what worked and about what did not work, we also developed the necessary human capacity to carry the reform forward and to sustain it beyond its funding period. We also learned what key elements of the reform could be transferred elsewhere through projects such as the Puerto Rico/NYC Educational Linkages Demonstration Project.

Another lesson involves the importance of establishing and maintaining strong connections between the different sectors of the educational system so it can be truly systemic. Without those connections, our reform would have been another fragmented effort to solve a systemic problem¾destined by its design to fail. The virtual organization and pipeline models described in our paper are our answers to these issues, which include building strategic alliances to support the efforts.

Our systemic educational reform has transformed itself from its early days. Our experiences have led us to understand that “what got me here today will not get me where I need to be tomorrow” because of the dynamic nature of social systems. Therefore, we are constantly looking for new and better ways to catalyze change in the system. We are also embarked on a constant search for answers to questions such as the following:

· What theoretical and philosophical elements are shared by successful systemic educational reforms?

· What are the most important implementation issues faced by systemic educational reforms in addressing the needs of underserved students? How have the reforms faced these issues?

· What elements of systemic educational reform travel best to other sites? Which elements are the most difficult to transfer to other sites and why is this the case?

· How can we improve pre- and post-test designs to look at the value added by the reforms?

· Why do we still focus so much of our attention on teaching instead of learning?

· What can we do as researchers and educators to learn more about the process of learning?

· Why is student performance higher in open-ended questions than in multiple-choice questions?

Equity for Culturally and Linguistically Diverse Students in Science Education

“Research efforts generally involve identifying educational problems or describing instructional practices rather than implementing intervention strategies to promote teacher effectiveness or student achievement. Research is still at the stage of conceptualizing issues that need empirical testing.”

As the student population in the nation’s schools becomes more culturally and linguistically diverse, it is essential to establish a knowledge base to promote academic achievement and equity for all students. A pressing problem with diverse students in science education involves the disconnection and tension between their languages and cultures and the nature of science as it is traditionally defined in the Western science tradition (Atwater 1994 and Lee 1999).

Equity is distinguished from equality. Equity is associated with justice or fairness, whereas equality is associated with sameness. In addressing the tension between Western and alternative views of science with diverse students, this paper considers equity from the cultural anthropology perspective. According to this perspective, science learning occurs when students successfully participate in Western science, while they are also engaged in alternative views and ways of knowing in their everyday worlds. This balanced orientation considers the contributions and strengths of both Western science and alternative views. Students have access and opportunities to learn the “high-status” knowledge of Western science as it is practiced in the science community and taught in school science. At the same time, alternative views of science and ways of knowing in diverse backgrounds are recognized and valued. As a result, students are able to achieve both academic success and cultural identity.

This paper addresses equity issues about epistemology and pedagogy of science content, learning, and teaching for students from diverse languages and cultures (see Lee 2000 for the full text). Highlighting the pressing problem of disconnection and tension as the key theme, the paper offers a synthesis of major issues and research findings for effective practices (i.e., what we know) and recommendations for a research agenda (i.e., what we need to know). Much of the literature is recent, published mostly during the 1990s. Research efforts generally involve identifying educational problems or describing instructional practices, rather than implementing intervention strategies to promote teacher effectiveness or student achievement. Research is still at the stage of conceptualizing issues that need empirical testing. Some innovative research provides important insights to enhance instructional practices and student outcomes.

What counts as science or what should be taught in school science is critically important because this definition determines school science curriculum. Western science, as traditionally practiced in the science community and taught in school science, presents high-status knowledge to which every student should have access. At the same time, students from diverse backgrounds bring alternative views of science and ways of knowing to the science classroom. This presents a challenge. On the one hand, an emphasis on the high-status knowledge without consideration of alternative views makes science less accessible, relevant, or meaningful for diverse students who have generally been bypassed in science education. On the other hand, an emphasis on alternative views, which are culturally and socially significant but may be unimportant topics in the science community and in school science, does not promote equitable outcomes. Research may address ways to incorporate alternative views in defining what counts as science and what should be taught in school science. This topic is as much a political issue as an empirical question.

Students with diverse languages and cultures bring to the science classroom ways of knowing, talking, and interacting that are different from those in the mainstream. Efforts need to be made to bridge the gap between students’ home cultures and the culture of science. The wider the gap, the more difficult it is to bridge. When disparities abound, there is no equity if Western science is imposed on students who do not share its system of meaning, symbols, and practices. Similarly, there is no equity if students are not provided with opportunities to learn Western science. Research may examine how diverse students learn (or fail to learn) to connect cultural norms (e.g., respect for authority) with mainstream expectations (e.g., questioning and argument). Research may also examine how diverse students achieve (or fail to achieve) academic outcomes as well as language and cultural identities.

Effective instructional scaffolding for diverse students involves consideration of many factors. Two issues emerge as central: (1) integration of the nature of science with students’ languages and cultures (i.e., instructional congruence) and (2) teacher-explicit or student-exploratory approaches. Equitable science instruction meets the learning needs of diverse students while preparing them to function competently in the mainstream.

Teachers often do not have both knowledge of science and understanding of students’ languages and cultures. Instead, some have adequate science knowledge but limited understanding of students, others have understanding of students but limited science knowledge, and still others have limited knowledge in both areas. While establishing instructional congruence is a demanding endeavor, it is particularly challenging when cultural norms for students’ classroom participation (e.g., respect for teacher authority) and mainstream expectations (e.g., independence and individuality) are incompatible.

Challenges also occur when culturally based instruction (e.g., teacher-explicit instruction in meaningful contexts) and mainstream expectations (e.g., student-exploratory instruction) are incompatible. For diverse students, the discourse at home is inconsistent with the discourse in school. Multicultural education literature suggests that teachers provide explicit instruction about the rules of discourse in school rather than expect students to figure out the rules on their own. A danger is that teachers may misinterpret explicit instruction as drill and practice through didactic instruction and fail to promote critical and creative thinking with diverse students. This may become yet another stereotype that can potentially limit opportunities for diverse students to learn to function competently in the mainstream. The tension in competing pedagogical approaches deserves special attention. Consideration needs to be given to students’ language and cultural expectations, science experiences, and the demands of academic tasks.

Teacher professional development is critically important because teachers play a central role in providing effective instruction. Although teacher change is a demanding process, the process may be more arduous when involving diverse students. Research may examine the process of change in teachers’ knowledge, beliefs, and practices as they participate in professional development. Research may also examine the kinds of support required for initiating and sustaining teacher change. This information is essential in designing effective instructional interventions with diverse groups of teachers and students.

Research is also needed to relate teacher change to student outcomes. It is important to examine how teacher change influences students’ academic achievement, and how student outcomes in turn influence teachers’ knowledge, beliefs, and practices. In addition, how are different kinds of teacher knowledge associated with different student outcomes? The interplay of teacher change and student outcomes may provide valuable insights into effective instruction and student learning.

First, research needs to examine ways to integrate academic disciplines with students’ languages and cultures. Research generally focuses on one area while keeping the other as the context. Instead, the two areas need to be addressed simultaneously to develop pedagogy that is both subject specific and diversity oriented.

Second, it is important to link curriculum, instruction, and assessment to understand a more complete scope of classroom practices. Because these three components are closely interrelated, intervention research focusing on one often faces the need to incorporate the others. Research needs to consider the alignment of curriculum, instruction, and assessment in meeting diverse students’ needs in classroom practices.

Third, research needs to consider multiple theoretical perspectives, which are typically associated with particular methodological approaches. For example, in addressing conflicting pedagogical practices between culturally based instruction (e.g., teacher-explicit instruction) and mainstream expectations (e.g., student-exploratory instruction), multiple theoretical and methodological approaches (e.g., cultural anthropology and progressive education) need to be considered.

Finally, to improve educational practices, teachers must be involved in the development of a knowledge base. The practical knowledge of individual teachers from diverse languages and cultures can be incorporated into the development of the theoretical knowledge of teaching. This knowledge base can be shared with teachers from a variety of backgrounds in providing effective instruction for students who have traditionally had limited opportunities in science. Ultimately, what benefits students from diverse languages and cultures can also benefit mainstream students, making it possible to attain the vision of standards-based reform—high academic achievement for all students.

This panel cuts across a wide swath of diversity and equity issues in mathematics and science education, from Elizabeth Fennema’s paper on gender equity in mathematics education, to Okhee Lee’s discussion of equity for culturally and linguistically diverse students in science education, to Manuel Gómez’s discussion of his successful mainstreaming of underserved K–16 students. The panel presented a very interesting tapestry. Underlying the first two presentations, the issue of values, culture, and science¾or sometimes, culture versus science¾seemed to predominate. An important point is that this should not become a tug of war, or our students will be caught in the middle. Our students need to live in the world of science and mathematics as it is presently constituted, if we are to help them get better jobs, compete in the world market, and prepare for the future. The problem is how to marry the culture that students bring to schooling with the outside world for which we want to prepare them.

I was disturbed to hear the results of the further study on Cognitively Guided Instruction and the identification of gender differences very early on. This is certainly something we need to look at, and we need to look at other programs in the early grades to see if there is a similar effect there. I think, however, there is very good news in the gender equity situation. Gender differences in mathematics achievement over the past twenty years have been decreasing. Girls are taking more advanced mathematics courses in high school than they did twenty years ago, and a greater percentage of women are taking mathematics majors in college¾in fact, it is up to about 50 percent. However, a smaller number of women majoring in mathematics are planning to go into teaching than males, so this is not exactly equitable. Also, the number of women who go on to graduate school in math has remained stable over the years, and is not increasing significantly at this time. So we still have a problem, especially with the retention of women in graduate courses in mathematics.

I agree with Elizabeth that the effectiveness of intervention programs (of which there are many, and I’ve been in a few of them myself) have not been well documented; we certainly need to be considering good ways of documenting results as we set up intervention programs. We haven’t done a very good job of this. Consequently, some intervention programs proliferate that perhaps should not, and other good ones are not carried forward.

I was also very interested in Elizabeth’s concern for a feminist perspective in mathematics, and this certainly goes to the cultural issues too. I wouldn’t want to carry this so far that girls were not learning the mathematics that they may need to know to go into science and technology fields. We need to be very careful about how we phrase our research questions¾this can determine the results we get. We need to look very carefully at this feminist perspective in order to phrase our research questions in ways that will not prejudice the results from the beginning.

Lee’s discussion of the postmodernist feminist perspective was interesting, and her concern not to consider this as highly as an anthropological approach was also an interesting take. To look at how students can successfully participate in Western science while maintaining their own cultural identities and values and those of their parents is certainly very difficult for teachers to do. To look very clearly at how we can help teachers cross this very interesting terrain is of extreme importance. We think, at the Department of Education, that the approach to a problem-based research agenda is a very important one. We are working, as Kent McGuire mentioned, to set long-range research agendas in mathematics and reading.

Manuel Gómez presented a very interesting and practical case for a K–20 educational pipeline to mainstream underrepresented students. He talked about orchestrating, and, as I read his paper, I thought of him as an orchestra leader. I also thought, a bit discouragingly, that there are not too many wonderful symphony orchestra leaders in the United States, nor in the world. I’m concerned as to how we might grow more wonderful orchestra leaders in this area of systemic reform. It takes this leader to pull together people, organizations, money, and funding, and to also have the capacity to generate enthusiasm, interest, and dedication¾that is a tricky matter. Maybe we need more education in developing more orchestra leaders in our systemic efforts in education. I certainly liked his model of research and practice. I think this is something we’re looking at—starting small, implementing, continuing to assess, keeping the cycle going. We have not been too successful at this in the past, as a whole or as a community, and we need to do more about it.

Will the increased emphasis on changing the way we teach¾that is, more emphasis on group work, more emphasis on teaching for understanding¾help the equity issue with regard to educating girls? Will this help women who are entering technical fields and raising, nurturing, and mentoring their daughters? I think that history has made a difference in the gender equity situation. And I see many good friends in the audience who are working on these issues too. I don’t mean to say the issues are solved, but I think history is on our side and that some of these issues have lessened over the years. Okhee, I am still concerned about how we can manage this juggling between the Western culture, or may I say (since we are becoming such a global society) almost a global look at what science is, with what children bring to school. For example, some children may bring to school from their culture the belief that the sun revolves around the earth, and some children may bring to school a creationist approach to science. How do we resolve this? How do we expect the teacher in the classroom to handle these problems? I think these are thorny issues.

“There has been little examination of unintended and/or negative effects of special programs…Unless the possibilities of unintended outcomes are explored and tested with students from different demographic groups, we may be doing more harm than good.”

Traditionally, there has been gross underrepresentation in SMET careers of female and male students of color, white female students, and students with disabilities. Although there has been progress, it has been uneven. Underrepresentation in SMET continues, but the issues surrounding it are somewhat different for members of different groups:

· When young women graduate from high school, they have basic science, math, engineering and technology skills and knowledge in numbers and percentages comparable to young men, although some gaps exist at the most advanced levels. However, young women are much less apt than young men to continue on in SMET.

· Although the science and math achievement and course-taking of students of color has been increasing, relatively few African American, Hispanic, and Native American students are graduating from high school with the skills and knowledge needed to continue in SMET. Even fewer go on in these areas.

· By the end of high school, students with disabilities have taken less science and math than other students. Little else is known about their SMET skills and knowledge.

Research and evaluation that has been done on special programs has tended to focus on program impact on delineated goals and outcomes. There has been little examination of unintended and/or negative effects of special programs, although individual evaluations have documented such findings as:

· a focus on barriers faced by women in science, causing female high school students to become less interested in going into science careers;

· hands-on science activities led by adults with no knowledge of equity strategies, causing students to become more stereotyped and limited in their opinions of who could do science; and

· students and teachers in an SMET enrichment program targeting minority students seeing it as remedial, in spite of strong evidence to the contrary.

Unless the possibilities of unintended outcomes are explored and tested with students from different demographic groups, we may be doing more harm than good. Student data must be broken out by race/ethnicity and sex to allow decisions to be made, not just about the effectiveness or ineffectiveness of strategies for “average” or “typical” students, but their effectiveness for different groups of students.

Special programs need to become laboratories for trying out new ideas, rather than providing remediation or “enrichment” for some underserved students. Instructional strategies for reaching all students with high quality content needs to be a part of every classroom, not a pull-out, after school or lunch program for a few students (Campbell and Kreinberg 1998).

There is a great need to learn more about the scaling-up process, to learn more about how to bring strategies that work into the mainstream of education in terms of content, pedagogy, and funding. It is particularly important that research and evaluation on scale-up efforts include state teacher certification programs, teacher educators, and others who work with preservice teachers, so that teachers come into the profession with the knowledge and skills to attract and keep students from underrepresented groups in SMET. Equally important is the need to address the areas of teacher rewards and reinforcements. It is key to determine what is needed to support teachers and others. This support should be not just in their initial efforts to change, but in their efforts to continue to implement and refine effective strategies to increase the participation and achievement of students from underrepresented groups in SMET.

To make the results of such research and evaluation useful, it is necessary to explore what acceptable evidence of effectiveness is. Issues to be covered in such an exploration include:

· the value of one well-controlled study versus many studies with different flaws; and

· the value of comparing the impact of one program to that of another (or to the impact of doing nothing) versus assessing program impact in terms of the degree to which it meets an acceptable criterion (i.e., 90 percent of students continue on in geometry).

As work continues on ways to scale up and institutionalize effective strategies, additional work needs to be done to develop and test new strategies and activities. However, it is difficult to develop innovative strategies without knowing what has already been done. Currently there is no comprehensive compilation of programs and strategies. Compilations that include lessons learned, effective and ineffective strategies, and even hands-on activities are badly needed. An easily accessible compilation would mean that program developers and implementers could build on and refine existing strategies and activities rather than reinvent them.

Another factor mitigating against the development and implementation of new and possibly risky strategies is an emphasis by funders on funding special programs that have the best chances of working. Both funders and developers need to see that finding out what doesn’t work and why can be as valuable¾and should be as valued¾as finding out what does work and why.

Challenge III: Rethinking the Roles of Evaluation and Research in Special Programs

There is an existing knowledge base that can and should be used in program and policy development. However, much is left to be learned. Typically, due to a lack of resources, few programs do the kinds of evaluation that can determine impact on student achievement, course-taking, or longer-term interest in SMET. There is a great need for longitudinal evaluation to better determine what works and what doesn’t in encouraging underrepresented students from different groups to continue on in SMET, particularly students with disabilities about whom so little is known. The major challenge for the research and evaluation agenda is to increase the amount of research and/or evaluation done using the following model:

Program participation ≥ intermediate effects ≥ long-term effects/student data for different demographic groups

To be successful, the research and evaluation agenda must be built on the belief that this is not about special programs, but about creating equal outcomes for students across all groups. Data-driven intervention must identify the variety of successful strategies that will allow us to serve the entire population, to close the gaps between groups without creating gaps within. To be high quality, education must serve all.

Effective Programs for Achieving Equity and Diversity in Mathematics and Science Education

“Another area of concern is the way in which programs define their relationship with students’ families; the ways in which programs that produce high student achievement outcomes work with parents…”

There is a great deal known about what works, what does not work, and¾to some extent¾why things do or do not have positive outcomes for student achievement. A research agenda for programs to promote equity in mathematics and science would do well to build upon and extend the existing knowledge base. This paper presents several key issues and briefly discusses why they are important in the field.

The first key research issue is arriving at some sense of the definition of a “quality” program. There are those who feel that it is not reasonable to expect that a single teacher development, teacher research, student enrichment, or even a systemic program can bring about and sustain this outcome beyond a pilot or an experimental program with the ability to control the environment tightly. Further, “student achievement as a reasonable outcome also awaits the development of reasonable and reliable assessment tools and methods that measure indepth, conceptual understanding of mathematics and science” (Kaser and Bourexis 1999, p. 110). Hence, some sense of how mathematics and science programs themselves define “effectiveness” and “reasonable outcomes,” the basis for their criteria, and the implications of their definitions of “success” and “effectiveness” are all reasonable questions which would bolster the work done by Malcom (1983); Clewell, Taylor-Anderson, and Thorpe (1993); and others who have sought to obtain evaluation criteria and data from mathematics and science enrichment programs at all levels. Once data have been obtained on the ways in which programs define effectiveness, some analysis of their outcomes in relation to their definition might provide useful information about program design and implementation and their relationship to student outcomes.

Another area of concern is the way in which programs define their relationship with students’ families, the ways in which programs that produce high student achievement outcomes work with parents, and whether these interactions differ from those in programs that do not produce gains in student achievement and other measures of academic progress. These findings would be useful in helping programs design and implement family interactions that create endorsements on the part of families for the work that will be required if students are to be successful. Funders are rightly concerned that resources produce results. By paying attention to student outcomes, programs assure those who are involved with their work that resources are being spent wisely.

A great deal of work has been done focusing on the strengths of traditionally underserved students and their families. This body of work is commonly referred to as the “resiliency frame.” This frame has potential utility because it points educators to specific traits of individuals, families, workgroups, and organizations that create a positive environment that nurtures and sustains young people. When these characteristics are applied to workgroups and organizations, they are particularly compelling. Research, which determines the extent to which “effective” programs also possess the characteristics and traits of “resilient” organizations, could be very useful in program design and implementation.

Another critical research area is the examination of programs and projects that have indeed successfully scaled up beyond the original pilot or site to other sites, to sites outside the district, to the district as a whole, or to other states. Quality ethnographic data on what was done and on how and why things were done could provide useful information to practitioners and researchers alike.

A related area of concern is the extent to which such programs have been institutionalized and have effected significant change in the ways schools design and implement programs for traditionally underserved students. If indeed these programs have been institutionalized and have become a part of the host organization, which entity has adapted and changed, and what have been the effects on student outcomes? Research could support practitioners in examining effective strategies for translating relevant research findings into practices that can actually be used in real-world settings. Research should be reported in ways that are practitioner-friendly as well as suitable for referred journals.

There are a fair amount of data that seem to indicate that programs and staff of schools that are successful¾as defined by student achievement, academic self-confidence, enrollment, and success in advanced mathematics and science classes¾differ from other schools in some very important areas. It appears that successful programs differ from what ineffective programs in these four critical areas. First, they seem to view the student and what he or she brings to the classroom differently. Second, there appears to be a difference in the relationship between the staff and students and between the staff and the families of the students. Third, it appears that the kind of content presented and the ways in which it is taught often differ dramatically from presentation and delivery in conventional classrooms and programs. These are all areas that could benefit from research and, as stated earlier, made user-friendly so that they may be adopted by other programs. Finally, the extent to which these four areas might inform preservice teacher training programs is certainly worthy of exploration.

“While these initiatives seem promising approaches to tackling some of the core factors affecting the low female participation in information technology, more research and programs are clearly needed. One important area of research is to find out whether the findings for females also apply to other underrepresented groups in information technology.”

The low participation by women in both the information technology (IT) industry and in computer science courses in secondary and postsecondary education is an important equity issue in science education. In addition to the increasingly intense need for more highly skilled people in the IT sector, women are missing out on many of today’s most attractive career opportunities. Equally importantly, the IT field is missing out on the broader range of perspectives and talents that would result from significantly increased participation by women. Both the percentage and total number of bachelor’s degrees awarded in computer science to women decreased almost every year over the last decade. This is in direct contrast to almost every other area of science and engineering, where participation by women has significantly increased. Only 14.4 percent of employees in IT are female (Myers 1999). The current percentage of computer science B.S. graduates from U.S. research-intensive universities is approximately 17 percent (Camp, Miller, and Davies 2000). Moreover, course-taking patterns in high school indicate that the gender imbalance in computer science has already established itself prior to university.

Research by many groups, including the SWIFT (Supporting Women in InFormation Technology) project at the University of British Columbia (see http://taz.cs.ubc.ca/swift), has identified several factors contributing to the lack of female participation in IT careers:

· In North America, socialization has the effect of labeling computer science as a hard-core male activity. This gender stereotype is entrenched by the male-oriented computer games that form the primary computer experience for most children. In addition, the lack of female IT teachers and the small number of females involved in the IT industry result in an absence of female role models.

· There is a significant gender imbalance in access to and ownership and usage of computers. The tendency for boys to dominate computers at school and at home and the scarce use of computers as a teaching tool in schools results in girls graduating from high school with less experience in computer usage, especially in programming.

· The combined result of the above two factors leads to more computer anxiety and lower levels of self-perceived ability in computer science among female students.

· The images of IT jobs (programming sixteen hours a day with little human contact) and of the people who work in them (geeks with no life) are not appealing to females.

· Introductory computer science courses usually focus on the technical elements of programming and computers rather than on computing applications. This approach is not appealing to females who tend to value computers for their uses rather than their intrinsic technical interest.

These findings point to the need for more research and programs related to gender-inclusive computer science education and software development. In addition, there needs to be significant improvement in the awareness of students, parents, and teachers about IT careers. Initiatives in these directions by the SWIFT project include:

· Research and development of gender-inclusive educational computer games (see http://taz.cs.ubc.ca/egems). Our findings indicate that female students are as interested and as competent as male students in playing educational computer games when the games are designed to take a broad range of playing and learning preferences into account. Simple interventions in the classroom such as providing one time per week when only girls are allowed to play at the computers (and an analogous time for boy-only play) have had dramatic results in increasing girls’ time on the computers.

· Development of the Virtual Family software activity which provides a fun introduction to programming in Java via the programming-by-example method.

· Delivery of hands-on workshops for girls in grades 7–10 that are designed to raise their awareness of IT careers and engage girls in interesting and enjoyable creative computer activities.

· Development of a university computer science course that introduces computer science concepts through applications in biology, psychology, and the fine arts.

· Development of ARC, a two-year postbaccalaureate program combining academic computer science courses and work experience that is designed to be attractive to women. The key “women-friendly” elements in ARC’s design include requiring no prior programming experience; aiming the program at students with an outstanding previous academic record in any discipline (e.g., English, music, political science, psychology, biology); providing smaller classes, extensive additional tutoring, and nurturing instructors; explicitly specifying that the program aims to have women be at least 50 percent of each cohort; and providing an opportunity for a paid work experience. So far approximately 60 percent of ARC students have been women.

While these initiatives seem promising approaches to tackling some of the core factors affecting the low female participation in IT, more research and programs are clearly needed. One important area of research is to find out whether the findings for females also apply to other underrepresented groups in IT. Another important area is to upgrade computer science and equity knowledge and skills of in-service and preservice teachers and counselors. A third area is the development of more culturally diverse and application-oriented approaches and resources for teaching computer science. Although a number of different groups have made significant efforts to find ways to make IT more attractive to female students, much remains to be done in making the various programs systematic and integrated with the school and university system. Perhaps the most important area is to change the popular image of computer professionals. The image of computer professionals as nerds who work in isolation in front of computer screens is neither accurate nor appealing to females.

I am a program officer at the Joyce Foundation, and systemic reform is what the education program does at the Joyce Foundation. We put money out there to help school systems change. It is not an easy field to be in, and it is a very expensive endeavor; sometimes it doesn’t look like you’re getting anywhere.

But let us think for a second about why I know that I’m going to win and the systems are going to lose. There are two very simple reasons I can stand up here very proudly and say, “Yes, I do systemic school reform and I pay for it.” They have nothing to do with the amount of money I spend, nor with the politics going on in schools right now. They have to do with many other things, and these things relate to why the issues brought up in these three papers are important.

The first one is inevitable; it is happening right now. On December 31, 1999, the United States ended the second largest baby boom in its history. Twenty-three percent of the children born in the last baby boom, from WWII to its peak in 1972, were minority children. Around 2020, 40 percent of all children attending school in this country will come from minority backgrounds. That is a huge increase, and what it says is this country’s educational system, whether it be public or private, will have to educate the largest number of minority students in history. The second reason is that the economy is global. It’s that simple. We no longer compete between Detroit and Chicago. Chicago and Detroit compete against Calcutta, Singapore, Paris, and Stockholm. If you want to make money, you have to compete on those fields. Therefore, when we think about diversity and equity in science, math, and technology education programs, we have to ask ourselves, what does it mean? To me, it means the future. The numbers are there, the economy is there, and we have to do things differently than we’ve done before.

There are three questions I want to go over; also, I want to talk about some things these papers have in common. The first question is that we must think about a new definition of success in education. It doesn’t mean success has to be less than it is¾it just means we have to look at it differently. The second question is that we have to think hard about how we measure success. How do we take these programs and successfully scale them up? As a historian of education, I don’t know if we’ve ever scaled up an educational program. We’ve mandated and put a lot of money behind it and done it. For example, look at what Sputnik did for math and science in schools. We put a lot of money behind it and just did it¾not a lot of research, but a lot of money. Third, how does this conversation of special programs no longer become special programs, but become the norm, or the culture, of the schools? Because it will have to become the norm to be successful.

I think the others did a very good job of capturing some points here, and describing some good things which are happening in terms of “these are successful practices.” And let me tell you that as a program officer who is responsible for giving away about $10 million a year in grants, I have never had anyone come into my office and say, “I’d like for you to fund a bad program.” All programs are good. But there are some common things here that we must pay attention to when we try to answer these three questions. The first one, as made clear in the papers, is that kids may not feel invited to learn the subject area. Women didn’t feel they were invited into this field. Whether it is a math or science program or any other program out there, what we begin to ask is: Are the students welcomed into the class? Are they expected to learn? And is the curriculum rigorous? Do they get a sense of efficacy from the faculty that is teaching them, from the teacher in front of them? Do they get the message “I am here to teach you, I believe you’re going to learn; I’m going to teach you this subject area, and you really belong here”? We also have to figure out, then, if they’re not invited, how do we invite them?

Another issue is that these three studies show a lot about what we have learned, and we’ve learned an awful lot in this field. Have we learned it all? No. But we have some knowledge. One of the problems we have is that the knowledge may exist here in this room with two hundred of us, and maybe you can multiply that three or four times, but how does that knowledge begin to spread into practice? What I didn’t see here was the hard wiring that takes the knowledge and puts it into the practice. You really brought back home some studies I have been reading over the last few years about minority achievement. Look at the differences in the Equity 2000 cities. In Milwaukee, with which I am very familiar, and other cities, look at what happened over the years. Do we have any indication of what was learned in those cities, about how they took an idea from its gaseous state and made it into a solid entity? And if we did, do we have teachers who can talk to other teachers about how they did it, how they learned it, how they took it into the classroom, and what they’ve learned from it¾and then carry it on? These are important things, because if we’re going to translate this knowledge from one place to another, we’ve got to have translators. This is some of the work that still needs to be done.

Also, it may be as important to discuss how we teach as it is what we teach. That one and one equals two, we will give that as a common thread. We can teach that over and over¾but how do we teach that? Maybe it goes back to the question about how to take what we’ve learned and make it into the norm, because the norm probably needs to be a new norm. How do you take the knowledge you have and integrate that with other areas? Now I’m going to push this a little. Maybe if the hands-on experimental approach, if the very active learning that must go on and that arose in almost all of your studies were necessities, and if the mentoring for all those things that seem to be common denominators across the board really produced strong results in academic achievement by children who were learning nontraditionally, then we must begin to look into other areas, other fields, about how to take this and integrate it.

Let me give you an example. A very interactive field of learning is the arts. Whether it’s theater, drawing, speech, filmmaking, painting, sculpture, or poetry, it is a very interactive field. Who says you can’t take one area of math or science and begin to integrate arts with it in a very interactive stance? I’m not saying we have the answer about how to do that or that we can demonstrate it. I have seen some schools that have used dance to teach geometry¾and it worked, the scores were up. The mayor was pleased. The superintendent was pleased. Therefore, the program was a success. Did those programs become a norm in the city? No. The trick here is that even if you raised the scores, the politicians don’t understand what happened. They just want to make sure the scores are up. All they will do is make sure that the person at that school or set of schools continues to do what he or she is doing, so when they bring the other politicians, they can demonstrate it. The trick is for us to take that unique piece of learning that happened there in teaching and begin to expand upon it in different ways.

The real issue once again is how do we take this and move it on. There is knowledge out there of substantive value. It is deep¾although perhaps not deep enough. It is broad¾although perhaps not broad enough. Most educational innovations in this country were not based upon anything much deeper than the will of the people, and the will of the folks who had the power to vote in legislative chambers to make change, followed by the acceptance and hard work of folks in classrooms to accept that change to create a new norm. What we have here is evidence that we need to build upon, and the studies talked about here lead me to believe that there is more work to be done. The job is not over, but we have begun to do that work.

Good Grades/Low Test Scores: A Study of the Achievement Gap in Measures of Quantitative Reasoning

“Perhaps the single biggest instructional challenge in all of high school mathematics is the difficulty teachers have in moving students from being able to solve well-structured problems to being able to solve verbally presented tasks (i.e., “word problems”). The most pressing immediate research imperative, I feel, is of a more ethnographic nature. What antecedent instructional conditions facilitate and what antecedent conditions frustrate the development of proficiency in quantitative problem solving?”

This investigation,[2] although it identifies a limited number of aspects of a complex problem, points up fruitful areas for future research. We have seen that students who have the requisite declarative knowledge to solve a class of quantitative reasoning problems nevertheless fail to use that knowledge when it is required. Additional research is needed to describe more completely the nature and structure of the mathematical knowledge that students with the “good grades/low test scores” achievement pattern have. More research is also needed on the features of quantitative reasoning problems that make it likely that students who have the required knowledge will correctly solve them. It was noted earlier that one relevant task feature appears to be the extent to which the problem is “concrete” (i.e., employs specific numbers) versus “abstract” (i.e., employs unknown variables). Kintch and Greeno (1985), Mesa and Herbst (1997), and others have stressed the verbal processing demands of many problems that are intended to measure quantitative ability. Perhaps the single biggest instructional challenge in all of high school mathematics is the difficulty teachers have in moving students from being able to solve well-structured problems to being able to solve verbally presented tasks (i.e., “word problems”).

The most pressing immediate research imperative, I feel, is of a more ethnographic nature. What antecedent instructional conditions facilitate and what antecedent conditions frustrate the development of proficiency in quantitative problem solving? Two general categories of studies come immediately to mind: studies of the instruction taking place in actual classrooms and studies of student non-classroom engagement and time spent on things academic.

It is axiomatic that good teaching is essential for good learning. To be sure, some students can, on their own, achieve remarkable levels of proficiency in certain domains, but for the vast majority, quality teaching is prerequisite for high or even adequate achievement. What constitutes “quality teaching” in elementary, middle, and high school mathematics? Eminent scholars such David Berliner, Lee Shulman, and Gaea Leinhardt have studied expert teachers in action. Their work, along with the highly influential series of standards and associated performance assessments of the National Board for Professional Teaching Standards for certifying public school teachers has contributed significantly to our understanding of precisely what constitutes good teaching. One surprising, but consistent, finding is that credentials per se (e.g., number of advanced degrees) are largely unrelated to teaching expertise. Teachers with bachelor’s degrees do as well on the board’s rigorous assessments as do those with master’s and doctoral degrees. The inference that one is a good teacher must be made on the basis of his or her actual classroom practice: What does the teacher assume about the state of knowledge of his or her students? Is instruction appropriately paced? Does the teacher sequence hierarchically ordered concepts in a rational and coherent way? How does he or she respond to individual differences in readiness? What kinds of assignments does he or she give the class, and what is the nature and quality of his or her individual student feedback? How does the teacher monitor and assess student progress, and what level of student proficiency do his or her grades reflect? A related set of questions involves the extent to which teachers take the easy way out by limiting instruction and assessment to the developmental level of their students, thereby avoiding more challenging tasks altogether. If we are to relate student achievement to teaching expertise in any defensible way, this level of specificity is essential. A well-designed ethnographic study of actual classrooms would be an enormous contribution to our understanding.

Issues of readiness and “social promotion” must also be systematically studied. Many students, especially those in overcrowded urban schools where many math and science teachers are certified in areas other than math and science, may advance through the mathematics sequence with acceptable grades but fundamentally unprepared for the next level of math instruction. As a consequence, much of the knowledge they have may be precarious and almost entirely verbal. The present investigation would suggest that this is precisely the case. To repeat, a well-designed ethnographic study of actual classrooms, of the content of instruction, of the assignments given and the grading criteria used, would be an enormous contribution to our understanding.

In addition to in-classroom studies, research is needed on exactly how students spend their non-classroom hours. Other things being equal, can individual differences in proficiency be traced systematically to the amount and quality of non-classroom time that students are engaged in relevant tasks? Student self-reports are often unreliable and generally insufficient. Observational studies of non-classroom activities, while expensive, are not impossible.

Finally, it should be noted that, although I have deliberately omitted discussion of social and psychological factors involved in performance on cognitive measures, including measures of quantitative reasoning, such factors are clearly important. Claude Steele’s highly original and insightful investigations into the phenomenon of “stereotype threat” are a case in point (Steele 1997). He convincingly demonstrated that individuals who are the object of a negative stereotype (as African American students are with respect to measures of intelligence and scholastic ability) tend to so internalize the stereotype that it adversely affects their performance on such measures. When students were randomly asked to solve a series of problems under stereotypically threatening conditions (“this is a test of intelligence”) and under innocuous conditions (“this is an experiment on processing strategies”), African American students randomly assigned to the former conditions did significantly worse. We need to know how pervasiveness this phenomenon is and to devise effective ways to counter its potentially harmful effects on student academic growth.

“How can states, districts, and schools undo the structural impediments to equitable course participation—e.g., uneven resources for high-level math and science among schools, tracking practices within schools, and the uneven assignment of teachers to schools and to tracks within schools?”

Our review of research on equity in mathematics and science course-taking and achievement reveals that, in a decade of policies pressing for high standards in schools that remain separate and unequal, we’ve made some progress in raising the levels of course-taking and achievement for all racial groups. However, we’ve done little to reduce the gaps among them. Although the increases are encouraging, they have served to raise standards for admission to competitive colleges in ways that prevent most low-income and minority students from translating their improved accomplishments into enhanced educational and life chances. However, our review also supports the claim made last year by the Task Force on Minority High Achievement that we have learned a great deal “about how minority educational outcomes can be improved, despite having made only modest investments in educational R&D” (College Board 1999, p. 14). We conclude with the task force that we must “redouble our efforts and our investments” to promote minority opportunities and high achievement (p. 14). To forward this agenda, we offer a set of research questions about the general educational system as well as questions specific to math and science education. We believe that both types of questions are necessary, as researchers and policymakers implement what we already know and mount new, vigorous initiatives to learn more and do more to achieve equitable course-taking and achievement.

Currently we are unable to draw on the full range of talents in our diverse population due to our lack of specific understanding of the value of diversity to learning and social advancement. We must dismiss the idea that we value diversity for diversity’s sake and start believing in the idea that diversity is needed to better us all. We must take the challenge posed by Rita Colwell in her foreword to Women, Minorities, and Persons with Disabilities in Science and Engineering: 1998, in which she writes:

A challenge for our country is to attract the best talent from all sources to science and engineering to stimulate creativity, innovation, and change; contribute to the advancement of science and engineering; and foster a scientifically literate population. Different perspectives, talents and experiences produce better ideas (NSF 1999, p. ii).

The answers to the following questions will help us better engage a broader section of our population in learning and contributing to science and mathematics:

· What can educators learn from science about the advantages of diversity in the natural world?

· What contributions do diversity and heterogeneity make to learning and change in social institutions?

· What could constructs from sociocultural perspectives on learning, such as learning in “communities of practice,” through “apprenticeship,” as “changing participation over time,” as “identity development,” etc., contribute to our general understanding of learning in diverse settings?

· How can math and science courses capitalize on diversity and heterogeneity to maximize learning? How, for example, might a greater emphasis on diversity contribute to all students’ multiple ways of knowing math and science?

We have specific evidence from research and equity interventions about school conditions likely to promote more equitable course-taking and achievement. A college-going culture at school, high-quality curriculum, well-prepared and knowledgeable teachers, special academic assistance when needed, supportive relationships with caring school adults, and connections with families focused on high achievement and college-going all seem to foster the outcomes we seek for low-income students and students of color. But to translate these features of exemplary schools and effective special programs into the routine, everyday practices experienced by low-income students of color presents enormous challenges. Research focused on the following questions should help:

· How can states, districts, and schools undo the structural impediments to equitable course participation—e.g., uneven resources for high-level math and science among schools, tracking practices within schools, and the uneven assignment of teachers to schools and to tracks within schools?

· How do schools create academic, college-going cultures where adults and peers see college-going as expected and attainable, and where they see the effort and persistence that preparation for college requires as normal?

· How can we piece together what we know from effective “equity programs”—including their provision of intensive academic and college-going support and close relationships between students and adults—to create an equitable science and mathematics educational system?

· How can schools, working with community organizations, develop connections with parents and neighborhoods that enhance their knowledge of and access to mathematics and science courses, high achievement, and college preparation?

How can we create courses that make mathematics and science content more accessible to all American students? In contrast to commonly held views that low-income and minority students devalue education, studies suggest that they more likely to turn away because of a real or perceived lack of opportunities (Steinberg 1996). A recent RAND study of low-income high school graduates who were eligible to attend the University of California but chose not to found that the students were most deterred by their beliefs that the university is “not for people like me” (Krop et al. 1998). These perceptions arise, in part, as students internalize negative labels assigned to their racial and cultural groups—what Claude Steele (1997) terms “a stereotype threat.” Creating courses where minority students can see the connections between themselves and the content of science and mathematics and where teachers use pedagogy that builds on students’ cultures and languages is one way to counter this threat. However, we need to know far more about what such courses might be like. Research into the math and science education questions below should help us develop a system in which students hold identities that are simultaneously multicultural and academic:

· How can science and mathematics be treated as everyone’s “everyday practices”?

· What are multicultural curricula and culturally relevant pedagogies in mathematics and science?

· Does the absence of multicultural and diversity issues in the National Science Education Standards prohibit equitable implementation of the standards?

· What assessments capture and respect multiple ways of knowing mathematics and science?

· Is Advanced Placement and the pipeline of courses that lead to it an equitable (or even the “best”) approach to advanced study in mathematics and science?

The National Task Force on Minority High Achievement puts it simply: “America is a diverse society in which educational differences have the potential to become a progressively larger source of inequality and social conflict” (College Board 1999, p. 1). Efforts to construct the math and science education system in ways that the literature suggests are necessary to make participation and high achievement possible for low-income students of color will inevitably bring political resistance from powerful forces bent on preserving the status quo. California’s recent rejection of affirmative action provides a sobering example. This response is understandable in a stratified educational system where opportunities are based on ideologies of intelligence and merit that disadvantage some groups and favor others. Are we to just sit by and let conflicts build? Or could research on the following issues reveal ways that Americans might move more harmoniously toward a diverse, high-achieving, and equitable society?

· What is the impact of our culture’s framing of mathematics and science achievement as “culture free” and ideologically neutral? How do we dismantle the elite and esoteric status of science and mathematics as fields of study?

· How can we change prevailing attitudes about who can learn mathematics and science? What alternative measures of competence and potential help reduce race and social class sorting?

· How do we develop norms whereby Americans see deep engagement, high achievement, and hard work in mathematics and science as normal and expected of all?

· How can we counter the often-unspoken race and social class fears that complicate efforts to create equitable course-taking and achievement?

· How can we unseat ideologies of competition and merit in schools that perpetuate social and racial stratification in school and beyond?

· How can more equitable schools hold on to children from families used to having a competitive advantage?

· How can we make salient that our goal must not simply be to accommodate “minorities,” but to educate everyone well in an increasingly diverse society?

“We often see ‘minorities’ as one huge homogeneous class of people. This is especially true of Hispanics, but vast disparity exists among different Hispanic groups. We would like to see more research on meaningful classifications of minority subgroups with meaningful data collection based on those groups.”

How could so much time, money, and effort be put into a problem with so little success? For at least three decades, this nation has attempted to increase the participation of underrepresented minorities in science, mathematics, engineering, and technology with dismal results. Countless research projects have been conducted with numerous programs implemented with so little improvement that one wonders if the same increase would have occurred if none of these efforts had ever been expended.

In this paper we will resist lamenting the failures of the past but will rather call for a major reexamination of the system. It is time to ask if new programs grafted onto an ailing system will solve the problem. Thirty years of failure should tell us that for underrepresented minorities, the system is irreparably flawed, and no fine-tuning of existing structures is going to fix it. We call for research to define an educational system that supports minority success.

Have you noticed that as long as teen violence remained confined to cities, it was not seen as an American problem? We expect urban minority kids to be violent. Not until teen violence emerged in rural and small-town white America did the country see it in crisis proportions. What do we as a nation believe about cities (and we really mean inner-cities), and how much of the cities’ failures do we merely accept as a consequence of minority culture? None of us see cities as just very big small towns. Cities and city schools are driven by parallel minority cultures. Large urban school districts are 85 to 90 percent underrepresented minorities. (See table.) Do we just expect them to be bad?

Ethnicity of Selected Urban School Districts

Sources: http://www.cps.k12.il.us/AboutCPS/Statistical_Information/statistical_information.html; http://www.lausd.k12.ca.us/lausd/demographics/; and http://www.houstonisd.org/About/Pubs/AboutHISD/index.htm.

We have to learn more about these cultures and our expectations if we are ever going to invent an educational system that is a good fit for urban America. How can we invent a new educational model for our cities until we expect better for them?

As a nation, we understand so little of the issues of diversity. We often see “minorities” as one huge homogeneous class of people. This is especially true of Hispanics, but vast disparity exists among different Hispanic groups. Two-thirds of the American Hispanic population are Mexican American, yet the majority of Hispanic scientists in America are European, South American, Central American, or Cuban American. We would like to see more research on meaningful classifications of minority subgroups with meaningful data collection based on those groups.

What has happened to minority males? After examining 1997 SAT data, Donald Stewart, former head of the College Board, identified African American and Mexican American males as the lowest performing subgroups (College Board 1997). Walk into historically black universities and colleges, and you will see a majority female population. Consequently, gender equity programs directed toward females in cities with majority “minority” populations make little sense. What kinds of programs are needed to engage minority males in science?

Over roughly the same time period that the nation has attempted to increase participation of minorities, we have also attempted to increase the participation of women. It is clear that in most areas, the women’s movement has been more successful than the minority movement (Mortenson 1999). We hasten to add that this work is not complete; women have made great strides in many fields, but are not at parity in engineering or computer science, for example. Yet there may be lessons learned that can be exported to the minority movement.

I found these three papers to be very stimulating. There were a lot of data, and a lot of facts¾and I like data, being in the evaluation business. Looking across the three studies, I would conclude the following: that schools with underrepresented students need to provide them with access to a full range of college preparatory mathematics. In these courses, teachers need to strive for their students to gain both clarity and procedural knowledge, and we need to think about students as individuals and not as groups or broad categories. Gloria Ladsen-Billings at the University of Wisconsin says that student performance is not predetermined. As we look at indicators and we put people into groups¾and Professor Tapia said this well, “diversity within diversity”¾we have to look at individual students and see what their needs are, and educate them. As Jeannie Oakes also said, we need to educate a diverse society. That is a key point¾we need to think of students as individuals and not think of their performance as predetermined. I think of the article by Oakes, Muir, and Joseph as being a cross-section of a problem. I think of the Tapia and Lanius paper as looking at the longitudinal question. And I think of the Bond paper as looking at learning needs and issues at the microlevel. All of them get at the learning issue, which I think is very critical.

In Oakes, Muir, and Joseph’s paper, there were many important ideas. Many students reside at high schools that do not offer the necessary set of mathematics and science courses for them to be eligible for higher education. That is a problem we need to deal with. Low-progress schools offer a higher number of below algebra courses and less AP courses. Students in poverty have less-qualified teachers¾this is a very important issue that none of the speakers discussed but which was included in their papers. The papers provided very compelling evidence of the ills of tracking.

The Tapia and Lanius paper showed very little increase in the percentage of African Americans, Hispanics, and Native Americans who have obtained bachelor’s, master’s, and Ph.D.s in science and engineering. This is a problem. I thought they provided very compelling evidence of that. I like the idea of “diversity within diversity”¾I think this is a very strong and important point in this paper.

College admissions need to consider other criteria than less-biased mechanisms, such as using only the SAT¾this notion of the SAT as a threshold is a very important point. You give a baseline of value, but anybody who can reach above that¾then you make a variety of decisions. Tapia and Lanius also use the words “affirmative development”; this is another critical idea in their paper, another area in which we need to take action. They point out that the three particular barriers [to what?]were recruitment, admissions, and retention, things that have policy and learning implications¾and also implications in trying to address the problems.

Concerning Lloyd Bond’s paper, I thought Larry Suter did a good job of pointing out some issues differences between declarative knowledge and procedural knowledge.[3] He thinks you must look at what students learn. What was so powerful about this paper is that Bond really identifies some learning issues. He says that these students have the clarity of knowledge and they have the facts, but how can they put the facts together? In his paper, he shows the protocols and interviews students on how they are thinking; he demonstrates that, yes, the students have the facts, but they don’t have the procedures to put them together to solve complex problems¾this gets back to education, learning, and qualified teachers. Another important issue to look at is what is the teachers’ knowledge? What is the procedural knowledge and what is the declarative knowledge of the teachers? This is something people don’t like to talk about, but we need to. We need to open things up as we look at education for a diverse society; we need to think about many different issues.

I’d like to draw on some of my experience in past work I’ve done with NISE. One is this issue of alignment. The Texas state assessment issue that Professor Tapia pointed out is very critical, and part of the issue is alignment. We’ve done a number of studies of state assessments as they match standards. Basically, large-scale assessments cover only about 33 percent¾or even less than half¾of the standards, so it is very difficult on any large-scale assessment to cover everything. In our assessment world, where we talk about multiple measures, you need multiple measures and you need monitoring of any large-scale assessment program. This is costly, and this is something we are looking at in Milwaukee. Recently, Milwaukee adopted a yearly testing program because the state wanted more information on value adding. But we wondered how you can guard against standardized tests becoming the curriculum, which it sounds like the Texas state assessment has become? Well, you can do it. You need other kinds of mechanisms. You also need to look at the monitoring process. One thing they built into the Milwaukee program is classroom-based assessment¾that is, there will be more open-ended assessments, assessments that will be large scale, and that will be more aligned with what you want instruction to be. The idea is to have more than one mechanism with which to make decisions.

We also have looked at the issue of curriculum reform. When we look at transcript data of schools with a large proportion of minority students, we see that this leads to more of the underrepresented students taking four years of high school, college-qualified mathematics. We compare that with students who began high school in the algebra strand. This raises the question of what we mean by high school mathematics, and how it should be organized. The European countries think of math as 1, 2, and 3; they don’t divide it into algebra, geometry, algebra 2, precalculus, and calculus; focusing on AP as the pinnacle of education and of high school is a problem. It is something we have to think about, although I know it is part of our culture.

One of the things this leads to¾and something that the speakers have talked about¾is looking at college as the reason for high school. We clearly need more representation among college graduates, and also in graduate education, but we’re not serving our students well if we only provide them with a college track. I think that this is a form of tracking. I recently heard statistics that stated that, for the first time in a long time, the United States does not have the highest percentage of students with college degrees; England does. England has about 35 percent of its students with college degrees, while the United States has about 33 percent. We’re talking about a third of our population with college degrees. What about the other population, the other two-thirds? Are they not doing anything? There are a lot of things available, and there are a lot of jobs available that are not being filled by trained people.

[1]While this short paper emphasizes research issues in evaluating programs, the longer paper by Campbell and Hoey (2000) ~~available at www.nise.org~~. also reviews the effectiveness of programs targeting gender and race. [I’ve added this to the refs; note tho’ that I couldn’t find this publication on the web site. Also note that even tho’ this link works, the URL is technically http://www.wcer.wisc.edu/nise/; please advise]

[2]Bond refers to original research that he describes in a more extensive paper, which is available at www.nise.org along with the more extensive background papers for each of the panelist’s short papers provided in this report.

[3]Larry Suter, chair of this Forum session, presented Lloyd Bond’s paper since he was unable to attend.

Ethnicity	Chicago	Houston	New York	Los Angeles
Latino	34.2%	52.5%	37.7%	69.1%
African American	52.5%	34.1%	35.7%	13.6%
Asian/Pacific Islander	3.2%	2.8%	10.8%	6.5%
Native American	0.2%	0.1%	0.3%	0.3%
White	10.0%	10.6%	15.5%	10.5%