Mathematical Thinking CATs


WHY USE THE MATH CATs?
The Mathematical Thinking Classroom Assessment Techniques (Math CATs) are designed to 
promote and assess thinking skills in mathematics. 

WHAT ARE THE MATH CATs?
The Mathematical Thinking Classroom Assessment Techniques (Math CATs) are designed to 
promote and assess thinking skills in mathematics. Few faculty have difficulty finding or 
developing tools which assess the specific mathematical techniques which they teach; a challenge 
which faculty do face is to find ways to promote and assess the development of mathematical 
thinking - notably to help students know what to do when faced with problems which are not 
identical to the technical exercises commonly encountered in mathematics classes. 

Here we define mathematical thinking as 
...the development of a mathematical point of view - valuing the process of 
mathematization and abstraction and having the predilection to apply them; and the 
development of competence with the tools of the trade, and using those tools in the service 
of the goal of understanding structure. (Schoenfeld, 1992)  

The Math CATs are designed to address this challenge by offering ways to assess and instill a 
broad range of the mathematical thinking skills important for students in the science, mathematics, 
engineering, and technology disciplines. These skills include: 
· checking results and correcting mistakes (Fault finding and fixing) 
· making plausible estimates of quantities which are not known (Plausible estimation) 
· modeling and defining new concepts (Creating measures) 
· judging statements and creating proofs (Convincing and proving) 
· organize unsorted data and draw conclusions (Reasoning from evidence) 


WHAT IS INVOLVED?
Instructor Preparation Time:
Minimal if use existing tasks.

Preparing Your Students:
Students will need some coaching on their 
first task.

Class Time: 
Some tasks take 5 minutes; others as much 
as 45 minutes.

Disciplines: 
Varies with specific CAT.

Class Size: 
Any.

Special Classroom/Technical 
Requirements: 
None except for Reasoning from Evidence 
CAT.

Individual or Group Involvement: 
Either.

Analyzing Results: 
Varies with specific CAT.

Other Things to Consider: 
Fairly demanding task for students who are 
unfamiliar with open-ended problems.



Description of the 5 Math CATs

1. Fault finding and fixing
There are a number of well-known misconceptions held by students of mathematics, many 
of which persist undetected into the college years. These misconceptions need to be 
identified and remedied to avoid major conceptual problems later. Many of the examples 
make use of common misconceptions, and so the task can play a diagnostic role. 

The tasks in this package offer students a number of mathematical mistakes which they are 
asked to diagnose and rectify. These require students to analyze mathematical statements 
and deduce from the context the part that is most likely to contain the error (there may be 
more than one possibility), explain the cause of the error and rectify it. Such tasks can be 
quite demanding. It is often more difficult to explain the cause of another's seductive error 
than to avoid making it oneself. Contexts include percentages, graphical interpretation, and 
reasoning from statistical data (download tasks). 

Example: Double Coin Toss
- I'll toss two coins.
- If they both come up heads then Jane wins.
- If they both come up tails then Ben wins.
- If we get one head and one tail then I win.

Explain why this is not a fair game. 
(Answer.) 

2. Plausible estimation (Fermi problems)
Plausible Estimation consists of a one or two easily-stated questions which at first glance 
seem impossible to answer without reference material, but which can be reasonably 
estimated by following a series of simple steps that use only common sense and numbers 
that are generally known or are amenable to estimation. 

Plausible Estimation tasks involves students in an activity central to modelling in science, 
other areas of intellectual activity, and in everyday life. The core skill is to create (or check) 
estimates of quantities that, at first glance, seem unknowable. Students are also required to 
communicate their assumptions and results and check the plausibility of their answers. In 
addition, Plausible Estimation requires students to practice arithmetic fluency, ability to 
handle large numbers, and conversion of units (download tasks). 

Example: How many babies are born in the United States each minute?
(Answer.) 

3. Creating measures
Creating Measures consists of a series of questions that prompt students to evaluate an 
existing measure of an intuitive concept , and then create and evaluate their measure of this 
concept. 

We constantly "mathematize," or construct measures for, physical and social phenomena 
and use these models to make decisions about our everyday lives. These can vary from 
measures of simple quantities (such as "speed" or "steepness") to complex and subjective 
social ones (such as "quality of life" or "best universities"). Since these measures are 
mathematical models of some phenomenon, they are open to criticism and improvement, 
especially when considering their usefulness. These tasks provide a fun and interesting 
way to assess your students' abilities to "mathematize" concepts and show students that 
there can be many different formal, quantitative measures of such concepts. More 
importantly, they emphasize that measures differ in their utility; some are more useful than 
others in representing concepts (download tasks). 

Example: Steepness 
 





Without measuring anything, put the above staircases in order of steep-ness.










4. Convincing and proving
This CAT introduces the notions of convincing and proving and illustrates several kinds of 
proofs commonly encountered in mathematics. These tasks are intended to assess how well 
students are able to argue logically, use examples and counterexamples to support their 
reasoning and identify breakdowns in rational argument. In addition, some tasks reveal 
common student misconceptions students make in their reasoning (download tasks). 

There are two types of tasks: 
1. Evaluate a set of statements as "always, sometimes or never true".
 Students are expected to offer examples, counterexamples, and reasons for their 
decisions. 
2. Evaluate "proofs" and distinguish the correct from the flawed. 

Example: If two rectangles have the same perimeter, they have the same area.
Is this always, sometimes or never true? 
(Answer.) 

5. Reasoning from evidence
This CAT requires students to analyze unsorted data. The tasks will assess students' 
abilities to organize information, represent it in a meaningful way, and draw sensible 
conclusions. It is an important skill especially for students in a SMET discipline, to be able 
to analyze and interpret data, and argue critically and make informed decisions based on 
sound reasoning obtained from this data (download tasks). 

Example: In this example, you are a Road Safety Advisor.
Your task is to produce some suggestions about how road safety in Smallville might be 
improved. 

To help you, below you have a map of Smallville and a database of traffic accidents that 
took place during the last year. These figures show the time and place of the accident, 
details of the victim and the type of vehicle that caused the accident. (Times are given as 
decimals, to make graphing easier). 

Your task is to: 
1. Find the trouble spots in the town. 
2. Try to decide why they are trouble spots. 
3. You have $100,000 to spend on improving road safety. 



Goals

Instruction in mathematics should help students: 
· become independent learners, interpreters, and users of mathematics; 
· feel confident in their ability to do mathematics; 
· develop mathematical thinking, to analyze and understand, and to perceive 
structure and structural relationships; 
· develop analytical skills, and the ability to reason in extended chains of
· argument; 
· understand important concepts; 
· to have a broad range of approaches and techniques; 
· by providing a broad range of problems; 
· focus on conceptual understanding and technical skills; 
· apply what they know to new contexts; 
· present clear, coherent arguments; 
· develop precision in written and oral presentations; 
· with a sense of what mathematics is and how it's done. 

If students are to achieve these goals, then an appropriate intellectual environment in which they 
learn mathematics must be created. The MathCATs provide materials which support the creation of 
such learning environments. 


Theory & Research

On the nature of mathematics
Mathematics...today is a diverse discipline that deals with data, measurements, and observations 
from science; with inference, deduction, and proof; and with mathematical models of natural 
phenomena, of human behavior, and of social systems...

In addition to theorems and theories, mathematics offers distinctive modes of thought which are 
both versatile and powerful, including modeling, abstraction, optimization, logical analysis, 
inference from data, and use of symbols. Experience with mathematical modes of thought builds 
mathematical power -- a capacity of mind of increasing value in this technological age that enables 
one to read critically, to identify fallacies, to detect bias, to assess risk, and to suggest alternatives. 
Mathematics empowers us to understand better the information-laden world in which we live.  
(National Research Council, 1989, pp. 31-32).

This quotation describes a number of distinct features of mathematics; a body of knowledge; a set 
of tools to be used in everyday life and by the scientific community; and a set of powerful thinking 
tools.  Mathematics is seen as discipline which empowers its students.  It follows that education in 
mathematics should set out to address each of these features.

Challenges facing teaching
Teachers of undergraduate mathematics and other disciplines that rely on mathematics face a 
number of challenges. Serious conceptual problems have been documented in students who seem 
appropriately qualified.  Armstrong and Croft (1999) set diagnostic tests of mathematics on entry 
to engineering programs, and showed several areas of weakness. For example, around 20% of 
students had problems dealing with significant figures, and around 15% had problems with 
decimals. Student ratings of their confidence that they understood and could apply mathematics 
ranged from about 60% for the graph of a linear function, to about 10% for polar co-ordinates.  
Faculty are often dissatisfied with student knowledge on entry to freshman mathematics classes 
(the Royal Society, 1998).  Particular problems are associated with: student fluency and accuracy; 
failure to understand the connectedness of mathematics; a lack of understanding of proof and the 
need for rigorous argument; and an inability to solve non-standard problems (e.g. Tall, 1992).

A number of commentators have identified problems with conventional approaches to the teaching 
of mathematics in high schools and colleges.  Weaknesses are related to pedagogy and assessments 
which emphasize the mastery of mathematical techniques, but emphasize neither the conceptual 
side of mathematics nor the development of the habits of mind that characterize mathematical 
thinking.  Traditional testing methods in mathematics have often provided limited measures of 
student learning, and equally importantly, have proved to be of limited value for guiding student 
learning. The methods are often inconsistent with the increasing emphasis being placed on the 
ability of students to think analytically, to understand and communicate, or to connect different 
aspects of knowledge in mathematics (e.g. Ridgway, 1988; Brown, Bull and Pendlebury, 1997). 

One consequence of this type of curriculum and assessment system is that students learn in school 
that problems mostly have neat, unique solutions, and that methods to solve problems will be 
provided to them.  For example, in the 1983 National Assessment of Educational Progress, nine 
students out of ten agreed with the statement "There is always a rule to follow in solving 
mathematics problems" (NAEP, 1983, pp. 27-28).  Over time students come to adopt a passive 
role, and think of mathematics as a dead body of knowledge which they have to memorize, rather 
than as a set of higher-order thinking tools which will increase their abilities to deal with a complex 
world. (e.g. Carpenter, Lindquist, Matthews, & Silver, 1983; Schoenfeld 1992). 


Developing Mathematical Thinking
...the reconceptualization of thinking and learning that is emerging from the body of recent work 
on the nature of cognition suggests that becoming a good mathematical problem solver - becoming 
a good thinker in any domain - may be as much a matter of acquiring the habits and dispositions of 
interpretation and sense-making as of acquiring any particular set of skills, strategies, or 
knowledge. (Resnick, 1989, p. 58).

Thinking mathematically depends on a number of different components (Schoenfeld, 1992), 
notably core knowledge, problem solving strategies, effective use of one's resources, having a 
mathematical perspective, and active engagement in the practice of mathematical thinking.  
Mathematics instruction must present experiences which develop student knowledge in each of 
these areas.

Mathematics in the classroom should model these elements if students are to come to understand 
and use mathematics and to learn to think mathematically.  Learning mathematics is about learning 
to work in the ways that mathematicians work, and is about acquiring the thinking skills that 
mathematicians use.  These skills are important for scientists as well as mathematicians.  P—lya 
(e.g. 1954, 1957) argued that mathematics resembles the physical sciences in its dependence on 
conjecture, insight, and discovery.  He argued that for students to understand mathematics, their 
experience with mathematics must be consistent with the way mathematics is done by 
mathematicians.

There is an extensive body of knowledge comparing the knowledge of experts and novices (e.g. 
Ericsson and Charness, 1994 for a review across disciplines; Schoenfeld, 1985 for studies in 
undergraduate mathematics) which can be mined for ideas on appropriate teaching strategies; and a 
great many studies which show the effectiveness of particular teaching methods (e.g. Palinscar and 
Brown, 1984).  Accessible accounts of the literature are provided by Schoenfeld (1983, 1985) and 
Bransford, Brown, and Cocking (eds.) (1999).

Developing Assessment
Improved assessment systems may help with these problems. There is evidence that educational 
attainment can be raised by better assessment systems (Black and William, 1998; Dassa, Vazquez-
Abad, and Ajar; 1993).  Such assessment systems are characterized by: a shared understanding of 
assessment criteria; high expectations of performance; rich feedback; and effective use of self-
assessment peer assessment, and classroom questioning.

The intention of the mathCATs is to improve the quality of both formative and summative 
assessment systems, and thereby to improve the quality of undergraduate mathematics teaching and 
learning.


Links
MARS web site (http://www.nottingham.ac.uk/education/MARS/)

Mathematics Association of America (http://www.maa.org/Welcome.html)

Cooperative Learning in Undergraduate Mathematics Education (Project
CLUME) (http://www.uwplatt.edu/~clume/)

Association for Research in Undergraduate Mathematics Education 
(http://www.maa.org/data/t%5Fand%5Fl/Arume1.html)

The Math Forum (http://forum.swarthmore.edu/)


Sources
Balanced Assessment Group (1998, 1999). Balanced Assessment for the
Mathematics Curriculum: High School Assessment (2 volumes). White
Plains, NY: Dale Seymore. 

Balanced Assessment Group (1998, 1999). Balanced Assessment for the
Mathematics Curriculum: Advanced High School Assessment (2 volumes).
White Plains, NY: Dale Seymore. 

Gold, B., Keith, S., and Marion, W. (Eds.). (1999). Assessment Practices in
Undergraduate Mathematics. Washington, DC: MAA. 


References
Armstrong, P., and Croft, A. (1999). Identifying the learning needs in mathematics of entrants to 
undergraduate engineering programmes. European Journal of Engineering Education, 14(1).

Black, P., and William, D. (1998). Assessment and classroom learning. Assessment in 
Education, 5(1), 7-73.

Bransford, J., Brown, A., and Cocking, R. (eds.) (1999). How People Learn. Washington, DC: 
National Academy Press.

Brown ,G., Bull, J., and Pendlebury, M (1997). Assessing Student Learning in Higher 
Education. London: Routledge.

Carpenter, T. P., Lindquist, M. M., Matthews, W., & Silver, E. A. (1983). Results of the third 
NAEP mathematics assessment: Secondary School. Mathematics Teacher, 76 (9), 652-659.

Dassa, Vazquez-Abad, and Ajar; 1993


Ericsson, K., and Charness, N. (1994). Expert performance: its structure and acquisition. 
American Psychologist, 49, 725-745.

The London Mathematical Society (1995). Tackling the Mathematics Problem. London: London 
Mathematical Society.

Marton, F, and Saljo, R. (1976). On qualitative differences in learning: I Ð outcome and process. 
British Journal of Educational Psychology, 46, 4-11.

National Assessment of Educational Progress. (1983). The Third National Mathematics 
Assessment: Results, trends, and issues (Report No. 13-MA-01).  Denver, CO: Educational 
Commission of the States.

National Research Council. (1989). Everybody Counts: A report to the nation on the future of 
mathematics education. Washington, DC: National Academy Press. 

Palincsar, A., & Brown, A. (1984). Reciprocal teaching of comprehension-fostering and 
comprehension-monitoring activities. Cognition and Instruction, 1(2), 117-175.

P—lya, G. (1954). Mathematics and Plausible Reasoning (Volume 1, Induction and analogy in 
mathematics; Volume 2, Patterns of plausible inference). Princeton: Princeton University Press.

Polya, G. (1957). How to Solve It: a new aspect of mathematical method. Princeton, 
NJ:Princeton University Press.

Resnick, L. (1989). Treating mathematics as an ill-structured discipline. In R. Charles & E. Silver 
(Eds.), The Teaching and Assessing of Mathematical Problem Solving. Reston, VA: National 
Council of teachers of Mathematics.

Ridgway, J. (1988).  Assessing Mathematical Attainment. Slough: NFER-Nelson.

Ridgway, J., Swan, M., and Burkhardt, H. (2001, in press). Assessing Mathematical Thinking 
via FLAG. In D. Holton and M.Niss (eds.): Teaching and Learning  Mathematics at University 
Level - An ICMI Study. Dordrecht: Kluwer Academic Publishers.

The Royal Society (1998). Mathematics Education pre-19. London: The Royal Society.

Schoenfeld, A. (1983). Problem solving in the mathematics curriculum: A report, 
recommendations, and an annotated bibliography.  Washington, DC:Mathematical Association 
of America.

Schoenfeld, A. (1985). Mathematical Problem Solving.  New York: Academic Press.

Schoenfeld, A. H. (1992). Learning to think mathematically: problem solving, metacognition, and 
sense-making in mathematics. In D. Grouws, (ed.). Handbook for Research on Mathematics 
Teaching and Learning. New York: MacMillan. pp. 334-370.

Tall, D. (1992). The transition to advanced mathematical thinking: functions, limits, infinity and 
proof.  In D.A.Grouws (ed.) Handbook of Research on Mathematics Teaching and Learning. 
New York: Macmillan.


Malcolm Swan
Mathematics Education
University of Nottingham
Malcolm.Swan@nottingham.ac.uk

 
Most assessment practices seem to emphasise the reproduction of imitative, standardised 
techniques. I want something different for my students. I want them to become mathematicians - 
not rehearse and reproduce bits of mathematics.

I use the five 'mathematical thinking' tasks to stimulate discussion between students. They share 
solutions, argue in more logical, reasoned ways and begin to see mathematics as a powerful, 
creative subject to which they can contribute. Its much more fun to try to think and reach solutions 
collaboratively. Assessment doesn't have to be an isolated, threatening business. 

Not just answers, but approaches.

Malcolm Swan is a lecturer in Mathematics Education at University of Nottingham and is a leading 
designer on the MARS team. His research interests lie in the design of teaching and assessment. 
He has worked for many years on research and development projects concerning diagnostic 
teaching (including ways of using misconceptions to promote long term learning), reflection and 
metacognition and the assessment of problem solving. For five years he was Chief Examiner for 
one of the largest examination boards in England. He is also interested in teacher development and 
has produced many courses and resources for the inservice training of teachers. 


Jim Ridgway
School of Education
University of Durham
Jim.Ridgway@durham.ac.uk

 
Thinking mathematically is about developing habits of mind that are always there when you need 
them - not in a book you can look up later.

For me, a big part of education is about helping students develop uncommon common sense. I 
want students to develop ways of thinking that cross boundaries - between courses, and between 
mathematics and daily life.

People should be able to tackle new problems with some confidence - not with a sinking feeling 
'we didn't do that yet'. I wanted to share a range of big ideas concerned with understanding 
complex situations, reasoning from evidence, and judging the likely success of possible solutions 
before they were tried out. One problem I had is that my students seemed to learn things in 'boxes' 
that were only opened at exam time. Thinking mathematically is about developing habits of mind 
that are always there when you need them - not in a book you can look up later.

You can tell the teaching is working when mathematical thinking becomes part of everyday 
thinking. Sometimes it is evidence that the ideas have become part of the mental toolkit used in 
class - 'lets do a Fermi [make a plausible estimate] on it'. Sometimes it comes out as an anecdote. 
On graduate told me a story of how my course got him into trouble. He was talking with a senior 
clinician about the incidence of a problem in child development, and the need to employ more 
psychologists to address it. He 'did a Fermi' on the number of cases (wildly overestimated) and 
the resource implications (impossible in the circumstances). He said there was a silence in the 
group...you just don't teach the boss how to suck eggs, even when he isn't very good at it. He 
laughed.

Jim Ridgway is Professor of Education at the University of Durham, and leads the MARS team 
there. Jim's background is in applied cognitive psychology. As well as kindergarten to college 
level one assessment, his interests include the uses of computers in schools, fostering and testing 
higher order skills, and the study of change. His work on assessment is diverse, and includes, the 
selection of fast jet pilots, and cognitive analyses of the processes of task design. In MARS hhe 
has special responsibility for data analysis and psychometric issues, and for the CL-1 work. 


About MARS
The Mathematics Assessment Resource Service, MARS, offers a range of services and materials in 
support of the implementation of balanced performance assessment in mathematics across the age 
range K to CL-1. MARS is funded by the US National Science Foundation, and builds on earlier 
funding which began in 1992 for the Balanced Assessment Project (BA) from which MARS grew. 

MARS offers effective support in: 
The Design of Assessment Systems: assessment systems are tailored to the needs of specific 
clients. Design ranges from the contribution of individual tasks, through to full scale 
collaborative work on test development, scoring and reporting. Clients include Cities, States, 
and groups concerned with educational effectiveness, such as curriculum projects and 
professional development initiatives. 

Professional Development for Teachers: most teachers need help in preparing their students for 
the much wider range of task types that balanced performance assessment involves. MARS 
offers professional development workshops for district leadership and 'mentor teachers', built 
on materials that are effective when used later by such leaders with their colleagues in school. 

Developing Design Skills: many clients have good reasons to develop their own assessment, 
either for individual student assessment or for system monitoring. Doing this well is a 
challenge. MARS works with design teams in both design consultancy and the further 
development of the team's own design skills. 

To support its design team, MARS has developed a database, now with around 1000 interesting 
tasks across the age range, on which designers can draw, modify or build, to fit any particular 
design challenge.