Go to Collaborative Learning Go to FLAG Home Go to Search
Go to Learning Through Technology Go to Site Map
Go to Who We Are
Go to College Level One Home
Go to Introduction Go to Assessment Primer Go to Matching CATs to Goals Go to Classroom Assessment Techniques Go To Tools Go to Resources




Go to Why do Assessment?
Go to An introduction to assessment - the basics
Go to Assessment within the context of course development
Go to Assessment references
Go to Assessment authors
Go to Entire Primer
Go to Download Primer

Go to previous page.

Assessment Primer

(Screen 3 of 5)
Go to next page.

Assessment within the Broader Context of Course Development: A More In-Depth Look at Assessment - The Holistic View

Assessment Drives Course Development

Assessment plays a dual role: It drives student learning and it provides important feedback for both students and instructors. In An Introduction to Assessment: The Basics, we focused primarily on the former of these two roles. Students want to "do well" (receive high grades) in their courses, and they use our assessment techniques as the means for determining what it is we expect them to learn and for diagnosing how well they are learning it. Students will adapt their mastery of course content to what our assessment techniques require of them. Thus, if we want students to achieve our course goals, we must choose appropriate assessment techniques that guide our students to those goals.

In this document, we build upon the feedback role of assessment. In particular, we discuss the feedback that assessment provides the instructor, and the ways in which assessment drives course development. We want our students to "do well" (attain our course goals), and we use our assessment techniques as the means for diagnosing how well our students are doing. We can modify our curriculum and instructional methods based on what our assessment techniques tell us about what students are learning compared to what we want them to learn. We must, therefore, choose assessment techniques that will give us useful feedback for refining our curriculum and our instructional methods so that we achieve our course goals.

What follows is a generalized model for course development, within which assessment plays a critical role. Not surprisingly, this course development model centers on course goals. As we will see, our goals determine the curriculum, instruction, and assessments that are best suited for the course, and we will see that assessment serves as the "feedback loop" wherein we evaluate the extent to which our curriculum and instruction choices are leading to the attainment of course goals so that we may then modify the curriculum and instructional methods based on this evaluation.


A Generalized Model for Course Development

Circular model of curriculum, instruction, and assessment connected by goals, the CIA model.  An icon in the middle of the circle denotes the students in this CIA model.
Figure 1

Curriculum, Instruction, and Assessment
The three primary components of any course are the curriculum (the "content"), the instructional methods used to deliver the curriculum, and the assessment techniques with which our success in attaining course goals is evaluated. These three components (curriculum, instruction, assessment-CIA) are inextricably linked, and are bound together by the goals we set for the course. The CIA model presented here requires that goals be formalized at the outset, which is to say that goals be clearly articulated. Ultimately, it is our course goals that set the standard against which the success of the course development effort must be measured. In this context, the role of assessment is to measure the efficacy of our curriculum and of our instructional methods with respect to stated course goals. This is how curriculum, instruction, and assessment are linked in the CIA model.

While formalizing goals is the most important part of course development, it is only the first step. The path through the course development process can be envisioned as a "road map", with goals at the beginning, pointing the way, to assessment at the end, telling us if we have reached our destination or if we need to retrace our steps. This course development "road map" (Figure 1) provides a detailed set of directions, with specific actions to be taken at several signposts along the way. Starting from formalizing course goals, the "directions" are as follows:

  • translate goals into Measurable Student Outcomes
  • determine Levels of Expertise required to achieve outcomes
  • select both Curriculum and Classroom Assessment Techniques
  • choose and implement Instructional Methods
  • conduct Assessment and evaluate--were Measurable Student Outcomes realized?

Let's consider these steps in turn.

Translating Course Goals Into Measurable Student Outcomes
Assessment can measure the extent to which course goals have been achieved, but only if those goals are measurable. For the most part, however, course goals are too broad or too abstract to measure directly. This is one of the first difficulties often encountered in the course development process. For example, one course goal in an introductory astronomy course might be that "students understand the seasons." But how does one measure "understand"? This goal can be made more measurable by identifying specific outcomes one would expect from a student who "understands" the seasons. For example: The student can "define seasons" and can "distinguish the importance of different factors such as tilt and distance."

Thus, once goals have been formalized, the next step in course development is to translate the often abstract language of course goals into a set of concrete measurable student outcomes. Measurable student outcomes are specific, demonstrable characteristics--knowledge, skills, values, attitudes, interests--that will allow us to evaluate the extent to which course goals have been met. For each course goal, identify the principal outcomes one would expect from a student who has achieved that goal, keeping in mind that our ability to measure the achievement of course goals will be determined entirely on the basis of these measurable student outcomes. Of course, knowing what kinds of outcomes are actually measurable requires knowledge of the kinds of assessment techniques that are available, and what each technique can and cannot measure. Don't worry: we'll discuss different assessment techniques later, and the section on Classroom Assessment Techniques presents a variety of these assessment methods and how they connect with different measurable student outcomes. Figure 2 gives an example of translating a specific course goal (in the context of dental health) into measurable student outcomes.

The goal for the student is to understand proper dental hygiene.  The measureable outcomes include identifying the active ingredient in toothpaste, explaining why teeth should be cleaned twice a year, and describing how poor dental hygiene can lead to poor overall health.
Figure 2 - An example of translating a course goal into measurable student outcomes.

Determining Levels of Expertise Required to Achieve Measurable Student Outcomes
Having translated course goals into measurable student outcomes, we are one step closer to selecting the curriculum, instructional methods, and assessment techniques that will constitute the course. In order to select the CIA that are best suited for the course goals we have identified, it is necessary to determine the levels of expertise that are required for achieving the measurable student outcomes that go with each course goal. The levels of expertise that we assign to measurable student outcomes are important because they are the factors that most directly determine the appropriate choices of CIA for the course.

What do we mean by "levels of expertise"? The various student outcomes that we assign to each course goal require different levels of mastery of course content. Some student outcomes require no more than students simply knowing the correct answer. However, many student outcomes require more sophisticated levels of understanding--or levels of expertise. Consider again the dental hygiene example above (Figure 2): The measurable student outcome of "knows the active ingredient in toothpaste" requires only that students memorize the correct answer (fluoride), while the outcome of "can describe how poor dental hygiene can lead to poor overall health" requires a much more sophisticated level of understanding, involving synthesis of multiple facts and concepts. Again, measurable student outcomes vary in the levels of expertise required to achieve them. Accordingly, the criteria by which we measure student success in achieving desired measurable student outcomes--our classroom assessment techniques--should be capable of assessing a variety of levels of expertise. In general, this means using a variety of classroom assessment techniques. Let's consider how to go about determining levels of expertise for our measurable student outcomes.


Bloom's Taxonomy of Educational Objectives.

One of the most widely used ways of organizing levels of expertise is according to Bloom's Taxonomy of Educational Objectives.3 Bloom's Taxonomy (Tables 1-3) uses a multi-tiered scale to express the level of expertise required to achieve each measurable student outcome. Organizing measurable student outcomes in this way will allow us to select appropriate classroom assessment techniques for the course.

There are three taxonomies. Which of the three to use for a given measurable student outcome depends upon the original goal to which the measurable student outcome is connected. There are knowledge-based goals, skills-based goals, and affective goals (affective: values, attitudes, and interests); accordingly, there is a taxonomy for each. Within each taxonomy, levels of expertise are listed in order of increasing complexity. Measurable student outcomes that require the higher levels of expertise will require more sophisticated classroom assessment techniques.

The course goal in Figure 2--"student understands proper dental hygiene"--is an example of a knowledge-based goal. It is knowledge-based because it requires that the student learn certain facts and concepts. An example of a skills-based goal for this course might be "student flosses teeth properly." This is a skills-based goal because it requires that the student learn how to do something. Finally, an affective goal for this course might be "student cares about proper oral hygiene." This is an affective goal because it requires that the student's values, attitudes, or interests be affected by the course.


Table 1: Bloom's Taxonomy of Educational Objectives for Knowledge-Based Goals
Level of Expertise
Description of Level
Example of Measurable Student Outcome
1. Knowledge Recall, or recognition of terms, ideas, procedure, theories, etc. When is the first day of Spring?
2. Comprehension Translate, interpret, extrapolate, but not see full implications or transfer to other situations, closer to literal translation. What does the summer solstice represent?
3. Application Apply abstractions, general principles, or methods to specific concrete situations. What would Earth's seasons be like if its orbit was perfectly circular?
4. Analysis Separation of a complex idea into its constituent parts and an understanding of organization and relationship between the parts. Includes realizing the distinction between hypothesis and fact as well as between relevant and extraneous variables. Why are seasons reversed in the southern hemisphere?
5. Synthesis Creative, mental construction of ideas and concepts from multiple sources to form complex ideas into a new, integrated, and meaningful pattern subject to given constraints. If the longest day of the year is in June, why is the northern hemisphere hottest in August?
6. Evaluation To make a judgment of ideas or methods using external evidence or self-selected criteria substantiated by observations or informed rationalizations. What would be the important variables for predicting seasons on a newly discovered planet?


Table 2: Bloom's Taxonomy of Educational Objectives for Skills-Based Goals
Level of Expertise
Description of Level
Example of Measurable Student Outcome
Perception Uses sensory cues to guide actions Some of the colored samples you see will need dilution before you take their spectra. Using only observation, how will you decide which solutions might need to be diluted?
Set Demonstrates a readiness to take action to perform the task or objective Describe how you would go about taking the absorbance spectra of a sample of pigments?
Guided Response Knows steps required to complete the task or objective Determine the density of a group of sample metals with regular and irregular shapes.
Mechanism Performs task or objective in a somewhat confident, proficient, and habitual manner Using the procedure described below, determine the quantity of copper in your unknown ore. Report its mean value and standard deviation.
Complex Overt Response Performs task or objective in a confident, proficient, and habitual manner Use titration to determine the Ka for an unknown weak acid.
Adaptation Performs task or objective as above, but can also modify actions to account for new or problematic situations You are performing titrations on a series of unknown acids and find a variety of problems with the resulting curves, e.g., only 3.0 ml of base is required for one acid while 75.0 ml is required in another. What can you do to get valid data for all the unknown acids?
Organization Creates new tasks or objectives incorporating learned ones Recall your plating and etching experiences with an aluminum substrate. Choose a different metal substrate and design a process to plate, mask, and etch so that a pattern of 4 different metals is created.


Table 3: Bloom's Taxonomy of Educational Objectives for Affective Goals
Level of Expertise
Description of Level
Example of Measurable Student Outcome
Receiving Demonstrates a willingness to participate in the activity When I'm in class I am attentive to the instructor, take notes, etc. I do not read the newspaper instead.
Responding Shows interest in the objects, phenomena, or activity by seeking it out or pursuing it for pleasure I complete my homework and participate in class discussions.
Valuing Internalizes an appreciation for (values) the objectives, phenomena, or activity I seek out information in popular media related to my class.
Organization Begins to compare different values, and resolves conflicts between them to form an internally consistent system of values Some of the ideas I've learned in my class differ from my previous beliefs. How do I resolve this?
Characterization by a Value or Value Complex Adopts a long-term value system that is "pervasive, consistent, and predictable" I've decided to take my family on a vacation to visit some of the places I learned about in my class.


To determine the level of expertise required for each measurable student outcome, first decide which of these three broad categories (knowledge-based, skills-based, affective) the corresponding course goal belongs to. Then, using the appropriate Bloom's Taxonomy, look over the descriptions of the various levels of expertise. Determine which description most closely matches that measurable student outcome. As can be seen from the examples given in the three Tables, there are different ways of representing measurable student outcomes, e.g., as statements about students (Figure 2), as questions to be asked of students (Tables 1 and 2), or as statements from the student's perspective (Table 3). You may find additional ways of representing measurable student outcomes; those listed in Figure 2 and in Tables 1-3 are just examples.

Bloom's Taxonomy is a convenient way to describe the degree to which we want our students to understand and use concepts, to demonstrate particular skills, and to have their values, attitudes, and interests affected. It is critical that we determine the levels of student expertise that we are expecting our students to achieve because this will determine which classroom assessment techniques are most appropriate for the course. Though the most common form of classroom assessment used in introductory college courses--multiple choice tests--might be quite adequate for assessing knowledge and comprehension (levels 1 and 2, Table 1), this type of assessment often falls short when we want to assess our students knowledge at the higher levels of synthesis and evaluation (levels 5 and 6).4 Multiple-choice tests also rarely provide information about achievement of skills-based goals. Similarly, traditional course evaluations, a technique commonly used for affective assessment, do not generally provide useful information about changes in student values, attitudes, and interests.

Thus, commonly used assessment techniques, while perhaps providing a means for assigning grades, often do not provide us (nor our students) with useful feedback for determining whether students are attaining our course goals. Usually, this is due to a combination of not having formalized goals to begin with, not having translated those goals into outcomes that are measurable, and not using assessment techniques capable of measuring expected student outcomes given the levels of expertise required to achieve them. Using the CIA model of course development, we can ensure that our curriculum, instructional methods, and classroom assessment techniques are properly aligned with course goals.

Note that Bloom's Taxonomy need not be applied exclusively after course goals have been defined. Indeed, Bloom's Taxonomy and the words associated with its different categories can help in the goals-defining process itself. Thus, Bloom's Taxonomy can be used in an iterative fashion to first state and then refine course goals. Bloom's Taxonomy can finally be used to identify which classroom assessment techniques are most appropriate for measuring these goals.

Selecting Course Curriculum (Content) and Classroom Assessment Techniques
We are now at the point in the course development "road map" where concrete decisions must be made about what will be included in the course curriculum. As the instructor of the course, choices about course content are entirely yours to make based upon what you want your students to take from the course. It would be beyond the scope of this document to attempt to discuss specific choices about curriculum in detail. We do comment, however, that here again goals are paramount. If you are working from an existing syllabus (either your own or someone else's), take this opportunity to critically re-examine each component of the curriculum with respect to your course goals. Are there topics in the syllabus that are not related to one or more course goals? Don't include content merely because "it's always been done that way" or because "it's important". If a topic is important, it will be reflected in your goals. If an "important" topic is not reflected in your course goals, you may wish to re-visit the goals themselves. In any case, each and every aspect of the curriculum should connect clearly to course goals.

This is also the point in the course development "road map" where specific classroom assessment techniques (CATs) must be selected based upon measurable student outcomes and their associated levels of expertise. The FLAG site was designed primarily to help you through this step in the course development process. Provided for your use are a variety of CATs that have been tested in the field and which are authored by national experts in the use of that particular technique. To be sure, the CATs provided in the FLAG site are but a subset of innovative CATs available from a variety of resources. The CATs provided in the FLAG site should give you a good start.

The CATs selected in this step will provide the feedback you need to evaluate the extent to which yours course goals have been achieved. Thus, it is imperative that the CATs you select be properly matched with your measurable student outcomes. The FLAG site provides a facility for helping you do this.

Choosing and Implementing Instructional Methods
With your course goals formalized, having translated those goals into measurable student outcomes and assigning to each appropriate levels of expertise, and having selected the course curriculum and CATs, we are finally in a position to actually teach. That is, we are at the point in the course development "road map" where we choose and implement the instructional methods that will best guide our students to attainment of our course goals.

As with choosing curriculum and assessment, the choice of instructional methods must be guided by our course goals and, perhaps even more so, by the levels of expertise associated with measurable student outcomes. For example, suppose that two of the course goals in an introductory engineering course are that (1) students learn how to "design simple machines that satisfy realistic constraints" and (2) that students "can work effectively as part of a design team". A common measurable student outcome for these goals might be that "students, working in a team, can design a device, using simple raw materials, that protects an egg when dropped from a height of two stories". This measurable student outcome is at the "Organization" level of expertise (Bloom's Taxonomy of Educational Objectives for Skills-Based Goals; Table 2) because it requires students to "create new tasks or objectives incorporating learned ones". Traditional lecturing alone would not be a sufficient instructional method in this case. Instead, an instructional method that emulates teamwork and that promotes creative thought would be more appropriate. That is, a more collaborative instructional method is called for.

A variety of instructional methods have been developed for guiding students to the different levels of expertise represented by the goals of our course. One commonly used instructional method--collaborative learning--is described in detail in the College Level One's Collaborative Learning website. In fact, you will find that collaborative learning instructional methods are appropriate and useful for a wide variety of goals, outcomes, and levels of expertise.

Conducting Assessment and Evaluating Attainment of Goals: Closing the Feedback Loop
It is in this final step of the course development process that we harness the power of the data provided by the CATs used during the implementation of the course. Some of this assessment data may be used for assigning grades to our students. Ultimately, however, the real value of this assessment data comes when we use it for improving the course. That is, our assessment data provide us with critical feedback for evaluating what we've done: What works and what doesn't? Depending upon the CATs we have chosen, this feedback may be used either at the end of the course (to summarize the efficacy of our course development efforts) or along the way (to inform our course development efforts in progress). When assessment data is used to evaluate the course in summary fashion at the end, it is called summative. When assessment data is used to modify the course while it is in progress, it is called formative. Either way, the point of assessment is to give us the information we need for evaluating achievement of our course goals.

How, specifically, do we perform this evaluation? By what criteria do we know if we have achieved our goals? Our measurable student outcomes are the key: If these outcomes are realized, we will know that we have attained our course goals. Look at the assessment data. Did your students achieve the hoped-for outcomes, and at the desired levels of expertise? Using the engineering example from above, perhaps the egg survived but only because one student in the team did all of the work, i.e., the outcome related to teamwork was not realized. How might you modify the course to better foster effective teamwork? Perhaps students needed more guidance on how to work collaboratively; consider how the curriculum and/or instructional methods might be changed to accomplish this. Or perhaps it is the teamwork goal itself that needs refining. It is this important, evaluative step that allows you to determine the extent to which you are reaching your course goals and to decide if there are changes you would like to make.

We close this section by noting that the terms assessment and evaluation are often incorrectly used interchangeably. Assessment is the collecting of data to inform both the instructor and the student as to how the course is progressing (formative) or how it has ended (summative). Assessment involves gathering data via one or more classroom assessment techniques. Evaluation is what we do with these data once we have them. Once we have acquired the assessment data, it is up to us to judge the efficacy of our instructional methods, the content of our course, and the achievement of our course goals.

Course development flowchart. Course development goes to course goals which flows to measureable student outcomes which goes to levels of expertise. This splits to curriculum and CATs. These re-merge and go to instructional methods which flows to assessment which feeds back to level of expertise.
Figure 3 - Road Map of Course Development.


Summary

Assessment Is Feedback for both Students and Instructors
The perspective that has been advocated here is that we can use carefully constructed classroom assessment techniques as a means of determining whether or not we are meeting our stated course goals, not just for assigning our students grades. For us, classroom assessment can help us answer the following questions:

  • To what extent are my students achieving the stated course goals?
  • How should I allocate class time for the current topic?
  • Can I introduce this topic in a more effective way?
  • What parts of this course are my students finding most valuable?
  • How will I change this course the next time I teach it?
  • Which grades do I assign my students?

For our students, classroom assessment answers a different set of questions:

  • Do I know what my instructor thinks is most important?
  • Am I mastering the course content?
  • How can I improve the way I study in this course?
  • What grade am I earning in this course?

Answering these questions and others can inform and improve the quality of student learning in our classes.


A Charge to Change

We can not emphasize enough how important it is to actually write down your course goals and share them with your students. Our goals are what bind the course together (Figure 3). Our choices of curriculum, instruction, and assessment are all guided by--and held together by--our goals. Once your course goals are set, questions about instruction, assessment, and grading will be much more focused. This is a small step beyond the assessment strategies that most faculty are already doing; yet with a small investment in planning, the data acquired can provide valuable feedback for improving the quality of student learning. And ultimately, our students are what course development is all about.


Resources




3. Bloom et al., 1994; Gronlund, 1991; Krathwohl et al., 1956

4. Bloom et al., 1994; Tobias & Raphael, 1997


Go to previous page Go to next page



Tell me more about assessment:

Go to top of page.



Introduction || Assessment Primer || Matching Goals to CATs || CATs || Tools || Resources

Search || Who We Are || Site Map || Meet the CL-1 Team || WebMaster || Copyright || Download
College Level One (CL-1) Home || Collaborative Learning || FLAG || Learning Through Technology || NISE