Do you know what your students are learning?
(And do you care?)


Printer friendly version in PDF

Introduction

In higher education we sometimes pause at the end of a lecture and ask “Any questions?” or “Is everything clear?” Usually there are no responses, and as students leave we are satisfied that we did allow students the opportunity to ask questions. As there were no questions, they must have understood, or so we think. However, how do we really know what students are actually learning when they are in the process of learning something new for the first time? As lecturers we are not mind readers. We need to check in with our students to find out what they are learning and what they don’t understand fully.

Lecturers who care about what students are actually learning have often found informal ways to ask students what they have learned. However, by systematically and thoughtfully asking students about their learning as a normal integrated part of a lecture, we can gain valuable feedback about any gaps in their understanding of a particular topic. The goal for lecturers is to gain an understanding of what students know (and don’t know) in order to make responsive changes in teaching and learning (Boston 2002). This is a very specific set of strategies which use the principles of student-centred learning as described in a previous chapter.

This chapter will provide strategies that will help lecturers to determine what students have learned in any in-class situation: lecture, lab, tutorial, etc. Using these in-class strategies, lecturers can monitor the learning progress of a group of students and address or review difficult topics.

In this process, students become more skilled at evaluating their own learning progress, an essential skill for lifelong learning. Lecturers learn whether or not the teaching and learning strategies are actually helping students to learn. This can stimulate greater creativity in teaching and greater responsiveness to learners as lecturers seek to find new ways to help students understand particularly challenging concepts.

This chapter will provide background on the use of strategies to check on student learning in higher education, examples of six practical ways to implement these strategies in teaching, and a discussion of how student feedback on their learning can help lecturers to improve their teaching.

Background

Terminology

These strategies to check on student learning have been called “Classroom Assessment Techniques” (CATs) in the literature of North American higher education. However, this name might be interpreted differently by those in Irish universities. “Assessment” in this case does not mean graded exams, but rather a set of teaching strategies aimed at improving the quality of student learning. And “Classroom” does not refer to secondary school but rather to in-class sessions (lectures, labs, tutorials) in higher education.

The literature of assessment includes distinctions between “summative” and “formative” assessments. “Summative” assessments include any assessments occurring after the learning has taken place, such as end of year exams or projects that are graded to make a judgement about the extent and quality of learning that is demonstrated. “Formative” assessments generally involve providing feedback to students on work in progress, such as an essay or a project, after students have learned enough about a topic to work on an essay or project. Although usually not graded, the formative feedback from the lecturer or from peers (other students) is usually a critique of the work which is advisory or evaluative.

In contrast, Classroom Assessment Techniques (CATs) are used at a very early stage in the learning process, when students are first learning about a new topic. The CATs are anonymous and non-graded, and mainly aimed at gathering feedback from a group of students about what they have learned and what they find confusing about a topic. You can use them to help students in the process of learning a new subject.

Research

The origins of “Classroom Assessment Techniques” were in the late 1980s in two well-respected American universities: Harvard University (Mosteller 1989Light 1990Roueche, S. (ed.) 1993) and University of California at Berkeley (Cross 1987Cross and Angelo 19881993aCross and Steadman 1996Davis 1999). Since the beginning, “Classroom Research” has been done in the way that K. Patricia Cross originally envisioned (1987): lecturers use “Classroom Assessment Techniques” to systematically find out what and how well their students are learning and then use the results to improve their teaching practice. This fits with the notion of “The Scholarship of Teaching” (Boyer 1990) in encouraging lecturers in higher education to research the teaching and learning of their subjects.

More comprehensive research studies have examined the larger effects of CATs on student learning (Ang 19911998Kelly 19911993Cross and Steadman 1996). Results of these studies indicate that overall, Classroom Assessment Techniques have a positive impact on student learning, including deeper learning and greater involvement in the learning process. However, the impact of these strategies would probably be greater if used in a cohort group in which students travelled through a programme together and all lecturers on the course were using CATs routinely (Kelly 19911993).

The use of CATs has also had a strong positive impact on the professional development of lecturers as teachers Kelly (19911993). There is no question that Classroom Assessment has helped many lecturers to re-think how they teach their classes (Cross and Steadman 1996Kelly 19911993College of Marin  1990). This can result in rejuvenation among long-term lecturers and more confidence among new lecturers.

Learning Theory

The notion of checking on student learning using CATs supports Constructivist theory, Adult Learning Theories, Experiential Learning, and Deep Learning. CATs encourage students to think about what and how they are learning, construct their own knowledge, link their learning with their experiences, and move toward a more self-directed approach to learning. As Jarvis et al. points out (1998), learning is regarded as constructed by the learner rather than received from the teacher. When learning something new, students try to understand the new information as it relates to other things they already know. For this reason, the individual learner’s role is central. By asking students about what they have learned in the very early stages, they have the opportunity to reflect upon their understanding of the new thing they have just learned. It’s possible that misunderstandings can occur or there might be some confusion in the minds of the learners because what they have just learn doesn’t “fit” with their prior experience. Using CATs, a lecturer can quickly see how students have interpreted what they have learned and can take any needed corrective action to help students to learn.

Students go through developmental stages as they become more confident as learners in the subject area. As students gain confidence they should be less dependent on the lecturer and more dependent on their own abilities as learners, so that ultimately they become self-directed lifelong learners. One of the purposes of using CATs is to help students to develop their ability to self-assess their own learning, monitoring their learning progress. They also actively construct their own learning in their responses to the CATs and in the discussion of feedback from the lecturer. As a result, it is not surprising that research into the use of CATs has indicated that these strategies help students to become more reflective and confident as learners (Cross and Steadman 1996Angelo 1999).

“Best Practice” in Teaching and Learning

In the 1980’s a research project was set up by the American Association for Higher Education (AAHE) to summarize best practices in teaching and learning in higher education. The results of that research project were summarised into “Seven Principles of Good Practice in Undergraduate Education” (Chickering and Gamson 1987). This publication was based on results from many research projects into teaching and learning in higher education. Several years later, the AAHE Assessment Forum developed “Nine Principles of Good Practice for the Assessment of Student Learning” (Astin et al. 1995). More recently, the Higher Education Academy Generic Centre in the UK has published the following seven principles for good feedback practice:

  1. Facilitates the development of self-assessment (reflection) in learning.
  2. Encourages teacher and peer dialogue around learning.
  3. Helps clarify what good performance is (goals, criteria, standards expected).
  4. Provides opportunities to close the gap between current and desired performance.
  5. Delivers high quality information to students about their learning.
  6. Encourages positive motivational beliefs and self-esteem.
  7. Provides information to teachers that can be used to help shape the teaching.

(Juwah et al. 2004)

Professional Development

Most lecturers start using CATs because they want to find out what their students are learning. However, most continue to use these strategies because it stimulates greater creativity in teaching and helps them to find ways to improve their own teaching. The feedback from students often provides lecturers with a stimulus to try new teaching methods aimed at enhancing student learning. These strategies provide valuable input to all lecturers about what is working and what needs to be changed in their teaching in order to enhance student learning.

Implementing classroom assessment techniques

Classroom Assessment Techniques (CATs) are systematic on-going strategies for collecting student feedback about their learning which answer these questions:

  1. What are students actually learning in my lecture/lab/tutorial?
  2. How are the students progressing toward the learning objectives?
  3. Where are they having difficulties in learning?

CATs allow lecturers to determine the learning progress of a group of students through anonymous written responses to questions posed by the lecturer. The emphasis is on what students are learning rather than on feedback about the lecturer. Some lecturers gather feedback by using student surveys at the end of the year. Although this may be helpful in planning next year’s class, it does not directly benefit those who are currently in the course. However, using CATs the lecturer may obtain feedback from the students as often as every session, or at critical points in the term. In this way, it is immediately apparent if students are having problems understanding a concept or if they have missed an important point. It is possible to take corrective action at the next class session rather than waiting for an exam to find out what students did not understand. These strategies help lecturers to focus on student learning rather than on their own teaching. By finding out what students have learned and what is unclear, lecturers can focus the class more effectively to meet the learning needs of that group. This may mean reviewing some areas, or spending less time in other areas.

Before starting to use Classroom Assessment Techniques, it is important for lecturers to clarify their own teaching goals, using the Teaching Goals Inventory (Cross and Angelo 1993b). Learning objectives will flow from the teaching goals, and should be clearly communicated to students (Sadler 1989).

Examples of Classroom Assessment Techniques

CATs may be used in any type of class session: traditional lectures, laboratories, tutorials, seminars, etc. Some CATs are for individual students, others are for use in small groups. Some are designed to check students’ immediate understanding, others are for application and critical thinking. The following are several examples of CATs that may be used to enhance student learning (Cross and Angelo 1993a).

The Minute Paper

This is often used at the end of a lecture to give students the opportunity to reflect upon what they had learned. Their anonymous responses provide valuable feedback to the lecturer that may be used in planning the next class session. There are six main steps for a lecturer:

The Minute Paper may also be used at the beginning of a class session to ask students questions about a reading assignment or a project they are working on. Minute Papers have also been used in the middle of a lecture to encourage students to reflect on a particular point that has been raised, or to check on their comprehension of a new concept. In these cases it may be more expedient to get immediate feedback by combining the Minute Paper with a “Think-Pair-Share” activity in which students first write briefly, then they pair up and share what they have written for about two or three minutes, and then the lecturer calls on a few students to get a variety of responses to the Minute Paper. The lecturer may also choose to collect the cards for review after class. This is an efficient way to gather quick feedback from students, even in very large lectures.

Background Knowledge Probe

The “Background Knowledge Probe” allows lecturers to learn about students’ prior knowledge or experience in the subject. It may sometimes take the form of a survey at the beginning of the course, but could also be used as new topics are introduced. This strategy is useful for the lecturer to know the variation in background of that particular group of students. The responses may also be used in measuring the overall learning progress of the group at a later stage in the course. A Background Knowledge Probe is usually in the form of a survey which might include the following areas:

Although asking about apprehensions may appear to be negative, in reality, addressing these fears from the beginning is actually helpful to students (Kelly 1993), particularly when they find out they are not the only one with this concern. This also lets students know that the lecturer wants to help students to overcome these fears.

Focused Listing

A Focused Listing exercise may be used at any time to ask students to recall a set of terms, facts, or concepts that they should know. Although this strategy might appear to be at the lowest level of Bloom’s Taxonomy (1956), it can be used to check on recall as well as understanding of terms if they are particularly important for the topic of a lecture. Some lecturers have used a focused listing exercise at the beginning of a class session to measure students’ recall of a reading assignment, and again at the end of the class session to see the extent to which their recall and understanding of terms has improved.

Directed Paraphrase

This strategy is particularly useful in measuring students’ level of understanding of a particular set of procedures or methods to be followed, although it can also be used to check for students understanding of a complex concept or theory. This assessment may be given as an assignment to be completed outside of class, or it may be done during a class session individually, in pairs or in small groups. Students are asked to write an explanation of a concept or a set of instructions in their own words as if writing for someone who is not on the course. This paraphrase provides a way for the student and the lecturer to assess the degree to which students have understood an important concept or procedure and if there are any gaps in their understanding.

Memory Matrix

The Memory Matrix is particularly helpful if students will be asked to compare and contrast various items for a higher level analysis. A listing of items is provided by the lecturer down the left side of the matrix, and several key characteristics are listed across the top of the matrix. The students then fill in the blank boxes with their understanding of how the items are different. This method could be used to help improve students’ analysis of different types of cells in biology, different types of government structures, different types of economic theories, or different authors of a particular literary genre. It helps students to construct their own knowledge for a deeper level of understanding. The following Memory Matrix shows how students might use this CAT to analyze the writing styles of several different authors of short stories.


Authors characters plot perspective setting
author 1
author 2
author 3
author 4

Process Self-Analysis

Students are asked to write down all of the actual steps they take in carrying out an assignment or project, and the length of time it takes them to complete each part of the project. They then analyze how they have done the work, and which areas they found most difficult or most time consuming. This helps students to pinpoint the areas in which they may need more work to develop the skills needed to improve the process of completing this type of project. Process Self-Analysis could be used with any type of assignment, including project work, individual essay writing, or research work.

Common Questions about implementation

Lecturers who want to begin using Classroom Assessments often have questions regarding the details of implementation. Concerns are expressed in the following areas:

How much time does it take?

The most common barrier to the implementation of these strategies is the perception among lecturers that they will take too much time (Juwah et al. 2004Cross and Angelo 1993a). Lecture sessions are short, and lecturers are concerned that it is already difficult to find sufficient time to “cover the content” without adding Classroom Assessments. However, those who have used CATs find that they actually save time in lectures by focusing the lecture on areas of greatest importance for student understanding. In addition, students take on more responsibility for their own self-assessment of learning which also saves time (Boud 1986). By linking the learning objectives to what students are actually learning, it is possible to direct the teaching sessions to the areas in which students need more help rather than attempting to “cover the content.” Different CATs take differing amounts of time during a lecture, from very few minutes at the beginning and ending of a class session for the “Minute Paper” to perhaps 15 or 20 minutes for a complex CAT such as the Memory Matrix. Some CATs, such as Process Self-Analysis are more effective if they are done outside of the class session and discussed in the next session. The most important thing to remember in using these strategies is that they are actually integrated into the learning process rather than an “add-on” activity.

Why use these strategies at all? Is this “hand-holding” necessary in higher education?

Classroom Assessment Techniques provide opportunities to check on student learning before a critical stage: before the final exam, before a major project, or before a transition to a new subject upon which knowledge of a prior topic is crucial. Some lecturers might consider this “hand-holding.” However, as Jarvis points out, “Involving the student in judging what he or she has learnt encourages a more positive attitude to learning and increases the degree of student direction of the learning process.” (Jarvis et al. 1998:144) In other words, these strategies for checking on student learning are actually more rigorous for students because they put greater responsibility on the students for monitoring and constructing their own learning.

How do lecturers convince students that checking on their learning is worthwhile?

Lecturers have found that in the beginning students may be sceptical or may not take these strategies seriously because they are not graded. For this reason it is important to explain what you are doing and why you are doing it: their responses to the CATs will allow you to help them to learn. If a significant number of the students clearly did not understand a key concept, the fact that the lecturer spends more time on it or approaches it in a different way will convince students that the CATs are important and will help them to do well later on the graded assessments.

Students also appreciate having opportunities to provide anonymous feedback to lecturers about what they are learning and what is confusing. Often students are hesitant to ask questions during a lecture. Studies on the use of Classroom Assessment Techniques (Kelly 1993Cross and Steadman 1996) indicate that students feel that the teacher ”really cares” about whether or not they are actually learning. This often makes them more motivated to learn. Students also believe that the CATs made them more involved in learning because they were forced to think about what they had learned. In addition, CATs encourage students to reflect on their learning, taking a deep (rather than a surface) approach to their learning during lectures.

Is it necessary to always give feedback to students about their responses to the CATs?

It is essential! Closing the feedback loop with students as quickly as possible is the most important part of the process. When students get feedback from the lecturer, they know that the lecturer is paying attention to their responses. As the lecturer reviews the student feedback, usually at the beginning of the next lecture, students often find that others had similar questions. This can be comforting and can raise self-confidence among students who are having difficulties. Some lecturers base the entire next class session on the feedback to the students. This works particularly well for review/revision sessions. However, even taking five minutes at the beginning of a class session is beneficial to the learning process. The most important thing is to ensure that students understand the feedback from the lecturer, and that they know what to do with it (Sadler 1989).

How often should these strategies be used?

Some lecturers ask students to respond to a question at the end of every class session, and others use CATs at the most critical points in the course or before a major exam or project. Most lecturers integrate the CATs as regular class activities. Others use these strategies to evaluate the effectiveness of class activities or tests. And some use CATs to encourage students to evaluate their own learning progress. The frequency and type of Classroom Assessment Techniques depends upon the group of students, the lecturer, the subject, the learning objectives, and the reasons for asking students about their learning.

Do the student responses have to be anonymous?

Anonymous feedback results in responses that are more candid. However, if the assessments are used in the form of homework assignments or small group activities within the class, anonymity is not possible, and may not be necessary. Research on the use of these strategies (Kelly 1993) indicates that students generally feel more comfortable if their responses are anonymous.

What kind of questions should I ask?

It is best to ask students learner-centred questions (“What have you learned?”) rather than teacher-centred questions (“How do you like my teaching?”). The learner-centred questions will show clearly whether or not the teaching is effective. Thoughtful, reflective questions are better than simple, factual ones. Questions should be asked only if you really want to know the answer and are willing to respond to the feedback to meet student needs.

Is it necessary to undertake professional development before using these strategies?

Many lecturers have started using Classroom Assessment Techniques on their own by reading the Cross and Angelo handbook (1993a) and selecting and adapting a few CATs for their lectures. However, CATs are even more effective for students if they are undertaken by a department or a course team. In this way, students have opportunities to respond to questions about their learning as they go through different lectures, seminars, tutorials, and labs. Lecturers working together to integrate CATs into the curriculum find it very rewarding to share their experiences and work collaboratively as a group to improve teaching and learning. The “Teaching Goals Inventory” is a particularly useful exercise for a group of lecturers who plan to implement CATs (Cross and Angelo 1993b). This inventory helps to clarify teaching goals and learning objectives for individual lecturers or a group of lecturers working together in a department or course team. It can be an advantage to undergo some professional development as a group to learn about the various forms of CATs and which ones might be most appropriate and most useful to enhance learning in the course.

Conclusion

Using Classroom Assessment Techniques has benefits for both the students and the lecturers. Students appreciate being asked what they are learning and what they don’t understand. Using these strategies demonstrates to students that we care about their learning and want to help to facilitate their learning. However, lecturers using CATs also experience clear benefits. It is easy to begin using these strategies in a small way, starting with a simple “Minute Paper” at the end of a lecture. By asking students about their learning, lecturers find out about their own teaching and become more interested in the teaching and learning process in their own subject. This often leads to more discourse and enthusiasm about teaching and learning in departments and course teams and can ultimately result in enhanced teaching and learning.

References

   (1991). Classroom Research: Early Lessons from Success. In T. Angelo (Ed.), New Directions for Teaching and Learning, Number 46. San Francisco: Jossey-Bass Publishers.

   (1998). Classroom Assessment and Research: An Update on Uses, Approaches, and Research Findings. In T. Angelo (Ed.), New Directions for Teaching and Learning, Number 75. San Francisco: Jossey-Bass Publishers.

   Angelo, T. (1999, May). Doing Assessment as if Learning Matters Most. AAHE Bulletin.

   Astin, A. et al. (1995). Nine Principles of Good Practice for Assessing Student Learning. Assessment Forum, American Association of Higher Education (AAHE). http://www.aahe.org/assessment/principl.htm

   Bloom, D. (Ed.) (1956). Taxonomy of Educational Objectives -- Book 1, the Cognitive Domain. London: Longman.

   Boston, C. (2002). The Concept of Formative Assessment, ERIC Digest. College Park, Maryland: ERIC Clearinghouse on Assessment and Evaluation. (ED 470 206). http://www.ericfacility.net/databases/ERIC˙Digests/ed470206.html

   Boud, D. (1986). Implementing Student Self-Assessment. Higher Education Research and Development Society of Australia (HERSDA), Kensington, New South Wales.

   Boyer, E. (1990). Scholarship Reconsidered: Priorities of the Professoriate. Princeton, New Jersey: Carnegie Foundation for the Advancement of Teaching.

   Chickering, A. and Z. Gamson (1987, March). Seven Principles of Good Practice in Undergraduate Education. AAHE Bulletin (39), 3-7. Washington D.C.: American Association of Higher Education (AAHE), http://aahebulletin.com/public/archive/sevenprinciples1987.asp

   College of Marin (1990). Teacher Directed Classroom Research. (Videotape) College of Marin, California.

   Cross, K. P. (1987). The Need for Classroom Research. In J. K. Kurfiss (Ed.), To Improve the Academy. Stillwater, OK: POD Network in Higher Education and New Forums Press Inc.

   Cross, K. P. and T. Angelo (1988). Classroom Assessment Techniques: A Handbook for Faculty. Ann Arbor: National Center for Research to Improve Postsecondary Teaching and Learning. University of Michigan.

   Cross, K. P. and T. Angelo (1993a). Classroom Assessment Techniques: A Handbook for College Teachers (second ed.). San Francisco: Jossey-Bass Publishers.

   Cross, K. P. and T. Angelo (1993b). Teaching Goals Inventory. Available online at University of Iowa, Teaching Center. http://www.uiowa.edu/~centeach/tgi/index.html

   Cross, K. P. and M. Steadman (1996). Classroom Research. San Francisco: Jossey Bass Publishers.

   Davis, B. (1999). Fast Feedback. In Tools for Teaching. Berkeley, California: University of California, Berkeley.

   Jarvis, P., J. Holford, and C. Griffin (1998). The Theory and Practice of Learning. London: Kogan Page.

   Juwah, C., D. Macfarlane-Dick, B. Matthew, D. Nicol, D. Ross, and B. Smith (2004). Enhancing Student Learning through Effective Formative Feedback. U.K.: The Higher Education Academy Generic Centre.

   Kelly, D. K. (1991). The Effects of Classroom Research by Part-time Faculty upon the Retention of Adult Learners. Saratoga Springs, N.Y.: National Center on Adult Learning, Empire State College, SUNY.

   Kelly, D. K. (1993). Classroom Research and Interactive Learning: Assessing the Impact on Adult Learners and Faculty. Unpublished doctoral dissertation, The Claremont Graduate University.

   Light, R. (1990). The Harvard Assessment Seminars, First Report. Cambridge, Massachusetts: Harvard University Graduate School of Education and Kennedy School of Government.

   Mosteller, F. (1989). The “Muddiest Point in the Lecture” as a Feedback Device. On Teaching and Learning 3. Cambridge, Massachusetts: Harvard University, Derek Bok Center for Teaching and Learning, http://bokcenter.fas.harvard.edu/docs/mosteller.html

   Roueche, S. (ed.) (1993). What I learned about quality in the classroom from a Harvard Business School professor and a group of freshman students. Innovation Abstracts XV (22). Austin, Texas: National Institute for Staff and Organizational Development (NISOD), College of Education, The University of Texas at Austin.

   Sadler, D. (1989). Formative assessment and the design of instructional systems. Instructional Science 18, 119-144.

Websites on Classroom Assessment Techniques

American Association of Higher Education (AAHE) Assessment Forum]
http://www.aahe.org/assessment/assess˙links.htm

National Teaching & Learning Forum (NTLF)
http://www.ntlf.com/html/lib/bib/assess.htm

Technology applied to Classroom Assessments
http://www.ntlf.com/html/sf/vc75.htm

Classroom Assessment Techniques designed for Technology
http://www.mtsu.edu/~itconf/Proceed99/Martin.htm

University websites on Classroom Assessment Techniques

Southern Illinois University
http://www.siue.edu/~deder/assess/catmain.html

Hawaii Community College
http://www.hcc.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-1.htm
http://www.hcc.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-2.htm

Indiana University
http://www.iub.edu/~teaching/feedback.html#sfcats

Iowa State University
http://www.cte.iastate.edu/tips/cat.html

Pennsylvania State University
http://www.psu.edu/celt/CATs.html

Portland State University
http://www.fd.pdx.edu/workshops/cat/examples.html

Syracuse University
http://cstl.syr.edu/cstl/t-l/cls˙asmt.htm

University of New Orleans
http://ss.uno.edu/ss/TeachDevel/Asses/AssemTechMenu.html

University of Washington
http://depts.washington.edu/cidrweb/CATools.htm