[Skip!] [Why?]

Case 4
Applying a Criterion and Standards Approach to Assessment by Examination in Law

Contributor

Dr. Clair Hughes

Email

clair.hughes@uq.edu.au

Telephone

+61-7-3365-2456

Affiliation

Teaching and Educational Development Institute, University of Queensland, St Lucia, Q 4072, Australia.

Contributor

Russell Hinchy

Email

r.hinchy@law.uq.edu.au

Telephone

+61-7-3365-2357

Affiliation

T C Beirne School of Law, University of Queensland, St Lucia, Q 4072, Australia

Contributor

Clare Cappa

Email

c.cappa@law.uq.edu.au

Telephone

+61-7-3365-2252

Affiliation

T C Beirne School of Law, University of Queensland, St Lucia, Q 4072, Australia

4.1 Context

This case study describes the application of a criterion- and standards- referenced assessment (CSRA) policy to an examination in a School of Law. Though CRSA had been institutional policy for some time, the requirement to give students advance notice of the criteria and standards to be used in making assessment judgements had been most commonly observed for tasks such as essays that were not undertaken in invigilated or timed conditions. By contrast, application of CSRA to examinations had been far less common. In 2005, the availability of a newly developed assessment resource provided both motivation and opportunity for reconsidering how student learning could be developed and assessed through the application of CSRA to examinations.

During 2005, two lecturers from the TC Beirne School of Law (The University of Queensland, Australia) worked with an educational consultant to develop a detailed set of generic criteria and standards—a Law Assessment Framework (LAF)—from which selections could be made to support assessment in individual subjects. As part of the development process, the lecturers conducted a trial to investigate whether the LAF could be successfully applied to an examination planned for a large first-year student cohort.

In previous years, students in this subject had been given access to past examination papers and explanations of the type of questions that would be asked. The trial involved providing students with additional detail about examination requirements in the form of a set of criteria and standards derived from the LAF.

4.2 Learning Outcomes being Assessed

The assessment task selected for the trial was a short-answer, closed-book examination to be conducted early in the semester to assess learning objectives related to the students’ ability to:

4.3 Assessment Procedures/Details

The examination (weighting 20%), which was conducted in Week 5 of the semester in an invigilated examination room, required students to answer four short questions. Marking criteria and standards, the same for each question, were included in the print and online subject informational materials made available prior to the commencement of classes and also discussed in lectures. The presentation format was designed for ease of annotation for marking and feedback purposes. Table 4.1 illustrates the type of wording used.

Table 4.1: Illustrative standards for one criterion derived from the Law Assessment Framework (LAF)

Marks

1

2

3

4

5

Criterion 1: Knowledge of foundation facts and concepts

Evidence of limited reproduction of required knowledge of facts and concepts with significant inaccuracies

Evidence of expression of required breadth of knowledge of facts and concepts in own words—generally accurate but with some inappropriate emphasis (i.e., over-emphasis on minor areas and under-emphasis on others of greater significance in the context)

Evidence of:

  • required breadth of knowledge of facts and concepts expressed in own words with consistent accuracy
  • elaboration and emphasis provided with consistent discernment
  • consistently succinct expression (i.e., the ‘packaging’ of information through selection of precise terminology and compact sentence construction)

4.4 Strengths and Limitations

4.4.1 Strengths

Lecturers perceived the use of the criteria and standards as useful in:

For students, the strengths of the approach were less clear. This assessment task had not been used before so comparison with the learning outcomes of previous cohorts was not possible. Lecturers observed fewer challenges to marks assigned, as students appeared to understand and accept the links between marks and verbal descriptions of standards.

4.4.2 Limitations

Though no obvious limitations emerged from the trial, concerns identified by lecturers included the possibility of:

4.5 Contributors’ Reflections on the Assessment

4.5.1 The Lecturers’ perspective

The trial helped us to identify some limitations of the LAF. We realised that in developing this resource we had unconsciously over-emphasised descriptions of learning demonstrated through the type of “problem-based” examination questions and assignments that are endemic to law assessment. This had resulted in an understandable focus on critical reasoning or “thinking like a lawyer” but had reduced the applicability of the resource to other forms of examination also in common use. In order to accommodate short-answer examinations the LAF was revised to boost its potential application to a broader range of assessment tasks. This process of adaptation and revision will be valuable as the use of the resource is expanded to include other subjects within the law discipline.

The development of criteria and standards helped us to clarify how we expected the students to respond to trial examination questions, which in turn helped in their formulation.

4.5.2 The Subject Coordinator’s perspective

Involvement in the development of the LAF has highlighted for me the importance of congruence between assessment and assessment criteria and subject material and learning objectives. Though the need for assessment to fit within the context of the subject material/learning objectives is obvious, this trial has increased my understanding of how the person with responsibility for the overall running of a subject can ensure that this occurs.

As a result of the 2005 trial, there will be an expanded use of the LAF in 2006 through revision of course content and teaching methodology in the trial subject. This revision has included a different approach to the delivery of lectures, a more conceptual approach to subject material and a reworking of tutorial problems. Greater emphasis will also be placed on reference to the LAF in the development of assessment tasks including one worth 30% and a final examination worth 70%.

The development and application of the LAF has been only the start of an ongoing learning experience for both lecturers and students. It is therefore far from the ‘end of the story’ and, as anticipated during the development phase, early efforts have not always been perfect and future adjustments will be necessary.

4.5.3 The Educational Consultant’s perspective

Involvement in the development of the LAF and the trial application to the examination assessment task has confirmed my belief in the utility of a generic resource such as the LAF. An expanded repertoire of language to describe learning has also been of benefit to my consultancy work across a range of diverse disciplines including Dentistry, Pharmacy and Nursing. This collaborative project has deepened my understanding of the challenges lecturers address in their assessment practice such as:

There are still many challenges to be addressed in finding effective ways to apply CSRA to examinations. The simple design of the examination used in this case study—four questions requiring similar responses—lent itself to a relatively straightforward set of criteria and standards. A more challenging task will be the development of criteria and standards of practical use to both lecturers and students for examinations comprised of complex combinations of short- and long-answer questions and various forms of multiple-choice items.

Perhaps the most significant outcome of this trial has been the change in the way lecturers view assessment as integral to subject design and, as a result, the emergence of a curriculum development process where assessment decisions exert a major influence on the design of teaching approaches and activities.

4.6 Bibliography