#aplec2013 Day 1 – First Parallel Session

The Importance of Assessments in Practical Legal Training Programs

Suruj Skarma

University of the South Pacific, Fiji

I could not attend all parallel sessions. I’m blogging about those I could attend.

This session picked up on similar themes expressed in the first keynote. What follows are my impressions, in terse form. Errors and omissions are mine.

Given PLT is focused on preparing graduates with usable professional skills on completion, it is important to examine the assessment design. Awarding marks in PLT assessment might not be appropriate, and give an unreliable picture of the graduates’ learning outcomes. Starting questions, when do you measure, how do you measure, what do you measure? Need good information about what learned-centred approaches are achieving. Need strategies to get feedback from trainees about what they are learning in their own time, to identify what needs to be followed up, as part of a daily conversation. Need to adapt the teaching and learning style to local circumstances, and trainees’ circumstances and attributes. Trainees from a range of professional and practical experiences, so space needed to accommodate trainees with different levels of ability and experience. Some learning accomplished through assessments, can sometimes be the most appropriate way to learn certain concepts or skills. Using a combination of instruments and interactions – sit-in and take home tests, short tests, practice runs, examinations and evaluations. Rankings can be an issue – consider need for continuity of external measure of learning outcomes, need to be weighed against more complex assessments less amenable to quantitative rankings. Assessments essential but difficult, usually the trainer’s responsibility, questions around ranking or pass/fail approaches, consider in-progress assessments, consider whether formal assessment by itself is incomplete assessment. Unavoidable logistical challenges might influence choice of assessment timing and type. Also consider accountability to trainees and to other stakeholders regarding assessment decisions. Consider whether we’re focused on “good grades,” or competent and capable entry-level lawyers. Should trainees who “fail” assessments be required to undertake whole program again, before re-attempting assessment? Might be preferable if you’re aiming for actual competence. Consider holistic assessment of overall performance vs strictly criteria-based approaches. Need to be clear about responses to missed/failed assessments, repeat assessments, moderation processes, and avenues of review.

Lots to say and discuss here – couldn’t quite cover it in the time available.


Taking into account ‘affect’ in Practical Legal Training

This post is an edited version of a comment that I posted to the Practical Legal Training Educators Australia discussion group.

I recently finished re-reading Julian Webb’s chapter, ‘The Body in (E)motion: Thinking through Embodiment in Legal Education’ in Paul Maharg and Caroline Maughan (eds), Affect and Legal Education – Emotion in Learning and Teaching the Law, Emerging Legal Learning (Ashgate, 2011) 211.

I think Julian Webb makes a compelling argument (at p 227) that, ‘By enabling our students to get social in the classroom, to come together in a more structured and reflective way, group learning can actively support their social and moral development, and – maybe, just maybe – enhance their ability to become ‘better’ social actors…’

This chapter follows nicely from Graham Ferris and Rebecca Huxley-Binns’ chapter, ‘What Students Care About and Why We Should Care’ in the same book. They rightly argue at p 195, ‘…that those delivering education should explicitly and deliberately consider the purposes of learners, meaning the things they do or might value, or care about, or strive for. Whilst the choice of purpose is that of the learner, we can use our experience and knowledge of teaching law in higher education to facilitate purpose, choice or value adoption or rejection.’

It seems to me that these positions are applicable to the practical legal training environment, not just the academy. That said, some might be understandably concerned about leaning too far toward what students care about and losing sight of the integrity of the training and the PLT accreditation. This is the ‘springboard’ for my following comments.

Taking the skills workshop situation as an example, we can design the instruction and plan certain learning objectives for this experiential learning experience.
There may be ‘global’ objectives embodied by ‘global’ statements in the Competency Standards for Entry-Level Lawyers; ‘instructional’ objectives the students’ performance goals, the conditions for that performance, and the the criterion for satisfactory performance. We can frame specific educational objectives as a subject-verb-object sentence: ‘[During the role-play interview] the student will be able to obtain all instructions necessary [to commence work on the client’s problem and to provide preliminary advice in plain language]’. We could specify what ‘plain language’ means in this context (e.g. we could decide to exclude ‘txt-speech’, and explain why – notions of professionalism, regularity, respect, integrity, etc.) Here, I’ve drawn on Mager, R.F., Preparing Instructional Objectives. 1997, Atlanta, Georgia, USA: CEP Press.

It is also possible to explicitly plan the learning across different levels of processing (e.g. retrieval, comprehension, analysis, knowledge utilisation, meta-cognition, and the self-system/affective level) across different domains of learning (information, mental procedures, psychomotor procedures). Here, I’ve drawn on Marzano, R.J. and J.S. Kendall, eds. The New Taxonomy of Educational Objectives. 2nd ed. 2007, Corwin Press: Thousand Oaks, California. I have previously blogged about this taxonomy.

Taking the above into account when planning the workshop, we can decide to adopt certain evidence-based teaching methods, such as advanced organisers, graphic organisers, whole-class interactive or co-operative learning approaches, and use a range of media to do this. I am drawing on Petty, Geoff, Evidence-based Teaching – A Practical Approach (Nelson Thornes, 2nd ed, 2009) here. Of course, it is important that the material and methods we use are ‘authentic’ and relevant to the learning objectives.

When we actually ‘perform’ or ‘deliver’ the workshop, ‘stuff’ comes up during discussions or arising out of the practice role-play interviews. It may be the student asks a question about how to handle a certain situation, or a student recalls an analogous situation from their volunteer legal work or graduate placement. We might respond by opening the question up to discussion, or share an illustrative ‘war story’ anecdote from our own practice to give an example of how we solved a problem. These are usually good opportunities to employ, and model, ‘reflection-in-action’, and ‘reflection-on-action’ approaches to teaching, learning and professional practice. I am thinking about the work of Chris Argyris and Donald Schön, but in particular Schön’s book., Educating the reflective practitioner, Jossey-Bass higher education series. (Jossey-Bass, 1st pbk. print. ed, 1990). Peter Senge is also good to read about this – Senge, Peter M., The Fifth Discipline – The Art & Practice of the Learning Organisation (Doubleday Business, 2nd ed, 2006).

These interactions have social as well as educational qualities; both of which involve the students emotionally to some extent.

Running through all of this, from planning to delivery, are considerations concerning the affective domain / self-system level of processing learning. What students want and feel is relevant to emotionally driven judgements about whether the learning task is important and relevant to their learning goals and their ability to complete the learning task: motivation to learn = value x expectancy. This can be especially important with adult learners who can resist what they perceive as ‘supplementary’ learning (Atherton, James, ‘Resistance to Learning: A Discussion Based on Participants in In-Service Professional Training Programme’ (1999) 51(1) Journal of Vocational Education and Training: The Vocational Aspect of Education 77).

Some tend to focus on these aspects in relation to initial engagement, but I think they’re equally important to student satisfaction with the learning and could possibly affect their feelings about continuing professional education. In other words, what we do during the interactions is relevant to the immediate educational objective, but also could affect young lawyers’ commitment to life-long learning, and either impinge or enhance their satisfaction with their professional development, and their chosen profession. I think these factors are also relevant in training entry-level lawyers to pursue thinking about ethics and professional responsibility.

So, I agree that it is important to be clear about what are our teaching and learning goals in facilitating our students construction of themselves as lawyers; I think it is important that we are able to justify our instructional decision-making; it seems to me that both of those propositions involve developing our understanding of the affective domain of learning in practical legal training so we can continually improve the way we train lawyers.


A pervasive approach to teaching legal ethics?

Recently I was teaching in an advocacy skills workshop. Something happened in the workshop that reminded me of on an interesting paper presented by Michael Robinson at 3ILEC in 2008: ‘Providing Ethics Learning Opportunities throughout the Legal Curriculum’ (Paper presented at the The Third International Legal Ethics Conference, Gold Coast, Australia, 13-16 July 2008)). That paper has since been published and is available at: http://eprints.usq.edu.au/6293/2/Robertson_LE_2009_PV.pdf

Michael proposed ‘a model that makes it possible to provide law students with legal ethics learning opportunities throughout the legal curriculum’ (p 61). The paper provides a detailed approach to curriculum design to provide such opportunities.

I like the word, ‘opportunity’, it reminds me of a term used in medical general practice, ‘opportunistic vaccination’. That refers to a situation where a child presents to a doctor with some complaint and during the consultation the doctor ascertains whether the child is up to date with her or his vaccinations. If the child is not up to date, the doctor offers to bring the vaccinations up to date at that session.

I agree that a directed pervasive approach to curriculum design is important, but I also think that adopting the ‘opportunistic vaccination’ approach could be a powerful and practical way to encourage graduate lawyers to recognise ethical issues when they arise.

Back to my workshop – the graduates were split into two groups with one acting for the plaintiff and the other acting for the defendant. For logistical reasons we had to keep all the graduates in the same room at this time. The graduates were enthusiastically engaged in the preparation and within their own group energetically debating factual issues and strategic approaches to examination-in-chief and cross-examination. It did not take long for the volume of the discussions to get very loud. There erupted a bit of good-natured fencing about members of each group eavesdropping on the discussions of the other group.

I stepped in at that point and suggest we chat about that for a moment. Imagine, I said, that you are at court taking instructions from a client in one corner of the foyer, when you realise you can overhear your opponent in the same matter taking instructions from her or his client close by. What should you do?

Most of the graduates immediately volunteered that it would be unethical to eavesdrop on a privileged discussion between a lawyer and her or his client. There was some further discussion about what practical steps should be taken to remedy the situation. One graduate also commented that it was a bit similar to finding your opponent’s brief of instructions, and that it would be wrong to pick up the brief and read it. There was a discussion about written and unwritten ethical rules, and the topic of personal and professional values.

This side discussion only took a few minutes, but it seemed to produce a great deal of interest given the graduates were ‘experiencing’ a potential dilemma. After that discussion, the graduates did quieten down a bit when they returned to their advocacy work.

It would be possible to ‘script’ learning opportunities like this, but I am really interested in the spontaneous ‘opportunistic vaccination’ approach, which connects to ideas of reflection-in-action and reflection-on-action as part of professional skills. It does mean that we as teachers need to stay engaged with what is happening in the room while the graduates are doing group work, be sensitised to opportunities, and be ready to provide an informed approach to discussion about the ethical issues.

Is this something we leave to lecturers to pursue at their discretion? Is it something with which lecturers might need some help to develop? What do you think?


Written examinations as summative assessment; are they appropriate in PLT?

In ‘Why Study Emotion?’ in Paul Maharg and Caroline Maughan (eds), Affect and Legal Education, Emerging Legal Learning (Ashgate, 2011) 11 at 27-28, Caroline Maughan asks, ‘to what extent can the negative affect associated with formal assessment remain unacknowledged?’, ‘What is the real purpose of our formal assessment systems?’. ‘Should some of the time and effort devoted to coverage and summative assessment be traded off against using that time to offer opportunities for students to improve their understanding…’?. These questions are framed in the context of student motivation and satisfaction and their need for feedback (formative assessment).

I think they are important questions, and I add that written examinations often only measure a fairly limited spectrum of students’ cognitive processing (to see what I mean by ‘cognitive processing’, I have previously blogged about Marzano and Kendall’s taxonomy of education objectives here).
What are your views on written examinations as a form of summative assessment in PLT?


Practical Legal Training Educators Australia

I have set up a new group on LinkedIn: Practical Legal Training Australia.

I hope that this will be a useful place for discussions about research and practice for educators involved in Practical Legal Training.


APLEC, Paul Maharg

I had a enjoyed a short but encouraging exchange with Paul Maharg re online legal education, I recommend a visit to his blog – APLEC, Friday am, presentations, 1 & 2 — Paul Maharg.