This week,  Chief Learning Officer Dr. Peter Looker dives into authentic learning and assessment and how AR/VR could be used to boost these outcomes.

The Problem with Higher Education Assessment

Somebody once asked me what I would change about higher education learning and teaching. My emphatic answer would be the same today as it was then: “Assessment”. In general, assessment fails to help students jump across the boundary between formal education and the world in which their learning is meant to be enacted, whatever that world might be. And it appears to be a problem across the whole sector.

In this post, I want to make a claim for thinking of both the process of learning and assessment as “performance” in real-world contexts, in the same sense that we perform jobs.

To be fair, it is not just the assessment itself – the formal exams and tests, the endless multiple-choice questions, the assignments, and so on – but the relationship between the actual process of learning and the assessment. Transmission teaching, which is still strong, combined with standardized “objective” tests are not a good combination for students to become real-world problem solvers, critical thinkers, or competent performers.

A mathematics teacher complained to me a few years ago about a student who had come to tell her that she (the student) was enjoying doing problems they had been set to applying an equation they had been taught. Pleased, the teacher said she assumed that meant the student understood the underlying concept. The student replied that she did not. The teacher told her that she needed to understand the concept, and the student bluntly (and honestly) told her she did not need to know it because it would not be on the exams. The exams measured the somewhat mechanical application of equations to abstract problems., not higher order thinking.

I repeat this anecdote often because it encapsulates exactly where teaching and assessment go wrong. Though the teacher had an eye-rolling attitude to the student, the student is surely right. If the teacher wants students to know the concepts, then perhaps they should not be allowed to pass the course unless they demonstrate it in assessment. Also, the teacher should find an instructional approach that directly addresses the concepts, signalling both in the teaching and the assessment that the concepts are important, rather than assuming students will just pick it up.

It is also precisely this situation that Harvard physics professor Eric Mazur refers to when he says that physics students can successfully pass the test but fail in the real world. The students can do problem-solving on paper, sometimes of quite a sophisticated nature, but they cannot relate the physics they have learned to real-world phenomena. They do not think in physics.

We could say that the students have only learned physics in an abstract way, not in a way that is applied. Abstract thinking is, however, an important part of education. As Diana Laurillard (1993) pointed out, abstract, symbolic, or theoretical thinking is an essential way to represent the world and arises from reflection on direct experience. “Much academic knowledge is second-order knowledge that can only be represented symbolically.” If we were able to learn everything from direct experience, we would not need formal education. It is symbolic thinking and theory that allow us to work out how to send a probe all the way to Pluto to take photographs.

The problem is that higher education frequently divorces abstract thinking and real-world learning-into-action. NASA’s New Horizons, on the other hand, is an example of real-world learning-into-action where theory and application go hand in hand.

On another point, we are required to be self-assessing as professionals. We must be able to judge our own performance, know what standards are expected of us, and be able to interact with the world dynamically. The problem with a lot of standard assessment is that it does not help students develop this capacity. At some universities I could name, students do not even get the results of their tests, or any feedback, just a final grade. How can they develop a sense of themselves? What they need is practice in real-world environments, and feedback on their performance.

We could think here about apprentices. An apprentice learns how to work in a trade by working, not through having the job described in a lecture. In this sense, learning is also performance of the job. Or performance and practise combined. And we can think of assessment, similarly, as a performance of learning, rather than a test of learning after the learning has happened. Think of surgeons, lawyers, or pilots. They are every day performing their learning, not describing it. The problem, as outlined by Eric Mazur, is that higher education has separated the telling from the doing.

Authentic Assessment, Authentic Learning, Authentic Pedagogy

All this leads to a reconsideration of the process of learning itself, and to authentic assessment.

Authentic assessment involves tasks that “replicate real-world challenges and standards of performance” expected of professionals in the field. “It requires students to demonstrate their deep understanding, higher-order thinking, and complex problem solving through the performance of exemplary tasks.” (Koh, 2017)

Other significant features of authentic assessment (according to Wiggins, 1989) are that the task should replicate how students’ knowledge and skills are assessed in a real-world context; it should require students to make good judgments; it should involve self-assessment; and it should involve open- rather than closed-topics or questions.

How can learning and assessment be linked to create “authentic” pedagogy?

A friend who practises law once said to me that he felt quite angry about his university education. He had gone to a top law school, always gone to classes, studied hard, and done well in his final examinations, but when he went into practice, he said he did not have a clue what to do. It took him several years to learn to put what he had learned through “book” learning into real-world practice.

I was happy to tell him that the university where I was working had recognised this problem and built a replica courtroom on campus. From year one, students were required to go into the courtroom and present evidence, as if they were in practice, with the teacher presiding as judge. Here is an example of learning and being assessed simultaneously – formative assessment – through professional performance, or apprenticeship performance.

The students are, of course, not in a real courtroom. They are in a simulated courtroom, where they can learn through performing (as if in the real world), make mistakes and receive feedback to improve their performance. Its educational value is that it is practice at real-world performance with guidance and feedback. One assumes that after four years of this, these students do not face the same problems my friend did when they graduate.

Learn, Train, Perform – EON XR and Authentic Pedagogy

There are many reasons that both AR/VR (in general) are suited to authentic assessment and authentic pedagogy: real-world, complex contexts, and a combination of visual, auditory, and (importantly) kinaesthetic learning. The dynamic nature of AR/ VR involves real-time actions and responses. Added to that is the capacity to use analytics to give feedback on real-time performance.

What is more, as Laurillard has noted, when lifelong learning is seen as vital to our future, students need to be producers of knowledge, not just reproducers of knowledge. The problem with current forms of learning and assessment is that students are asked to reproduce what is “taught” to them. (As in the example of the student who did not understand the underlying concept.) The ability to produce knowledge, in my view, is central to the notion of authentic pedagogy.

The EON XR Platform enables self-directed learning for students through the construction of “lessons”. Here, they are producing their own understanding through the requirement to find out for themselves and make a coherent learning activity. Using the Creator mode, students learn through researching, using critical discrimination and creativity to synthesise and focus their learning into either a 3D or 360 lesson. In Creator mode, students are producing, and they can then receive feedback both from peers and instructors on the quality of what the have produced.

The EON XR Platform also enables virtual training combining Virtual reality simulation-based learning with step-by-step integrated multi-user support. Learners enter a photorealistic virtual learning environment where the trainer can collaborate with the learner through a web browser, as well as pause and return at any time during the lesson while undergoing the learning. This is a good example of the apprenticeship model of learning, where students can learn and perform in a safe environment that is nevertheless a real-world context.

Similarly, using Augmented Reality, the EON XR Platform provides learners with the knowledge to perform complex tasks with a combination of step-by-step task management and knowledge markers that display relevant data. Learners are provided with contextual knowledge. Here is the combination of knowledge and theory, real world context and real-time performance in AR/VR that satisfy the conditions for authentic learning and assessment. It also allows for personalised learning on a mass scale. And importantly, analytics enable precise measures of actual performance.

We cannot build the equivalent of the campus courtroom for all subjects and disciplines, and even if we could, space limits usage, but AR/VR can address authenticity, personalised learning and assessment, and active production of knowledge to create graduates more able to face real-world problem solving.

References

Laurillard, D. (1993) Rethinking University Teaching: a framework for the effective use of
educationl technology Routledge

Koh, K.H. (2017) “Authentic Assessment” Oxford Research Encylopedias: Education
https://oxfordre.com/education/view/10.1093/acrefore/9780190264093.001.0001/acrefore-9780190264093-e-22

Wiggins, G. (1989). A true test: Toward more authentic and equitable assessment. Phi Delta Kappan, 70(9), 703‒713.