Brightpath to learning

Challenge

When assessing student work for tasks such as reading comprehension and mathematics, school teachers use multiple choice and short responses. In areas of learning that involve extended performances such as sport, visual arts, oral, musical and dance performances or those involving critical thinking such as information reporting, the process of assessing student work can be challenging. School teachers typically use a rubric or scoring guide to evaluate the quality of students’ responses against set criteria. However, the researchers have identified and published about issues associated with standard grid-like rubrics.

Innovative research on assessment methods began at The University of Western Australia around 2008 with its impetus stemming from a number of observations, including:

• the deficiencies identified in rubric assessment methods
• the lack of ‘fine-grained’ information about student learning provided by existing methods
• a complete lack of valid and reliable assessment methods for some difficult-to-assess learning areas
• concerns from the teaching fraternity about assessment methods such as multiple choice

The goal of the research was to help teachers to obtain valid, reliable, and comparable detailed information in a range of difficult-to-assess areas in a way they can use it to inform their teaching.

Solution

Stephen Humphry, Associate Professor in Educational Assessment, Measurement and Evaluation at UWA and Dr Sandy Heldsinger conducted preliminary research from 2008 to 2013, with grant funding from the Australian Research Council. In 2014, Associate Professor Stephen Humphry and Dr Sandy Heldsinger established the company Pairwise and successfully developed and commercialised the Brightpath software, a breakthrough in the assessment of students in a range of learning areas including those not amenable to large scale testing.

How does it work?

The method of pairwise comparisons goes back over a century. In educational contexts, for each comparison teachers simply judge which of two performances is better. There is a growing body of research in which teachers use long series of pairwise comparisons to assess student work. Although reliable results are produced in a wide range of areas, the process is very time consuming.

Harnessing their reliability in marking and the advanced abilities of teachers, the researchers developed a novel two-stage approach, built on pairwise comparisons:

• Stage 1: Pairwise comparison scales were developed by a small group of teachers for each area. The results were then calibrated and teachers were provided with exemplars. The WA Principals Association were instrumental to this process helping to coordinate work by teachers in many schools to collect student work samples and subsequently compare performances.
• Stage 2: Based on both statistical and qualitative analysis, sets of exemplars are then selected as anchor points on a scale, which is referred to as the “teacher’s ruler”. The teacher’s job then then becomes simpler and more efficient. When assessing work, they simply determine where the child slots in on the scale. Brightpath also supports teachers by using brief descriptors and annotations in conjunction with the exemplars (see figure 1).

By harnessing teacher judgement and involving them when setting up the scales, the outcome is an assessment method that is as straightforward as possible for teachers, accurate, reliable and applicable in a wide range of learning areas.

Impact Pathway

600 schools across Australia are now using Brightpath and as at October 2018, teachers have completed over 380,000 assessments within the platform.

Since its release late in 2014, Brightpath has proven to be leading educational reform in Australian education. In May 2018, the Gonski 2.0 review, titled ‘Through growth to achievement: Report of the review to achieve educational excellence in Australian schools’, recommended schools measure student growth, not student achievement. It also recommended that teachers be given access to a tool to help diagnose a student’s current level of knowledge, skill and understanding, to identify the next steps in learning to achieve the next stage in growth, and to track student progress over time against a typical development trajectory.

Brightpath provides teachers with real-time and detailed data on a student’s growth in learning. It provides teachers with empirically-derived information about learning progressions and their students’ next steps in learning (see figure 2).

Importantly, the assessment process is based on teachers’ own assessments of their students. As such Brightpath provides a proven way forward for successfully implementing the Gonski recommendations.

Fig. 2 Learning Progressions

The WA School Curriculum Standards Authority (SCSA) has purchased a license Brightpath available to all WA schools (Department, Catholic Independent schools).

The South Australian Department Eduction has undertaken an extensive trial of Brightpath in 2017 and 2018 and is in the process of finalising a license agreement for their schools.

A trial is about to start with the Northern Territory education department.

There is an appetite for such a solution—one that has impact both in Australia and even globally. As well, the impact of this research could translate across other industries, wherever performance assessments are used such as in job interviews, auditions and so forth. For now however, the focus is on the education sector.

The Pairwise and Brightpath story is one in which years of research led to the commercialisation of an assessment approach that has successful outcomes for students, teachers and parents alike.