Why ATAR averages are poor measures of school performance

Identifying and promoting success within our schools is critical, yet the media’s attempt to do this by ranking schools using league tables is worryingly inaccurate and simplistic.

Our early research shows that other schools, often those with lower achieving students, are far more worthy of acclaim.

Esteeming schools solely upon the magnitude of student scores assumes that high student achievement is indicative of high performance by that school, and not merely the abilities of the students when they entered those schools.

To improve our schools we must esteem not just those that demonstrate high levels of student achievement, but also those which can show high levels of impact upon their students’ learning

By comparing Year 9 NAPLAN scores with Year 12 ATAR scores we are beginning to see the value-add of schools on their students’ learning.

Identifying high performing schools

Each December thousands of students receive their ATARs. Tears of joy and sadness are shed and dreams are realised, unrequited, or, adjusted as opportunities for tertiary pathways unfold. As our students depart their schools forever, the impact of these results is only just beginning for schools.

An addendum to this ritual is the league-style tables published by media to highlight the so-called ‘top performing schools’. But school performance is not that easily identified.

Imagine two Year 7 students; Student A is a strong, confident, mathematician and Student B is competent but less confident. They are both tested at the conclusion of a unit of work and the strong student receives a score of 85 and the less confident student receives 75.

Student A is clearly better academically but it is impossible to say who demonstrated greatest learning growth over the course of the unit and consequently whose teacher had the greatest impact.

To make any inferences about learning growth we need to have pre-tested the students. If we had done so and Student A received 85 on the pre-test whereas Student B received 40, it would be clear that Student A’s results have not changed at all while Student B’s improved by 35 points. Strong growth over the course of the year shows that Student B’s teacher has had a much more significant impact.

This same principle can be applied to the impact of schools. High achieving students do not mean that a school has had a big impact on their learning.

Likewise the converse may also be true; low achieving students does not mean that a school’s impact upon their students was low. To judge the performance of our schools we must consider exactly how much of an impact they are having upon their students’ learning.

Moving the focus onto impact

The notion of identifying and esteeming high impact on student learning is not a new idea. John Hattie’s Visible Learning research has focussed the attention of educators on the merits of an impact focused approach to teaching and learning and has infused education systems around the world.

This is all very encouraging, however, how do we start esteeming impact at Year 12?

Preliminary research at the Melbourne Graduate School of Education shows that some schools, just as often those with lower achieving students, appear to have a much greater impact upon their students than other schools.

By assessing Year 9 NAPLAN scores against Year 12 ATAR scores, we can clearly see how much of an impact schools are having on their students’ learning. This promising early work is being further explored.

Celebrating true success

Identifying success within our education system is an important yet complex task. Our current method of celebrating success, however, is simply not useful, relevant or accurate. The average level of student achievement is not a useful measure

We need to invest far more time, energy and effort into identifying schools that have the greatest impact on their students. By identifying where hard work is paying off, we can study this success and upscale it across the entire state.

We need to start shifting the narrative away from achievement and move it towards student progression. While high academic achievement and ATARs are of course desirable, the way to get there is to measure progress and foster the expertise and collaboration that leads to growth.