Asking the right questions of big data in education

The big data revolution has delivered universities deep goldmines of information on how their students are tracking, with every click they make logged and analysed through online learning systems. But in the rush for gold the sector needs to stay focused on what it’s all for – actually helping students learn.

The sheer volume of online student learning data now available is very seductive, but it is difficult to know where in the landscape of big data we need to be digging to find meaningful insights. We need to first know more about what we are looking for, which means better understanding how we can improve student learning. In other words: if big data is the answer for education, what is the question?

The data gold rush in education began in earnest around eight years ago with the emergence of the field of learning analytics. Learning analytics focuses on the measurement and analysis of student data to improve learning and learning environments. Universities all around the world have started their hunt for gold by establishing learning analytics research groups and co-opting the services of their business intelligence teams to mine their data reserves.

For the majority of institutions, the main goal has been to address the issue of student retention. Big data is used to attempt to identify as early as possible those students who are ‘at risk’ of falling behind or dropping out so that interventions can be made to help these students stay on track.

Other explorations are driven by the perceived ability of big data to reveal new, otherwise indecipherable, insights about the operation of educational institutions and student learning. The huge streams of data about students’ interactions with learning systems are seen to offer a “true” picture of what students are doing simply because the scale of the data can make the picture seem more valid and generalisable. Yet, the realities and complexities of creating meaning and action from big data sets are not as simple as they might sound.

The challenges of big data

For a start the volume of data is difficult to handle. Data can be stored in a variety of formats across systems that are often incompatible due to their different data structures. Curating this data in a way that allows educators and researchers real-time access for analysis is something most institutions are still grappling with from a technical perspective.

Determining the priorities for the use of this big data is another challenge facing institutions. This includes the alignment of the outcomes of the analysis of big data with broader institutional priorities. As with any complex institution, these are not uncontested decisions. Does the institution want to focus on student retention? Or is perhaps the focus on providing students support for decisions on their study pathways? Alternatively, is the focus on providing students with feedback on their learning? These priority decisions set the “analytics tone” of the institution and determine what data from the goldmine is of value and where and how deep to dig.

Another challenge is ensuring the ethical use of data. A system-wide adoption of learning analytics should consider students’ awareness and consent as well as the ethical implications of how we design analytics algorithms and when and how we intervene.

But the crucial challenge is using big data to improve learning. In the excitement of the data gold rush, learning is often simply overlooked. Determining meaning from the clicks and counts of big data and what this means for student learning is a fundamental challenge of learning analytics. Big data tends to primarily capture students’ behavioural interactions. But to what extent can these data be an indication of students’ learning or cognitive processes? The required next step of “sense-making” can add a significant overhead as educators need to consider the intent, motivation and thinking behind students’ actions to divine the cognitive element of student learning. Not an easy thing to do.

Moving forward

To enable education to be enhanced using learning analytics, educators need to focus on the right questions. These questions need to be supported by strong analytics frameworks based on educational theories and an understanding of the pedagogical design of learning activities and assessments.

The sector needs to be careful that the rush to exploit the goldmine of big data doesn’t obscure the true value of data in higher education.

This is an adapted piece from Visions for Australian Tertiary Education, a collection of writing by leading education researchers on their visions for the future of the sector, published by the Melbourne Centre for the Study of Higher Education (CSHE). The book is now available online on the Melbourne CSHE website.

Banner Image: Shutterstock