Developing the big picture of a learning analytics system – a report from the international standardisation scene

0

This summer the international standards community made important progress in coming up with the basic concepts describing a learning analytics technical framework. The group in question is the ISO subcommittee 36, which is focussing on learning technologies; and the project is Learning Analytics Interoperability – Part 1 Reference model. The LACE project findings of the need to consider privacy requirements have been brought to attention of the new workgroup of SC36, and a follow-up meeting with the project lead editors worked on extending the draft model in Seoul beginning of July.

In this blog post we will take a look behind the scenes of this ISO committee and reflect on the importance of designing high level framework while companies and some national agencies are trying to come up with working code to improve learning through analytics on a practical level.

The world of standards
A deliverable of the LACE project gives an overview of the standards organisations addressing learning analytics interoperability. Industry consortia like ADL (Experience API), IMS Global (Caliper), and Apereo (OpenLRS) are more hands on and may answer to the the needs of the LA tools developers. ISO SC36, on the other hand, is a formal standards organisation driven by national bodies not always blessed with first hand contact with the implementation community. After the European standards organisation CEN managed to kill both its workshop and technical committee working on learning technologies, ISO stands alone as the only formal standards body trying to build international consensus beyond sector or membership interests.

This is not the time and place for laying out the pros and cons of formal vs industry led standardisation; we merely observe that there seems to be a need to build consensus on different levels. The ISO reference model for Learning Analytics Interoperability will potentially create some clarifications useful for policymakers and tools developers alike. As a minimum, it is always useful to agree on a common set of concepts and terms before embarking on the more thorny issues of interoperability. The project is led by Asian experts with the South-Korean professor Jaeho Lee as the lead editor.

Let the concepts talk
To give the new SC36 working group (WG8) a head start an ad hoc group had been working for nearly a year to collect and discuss use cases solicited from different constituencies around the globe. For the first draft of the reference model professor Lee had done an excellent job extracting all the processes identified in the use cases defining a number of “zoom-in diagrams” describing the different workflows of a learning analytics service: data collection, data storing and processing, analysing, and visualization (see figure below).

Fig 2 LA workflow - SC36

When zooming in on the first workflow, Data Collection, one starts to see the limits to this method of concept development and the challenges ahead for the ISO process of SC36/WG8. Issues of Personal Data Control, Identification, and Accessibility Preferences are noted from the use cases; nevertheless, the full set of all kinds of data sources is lumped together in a box queried by a Data Collection API clearly inspired by the IMS Caliper work yet to be revealed. The work done by LACE on data sharing and interoperability shows there is more to data collection than taking the data for granted and focussing on metrics to help the query. The troubling part of this observation comes when you know how ISO standards are developed. How is new input to be fed into the standards development?

wp8_editors

SC36/WP8 chair Yong-Sang Cho presenting a slide outlining structure and editors of the new standard on Learning Analytics Interoperability

The current draft is the result of a first development cycle that ideally should go through several iterations. Obviously there are use cases that are not submitted and therefore not taken into consideration. But this phase is now passed; now it is time for word smithing through national body comments to the current draft. The seven editors representing Korea, Russia, China, Norway, Australia, and Japan may embark on more development cycles, but the process is slow and totally dependent upon written contribution through national representatives. The next SC36/WP8 meeting will be in beginning of December in Hangzhou, China, co-located with the 2nd Workshop on Learning Analytics at ICCE, co-organised by LACE project.

Towards a Data Sharing Consent Service
Already the same week of the inaugural SC36/WG8 meeting I had the chance to present some ideas how to address data sharing challenges to the ISO SC36 community and the French standards community in the Open Forum (an open event usually organised during SC36 meetings). These ideas are further developed in a paper by me and Weiqin Chen titled Privacy in Learning Analytics – Implications for System Architecture to be presented at ICKM 2015 in Osaka in November. If transparency, ownership of data, consent, and trust are to be taken seriously in system architectures for learning analytics you have to design a workflow that goes before Data Collection.
These ideas were discussed in a meeting in Seoul in July with professor Jaeho Lee and the SC36/WP8 chair, Yong-Sang Cho of KERIS, a South-Korean educational agency. In this meeting we also had a look at the Jisc open LA architecture now being developed.

Hoel_OpenForum

Tore Hoel presenting at Open Forum in Rouen.

Korean Reference Architecture for LA
In the meeting in Seoul the Dr. Cho and Professor Lee demonstrated the reference architecture KERIS is developing using open source software. The architecture is visioned to give some proof of concept of the more abstract work being done in SC36/WP8. The architecture seems to be heavily influenced by IMS Caliper, which is under development, and other IMS specifications, in particular Learning Tools Interoperability.

seoul_lee_cho

Yong-Sang Cho, Tore Hoel and Jaeho Lee discussing the LACE input to ISO draft reference model.

The Koreans were able to demonstrate a solution that let you open an e-textbook from a list of resources held in a LMS, read through the book in a Readium application, and have the progress captured and reported to a separate visualisation tool through the use of an activity stream specification generating RDF formatted data. The plan is to develop this reference architecture more in the coming two years to cover all parts of a learning analytics system.

wude1

South-Koreans vendors are also eager to build in learning analytics capabilities in their solutions. Kyu Ha Lee is CEO of the start-up company WeDu communications (www.wedu.co.kr). She demonstrated a service providing content and analytics available to the users through a system of an innovative pico projector letting the users annotate on any surface soon ready to mass produced with a very low entrance cost. (Photos: Tore Hoel)

wude_tool

Share.

About Author

Tore Hoel is affiliated with Learning Centre and Library at HiOA, and has been working within the learning technology standardisation community for more than ten years. He is now working on Learning Analytics Interoperability within the LACE project.

Leave A Reply