Published: 15th January 2016
By: Dai Griffiths (University of Bolton) and Tore Hoel (HiOA)
Description: This project deliverable comprises the second compilation of the studies related to data sharing and interoperability that have been produced by LACE. These studies range from general introductions to the field through to more technical works dealing with specific kinds of learning activity for which data capture would be useful for learning analytics.
Persistent URL: http://www.laceproject.eu/deliverables/d7-3-compilation…rability-studies//
Link to PDF of deliverable cover document: http://www.laceproject.eu/wp-content/uploads/2016/01/d7-3-interoperability-studies.pdf
1 Standards to Support Learning Analytics – an Overview of Current Activity 2015
Many people in the learning analytics community recognise the significance of the idea of interoperability but are, for various reasons, not well informed about what is going on in the standardisation bodies and pre-standardisation communities. To address this knowledge-gap, LACE produced an overview of current activity in these bodies and communities. This has been published as an article on the LACE website and as a PDF version, included in D7.3.
The report was first published end of 2014 and has been updated at the end of 2015, capturing recent development and newly published specifications.
2 Learning Analytics Interoperability – The Big Picture in Brief
This white paper, which is described as an “introductory briefing”, is published as part of the Learning Analytics Review and is intended to outline some of the potential benefits of interoperability in the context of learning analytics. It is aimed at people who may be considering adopting or developing learning analytics solutions in their organisation, and the term “Big Picture” is used to capture the idea that learning analytics systems may be usefully supported by existing or emerging interoperability specifications that are not directly associated with learning, education, or training, in addition to specifications intended for this domain of application.
3 Specifications and Standards Quick Reference Guide
This document, which has been published as a Draft for Public Comment continues where the Big Picture white paper left off. It is intended to be a quick reference for people building systems or making decisions about what to specify in procurement. It identifies 34 relevant specifications and standards and indicates both where the technical documentation may be found, and what evidence is available for its usefulness. The document also draws attention to some of the complexity inherent in evaluating specifications and standards.
All of the evidence and technical documentation referred to in the Quick Reference Guide are also stored in the popular online citation management system, Mendeley. New resources are added as we discover them so that Mendeley serves as living repository of information. We have made this a public group to allow other people to make contributions.
The Guide has been updated with recent 2015 developments.
4 Learning Analytics Interoperability – Looking for Low-Hanging Fruits
Whereas the Big Picture in Brief was aimed primarily at potential adopters of interoperability specifications, the paper described in this section considers the process of prioritising attention to interoperability barriers and possible solutions. The paper (Hoel & Chen 2014) itself was created for the 1st Workshop on Learning Analytics at the International Conference on Computers in Education, held in Japan from 30 November to 4 December, 2014.
This paper emphasises the idea that the problem space for learning analytics interoperability, and hence the solution space, must account for a wide spectrum of stakeholder concerns that go well beyond the overt benefit of interoperability: moving data around and using it. These concerns include what are often termed “organisational” facets of interoperability, which include the issues of privacy, trust, and control. These are factors that can, and do, nullify even the most elegant of technical solutions.
Although the conclusions based on stakeholder interviews used in this paper need to be validated through further field work, the conceptual aspects of the paper have been used in debates within the International Organization for Standardization Ad-Hoc group on Learning Analytics Interoperability (see “Standards to Support Learning Analytics – an Overview of Current Activity”).
5 Ethics and Privacy for Learning Analytics – a Review of Current Issues and their Solutions
This LACE Review report gives an introduction to the issues surrounding ethics, privacy, data protection, data ownership and sharing and other concerns in the context of learning analytics. The issues are complex, and since learning analytics is a new field of interest to most professions this review is not aiming at promoting preconceived solutions, but more to frame the discussion and give ideas how to develop an agenda for a more sound development strategy of learning analytics where these issues are thought through.
6 Privacy in Learning Analytics – Implications for System Architecture
Privacy is identified by the LACE project as one of the most pressing issues to be addressed both in terms of best practices for learning analytics as well as for technical design. However, it is not clear what this means for system architectures.
This paper explores some of the implications for system architecture. Based on requirements of open architecture, transparency and trust, and ownership and consent the paper proposes a search architecture for learning analytics based on open and linked data. The proposed middle layer highlights dynamic usage agreements and student agency and represents an alternative approach to the LA architectures now being developed in international standardisation fora.
The paper (Hoel & Chen 2015) was presented at the 11th International Conference on Knowledge Management, ICKM 15, in Osaka, Japan, 4 – 6 November 2015.
7 Towards Learning Analytics Interoperability at the Workplace (LAW Profile)
This paper introduces the LAW profile developed by the LACE Workplace Learning work package. It introduces the needs and possible options for interoperating learning analytics within industrial and corporate scenarios, directly at the workplace. It first introduces general concepts of standardization roadmaps, abstract reference frameworks, application profiles and reference implementations as key steps towards a shared approach to interoperability. It then proposes a scenario-based method to drill down to interoperability needs and options for workplace learning, using a top-down approach. The paper suggests how the community could take action to develop specific profiles and recipes from existing and emerging specifications, with the aim of producing, managing, sharing and distributing standards-based and actionable analytics for improving workplace learning within industrial verticals.
8 Interoperability Study – Audio/Video Player Events
This was the first study produced that considers technical detail within a narrow area of application. Among the various kinds of educational content resource that could be considered in a learning analytics context, video (in particular) was identified as a priority for attention for two reasons. First, there is likely to be some useful information that can be gathered from in-video activity tracking. Second, there is growing use of video from lecture capture, through MOOCs, and the Khan Academy, yet there is no clear common approach to expressing video watching activity data.
The approach taken in the study is to inspect a selection of existing media players through the lens of learning analytics enquiry and to speculate on a common model to capture run-time events.
This study is available as a Draft for Public Comment, and has been taken as input into the ADL Experience API Video Community of Practice (Video CoP). The current version of the Audio/Video study has benefitted from material and feedback provided by members of the Experience API community. We intend to participate in the Video CoP and expect to revise the LACE study according to new information, critique, etc. An early draft was also made available to the IMS Caliper project team.
9 Interoperability Study – Assessment and Allied Activities
This is the second study to address activity data in depth. Assessment is an essential part of almost all realisations of education and training, and concepts such as formative assessment and “assessment for learning”, which offer good potential for learning analytics, are becoming widely practiced. Because of the way assessment and feedback is so embedded in education and training practice, it is a good starting-point for engaging practitioners with learning analytics. Assessment analytics also stands to benefit all learners across the spectrum of age, ability, and motivation.
The approach taken in this study is to consider a range of assessment practices that could be described as mainstream, including the use of eAssessment (even though eAssessment may not be very widely practiced, it is generally equivalent to paper-based testing instruments in terms of what is assessed). The widely implemented IMS Question and Test Interoperability specification is taken as a conceptual starting-point; it defines concepts from the point of view of exchanging assessment content and associated response processing rules.
An early draft has previously been made available to the IMS Caliper project team as input to discussion on the Assessment Metric Profile.
Twitter discussion about this page: [Topsy]