Standards and Learning Analytics – current activities 2014

0

The development of IT standards and other kinds of technical specifications to support interoperability in educational technology has been underway for more than 15 years, but the focus of attention has usually been on system integration or portability of educational content. The growth of interest in learning analytics has led to an increase in the number and diversity of the people who identify specifications and standards as being important because analytics frequently requires the use of data from multiple sources. It is not simply that learning analytics has widened interest in the products of interoperability and standards specialists; the practice of learning analytics introduces a new set of requirements for technical specifications.

In spite of the interest in interoperability and standards from learning analytics innovators and researchers, standardisation activity remains as something that goes on “elsewhere” and is often not tracked due to difficulty of access or lack of time. This report should help to bridge this gap between interest and knowledge; it provides a brief summary of activities related to learning analytics that are underway in different kinds of standards-related bodies as at the end of 2014. This report is also available formatted for print (PDF, 580k).

We find standardisation activities in three types of fora: 1) industry or stakeholder consortia, 2) national or regional standardisation organisations, and 3) international standardisation bodies. In addition, there may be more informal, and often open-access, collaborations working on pre-standardisation tasks. The output of pre-standardisation activity may look like a standard, and some stakeholders have a preference for these specifications because the time and effort to evolve them is substantially less than in more formal settings. Pre-standardisation work is useful when the requirements and design decisions are emerging, and it provides a basis of evidence required for effective standardisation. Learning analytics, being a new and “hot” phenomenon, has stimulated activity in formal standardisation fora as well as a variety of pre-standardisation studies and experimentation, with different levels of stakeholder engagement and with varying degrees of competition and collaboration between these organisations and informal initiatives.

The LACE publication, “Learning Analytics Interoperability – the Big Picture in Brief” outlines a view that the scope of learning analytics interoperability includes aspects that are generally applicable to any kind of analytics as well as aspects that are specific to learning, education and training. Standards bodies and groups have addressed all of these categories, resulting in a number of specifications and standards that could be adopted or adapted for learning analytics. The LACE Draft for Public Comment “Standards and Specifications – Quick Reference Guide” provides a technical-level summary of the range of existing work that may be relevant to learning analytics system developers.

At present, the general situation is that “work in progress” describes the level of maturity in standardisation work specifically intended to be used for learning analytics. A similar point may be made about the process of gathering evidence for the suitability of existing specifications and standards for learning analytics. In order to properly evaluate the impact and benefits of different approaches, we would need to see more evidence of mature work and software implementations in realistic usage situations. Hence, this report avoids such evaluations and focuses on background and activity. Furthermore, rather than consider the full “big picture” scope, we consider only initiatives with a defined objective of solving issues related to learning analytics.

Consortia and Specification Development Fora

Advanced Distributed Learning (ADL) and Experience API

The Advanced Distributed Learning (ADL) Initiative of the US Department of Defence is concerned particularly with flexible training for workplace and lifelong learning through the use of technology. ADL is best known for SCORM (Sharable Content Object Reference Model), the standard with the strongest brand name in the short history of learning technologies. To further develop SCORM to fit into the future “Training and Learning Architecture”, ADL sponsored the development of what became known as the Experience API (xAPI) through a research and development project called Tin Can (hence the alternative name for xAPI is Tin Can API). The xAPI specification is now in version 1.0.1 and is published under the Apache Licence (version 2.0), which means that the specification is free to download and use.

xAPI is designed to address a learning landscape with increasing use of mobile devices, games, augmented reality, social media, etc. It enables the reporting of a range of experiences, not just completions of actions using a Learning Management System, although most current uses of xAPI still rely on a similar kind of system. The central concept in xAPI is Activity Streams, which are statements about what someone has done using a generic format, <Actor>, <Verb>, <Object>, e.g., John Connor answered “Question 1” with “True”.

xAPI also specifies the properties of a Learning Record Store (LRS), which is where the activity records are stored.

In addition to supporting adoption and maintaining the version 1, current activity on the Experience API in ADL is focussed on two areas:

  • There is currently vigorous debate on requirements for, and scope of, xAPI v2, and the extent to which xAPI v2 should be backwards compatible with xAPI v1.
  • Communities of Practice (CoP) working groups are being established to consider the use of xAPI in various settings, or for particular applications. CoPs are intended to create best practices and to define controlled vocabularies for use with the xAPI core language.

IMS Global Learning Consortium

IMS Global Learning Consortium, often just referred to as IMS, is a membership organisation that issues consortium standards.

Caliper Analytics

IMS Global’s learning analytics initiative is called Caliper Analytics and is outlined in a white paper (pdf) published in September 2013. Caliper is being designed to be compatible with IMS Learning Tools Interoperability in particular, and comprises a set of “Metrics Profiles” to define the data that is to be sent to the analytics store, and the “Sensor API”, that defines the overarching communication model. IMS plans to include concepts from other existing IMS specifications such as Learning Information Services and also to incorporate it into the emerging EDUPUB eTextbook specification.

Three Proof of Concept demonstrations of IMS Caliper are due to be made at the annual IMS Learning Impact conference in May 2015. Some public documentation is anticipated during mid-2015. In the mean time, access to IMS specifications under development is restricted to IMS members.

Privacy

The development of Caliper, coupled with the recent high profile forced closure of InBloom and legislation in several US states, has led to increased interest in privacy among IMS members and within the IMS Board. A task force is currently considering the scope of work to be undertaken on privacy in IMS. It should be noted that IMS Learning Tools Interoperability specification already includes some useful features to protect the privacy of learners. It is expected that IMS will make a public announcement of the work to be undertaken during 2015.

HR Open Standards Consortium

HR Open Standards Consortium (previously “HR-XML”) is concerned with data exchange standards for the human resource community. Currently, there is no activity directly related to learning analytics. However, HR Open Standards have work groups and specifications on Assessment, Competencies, Performance Management, etc. These specifications may be relevant for learning analytics projects centred in the workplace, although they are not of immediate relevance to data collection for analysis.

National and Regional Standards Bodies

CEN (The European Committee for Standardization) TC353

CEN is the official standarisation body for the European Union and the formal standards it produces – European Norms – apply in all member states by default and member states are bound not to develop competing standards.

The Technical Committee 353, ICT for Learning, Education and Training, has discussed learning analytics, but no work item has been proposed by any of the National Standards Body “mirror committees” that also vote on proposals and draft standards.

International standardisation bodies

International Organization for Standardization

The International Organization for Standardization (ISO) is the standards body recognised by the UN. Information technology standards are developed by ISO/IEC JTC1, a Joint Technical Committee (JTC) with the International Electrotechnical Commission (IEC). Participation in ISO committees occurs through National Standards Bodies.

In June 2014 the ISO/IEC subcommittee on ICT for Learning, Education and Training (SC36) established an Ad Hoc Group on Learning Analytics to prepare scope and terms of reference for a new Working Group on Learning Analytics Interoperability. These have now been submitted into the ISO approvals process and, subject to approval by National Standards Body “mirror committees” of SC36, it is expected that the new Working Group will be established early in 2015. The first work item is likely to be a report describing a framework model for learning analytics; a draft work item proposal has been discussed in the Ad Hoc Group.

Within this ISO/IEC subcommittee there is also work on e-Textbook standards. Requirements describe e-Textbooks as adaptive and able to communicate with learning analytics engines, either embedded or through communication with external services. However, SC36/WG6, the working group specifying e-Textbooks, has yet to create a specification to meet these requirements.

The Institute of Electrical and Electronics Engineers

Within The Institute of Electrical and Electronics Engineers (IEEE), the IEEE-SA is the organisation that develops global IT standards within a broad range of industries.

IEEE standardisation of xAPI

During 2014, ADL submitted xAPI for standardisation with IEEE-SA. The proposal  was rejected for two reasons: IEEE requested a more modular structure of the specification and European IEEE members in particular requested a more clear discussion of issues of privacy. Following on from the first of these requests, it is expected that a new project submission will be made to IEEE in January or February 2015, comprising the Statement information model and JSON binding.

IEEE ADBook

In 2013 Arenas et al. [reference below]proposed the IEEE ADBook (Actionable Data Book), which is a specialised eBook, combining data exchange across multiple platforms (including Internet of Things) with specific support for STEM education and accessibility preferences. Industry Connections, an IEEE Standards Association program that facilitates the early exploration of potential interoperability solutions, has established an ADBook project. Although the website has not been updated recently, we are aware of continued work on ADBook from personal communications and expect a report entitled “Pedagogy and Architecture: The IEEE Actionable Data Book” to be released early in 2015 that will include analytics in the open architecture being envisaged for ADBook.

Informal and Pre-standardisation Initiatives

Open Learning Analytics Network

The idea of an Open Learning Analytics (OLA) platform was first advanced by a group of leading thinkers on Learning Analytics from Europe, Australia, and North America in a 2011 visioning paper published by the Society for Learning Analytics Research (SoLAR).  Since then, a summit meeting was held in Indianapolis (USA) in March 2014 to promote networking, to develop thinking, and to explore possibilities for collaborative research and innovation. This meeting made progress towards the idea of Open Learning Analytics as a technical and conceptual framework around which multiple stakeholders could coordinate their activity around areas of common interest. It also emphasised the idea of an Open Learning Analytics Network as an informal community of collaboration. A European summit meeting was organised on December 1st, 2014, by LACE in collaboration with Apereo Learning Analytics Initiative, and Alan Berg and colleagues at the University of Amsterdam.

Throughout the development of the concept of Open Learning Analytics, from the initial visioning paper, through to the recent European Summit, interoperability, standards and open APIs (application programming interfaces, the means by which data may be exchanged and instructions sent) have been a recurrent theme. As yet, however, there is no organised attempt to undertake pre-standardisation work in the Open Learning Analytics Network. Indeed, the focus of activity in the OLA Network is likely to be sharing experiences in using various candidate interoperability specifications, and tentatively moving towards a set of preferred specifications in an emerging Open Learning Analytics Architecture. In doing so, it is expected that useful evidence will be gathered, both for the benefit of potential adopters of these specifications, but also to inform the ongoing standardisation processes.

CEN (The European Committee for Standardization) WSLT

Workshops are a less formal component of European standardisation than the Technical Committees. They produce Workshop Agreements, which are pre-standardisation documents arising by consensus in an expert-based forum which any interested parties are able to apply to join.

The Workshop on Learning Technologies (WSLT) members had plans to put learning analytics on the agenda and LACE project partners had intended to use the WSLT as a forum to identify European requirements, and as a focal point for knowledge exchange. Unfortunately, during 2014, CEN dis-banded the WSLT due to an unresolved disagreement with its members about process; its members wished to continue operating the WSLT as an open access and open process pre-standardisation initiative, as it had been accustomed to operate for many years, although not consistent with standard CEN policies.


Arenas, E., Richards, T., & Barr, A. (2013). The IEEE Actionable Data Book: A platform for inclusive education (pp. 63–67). Presented at the Global Humanitarian Technology Conference (GHTC), 2013 IEEE, IEEE. doi:10.1109/GHTC.2013.6713655

This report was written by Tore Hoel with contributions by Adam Cooper. We expect to produce an updated version of this report at the end of 2015, but will also add comments to this post for significant developments.


View Twitter conversations and metrics using [Topsy]

Share.

About Author

Tore Hoel is affiliated with Learning Centre and Library at HiOA, and has been working within the learning technology standardisation community for more than ten years. He is now working on Learning Analytics Interoperability within the LACE project.

Leave A Reply