Evidence of the month: “Scaling up” learning design


Each month, we highlight one of the new additions to the LACE Evidence Hub, which brings together evidence about learning analytics. You are welcome to add to the Hub site, which you can visit via a tab at the top of this page.

The evidence of the month for January 2016 is a paper from Rienties, Toentenel and Bryan (2015): “”Scaling up” learning design: impact of learning design activities on LMS behavior and performance”.

This paper is the first empirical study linking learning design information for a substantial number of courses with Learning Management Systems (LMS)/Virtual Learning Environment (VLE) data and with learning performance.

The paper used hand-coded learning design data for 87 modules: the amount of time learners were expected to spend on each of seven types of learning design activity: assimilative; finding & handling information; communication; productive; experiential; interactive/adaptive; assessment. A cluster analysis of design of these modules suggested they fell in to four broad groups:: constructivist, assessment-driven, balanced-variety and social constructivist modules.

The authors found significant correlations between aspects of learning design and LMS/VLE usage data. Learners on the constructivist modules spent significantly more time on the LMS/VLE than learners on the other modules. The learning design decisions of teachers seemed to strongly influence how students were engaging with the LMS/VLE. Specifically, when more inquiry- or social constructivist learning activities were included in the learning design, learners tended to use the LMS/VLE more.

Finally, they explored the link with learning performance (completion and pass rates). They found no correlation between LMS/VLE activity and learning performance. However, there was a significant negative correlation between assimilative learning activity and learning performance, i.e. modules that had more time devoted to reading/watching/listening activity had significantly lower completion and pass rates.

The authors note that this was a small sample: full data was only available for 40 modules. However, it is a good first example of a study linking learning design with VLE/LMS data and with learning performance, and is evidence that the choices teachers make in learning design terms do have an effect on the learners, both in terms of VLE/LMS activity and learning performance.

(A further study with 157 modules is in press: Toetenel, Lisette and Rienties, Bart (2016).Analysing 157 Learning Designs using Learning Analytic approaches as a means to evaluate the impact of pedagogical decision-making. British Journal of Educational Technology (in press).)

Citation: Bart Rienties, Lisette Toetenel, and Annie Bryan. 2015. “Scaling up” learning design: impact of learning design activities on LMS behavior and performance. In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge (LAK ’15). ACM, New York, NY, USA, 315-319.

Url: http://dl.acm.org/citation.cfm?doid=2723576.2723600


About Author

Academic in to new technology in teaching, particularly learning analytics, at the Open University in the UK.

Leave A Reply