Student interaction with online IL content: The impact of exam structure and course design

Abstract

In 2015, UiT launched a MOOC on information literacy, iKomp (www.ikomp.no). Built on Open EdX, iKomp consists of four modules: learning strategies, information evaluation, information searching, and academic formation. The final exam consists of a 40 question multiple-choice test, originally allowing an unlimited number of attempts and with easily accessible explanations of the questions. An analysis of the exam results and learner behaviour (N=399) revealed that only 22,6% passed on their first attempt, and 24,8% used more than 10 attempts. Further, engagement with in-course activities correlated only weakly (r = .18) with exam score (Andreassen et al., 2015). These findings might indicate either that the exam is too difficult, or that students do not engage fully with the course content before taking the exam. An analysis of error rates and response distribution for the separate questions leads us to hypothesise that the second explanation is the correct one. To improve student learning, we consider it necessary to enhance their engagement with the course content. In an attempt to achieve this, we have modified the settings on the exam, thereby imposing soft constraints on learner behaviour. Number of possible attempts have been reduced to 4, and elaborate explanations have been replaced by hints cross-referencing to course content. In this paper, we evaluate the impact of the revision on student engagement with the course content. We first present data collected after revision of the exam (Spring-2016) and compare them to the data from our previous analysis, on the same parameters. Second, we present data (Spring-2016) indicating student interaction with the course content between the first and final exam attempts. Specifically, by examining their time spent on chapters and units, we aim to see whether the students follow the hints provided for the failed answers. In light of these findings, we discuss whether or not we have achieved the balance between student control and constraints imposed by course design.