Augmented Reality Application for Training in Maritime Operations. A Proof of Concept AR Application Developed for Microsoft HoloLens

Abstract

Augmented reality (AR) is an advanced technology that integrates augmentations with the real world. This technology has been used to provide training and education along with other purposes. This work has been focused on enriching the learning experience of the maritime trainee by applying AR technology. In this work, a proof of concept AR application (App) is developed for the training of the maritime students. The App was designed to introduce the selected stations and panel in the Kongsberg Ship Bridge Simulator in Department of Engineering and Safety, UiT The Arctic University of Norway. The App is designed with four main options namely: Stations, Panel, Help and Quit. Microsoft® HoloLens has been chosen as a wearable AR device for the execution of the App. The primary reason for selecting Microsoft® HoloLens was its standalone ability. In this App design, marker-based tracking and markerless-based tracking were compared. The marker-based tracking was found to be appropriate for the maritime training. The main reason is that marker-based tracking allows to adapt to the different Ship Bridges without much of a hassle. Within marker-based tracking method, various types of markers were considered. After thoughtful consideration, AR image markers and 2D-barcode markers were chosen. AR image markers are effective within larger range (up to 1.5m) hence useful to highlight the stations on the Ship Bridge. However, 2D-barcode markers are easy to detect (consume less processing power and allow multiple markers to be detected at the same time) and hence useful on the control panels. The camera on the HoloLens has to be calibrated for the accurate tracking of the markers. The App was designed in Unity 5.5.1f1 Personal version, Visual Studio 2015 Update 3 and Windows 10 environment. The scripts were written in C# Programming Language and libraries from ARToolKit SDK, OpenCV, HoloToolKit were incorporated. Developed App is limited to operate only with the Microsoft® HoloLens. Workings of the App can be viewed here: https://youtu.be/zzJ6mEH91F4.