Automatic verification of UI tests

Abstract

The purpose of this thesis is to investigate whether or not it is possible to perform automatic verification of UI Tests using Neural Networks. The problem being looked at is variance tied to the operating system, graphics card, or other hardware. This can cause false positives during UI testing, and thus we wanted to find a solution that could learn to ignore this, while still verifying the result. The main technique used was Convolutional Neural Networks, since this task was tied to verifying images of results. The neural network model used was based on the VGG16-model. The models were trained on recognizing 3D-rendered objects in a geological modelling program, with varying translation, rotation and zoom to simulate various different valid UI-test results. The results part of the thesis takes the form of the classification reports generated after training. In addition to this, the models were verified on an additional image, taken from outside the datasets they were trained on. With the one model having an accuracy of 89%, and the two others having around 100% accuracy, we concluded that it is possible to perform automatic verification of UI tests with neural networks.