Recognizing Hand-Based Micro Activities Using Wrist-Worn Inertial Sensors: A Zero-Shot Learning Approach

Abstract

Zero-shot learning (ZSL) is a machine learning paradigm that enables models to recognize and classify data from classes they have not encountered during training. This approach is particularly advantageous in recognizing activities where labeled data is limited, allowing models to identify new, unseen activities by leveraging semantic knowledge from seen activities. In this paper, we explore the efficacy of ZSL for activity recognition using Sentence-BERT (S-BERT) for semantic embeddings and Variational Autoencoders (VAE) to bridge the gap between seen and unseen classes. Our approach leverages wrist-worn inertial sensor events to capture activity data and employs S-BERT to generate semantic embeddings that facilitate the transfer of knowledge between seen and unseen activities. The evaluation is conducted on datasets containing three seen and three unseen activity classes with an average duration of 2 s, as well as three seen and three unseen activity classes with an average duration of 7 s. The results demonstrate promising performance in recognizing unseen activities, with an accuracy of 0.84 for activities with an average duration of 7 s and 0.66 for activities with an average duration of 2 s. This highlights the potential of ZSL for enhancing activity recognition systems which is crucial for applications in fields such as healthcare, human-computer interaction, and smart environments, where recognizing a wide range of activities is essential.