Real-time annotation of video streams using staged processing

Abstract

Real-time media rich applications rely on live streams of rich and accurate meta-data describing the video content to provide personal user experiences. Unfortunately, the general amount of video meta-data today is often limited to titles, synopsis and a few keywords.

A wildly used approach for extraction of meta-data from video is computer vision. It has been developed a number of different video processing algorithms which can analyse and retrieve useful data from video. However, the computational cost of current computer vision algorithms is considerable.

This thesis presents a software architecture that aims to enable real-time annotation of multiple live video streams. The architecture is intended for use within media rich applications where extraction of video semantics in real-time is necessary. Our conjecture was that staging video processing in levels will make room for a more scalable video annotation system. To evaluate our thesis we have developed the prototype runtime Árvdadus.

Our experiments show that staged processing can decrease the computation time of meta-data extraction. The evaluation of the architecture suggests that the architecture is applicable in a wide range of domains where extraction of meta-data in real-time is necessary