Context Centric Approach of Semantic Image Annotation and Retrieval

Abstract

This thesis proposes a novel approach to explore and extract context information attached with images, mainly gathered from social network sites. I first performed a user study, to understand the user behavior on social network sites. I inferred that the relationship among users have central importance.<p>

<p>To assist users to annotate images in social network, I use existing metadata gathered from already annotated images on social networks, to generate metadata for non-annotated images. Social network analysis techniques together with image metadata are used to automatically annotate images. As context for an image, I consider temporal and geographical values. In addition to that, I consider three basic social entities associated with images; user relationships, user activities (comments and likes) and annotations.<p>

<p>To retrieve the most relevant images from social network, I proposed Relation-Based Image Retrieval (RBIR). For each user I calculate their relationships with other members in the network, and a ranked list of the closest and most reputed friends is compiled by analyzing the mutual activates between two users and their overall individual reputation in the social network. Comments and likes made by highly ranked members hold more weight, and photos are ranked in accordance with the number and weight of likes and comments they receive.<p>

<p>To test our approach, I developed a prototype based on the Facebook platform, to annotate images and allow users to search for images among their Facebook friends. The results demonstrate that our techniques are useful for annotation and retrieving relevant images.