A Utility Centered Approach for Evaluating and Optimizing Geo-Tagging

Geo-tagging is the process of annotating a document with its geographic focus by extracting a unique locality that describes the geographic context of the document as a whole.
Accurate geographic annotations are crucial for geospatial applications such as Google Maps or the IDIOM Media Watch on Climate Change, but many obstacles complicate the evaluation of such tags. This paper introduces an approach for optimizing geo-tagging by applying the concept of utility from economic theory to tagging results. Computing utility scores for geo-tags allows a fine grained evaluation of the tagger's performance in regard to multiple dimensions specified in use case specific domain ontologies and provides means for addressing problems such as different scope and coverage of evaluation corpora. The integration of external data sources and evaluation ontologies with user profiles ensures that the framework considers use case specific requirements. The presented model is instrumental in comparing different geo-tagging settings, evaluating the effect of design decisions, and customizing geo-tagging to a particular use cases.