Automatic analysis and interpretation of multimedia user generated content for emergency management

Traditional Emergency Management Systems (EMS) are mainly focused on the institutional warning response and do not fully exploit the active participation of xitizens involved. In the case of emergency events, citizens are usually considered as people to be rescued rather than active participants. Nowadays the widespread adoption of digital media and the production of content by ordinary people have marked a significant change in the study of the disaster context and have allowed analysis of the event from a completely new perspective: that of citizens involved. Thanks to the use of blogs, social networking sites, and video/photo-sharing applications, a large number of citizens are able to produce, upload and share content related to the impact of a disaster, the emergency response, the search and rescue operations, the restoration phase, etc. All this social content can be exploited in order to provide a more accurate situational awareness of the event from below, in addition to the traditional EMS. This thesis focuses on a Smart Multimedia User Generated Content Retrieval system (SMR) expressly conceived for event detection and situational awareness applications. Based on state-of-the-art clustering algorithms, it is able to locate an event and extract the most significant multimedia content. Contrary to already existing EMS, the proposed SMR system is able to analyse not only the textual content posted by users during an event, but also the visual context. To perform such a task, specific computer vision algorithms have been exploited in order to evaluate images retrieved from social platforms. Retrieved images are then displayed by emergency operators through a user-friendly graphical interface. Important results have been obtained by testing the system with over 60 events that occurred in 2015. More than 130K images were retrieved and analysed by the proposed SMR system. Results obtained are really promising and show the feasibility and the interest of the proposed SMR system.