The Institute for Medieval and Early Modern Material Culture (IMAERAL), a cultural studies research institute of the University of Salzburg, organised an event under the title “TRANSMEDIAL-DIGITAL? The potential of current approaches to image and text research in the pre-modern era” on 26-27 May 2022. The conference brought together current positions in artistic, literary, comparative, historical and cultural research on premodern forms of image and text medialities.
Artem Reshetnikov, Research Engineer from the CASE Department at the Barcelona Supercomputing Center, presented the talk “Enrichment of textual metadata for cultural heritage objects using deep learning” on the first day of the conference within the framework of the Saint George on a Bike project. Firstly, Reshetnikov introduced the project showing the audience its video. He also explained the motivation of the project, its objectives and its research lines such as the object detection, inferring visual relationships between detected objects in images with complex scenes using bounding box analysis and the NLP for description generation. In his speech he also introduced the sources of dataset the project is using, for example the Europeana Collection, Wikiart and also datasets from museums such as the British and Prado museums.
Moreover, the analysis methodology used in the project was also presented as well as the crowdsourcing campaign carried out to collect manual descriptions of paintings. Finally, Reshetnikov also talked about the project’s challenges so far and the next steps until the end of the project, for example improving the current methods for tagging and description generation, deploying enrichments and implementing search and browse functionalities and carrying out an exhaustive quantitative evaluation.
