From 7 to 11 February 2022 the Barcelona Supercomputing Center organised a course called “PATC: Introduction to Big Data Analytics”, which was addressed to trainees who already had some theoretical and practical knowledge in the topic.
Maria Cristina Marinescu, Joaquim Moré and Artem Rashetnikov, researchers from the Saint George on a Bike project, carried out a training session within the course focused on Cultural Heritage as an example of a field that can really take advantage of integrating, analysing, and reasoning with large amounts of data from many heterogeneous sources. Overall, they explained how to improve the quality and quantity of open metadata associated with European Cultural Heritage (CH) imagery, starting from images of paintings and text. All of it in order to transcribe insights about culture, symbols and traditions in a knowledge representation accessible to machine learning and artificial intelligence.
First of all, Maria Cristina Marinescu presented the goals of multidisciplinary research and data analytics and introduced Saint George on a Bike among other projects. She also focused on the main challenges of the project, for instance that the current approaches work well for recent pictures but fail for ancient paintings since there can be old objects not in use anymore, unusual actions for everyday life (e.g. man killing a dragon), etc.
Then Artem Reshetnikov explained that currently the researchers are working on improving object detection in paintings by placing them in time context and he highlighted how they generate captions taking into account the context, the content and the form.
Finally, Joaquim Moré introduced natural language processing and how it is applied in the project. In addition, he described how they use a language model to generate more precise labels and also showcased some examples of different classifications from visual relations.
