Interactive technologies, such as virtual and augmented reality and Artificial Intelligence (AI) techniques, are increasingly present in our lives, but there are many people who insist on questioning whether these are really “forces of good” that will help us to improve our planet and our society. The MEMEX project answers some of these questions, and shows how interactive technologies can be a strong ally in the fight against social exclusion. The project is coordinated by Dr. Alessio del Bue from Istituto Italiano di Tecnologia, and will last three years.
MEMEX will help marginalised people and communities to participate in society. “MEMEX project promotes social cohesion through collaborative tools related to cultural heritage (CH) and, at the same time, promotes meetings, discussions and interactions between communities at risk of social exclusion”, explains Valentina Nisi, Técnico professor and ITI – LARSyS researcher. Using tools such as augmented reality and AI techniques to detect the surrounding context, the project will give communities the possibility to turn “their experiences and fragmented memories into convincing and geolocalized stories, using new digital content”, explained the Técnico professor.
This project, which is part of the EU Research and Innovation programme – H2020, will allow the development of new interactive and AI technologies over the next three years, thus improving the inclusion of communities. “MEMEX focuses on three essential reusable technologies: knowledge graph; automatic location (users and objects); and storytelling through Augmented Reality ”, highlights professor Valentina Nisi.
Using an application (app) installed on a smartphone, users will be able to view stories related to their personal memories and access various types of experiences. “MEMEX users will gather contemporary and historical data, including new materials related to their experiences and memories, personalization of cultural heritage and creative multimedia content”, says professor Valentina Nisi.
Initially, the MEMEX application will be powered by migrant communities that will play the role of storytellers. According to professor Valentina Nisi, mobilizing and involving the elements of these communities will not be difficult “because these communities include socially vulnerable people who are systematically deprived of various opportunities and cultural resources”. Initially, MEMEX’s target audience will be a set of specific institutions and NGOs that will contact migrant communities. In the near future, schools would also be important players in this social paradigm shift. “It would be great if teachers and educators used this App to present multicultural contents to their students”, says the ITI researcher.
The first research study under this project took place in Lisbon and was led by ITI researchers Vanessa Cesário, Dan Brackenbury, Valentina Nisi and Nuno Nunes, the company Mapa das Ideias and the NGO Instituto Marquês de Valle Flôr. The migrant participants (10 young adults) were invited to participate in a one-week photo challenge. Then, the participants attended a co-design workshop, in which they explored the photographs captured and co-created stories around the sites, linking them to their memories. “This research study aims to answer the following questions: How do participants tell a story? What makes your story powerfull? What is Cultural Heritage? How participants relate to each other’s stories?”, points out professor Valentina Nisi. “Qualitative data – audio, video and co-created stories – is being analysed in order to design end user scenarios, and also to define the MEMEX app requirements and the adopted methodology”, adds the professor.
The involvement of Técnico researchers in this project came naturally and due to “Técnico’s long tradition of cooperation with the main research institutions in Europe”. ITI will be responsible for managing the Human-Machine Interaction. “This means that ITI researchers will be responsible for understanding the needs of the three migrant communities [that will do the pilot tests] and translating them into actionable features”, explains professor Valentina Nisi. “In addition, and based on data collected, ITI-LARSyS will be responsible for the development of a storytelling tool that will facilitate the collection of stories from the migrant communities themselves,” adds the Técnico professor.
