Pushing the boundaries of eye tracking research, integrity and privacy

An Austrian art historian Raphael Rosenberg from the University of Vienna is trying to find an answer to the impact of culture on art. At the same time, he and his team developed a very interesting eye tracking solution where your eye’s can be tracked without you knowing it. In this project, funded by the Austrian Science Fund FWF, they have been pushing and testing the boundaries of eye tracking research but also of integrity and personal privacy.

”The questions as to whether and to what extent visual habits and works of art are informed by the cultural environment of a given period are still a central issue in art history.”

-Raphael Rosenberg, Art Historian

To find an answer to this question the idea from Rosenberg and his team was, to track the natural viewing behavior in an art museum, without the participants knowing that they were tracked.

This itself is quite revolutionary because the challenge with eye tracking is that you need VERY high precision eye tracking camera’s in order to be able reliably track the eye movements. In our research we always use the best eye trackers and we calibrate them for each participant, so we know the challenge in this idea.

Secret eye tracking camera’s

In the Vienna Museum for Art History they built two pedestals encased by semi-permeable black acrylic glass that appeared to belong to the museum inventory.

In these pedestals they secretly hid four infrared camera’s that were developed and highly adapted for this purpose by Fraunhofer IDMT . Two low-solution facetracking cameras, establishing the presence of a face and defining the coordinates of the eyes. And two high-resolution cameras focusing on the eye coordinates. The camera’s were connected to a pc and remotely connected through WiFi.

To make it work, the test subjects needed to always stand in the exact same place. Therefore, they built a wall with a window for this purpose, in the specific design of the rest of the museum walls. If a visitor wanted to view the picture, he/she had to stand exactly in front of the window.

Controlled by an app

Disguised as museum staff they recorded the visitors from a distance operating the recording process with an app on their smartphone. Now, as far as I know, this is the first time people’s eyes are being tracked without them knowing it. The researchers asked the participants afterwards for their consent and did a retrospective questionnaire in which the individuals provided data for additional evaluation to complete the research.

Post-hoc calibration

Not surprisingly, they couldn’t reach the promised quality in the eye tracking data. Therefore their partners Prof. Dr.-Ing. Peter Husar , Head of Bio-Inspired Technologies at the Fraunhofer IDMT Ilmenau and Jun.-Prof. Dr. Enkelejda Kasneci from the Perception Engineering Group at the Eberhard Karls Universität Tübinge developed a software solution for an post-hoc calibration. The process includes a semantic based adjustment of data point clouds. For example: If they see a cloud of data point next to a face, they assumed, that the cloud should be directly on the face and shift it. If they could do this for at least 3 clouds the software was able to make a calibration for the whole data set. A smart way to post-hoc calibrate the data.

The difference between people from Japan and Austria

The research has finished and they tracked the eyes of 800 people in four weeks and data analysis is ongoing. The question that seems to be confirmed in previous research from 2009 is that people from Japan look different at art compared to people from Austria. This means that is that there is a cultural difference in the tendency to concentrate on the foreground or background of images, which is connected to the theory of western-type individualism as opposed to Asian collectivism. The Japanese respondents clearly focussed more attention on the background of the paintings than the Austrian viewers, who tended to concentrate more on the figures in the foreground. It will be interesting to know what more findings will come out of this research.

Where do the boundaries go?

The future is here and this technique will further develop into such high quality hardware and software that it won’t be necessary to stand on the same spot and in the end we will be able to do very exiting research with this, but the question is where do the boundaries for integrity and privacy go in this type of research? What do you think?

For more information on the research project, please read the pressrelease: http://www.openpr.com/news/342835/Art-history-In-the-eye-of-the-beholder.html