In October 2016, ARoS Art Museum in Aarhus, Denmark unveiled ARoS Public, a reinvention of the art museum experience as an innovative public forum for learning and interpretation.
I developed “The Art of Looking” (Eye Tracking installation) and the “The Art of Creativity” (Portrait Machine installation) interactive installations.
The Art of Looking
The premise here is for the visitor to have a seat, and look at some artworks on a large screen. Eye Tracking technology is used to observe the eye movements of the visitors as they observe several artworks, to perform some analysis on where the user fouces his/her attention, highlighting the results.
The Eye Tracking is leveraged by the SmartEye software, which runs on its own hardware. They offer an RPC interface which is used to setup and re-calibrate the system for each new user. The system works with dual IR cameras matched with high power IR illuminators. Their software allows the visitor to move freely within a certain area. The software provides pretty accurate realtime information on where the eye is looking at (within the bounds of the screen).
This information is stored for each session, keeping track of where the user spent most time looking at, how fast are the user saccades, how long it lingers in a certain spot, etc. After each session, the user is provided with highlights regarding what’s unique on his/her approach to looking at the artwork vs everyone else’s.
Note that the interface runs entirely off the user’s gaze, a custom interface was built so that the user can look and hold into particular sections of the screen to achieve interaction.
The Art of Creativity
This is a two player cooperative interactive installation; the goal is to create a collage-artwork poster. One player takes the role of the designer, and the other one will pose to create the base of the poster. The players are separated by a transparent OLED screen, which effectively becomes an Augmented Reality interface. The two players can can keep direct eye contact through the transparent screen, and at the same time the designer-player gets an HUD guiding her/him through the process.
The installation has two detph cameras mounted around the players, allowing the system to know where the designer-player head exactly is (thus being able to infer a point of view) and also get a sekelton with the exact pose and position of the second player. There’s also a high res photo camera to take the final snapshot that will be used as a base for the users to design their posters.
With all this information, it is possible to use the OLED screen as an AR device.
Debug view of the “world” recreation. The software needs to know where each of the “players” is to faithfully adapt what’s displayed on the transparent OLED display for it to match the front player’s perspective. Two kinects are used to obtain the required data.
See through OLED + realtime adaptative perspective. Note how the arrow is always pointing at the plastic bottle, adapting to your point of view.
Creative Direction: Elvira Barriga