INTRODUCTION
In this exhibition, questions by the public are central. For this particular interactive experience it is: ‘do you think there is a soul?’. Visitors undergo a simple survey. Do they think a soul exists? If yes, where does it reside? Visitors can indicate where they think their soul is in relation to their body with intuitive gestures, and control this interactive experience with their body.
PROJECT DETAILS – What if: Do you think there is a soul?
Client
We The Curious
Realization
2021
Role
Lead designer
Responsibilities Project oversight, Vision, Gameplay, Prototyping, User testing, Concept, Interaction, Refinement
Partners
Employer Kiss the Frog (software and design)
Kossman & de Jong (exhibition concept)
Bruns (furniture)
Atatech (hardware)
CONTEXT – We The Curious
We The Curious is an educational charity and interactive science center which brings together people and ideas. From Bristol, they manage welcoming spaces and create inclusive experiences which fuel curiosity. This means that they bring together science, art, technology, culture and innovation to create positive social change for the community and environment.
Project What If is the first major science center exhibition in the United Kingdom inspired entirely by the curiosity of the city’s residents.
This project is inspired by 10,000 questions collected from every postcode in Bristol, which places people at the heart of science. Starting with the real question people ask themselves and using this as a springboard to delve into the real world of science. For this interactive experience, the question is ‘do you think there is a soul’?
CHALLENGE – Where is a soul?
Enable visitors to show if they think there is a soul and where it resides by using body-tracking technology.
INTERPRETATION – Make the soul tangible
Visitors indicate with body-tracking technology where they think their soul is.
First, visitors see a stylized figure that greets them. They then see themselves converted to a same stylized silhouette. This is their basis for interacting, as their body and hands control the interactive.
The visitors can place a soul icon where they think is appropriate in where it resides, and compare it to others. In the final stage they can encounter body activities that science can actually see using imaging techniques, because the soul is still elusive to science.
APPROACH – With heart and soul
STORYBOARDING – Visualizing the journey
FLOW – A linear and sequential build-up
VISUAL DESIGN – A stylish skeleton
PROTOTYPING – Tweaking the gestures
USER TESTING – A remote challenge
In this project, our client was in another country during the pandemic. With some digital aids, we completed remote user testing.
Remote user testing was a challenge but proved valuable in delivering a bespoke experience.
EXPLORATION & RESEARCH – Gestures don’t lie
Insight
Decision
People need a way to see what they are controlling.
The mirrored skeleton of the visitor’s silhouette provides direct feedback on what to do.
It is an open exhibition so people walk by or in front of the screen while someone else is playing.
By restricting the recognition area in the software and by putting a sweet spot sticker on the ground, only one person is able to control the interactive.
Complete body tracking is not accessible for all bodies.
By tracking what parts of the visitor do not move, we can lock these aspects.
Not all bodies are the same.
The amorphous, androgynous ‘skeleton’ makes it possible to capture the essence of a person as a silhouette.
FINAL EXPERIENCE – Place your soul
INTERACTION – Gestures move the soul
The body tracking-technology (Kinect V2) tracks the complete skeleton of the visitor and converts this to a stylized representation on screen.
The hands of the visitor are tracked so their position is the control. If they hold still at a certain position, grabbing or selecting is performed.
LOOK AND FEEL – An ethereal appeal
The colors, fonts and iconography are inspired by the brand book of the exhibition.
The amorphous, androgynous ‘skeleton’ makes it possible to capture the essence of a person as a silhouette. It allows people to recognize themselves, without the need for further detail such as body shape, gender or length. It also allows for direct comparison of the results.
The visual appearance of the silhouette changes. The closer the silhouette is to the screen, the more clear it can be seen. The further away it is from the screen (in, or in front) the more a milky and blurry effect is applied. The effect can best be described by pushing oneself onto milky or blurry glass.
The gestures of the guide are recorded with body-tracking technology and are therefore organic and natural.
The screen transitions indicate the flow of the experience as a linear walkthrough.
The soul representation has a lively atomized and stellar feel. The soul icons have a particle effect, underlining the ethereal and elusive nature of the soul.
PLAYTHROUGH – Feel the soul
Screensaver An amorphous and androgynous guide invites the visitor to step in front of the exhibit by making greeting gestures. Footsteps on the floor indicate the sweet spot for interaction.
Skeleton Now only the mirrored and stylized image of the visitor, based on skeleton tracking is shown. The visitor will see this silhouette take different shapes as they change posture, so the connection with themselves is established. If they move, the silhouette moves. It makes the screen act as a mirror. (The guide’s silhouette stepped out of the screen.)
Question The first question is ‘Do you think there is a soul?’. The visitor now points to either one of two positions, related to their thoughts on the existence of a soul. The selected side becomes active and the choice is submitted.
Public opinion Now the opinions of the public are shown, so if they think a soul exists in simple yes or no terms. These are indicated by percentages and in relation to a full circle. If the visitor answered “yes” earlier, the process of indicating the soul is entered. If the answer is no, the exhibit will continue to the last ‘what science can see’-part.
Being the canvas The guide reappears and the visitor is asked to copy the pose of the guide. This will be the basic pose for the visitor to relate the placement of the soul to. If the visitor stands correctly. a capture (with flash-effect) is taken. This will be the blueprint for the next step.
Placing the soul The visitor is now asked to place the soul to a position where he or she thinks it resides, anywhere in or outside of his or her body. This placement takes place within the capture of their silhouette, making it more personal. When holding a hand on the position of the soul icon it is grabbed. The visitor moves the soul to the position which corresponds to their opinion. By grabbing the soul, the soul representation moves around over the screen. By holding it still at the same position for a short while, the icon is let go and placed. The choice is saved to the dataset.
Heatmap Once it is placed, a heatmap covering all the input by other visitors is shown. This takes place in the silhouette of the guide, who steps in again. This makes it more generic. The intensity and size of the icons differ based on the amount of votes for that particular position.
Hotspots In the final stage, visitors can learn more about different body imaging techniques that show what can be seen, because the soul is still elusive for science. The guide’s silhouette acts as a placeholder for several hotspots. The visitor selects a hotspot by hovering over it with a hand. The hotspot becomes active, and the outline of the hotspot fills to indicate loading. The visitor is shown in a translucent way, in order to be able to display all of the hotspots and the guide. When all techniques are seen, the screen returns to the screensaver.
Imaging technique The technique will dictate the stance and posture of the androgynous guide. The silhouette becomes an underlayer for the imaging technique that will be shown. The guide will push the bodily segments of interest onto the (invisible) “glass”. This invisible glass consecutively scans that region, and shows what science is capable of seeing. The guide pushes a specific body segment onto the ‘glass’, where the image technique appears with information. After this, the screen reverts to the hotspot selection.
