Spatial Computing, the next paradigm of Human-Computer Interaction

15 June 2020

Jan Gugenheimer joined the LTCI Design, Interaction, Visualization & Applications (DIVA) research team on January 2020. After a Bachelor’s in Computer Sciences, he started focusing on HCI (Human-Computer Interaction) during his Master’s and PhD at the University of Ulm (Germany). He then worked abroad extensively and in particular spent a few months at the MIT Media Lab and Microsoft Research in Redmond, as a means to get a different perspective on his research field.

« For me, HCI is at the intersection of computer sciences, psychology and of sociology. Humans and technologies exist together, and I’m interested in how humans can interact with technology efficiently, and how understanding this technology impacts the individuals and society. My particular sub field is mixed reality (MR), an umbrella term for virtual reality (VR) and augmented reality (AR). I do not research specific technological difficulties as this should be done by the companies producing these devices: I am mostly interested in how we interact with the technology and how can we embed it into the fabric of our daily life, » says the young researcher.

From command line to spatial computing

Although there is no shared definition of what mixed reality actually is (everyone in the field uses a slight variation), the one given by Paul Milgram in a 1994 paper is often quoted. According to him, mixed reality can be described as a spectrum between two extremes: complete virtuality and complete reality.

Jan Gugenheimer explains: « Mixed reality is a new buzzword. In fact, I prefer to talk about the underlying paradigm, spatial computing. What interests me is how the ability to display digital media and interact with it inside our physical space will impact us on an individual and societal level. If we look back to the history of HCI, the first paradigm was command line interfaces: you could only communicate with the computer by typing commands. The second paradigm was the graphical user interface that we use daily on personal computers and smartphones. In the future, whether we talk about virtual reality, augmented reality or mixed reality, the point is that it’s not going to be in 2D anymore: that’s why the third paradigm will be spatial computing. »

The future of HCI

In the fast-changing field of HCI, international conferences are more relevant than scientific journals. The ACM CHI Conference is both the oldest and biggest conference in the field, with a very high level of selectivity and an acceptance ratio of around 20%. This year, Jan Gugenheimer had four papers accepted by CHI 2020 [1-4] and one workshop [5]. One of the papers even received a Best Paper Honorable Mention Award (given only to the 5% best publications) [3].

Three of the papers all address specific augmented and virtual reality problems such as an error in input [2], a problem of constraint input for drawing [4] and the potential to leverage augmented reality technology to enhance the ability to create 3D printed objects [1].

« In the same way that the smartphone did not make the personal computer obsolete, future spatial computing technologies, such as VR helmets and smart glasses will probably not stop us from using our smartphones. But they will provide new possibilities, additional services which we are still working on. »

For instance, the paper that received an Honorable Mentioned Award [3], explored how we can design communications between cars and vulnerable road users like blind people, so future autonomous driving systems are also considering these special groups in the population. The researchers conducted several studies using VR as a means to explore these future scenarios.

Ethical, social and political issues

 « Additionally, a workshop that was on ‘Exploring Potentially Abusive Ethical, Social and Political Implications of Mixed Reality Research in HCI’ was accepted. The idea was to reflect on our responsibility as the HCI community on how to shape this upcoming technology and what potential problems we might create. »

Usually VR and AR technologies are seen in a very positive way. However, as designers of these technologies, researchers bear a lot of responsibility. For example, smartphones generate a huge debate about how they get us addicted, in relation with the business models of social media companies. As a matter of fact, the “persuasive tech” or “nudge” they use was invented thanks to HCI research.

“What we want to do now is to explore some potential negative scenarios, with the goal of learning what could make these technologies abusive or result in negative outcomes for the users. We need to start this discussion now, because when the technology is out there, it will be too late.”

(c) head image Yanalya on Freepik

References

[1]Mix&Match: Towards Omitting Modelling Through In-situ Remixing of Model Repository Artifacts in Mixed Reality
https://dl.acm.org/doi/abs/10.1145/3313831.3376839
https://dl.acm.org/doi/fullHtml/10.1145/3313831.3376839

[2] Understanding the Heisenberg Effect of Spatial Interaction: A Selection Induced Error for Spatially Tracked Input Devices
https://dl.acm.org/doi/abs/10.1145/3313831.3376876
https://dl.acm.org/doi/fullHtml/10.1145/3313831.3376876

[3] Towards Inclusive External Communication of Autonomous Vehicles for Pedestrians with Vision Impairments
https://dl.acm.org/doi/10.1145/3313831.3376472

[4] VRSketchIn: Exploring the Design Space of Pen and Tablet Interaction for 3D Sketching in Virtual Reality
https://dl.acm.org/doi/10.1145/3313831.3376628

[5] Exploring Potentially Abusive Ethical, Social and Political Implications of Mixed Reality Research in HCI
https://dl.acm.org/doi/abs/10.1145/3334480.3375180