Physical Interactions in Virtual Space: A Workshop on VR in VR
Updated: Mar 10, 2021
My colleague Jill Walker Rettberg and I held a workshop in AltspaceVR, 'VR Narratives: A Workshop in VR, about VR', for the Electronic Literature Organization (ELO) Conference on the 16th of July 2020, where VR scholars and artists Caitlin Fisher, Illya Szilak, Scott Rettberg, Laryssa Whittaker, and Anna Nacher presented their work.
While much of the discussion on virtual reality (VR) remains about the future utopian or dystopian possibilities of VR, the participants to the workshop showed what is being done now with VR and in VR.
Caitlin Fisher (York University) mapped two tendencies in terms of virtual reality narratives, one that came from the cinematic tradition of imposing a certain perspective to the viewer, and one that elaborates as electronic literature, through multimodal, poetic, and participative elements. For Illya Szilak it is the possibility to create unmediated experiences that she values with making virtual reality (only mediated through our own senses). About her pieces 'Queerskins: A love story' and 'Queerskins: ARK' to be released in full later this year (a 360 video based on the VR work was viewable at the 2020 Marché du Film Cannes XR), she insisted on the aesthetic and narrative value of filming on location before starting any volumetric manipulation that will give rise to the final computer-generated images (CGI). Scott Rettberg told us about his experience when co-creating Hearts and Minds for a cave environment. In spite of the cartoon environment that VR imposes, Scott argued that it allowed an affective involvement of the viewer into a space of individual and cultural memory (of the atrocities during the Irak war).
Laryssa Whittaker exposed her ethnographic research with VR participants, as part of an Audience Insight team at StoryFutures in London. When looking at the before, during and after of VR experiences, she explains the complexity of audience experience --for example stating that rooms at home were often not big enough, and that people valued the possibility to share their experience with others as it was happening--, considerations that should be taken into account by creators and producers. Finally, Anna Nacher provocatively ended the panel by reminding us of the limitations of the technology --the restrictive participative possibilities and costly headsets--, while proposing open source possibilities and a diversity of narratives. All in all, an extremely rich conversation started with this exhilarating experiment of holding a conference panel on VR, in VR. The experience, however, also made me reflect on different aspects of VR, both its narrative/participative aspects, and its technical limitations and problems of accessibility.
Whether we think about virtual reality within the history of cinema or interactive art, VR is often discussed in terms of presence, and (un)mediated experience. If VR fictions and non-fictions aim to immerse us within a different reality than our everyday 'physical' reality, most pieces do not achieve the unmediated presence towards which they aspire (on this topic, also read Julia-Scott Stevenson 'Virtual Futures: A Manifesto for Immersive Experiences'). Just like Julia, I have stopped counting the number of 360 videos that have made me wonder about the purpose of the headset that I was wearing. Many pieces of work viewed in VR do not exceed the immersiveness of cinema, or even sometimes achieve less of it.
First, VR is physical and cannot be deemed to be entirely virtual, our experience in and of VR is mediated through our bodily senses (as Katherine Hayles also pointed out with regards to earlier instances of VR). This implies that movement within VR must respect physiology and the refreshing rate of the screen frame (see Philippe Fuchs' work), so as not to induce nausea or vision trouble. This also implies that the sound environment must be treated with as much --if not more-- attention as the visual. As film theorists explained, one cannot close off the ear (see Michel Chion's work, and Sean Redmond's video essay on sound and attention shown through eye tracking). Sound directs our attention, whether inside the virtual environment or outside of it. I have experienced VR in noisy exhibition halls with no noise-cancelling headphones, which ruined any attempt of immersion.
Second, the present form of the technology does not allow unmediated experiences even though body localisation, eye tracking, and what has been termed 6 degrees of freedom are becoming more commonly integrated within headsets (6DoF, referring to the degree to which the participant can move and to which extent their movements will be perceived by the device). The mediation between our senses and the virtual reality presented to us still exist and is well present. 'Being' in the virtual environment of AltspaceVR and listening to the speakers was hardly an unmediated or immersive experience. The headset is heavy and after an hour and a half of wearing it, my neck felt sore, and I felt nauseous. Being an avatar that floats around and must move through teletransportation (although that would be great in real life!) does not contribute to making the experience immersive; even less so than a video game where our avatar would move and run the way humans generally do. If the goal of AltspaceVR is to replicate reality, to offer a platform where people would feel more present than they feel while in teleconference (something that many will have become familiar with during the Covid-19 pandemic), among others rather than looking at each other, I would not say that it is successful. The numerous aural and visual glitches (whether because of the headset, of the AltspaceVR platform, or of insufficient bandwith) constantly reminds us of the technological mediation between our bodily experience of a world and the world itself. What it does successfully however is bring us in a different reality, a colourful one, one where we can choose to represent ourselves differently.
While the possibility of representing oneself differently than one is in the physical world exists, many of us chose to represent ourselves as closely as possible to what we generally look like in terms of gender, skin colour and hairstyle. This is partly due to the academic conference setting, where being (physically) recognised by others is key. A big part of academic life involves networking, and thus also shaping our working persona on the net. Another reason for this is being encouraged by social media platforms to use our real names and figures as username and avatars. AltspaceVR for example has made the choice of offering realistic avatars and removing robots (after the update on the 15th of July 2020). With the suppression of the cyborg, the platform left little space to play and experiment, with funky hair and skin colours, or designing a queer character instead of a sexed one.
Interacting in virtual reality --or cyberspace-- through the mediation of an avatar would presumably raise the possibility to escape racism or sexism. As Sally Pryor and Jill Scott ask, 'Could you feel pain if you had no body? Could you experience racism or sexism?' (qtd. in Bailey, 35). As AltspaceVR pushes for users to choose a realistic avatar, it also asks them to choose a skin color and inevitably a sexed representation of the self (through clothes, hair style, face shape). As Cameron Bailey emphasises, the body in cyberspace remains a 'powerful referent' of categories of identity, and to a certain extent of social power. Most of the participants of the workshop embodied fair skinned avatars (or blue for those conscious (or not) of the impact of skin colour), which makes me suspect that most of them were fair skinned in real life too. While the demographics of VR headset users broadens and this particular workshop aimed to be inclusive, the gaming community that uses VR still remains largely dominated by white middle-class males. On the internet -- just as in the physical world -- strategies of exclusion multiply, from the use of acronyms (such as one that could be applied to this very article: TL;DR, 'Too Long Didn't read'), to vitriolic comments, or even doxing. As many feminist works on online misogyny have pointed out, white male's visions of reality predominate on the web (see Emma Jane's article and Feminist Media Studies on online misogyny).
If the Internet has been recognised as the non-inclusive place it has always been, there still remains myths about its presumed accessibility. Internet is neither available everywhere around the globe and to everybody, nor has it become a human right (as some would like to believe!). When it comes to VR, the problem of accessibility becomes even worse, not only in terms of socio-economic resources, but also with hearing, vision impairment, or propioceptive disability. Many would have a computer or a smartphone --with increasingly available assistive technologies-- from which they could connect to the net (and thus join a teleconference and access a digital conference in this particular case), but only a few have access to a VR headset, or have heard about it altogether. For us, as organizers and panelists, the workshop was a way to experiment with the capabilities and limitations of a platform that we discuss in our work. On the one hand, the limited number of people with access to VR headset around the world makes our panel inaccessible (in spite of the desktop option that AltspaceVR proposes), and on the other hand, holding the panel in VR opened it to participants who had never heard of the ELO conference before. Many, of varied age and location, joined the virtual room where the workshop took place simply because it was one of the events with most participants going on at the time on the platform. In AltspaceVR, it was visible to all (contrary to the rest of the digital conference only accessible with the link). That is, visible to all with a VR headset.
If VR headsets are becoming more and more of a reality than accessories in sci-fi films, they do remain inaccessible to most... Inaccessible, or sometimes, merely unwanted. One last point that I think worth to be touching upon is the ethics when making a VR piece. As the researchers of the 'Virtual Realities: Immersive Documentary Encounters' (@VRDocsProject) project have pointed out, ethical considerations are a must, especially when making non-fiction. In their article 'Real Virtuality: A Code of Ethical Conduct', Michael Madary and Thomas K. Metzinger formulate recommendations for researchers and producers towards ethics in VR, regarding both psychological and physiological approach to the virtual environment, and the privacy of the user and collection of their data.
In particular, and for reasons mentioned earlier, it is the embodiment and habitation of space that should first be considered. As said, virtual experiences are mediated through the body, and one's body or avatar will influence the way we are in space; the way we occupy space, and the relations of power that constantly permeate our environment and interactions with others. As phenomenologists affirm we only inhabit space through our senses, and our bodies have been marked (or 'oriented' in Sara Ahmed's terms) by social relations and history from which we cannot be divided. Mandy Rose, co-investigator on the VRDocsProject, predicates phenomenology and experience design instead of textual analysis in order to address VR narrative forms and storytelling. Both Rose, and Kate Nash argue for a 'proper distance' when making VR non-fiction, respecting the documentary film tradition of creating proximity with the subject while also not conflating the audience with the 'other' in the story (I have talked about this elsewhere, with my co-author Chris Ingraham, in relation to the VR piece Traveling While Black). For Nash, VR non-fiction risks doing just that, creating what she calls an 'improper distance’. Similarly, neuroscientific experiments involving VR for clinical psychology, such as the ones made by Maria Sanchez-Vives, Mel Slater and their team, require serious and continual considerations of ethics, so as to ensure the bodily and psychological well-being of the participants. The embodiment or representation of gendered or racialised avatars comes with the responsibility of acknowledging the dimensions of power and systemic sexism and racism that may determine the participants' relations to their own or other avatars (as Nick Yee and Jeremy Bailenson show).
Ahmed, Sara. 2006. Queer Phenomenology: Orientations, Objects, Others. Durham; London: Duke University Press.
Bailey, Cameron. 1996. 'Virtual Skin: Articulating Race in Cyberspace'. In Mary Ann Moser, ed., Immersed in Technology: Art and Virtual Environments. Cambridge; London: MIT Press, 29-49.
Ceuterick, Maud, and Chris Ingraham. Forthcoming 2021. 'Immersive Storytelling and Affective Ethnography in Virtual Reality'. The Review of Communication 21(1).
Chion, Michel. 2005. L’audio-Vision: Son et Image Au Cinéma. Paris: Armand Colin Cinéma.
Fuchs, Philippe. 2018. Théorie de la réalité virtuelle: Les véritables usages. Paris: Presses des Mines.
Ging, Debbie, and Eugenia Siapera. 2018. ‘Special Issue on Online Misogyny’. Feminist Media Studies 18 (4): 515–24.
Hayles, N. Katherine. 1996. 'Embodied Virtuality: Or How to Put Bodies Back into the Picture'. In Mary Ann Moser, ed., Immersed in Technology: Art and Virtual Environments. Cambridge; London: MIT Press, 1-28.
Jane, Emma A. 2016. ‘Online Misogyny and Feminist Digilantism’. Continuum 30 (3): 284–97.
Madary, Michael, and Thomas K. Metzinger. 2016. ‘Real Virtuality: A Code of Ethical Conduct. Recommendations for Good Scientific Practice and the Consumers of VR-Technology’. Frontiers in Robotics and AI 3.
Nash, Kate. 2018. ‘Virtual Reality Witness: Exploring the Ethics of Mediated Presence’. Studies in Documentary Film 12 (2): 119–31.
Redmond, Sean. 2017. ‘The Ear That Dreams: Eye Tracking Sound in the Moving Image’. [In]Transition: Journal of Videographic Film & Moving Image Studies 4 (3).
Rose, Mandy. 2018. ‘Technologies of Seeing and Technologies of Corporeality: Currents in Nonfiction Virtual Reality’. World Records 1 (1).
Scott-Stevenson, Julia. 2019. ‘Virtual Futures: A Manifesto for Immersive Experiences’. Immersive.News (Medium), March, 8.
Seinfeld, S., J. Arroyo-Palacios, G. Iruretagoyena, R. Hortensius, L. E. Zapata, D. Borland, B. de Gelder, M. Slater, and M. V. Sanchez-Vives. 2018. ‘Offenders Become the Victim in Virtual Reality: Impact of Changing Perspective in Domestic Violence’. Scientific Reports 8 (1): 2692.
Yee, Nick, and Jeremy Bailenson. 2007. ‘The Proteus Effect: The Effect of Transformed Self-Representation on Behavior’. Human Communication Research, no. 33: 271–290.