Enhancing VR Communication
Enhancing Communication and Facilitation of VR Experiences in Collaborative Settings between HMD Users and Non-users
Co-authors: Mint Tanprasert, Ege Unlu, Oliver Jacobs
UBC Designing for People Project
UBC’s Designing for People (DFP) is a human-centered design cluster that integrates interdisciplinary research methods and teamwork to solve societal and technical problems (https://dfp.ubc.ca/). The CPSC 544K Designing for People Project course is one of the required courses for the Design for People trainees.
This project investigates the isolation problems between VR head-mounted display (HMD) users and non-users in collaborative settings and proposes a list of design solutions to improve communication and collaboration between VR HMD users and non-users.
Virtual reality (VR) is providing businesses and users unique ways of benefiting from the sharing of 3D immersive visualization. These interactive visual contexts enhance data-driven decision making and facilitate stakeholder collaboration. However, these VR experiences are not always straightforward and effective, especially between HMD users and non-users. For example, when a presenter is wearing the HMD to give a presentation, the display obscures communication with the audience: they can’t see what the presenter sees, and the presenter can’t see the audience. It becomes harder for headset users to observe and communicate with others situated in the same physical space.
The literature review of this project is divided into three areas: development of spatially immersive or physical displays, designs of tools that allow HMD users to perceive the real world, and designs of tools for presenting the virtual world to the non-users. Before interviewing experienced VR users to gauge the significance of each problem we found, my team and I conducted a cognitive walkthrough to explore the problem space. For the walkthrough test, there are two task examples of situations where there is one “presenter” wearing the VR HMD and several “outsiders” without the HMDs in the collaborative setting. The first task example is of a business meeting with approximately 10-20 people in an attendance. The second example is of an exhibition booth, where the audience may join and leave anytime during a presentation. The notes are taken of any problems that arise in the process both from the presenter’s and outsiders’ perspectives.
There are four problems derived and charaterised in this research: presenters cannot see outsiders’ reactions or participation, dizziness watching the stream with a lot of movement, trouble keeping track of current positions and locations of objects in the virtual space, and lack of knowledge on how to use VR equipment and its application in collaborative settings.
Based on the four identified issues, we followed a path of brainstorming and trying new solutions in the course of approximately 2 months following the empathize study.
For the dizziness problem, we proposed a design solution called hand-controlled camera, which emphasizes the congruity between the presenter’s movement to the stream change and forces the presenter to be deliberate with their camera movements. Instead of streaming the presenter’s vision to the audience monitor, the presenter will use their hand-controllers to control an imaginary camera to stream to the audience monitor.
For the audience-to-presenter visibility problem, we proposed a solution called picture-in-picture camera view, which emphasizes the enhancement of non-verbal communication between VR presenters and audiences. The Picture-in-picture (PIP) is a technology that displays visual content on the top of other visual content, which displays both the main screen of the VR HMD and a second screen that manifest the audiences’ reaction outside the VR world. We created a mock-up video to test this solution and collected two participants’ feedback from an interval interview.
For the solution to the lack of visual and temporal references, we proposed a solution that lets the non-users annotate objects from their streamed view, as well as lets them timestamp and save specific frames, i.e. screenshots, and let them transmit these annotations or teleport the VR user to saved screenshot frames.
The solution for the fourth problem is to develop a helpful and inviting deck of cards each with their own VR tooltip. The cards can be thought of as a set of best practices with some containing true solutions while others are just tips on how to minimize the identified problem.
Figure 2. The sketch of the overall experiment setting. Annotation (1) depicts the location of presenter’s camera in the baseline system and (2) depicts its location in the hand-controlled camera system.
To test our designs, we conducted two studies. The first study was for refining the design of the hand-controlled camera. Additionally, we conducted an interview with the participants from the second study to gauge their opinions on the PIP camera view based on a mock-up video and their reflections on the challenges they faced during the study. We sought to understand if the proposed hand-controlled camera solution can eliminate audiences’ dizziness, and if the participants feel the need for the PIP camera-view solution.
Our experimental setup is designed under the social-distancing constraint of the COVID-19 situation. Considering that participants might not have VR equipment at their homes and we could not set up a collaborative environment between several participants in a shared space, we prototyped our solutions using video call applications. The remote communication between participants, under careful arrangements, allowed us to recreate the isolation between VR HMD users and non-users and observe the dizziness, visibility, and reference problems that arise like in actual VR experiences (see Figure 2).
Figure 3. (left) A neck phone mount to holds phone on presenters’ forehead, which mimics the VR headsets, (middle) A cart with stick tripod on top, which the cart provides enough height and tripod provides stabilized rotation of the camera during the presentation, and (right) Presenter room.
To evaluate the effectiveness of the hand-controlled camera solution at eliminating audiences’ dizziness, we conducted the experiment with 2 groups of participants (2 presenters, 4 audience members). We transcribed the audio recordings and extracted quotes to further explain how the solution influences the responses we got from the satisfaction surveys and the participants’ performance on the study tasks. We analyzed this data along with the notes we took during the study and the video recordings for any notable behaviors or occurrences while the participants performed the study tasks.