FOVE
THE FIRST EYE TRACKING VIRTUAL REALITY HEADSET
Eye-Tracking in Virtual Reality
In recent years, VR industry is looking for the big leap in hardware manufacturing. The technology is taking steps to bridge gaps in virtual reality by building the next generation headsets. Many startups and companies are putting heavy investments on wireless headsets, enhanced visual fidelity, tracking and interactivity. Eye-tracking technology is promising to deliver a more immersive experience in virtual reality. After the first release of headset with eye-tracking support, many companies like Google, Apple and Facebook started to consider the technology in their next products, which will have a huge impact on the VR industry.
Incorporating the eye-tracking technology into the VR is expected to change the user interfacing with data the way that touch technology changed interactions with mobiles and displays. Although using the eye as an input can be frustrated to interact with the virtual environment, the eye-tracking in VR can work in the background for many applications. It would upgrade the levels of gameplay, interpersonal connections, VR-based research and VR efficient producing[2][6].
Virtual Reality is about immersion, true immersion [11]
The technology works by beaming an invisible headset-mounted laser into the eye. The laser passes through the eye and reflects off the iris to reveal the pupil edge to the sensors in real time. Incorporating eye-tracking technology into VR came at different levels. aGlass is a simple plugin of external lenses to set on the HTC vive headset for eye-tracking. The Swedish company Tobii built their eye-tracking solution into the HTC vive headset as assistive technology. In January 2017, FOVE released their first headset with eye-tracking support as all in one solution [1][2][3].
The technology works by beaming an invisible headset-mounted laser into the eye. The laser passes through the eye and reflects off the iris to reveal the pupil edge to the sensors in real time. Incorporating eye-tracking technology into VR came at different levels. aGlass is a simple plugin of external lenses to set on the HTC vive headset for eye-tracking. The Swedish company Tobii built their eye-tracking solution into the HTC vive headset as assistive technology. In January 2017, FOVE released their first headset with eye-tracking support as all in one solution [1][2][3].
Plug-in solution: aGlass for HTC vive [3] Built-in solution: tobii on HTC vive [3] All-in-one solution: FOVE headset [1]
FOVE Headset
FOVE [1], built the most advanced eye-tracking technology with low latency tracking and complement it with a high-resolution screen to produce the FOVE headset. The FOVE headset has the following features that change the user’s virtual experience:
- Infrared cameras that track the user’s pupils and register their movements
- Eye-tracking capability enables a natural communication with virtual characters
- Facilitate foveated rendering: a technique that focuses the fidelity where the user is looking leading to an improved performance
- Enable realistic depth of filed as the parallax between pupils can be calculated
- Natural user interface to be navigated using user’s gaze only
- Tracking accuracy with less than 1 degree
Source [1]
Why Is It a Good Use of VR?
With FOVE technology, eyeballs can be turned into a mouse, which greatly changes our interfacing with the data. It gives the ability to quickly navigate using person's gaze and activate selection with another glance. Although that can offer a more natural interaction, it can be cumbersome to use the gaze as input method. However, eye-tracking can be used to enhance the VR experience from different aspects. The following briefly explains why FOVE and eye-tracking technology is a good use of virtual reality.
Realistic eye movements of virtual avatars
With eye-tracking, the strange feeling of disbelief that is associated with virtual avatars will be disposed off when realistic eye movements are applied to virtual avatars and players in virtual reality. It’ll enhance the interpersonal communications with natural eye contacts and two-way interaction [6][7][8].
With eye-tracking, the strange feeling of disbelief that is associated with virtual avatars will be disposed off when realistic eye movements are applied to virtual avatars and players in virtual reality. It’ll enhance the interpersonal communications with natural eye contacts and two-way interaction [6][7][8].
Eye movements of virtual avatar - Source [12]
Foveated rendering
Natural focus of human eyes can be simulated realistically. The human eyes perceive the vision with the visual acuity is highest at the fovea area at center of our field of view. Objects outside the fovea area are perceived with less detail and blurry. Eye-tracking enables foveated rendering in which images at the center of the field of view are rendered with full resolution while images outside the field of view can be rendered with less rendering resources. Full high-resolution rendering can be very expensive. With foveated rendering technique, computing power can be preserved and used for other computation that would enhance overall performance [6][7][8].
Natural focus of human eyes can be simulated realistically. The human eyes perceive the vision with the visual acuity is highest at the fovea area at center of our field of view. Objects outside the fovea area are perceived with less detail and blurry. Eye-tracking enables foveated rendering in which images at the center of the field of view are rendered with full resolution while images outside the field of view can be rendered with less rendering resources. Full high-resolution rendering can be very expensive. With foveated rendering technique, computing power can be preserved and used for other computation that would enhance overall performance [6][7][8].
Natural User Interfaces
Users with disabilities will have a natural input method to use VR. FOVE presented two use cases in which people with disabilities can make a good use of FOVE headset. In the first case, a piano is mimicked in VR so it can be played using eyes only. The other case, is to navigate the web using eye movements [1].
Eye plays the piano in VR - Source [1]
Web Navigation Using eyes - Source [1]
New levels of gameplay
Adding a new interaction mechanism will enable new levels of interactions. Eye-tracking driven experiences will be easily and effectively added. The user’s focus is known and hence actions can be triggered as soon as they became under focus. Instead of waiting for an input to start an action, it can begin to progress when the user look at it [5][6][7].
Adding a new interaction mechanism will enable new levels of interactions. Eye-tracking driven experiences will be easily and effectively added. The user’s focus is known and hence actions can be triggered as soon as they became under focus. Instead of waiting for an input to start an action, it can begin to progress when the user look at it [5][6][7].
New level of responses to eye tracking - Source [1]
Study of human behavior using VR and eye-tracking
Scientists can measure users engagement in virtual reality by recording eyes movements and analyze their focus throughout the environment. Further analysis can be conducted on where users mostly focus their looks inside virtual worlds. Another studies can be conducted on users emotions and responses from their eyes tracking. Since pupils’ edges can be estimated using the infrared laser, their expansion and positions can be tracked [6][7][10].
Scientists can measure users engagement in virtual reality by recording eyes movements and analyze their focus throughout the environment. Further analysis can be conducted on where users mostly focus their looks inside virtual worlds. Another studies can be conducted on users emotions and responses from their eyes tracking. Since pupils’ edges can be estimated using the infrared laser, their expansion and positions can be tracked [6][7][10].
Eye tracking for human attention research - Source [4]
Issues with It
Technological challenges
Virtual reality headsets with eye-tracking support face technological challenges. The VR gear manufacturers need to avoid opening new gaps in virtual reality. Integrating a new technology to a gear that is light and consumes little power can be a challenge.
Integrating a technology that works and work well is another challenge. Tracking mechanism should work with infinite variations of eyes shapes and sizes. Therefore, computer vision based independent techniques like infrared-based mechanisms are required.
The FOVE at first step requires eyes calibration to the headset. Users claimed that they were required to wear the device lower on their faces than the usual HMDs, which was a bit inconvenient. However, their overall experience was thrilled [9].
Virtual reality headsets with eye-tracking support face technological challenges. The VR gear manufacturers need to avoid opening new gaps in virtual reality. Integrating a new technology to a gear that is light and consumes little power can be a challenge.
Integrating a technology that works and work well is another challenge. Tracking mechanism should work with infinite variations of eyes shapes and sizes. Therefore, computer vision based independent techniques like infrared-based mechanisms are required.
The FOVE at first step requires eyes calibration to the headset. Users claimed that they were required to wear the device lower on their faces than the usual HMDs, which was a bit inconvenient. However, their overall experience was thrilled [9].
Financial challenges
The biggest financial hurdle for VR consumers is the price point. The price of FOVE is almost hitting the $600, which is higher than what an average person would spend on virtual reality headsets [1].
The biggest financial hurdle for VR consumers is the price point. The price of FOVE is almost hitting the $600, which is higher than what an average person would spend on virtual reality headsets [1].
References
- https://www.getfove.com/
- Cava, Marco della. “Next big thing for virtual reality: lasers in your eyes.” USA Today, 2 May 2016, www.usatoday.com/story/tech/news/2016/05/02/new-mouse-vr-could-your-eyes/83716986/.
- BROWN, MATT. “Eye tracking is coming to HTC Vive: Here's what you need to know.” VRheads, 8 June 2017, www.vrheads.com/eye-tracking-coming-htc-vive-heres-what-you-need-know.
- Murison, Malek. “Tobii Pro combines eye tracking with VR to understand human behavioure.” Internet of business, 22 June 2017, internetofbusiness.com/virtual-reality-tobii-pro-eye-tracking/.
- Eng, Kynan. “Which New Areas of Virtual Reality Will Use Eye-Tracking?” Slat, 2 Mar. 2017, www.slate.com/blogs/quora/2017/03/02/which_areas_of_virtual_reality_will_use_eye_tracking_next.html.
- Kuerzel, Eric. “How Eye Tracking is Driving the Next Generation of AR and VR.” VR scout, 1 Sept. 2017, vrscout.com/news/eye-tracking-driving-next-generation-ar-vr/.
- Rizzotto, Lucas. “Why Eye Tracking is a Huge Deal for VR/AR.” Medium, 15 Aug. 2017, medium.com/futurepi/why-eye-tracking-is-a-huge-deal-for-vr-ar-683e971652ee.
- KARLÉN, JOAKIM. “Eye-Tracking is virtual reality’s next frontier.” Venture Beat, 6 Sept. 2017, venturebeat.com/2017/09/06/eye-tracking-is-virtual-realitys-next-frontier/.
- “A first look at FOVE, the world’s first eye-Tracking VR headset.” Virtual reality pop, 8 Jan. 2017, virtualrealitypop.com/a-first-look-at-fove-the-worlds-first-eye-tracking-vr-headset-705717997aa0.
- Lutz, Otto, et al. “Application of Head-Mounted Devices with Eye-Tracking in Virtual Reality Therapy.” De Gruyter, vol. 3, no. 1, 2017, pp. 53–56., doi: 10.1515/cdbme-2017-0012.
- https://www.tobiipro.com/
- Johnson, Eric. “My Eyes Are Up Here: Eye-Tracking Comes to Virtual Reality Social Network AltspaceVR.” Recode, 8 June 2015, recode.net/2015/6/8/11563334/my-eyes-are-up-here-eye-tracking-comes-to-virtual-reality-social/.