It takes a lot of foresight, commitment and courage to imagine and act upon the next potential move when your competitor is still at its infancy planning the first move. Now that’s corporate strategy working hand in hand with the R&D folks.

Move over FOVE. Here come’s SONY. As we get ready for 2016 to experience some of the best Virtual Reality headsets and games, SONY is already working on another game changer even before their first Playstation VR reaches the hands of the eagerly waiting fans. SONY is pushing the envelope to make its VR experience the most immersive!

As we were busy shopping for Christmas and gripping about the delay around HTC-VIVE’s release, SONY quietly got a patent approved. Based on the patent approved on December 8th, (P# US009207455), it appears that SONY is already planning its next generation of Playstation VR with one exciting feature that will silence some of the critics. SONY’s patent filing suggests the possibility of an eye tracking VR headset . As the enthusiasm builds around the yet to be launched devices such as FOVE that promise eye tracking, SONY is already positioning itself for the next battle in virtual Reality HMD and games by tackling headon some of the issues that will be experienced by the users in this generation of HMDs.

Brief History of Eye Tracking

The fascination around eye tracking is not new in the sciences. In fact, it has been around as early as the 1800’s. In the early 1900s, an educational psychologist named Edmund Burke Huey built an early eye tracker. He used contact lens with holes for the pupils. The contact lens were connected to aluminum pointers, which would move along with the eyes to track a test subject’s eye movement. In the 1950s, a Russian psychologist named Alfred Lukyanovich Yarbus conducted several eye tracking studies that resulted in important eye tracking research. His 1967 book is still referenced today, as it showed that the task given to the eye tracking test subject has a very large influence on the individual’s eye movement and fixation.

Why is Eye tracking important in Virtual Reality HMDs?

Expereince issues such as geometric distortion can be addressed via the processing power of the GPUs. There are other problems that are not solvable that easily. For example, consistency of focus across the visual field, no amount of software correction can solve for this and that one of the reasons why HMD manufacturers such as Oculus use powerful lenses in their headsets.

Niether the GPU or the powerful lenses can tackle the issue of pupil swim.

This happens when an image distorts as you move your eye around the lens in VR. This can only be corrected by using eye tracking mechanism and generating corrections to the images on the fly. That is where SONY is headed. It can dramatically improve the immersive experience.

How is SONY planning to use the eye tracking technology in its next generation Playstation VR devices?

SONY’s is pursuing a novel approach to the eye tracking functionality as described in this patent. The patent describes two cameras one for each of eye that will track the eye movements. The cameras can either rely on the illumination provided by the displayed images within the HMD, or could use infrared or other illumination directed towards the user’s eyes. The objective is to pinpoint the location of the pupils of the eyes and use appropriate image processing and correction techniques to provide the user a better immersive experience. The design is based on a recognition that optical systems such as lenses used in HMDs can suffer from optical errors such as chromatic aberrations which depend upon the position of the viewer’s pupils relative to the optical centers of the optical systems. This advanced process aims to provide a correction based upon the detected pupil position.

Using the eye tracking cameras can help the software detect the dominant eye of the viewer. This feature can be really helpful in shooter type games. When the game enters the weapon aim mode, where the user is aiming (using gaze tracking in respect of an HMD) a virtual weapon such as a virtual rifle, the system will automatically establish the dominant eye and it will use it for aiming processes in this example.

SONY is also planning to place a video camera in the front of the headset facing externally, that will allow the HMD to do better algorithmic motion sensing by comparing images from this camera. This will be in addition to the usage of any piezoelectric accelerometer and optical fibre gyroscopes. Another novel approach to enhance th user experience.

The eye tracking feature if implemented in the Playstation VR will allow for a far more error corrected immersive experience and will be a steP in the right direction to tackle issues such as pupil swim.

Lets hope the 30+ million fans of SONY PS4 say a quiet little prayer tonight before going to bed!