How will it perform? Answer to AR HMDs or an Alien tech?

The buzz is everywhere. The next version of HoloLens is coming this year. Industry is waiting for it anxiously. The hopes are high provided how well the current version of HoloLens has served in building up developer communities around the global. Being tagged a ‘prototype development device’ at its launch, the device has proven that it is much more than a prototype device or one of the best prototype of anything ever built.

On 21st January, 2015 Alex Kipman revealed to the world a tech which I and many others thought to be an alien tech with whom I discussed about it. 3 years from then, today HoloLens has set the industry standards for AR HMDs.

On 24th February, 2019 at Mobile World Congress, Microsoft may finally be revealing the next gen HoloLens. (Follow the Verge’s article to know more https://www.theverge.com/2019/1/16/18185318/microsoft-hololens-2-mobile-world-congress-press-event-date ). Though nothing is being confirmed yet but a reveal seems to be imminent.

With all the excitement about next gen HoloLens inside me, the expectations are very high. I have written this article to discuss the features which I expect to be in the HoloLensVnext. The features I am going to discuss can happen to be implemented in HoloLensVnext at different level of quality and magnitude. To help in describing the features I am opting for ‘Bronze, Silver and Gold’ pointer scheme to identify at what level I am expecting a certain feature to be implemented and how much worth it would be. It is more like rating HoloLensVnext features on scale of 3 with bronze being 1 pointer and Gold being 3 points. Now to start with, I will discuss about the most talked about thing of any AR HMD. You guessed it right, It’s the Field of View []-).

1: Field of View (Expectation- Silver)

It is the most talked about feature of HoloLens which every other AR user wants to see get improved. Having a limited FOV is one of the reasons why HMD AR feels like a gimmick to some of its users. Talking about how much improvement we might see is totally unknown but I am expecting it to be double of the existing FOV which will be 70˚. To setup the scorecard, anything above the Magic Leap One FOV will earn a bronze, if the FOV is doubled I will give silver and anything above 75˚ will earn HoloLensVnext a Gold.

2: Eye tracking (Expectation- Silver)

Talking about input system for AR and VR experiences, eye tracking is the feature which will revolutionize the entire user experience. It will offer the most natural way to interact with digital world and reduce GPU usage with foveated rendering. Currently, Magic Leap One supports eye tracking which is not fully implemented. If HoloLensVnext comes with eye tracking support but not fully implemented then it will earn a bronze, if it is working through-out the OS and open for developers then it will earn a silver. Keeping the hopes high, if the HoloLensVnext supports foveated rendering then nothing can be more golden then HoloLensVnext.

3: Hand Tracking (Expectation- Silver)

The most natural way to interact with digital object is simply grabbing them with your own hands. Many other AR HMDs have already implemented it and this is the region where I believe HoloLens lacks the most, even more then the FOV. The best in the market right now will be the Project North Star from Leap Motion. In 2017, due to a certain leaked video it was almost confirmed that HoloLensVnext will be able to do hand segmentation tracking. This prediction has the most chances to be true. Setting up the scorecard, if hand tracking implementation is similar to what Magic Leap One has then it will earn a bronze, if it is comparable to Project North Star then it will earn a silver. Achieving Gold will depend on at what level it has been implemented in the OS and how much liberty developers have in utilizing it.

4: Processor (Expectation- Almost confirmed!?)

Rumor has it that HoloLensVnext will be an arm based always connected Windows device. It is expected to be running on Snapdragon 850 SOC. This feature is also almost confirmed and there is a little less of what may change it because processor is something around which the entire device is designed for. If Microsoft plans to disappoint us and put some Intel chip inside it then I will have to give a bronze. If it is a Snapdragon 850 then it will definitely earn a silver. Although the chances are negligible but I will love to see HoloLensVnext running on Snapdragon 855 SOC (build on 7nm) or maybe new Snapdragon 8cx SOC. If this happens, then it’s all gold about HoloLensVnext.

5: Object tracking (Expectation- Bronze)

HPU 2.0 is already responsible for hand segmentation tracking and I want more out of it. If HoloLensVnext can support controllers like Mixed Reality Controllers then I will give a bronze. If it can track random objects natively like a ball or a pen then it will earn a silver. And if it can track human position and movements then it will be all gold for HoloLensVnext. My proposed features for Object tracking may seem a little less specific and this is because I have a very little knowledge of AI and machine learning stuff. So please, suggest me how else I could have proposed the features here in the comments below.

6: Scene Understanding (Expectation-Silver)

Scene understanding is a feature which brings realism in an AR experience. I have seen some great implementation of it on ARkit platform. Again HPU 2.0 will be responsible for this. Though, it also depends on the software development team to bring this feature in HoloLensVnext. Object recognition is the simplest way of implementing it. Having a real time and native support for object recognition will earn a bronze. I am expecting HoloLensVnext will have scene understanding support similar to ARkit platform. Though I am a little skeptical about it but if it happens then it will earn a silver. If it is better than ARkit platform then I will give it a gold.

7: Spatial Mapping (Expectation- Bronze)

Spatial mapping is what sets AR HMDs apart from smart-glasses. Our computers are finally able to understand the environment around them. HoloLens again is the king here so it’s hard to expect what improvements can be brought in spatial mapping. Microsoft has made the Fourth Generation Kinect sensor keeping HoloLens in target and it’s the only second thing confirmed to be the part of the HoloLensVnext by Microsoft itself other then the HPU 2.0. The collection of raw data for scene understanding, Hand segmentation tracking and Spatial Mapping will be done by Kinect V4. Some of the raw footage of the demos of Kinect V4 takes the expectations to the next levels. I have always find spatial mapping to be the key point of AR and have the most ambitious expectation from HoloLensVnext in this department. I am expecting real time spatial mapping at 60Hz refresh rate coupled with Scene understanding AI stuff to fill in the gaps and smooth out the edges. If this happens then real time occlusion will be made possible, further enhancing the realism of AR experience. I will give HoloLensVnext a silver for achieving this. Improvements will definitely be there but if they are less then what I expect them to be I will give a bronze only. I have already asked a lot and improving something which is already best in the market is quite tough. If HoloLensVnext can produce some amazing and ground breaking results with its Kinect V4 like 120Hz refresh rate then it will definitely get a Gold. I have kept my hopes low on the improvement of this feature because it is the place of the most variability and low controllability. Spatial Mapping is the most important and toughest of the features to be implemented in an AR HMD.

8: Ergonomics (Expectation- Bronze)

I can use HoloLens for hours with HoloLens Fan spirit inside me but you cannot because HoloLens is simply not designed well enough to be comfortable on the head for long duration of use. Asking someone to wear it for the first time is a total mess. The design needs to be improved a lot and I am expecting it to look completely different than the current version. A three year development cycle is enough or not to make HoloLens ergonomics better can only be known by trying it. Keeping my hopes low, I am only expecting it to be much lighter than the current version and it is the bare minimum for qualifying for a bronze. If there is a complete over-haul of the design and they look more like glasses rather than googles then it will earn a silver. Defining a standard for Gold will get extremely descriptive so I am leaving it for now. Hope HoloLensVnext can surprise us here.

9: Battery Life (Expectation: Silver)

One of the reasons behind the small FOV of HoloLens is the battery life. It has been confirmed by some of the Microsoft officials that they could have used a bigger FOV wave-guide in HoloLens but they choose not to on expanse of saving the battery life of the device. With an ARM based SOC and more R&D from the past three years, battery life will definitely improve. If the battery life remains the same (keeping in mind that FOV and resolution of the wave-guide will be more) I will give a bronze. Anything between 3–6 hours will earn a silver. More than 6 hours will be a Golden feature of HoloLensVnext.

10: Price (Expectation: Silver)

AR HMDs are not cheap. They are expensive piece of tech and that’s what stops them from going mainstream in consumer space. There is no doubt the next version will be cheaper because it will no longer be a development device anymore. There is the enterprise market which Microsoft can target this time and they can keep cost of production low by simply increasing the scale of production. Nevertheless, the R&D also have brought the production price down. I am expecting a price of anything between $1500 and $2000. At this price range it will definitely beat Magic Leap One by a great margin and I will happily give a silver. I am quite positive to see this price point but cannot be sure of anything. The price will definitely be competitive to the Magic Leap One, so anything between $2000 and $2500 will earn a bronze. If Microsoft thinks that there product is ready to hit the market of general consumers then we might see a price point below $1500 and nothing will stop HoloLensVnext from being the first mainstream general purpose AR HMD. And of course it will earn HoloLensVnext a Gold.

Milestones

There are several features and aspects of AR HMDs which are crucial for their mainstream adoption. These aspects are like door to the future of AR HMDs and basically tells us that future is here.

1: 90˚ FOV

We have seen 90˚ FOV from the Meta 2 but that device doesn’t gain much attention. Having a 90˚ FOV is a must to avoid the FOV limitation concerns from its user. I believe it to be the threshold where one will not complain about the limited FOV. Having a 90˚ FOV in a popular device like HoloLens will definitely ring the doorbells of the door to the future of AR HMDs.

2: All in one AR HMD glasses (not goggles)

HoloLens is already an all in one device but it looks more like goggles. Magic Leap One does come close to being glasses but it is not an all in one device. ODG showed us a device with all in one glasses design but the device never gained any attraction or more like never sold to anyone. Having an all in one design coupled with glasses form factor is essential for AR HMDs to be a mainstream consumer device. Possibility of such a device is higher now than ever before.

3: A $1000 question

A device costing thousands of dollars is not a device for everyone. Suppose HoloLensVnext is capable of replacing smartphones but if the cost is too high it won’t attract general consumer. If a device like HoloLensVnext is available at a price point below $1000, I would say doors to the future of AR HMDs are wide open. Pro consumers will definitely buy it and we will only be couple of years away from AR HMDs being the next big thing for media tech industry to grow on.

4: User Interface of future

Windows 10 is currently the only major OS which has a dedicated AR/VR optimized operating system. Interacting with the OS on daily bases I never felt for anything lacking but my expectations from OS of the future are much higher. AR/VR marks the timeline by which we will go beyond 2D screens. It will be the era of 3D user interface. A powerful OS will be required to demonstrate the full potential of these future devices. HoloLensVnext is expected to be running on the brand new concept of Windows called ‘Windows Core OS’. Windows Core OS being the most modern OS will play a crucial role to determine if we are ready for the new era of UI/UX.

Factors of surprise

One of the things which could surprise us is the dramatic change in the form factor. Having two units system seems feasible to make a powerful and lighter device but I am sure Microsoft doesn’t want to address the power vs comfort problem in this way.

Some of the patents from Microsoft tells about their plans to add a feature through which translucency of the wave-guide can increased or decreased in real time. This will help in watching through the HoloLens under sunlight. Also the level of realism will reach the next level as the holograms will no longer be transparent anymore.

There are other notable features like faster and more memory support. High resolution camera. Aesthetics and I/O are also important things but they are definitely meant to be improved.

Wrapping up

You may have noticed that most of my expectations are on Silver level. The reason for that is the HoloLensVnext is actually a 3rd generation device and so improvements are expected to be much more. My Gold level features are what might answer my question asked in the beginning of this article. If HoloLensVnext if able to achieve two or more Golds then I will be convinced that Microsoft is simply working on reverse engineering some alien tech they would have found and presenting it as HoloLens to the world. Jokes apart, I am riding the hype train for HoloLensVnext to the MWC 2019. Nothing is confirmed by Microsoft about the 24th Feb event. That event might just be about some other Mixed Reality stuff and HoloLensVnext might not be part of it. But the reveal is imminent this year, experts are positive on this news and I cannot wait for it anymore.

When it comes to achieving the mentioned milestones, the question is not when Microsoft will be able to do it but rather when a device like this will be possible. Keeping HoloLens aside, I want above milestones to be achieved as soon as possible for the sake of developers and industry. But when I look around in the market I found only one company and device to be closer to it. That company is Microsoft and the device is HoloLensVnext []-)