Since it was first revealed to a friendly group of reporters in January, HoloLens has enjoyed a great wave of positive comments, but depending on one’s own experience with Augmented Reality (AR), each person will have a different take on it.
I always had a huge interest for things related to 3D real-time graphics. A long time ago, I started coding 3D as a hobby in the demoscene, before making it a professional activity for many years. It was pretty awesome. I’ve seen quite a bit of VR and AR contraptions, so I think that I came into this with reasonable expectations based on what I thought is possible with the current technology. Here’s my take on HoloLens.
What’s the HoloLens experience like?
Let’s cut to the chase — you want to know how it feels to use HoloLens, and if the hype is warranted. Well, the short story is: HoloLens is amazing and well executed – but much more can be done. If you aren’t familiar with VR/AR devices, let me explain a bit more:
The biggest difference with regular modern AR experiences which run on phones and tablets is that your hands are free, and that the display is translucent, so truly feel like the 3D objects are present in the room. And that is a huge difference when compared to a tablet AR experience, which feels more like looking through a window.
HoloLens projects objects into your environment because Microsoft has managed to create a display that is truly frameless and provide a near-perfect match between real-world and 3D objects."NEAR-PERFECT MATCH BETWEEN REAL-WORLD AND 3D OBJECTS"
The tracking of the user movements is also flawless. That was one of the more impressive feat of the demo. When you use Virtual Reality, you are completely disconnected from the “real world”, so even when tracking is not perfect, it doesn’t show as much. With HoloLens the tracking has to be perfect, and I was impressed that Microsoft pulled it off the way it did. The whole thing feels very natural.
Finally, there was no sickness induced by HoloLens. I am not particularly sensitive to VR sickness, but I could always tell that something was slightly off. That’s because there is always a little difference/lag between what your body expects and what the eyes are seeing in VR 3D. Since HoloLens lets you see the real-world, the match is perfect, and I suppose that this is the reason behind the lack of sickness (at least, for me).
HoloLens brings true added-value from Day 1
Obviously, when we’re talking about new technologies, some ideas about how to apply them are great, while others aren’t. I personally do not think that virtual paintings, or Skype calls with a virtual screen with HoloLens bring a true value beyond novelty. These need a lot more work.
"SUCCESS METRIC: DO IT 10X BETTER THAN A 2D DISPLAY"But things like helping someone with instructions for repairs, or working on architecture and design projects with AR could truly benefit from something like HoloLens. You can imagine all kinds of scenario, but my metric for success is: if I can do it 10X better with HoloLens than with existing display technologies, then it’s a win.
The construction/architecture demo that I played with was probably the one with the most formidable added-value: if you look at how that industry works today, it’s quite painful: modifications and details are often discussed using text and, 2D plans at best, and everyone has to figure out what that means for the actual construction. Here’s Microsoft’s pitch for this demo:
With HoloLens, it is possible to take a virtual walk into the construction space, look at the virtual building and go over proposed changes. It is also possible to overlay information such as pipes, electric cables and other layers. Proposed changes can be visualized and it makes a lot more sense than discussing it on “paper”. HoloLens truly gives you a sense of “seeing with your own eyes”.
There are many scenarios to be tested and invented, but with the proper set of data, HoloLens can give users the equivalent of X-Ray vision and the ability to put oneself in a real setting. Because Microsoft tuned the field of vision and the overall integration of Geometry well, the integration of 3D over reality works very well.
Image quality (good)
The image quality was good. I’m not sure what kind of resolution it uses, but it (perceptibly) felt as if I was looking at 720p 3D rendering. You don’t have that feeling of being so close that you can see individual sub-pixels, which is quite common with VR headsets.
None of the demos featured advanced shading (think “PC gaming”), but that’s probably because the rendering seems to be done on the device itself, so graphics processing (GPU) power would be limited. In the current demos, the graphic rendering was relatively simple (see the videos).
Conceptually, it may be possible to render on PC and stream to HoloLens, but that’s another story…
Eye comfort (very good)
As I said before, Microsoft made HoloLens sync near-perfectly with one’s field of vision. The design of the projectors also makes it very comfortable because it’s a little bit farther from the eyes. This is really important because “comfort” translates into how long, and how productive, you can be with it.
A nascent technology
As it is, the HoloLens hardware could help many types of users from Day 1. However, there are a few things that should (and probably will) evolve going forward.
Limited field of vision
First of all, the field of vision isn’t as wide as you may think when you look at the device. The design is slick and it seems as if the whole visor was the display, but no. The current technology doesn’t allow projecting an image over the whole visor.
Instead, it’s like walking around with a transparent 30” monitor placed 30” away from your eyes (I’m “guesstimating” it). The 3D content will be clipped at the edges of that rectangle, so you have to turn your head to look at something large in its entirety.
I don’t think that one can do much better with current projection technology (assuming a reasonable price), so I would consider the current implementation to be top-notch.
If the field of vision was extended, the experience would feel more and more natural. They don’t need to cover the entire field of vision (the human field of vision is quite large – almost 180 degrees, including the peripheral vision), but I would be extremely happy with -45 to +45 degrees.
Better rendering needed
Yes, today’s HoloLens rendering may be simple, but given how fast graphics performance evolves, it’s certain that in coming years, things will look more and more real, which will add even more value to HoloLens.
I’m speculating, it is probably possible to use the on-board cameras to capture shots of the surrounding area to create basic Light Probes. It’s a technique used to better integrate 3D objects in real-world scenes.
Sooner or later, image-based lighting developed long ago by the movie industry (PDF link to ILM’s work) will be used with devices such as HoloLens. Given enough time, “better rendering” will happen, there isn’t any doubt about that.
3D models availability
Anything that involves 3D rendering requires someone to build content: models, textures, etc.… Industries that rely on computer assisted design (CAD) already have the data needed to make HoloLens useful. They are likely the first ones to make good use of this technology.
But if you don’t have the data, it’s not obvious to build it. For instance, HoloLens could be awesome to visualize a renovation project for your house. However, there’s quite a bit of work that needs to be done to create the house model at the exact scale. Hopefully, techniques to reconstruct 3D models from a pair of cameras can be put to use here.
Stores like Ikea already have models that you could virtually place in the home. etc… it’s not easy, but it’s possible.
How is it compared to Google Glass?
I get this question… a lot. In reality, HoloLens and Glass are very different. Google Glass is not built for AR and feels more like walking around with a computer monitor on the upper-right corner of your vision. You are supposed to look up+right to see the information.
HoloLens displays the information on top of what you are looking at – this is the “Terminator vision” that can be seen in many movies.
Both devices may overlap slightly when it comes to displaying information, but overall, glass remains a “side screen” that I would not consider as Augmented Reality, while HoloLens is the best implementation of Augmented Reality that I have seen.
Ironically, HoloLens probably embodies what most people “think Google Glass is”.
Conclusion
The HoloLens concept and current implementation are impressive. Could it be better? Absolutely – and in time, it will be, but I think that the HoloLens team has done a magnificent job.
I don’t believe that we will all have a HoloLens on our heads in the next 5 years, but I do believe that with the right apps, HoloLens can truly make some activities and tasks far better and efficient.
There is a real industrial value to AR, and companies like Fujitsu have proven that already. HoloLens takes the whole thing to the next level.
Filed in . Read more about Augmented Reality (AR), Build and Microsoft.