Into the Wild: An Immersive Adventure at ArtScience Museum in Singapore Delivered on the Lenovo Phab 2 Pro with Tango

A collaboration between Google, Lenovo, WWF and Singapore artist Brian Gothong Tan, and in association with Panasonic and Qualcomm, Into the Wild is a virtual adventure that turns over 1,000 square meters of the Singapore ArtScience Museum, into a virtual rainforest. Moreover, all that, using the world’s first Tango-enabled smartphone unveiled in June 2016, the Lenovo Phab 2 Pro.

Divided into different virtual scenes, the adventure highlights five endangered animal species and ends up into reality with a pledge to WWF (World Wide Fund for Nature), and for every virtual tree planted, a real tree will be planted in a rainforest in Indonesia,

The initial concept was created by Google Zoo, Google’s creative think-tank, and built with technology and virtual reality content developed by award-winning digital agency MediaMonks.

All the main partners collaborated on the decision-making process to deliver “Into the Wild.”

How does the experience work?

Using Lenovo’s Phab 2 Pro, with its embedded technology Tango, ArtScience Museum visitors will start their journey in Basement 2 as a ranger in a virtual rainforest where they will discover 5 animals of the Southeast Asian rainforest and the challenges they face.

Watch the video I shot right after the official press conference on site in Singapore, on February 8, 2017, to get a better sense of the mix-reality experience delivered by Into the Wild.

During the adventure that spans on different scenes and while observing the pangolin (on the exhibition poster), the tapir, mousedeer, orangutan, and the tiger, visitors will take action to defend the environment.

At the end of the tour, at Level 4, they will help to restore the real rainforest of Southeast Asia, by pledging to WWF and planting a tree in Rimbang Baling, one of the last pristine rainforests in Sumatra where the endangered Sumatran tigers live.

Into the Wild’s mixed reality experience is enhanced by a cinematic experience, directed by one of Singapore’s leading filmmakers, Brian Gothong Tan. At the end of the virtual tour completed using the Lenovo Phab 2 Pro, visitors are invited to a dark room at the 4th level where they seat in group in front of a giant vertical screen – I shot a few pictures during the screening, however, I will not publish since I do not want to spoil the surprise effect of this impressive artwork.

In his film, Tan depicts the fragile habitat of the five characters featured in the virtual adventure – pangolins, tapirs, mousedeers, orangutans and tigers.  Tan’s animation describes the animal journey from creation to destruction and rebirth, and the installation is combined with real-world information and animations about the animals and the forests of Southeast Asia, displayed on six additional monitors on the opposite side of the main screen. (see photo)

The Technology: what is Tango on the Lenovo Phab 2 Pro?

Lenovo unveiled the Phab 2 Pro in June 2016 at an event in San Francisco. The world’s first Tango-enabled smartphone is powered by a Qualcomm Snapdragon 652 SoC and features three camera sensors that help to delivers the Tango augmented reality technology designed by Google and implemented in partnership with Lenovo and Qualcomm in the Phab 2 Pro.

Basically, Tango enables Motion Tracking, Area Learning and Depth Perception.

Motion Tracking makes the device understand its position and orientation relative to the environment. It uses using a combination of accelerometer, gyroscope and the fish-eye camera to gather real-time information about the 3D motion of the device.

Area learning is about retaining the memory of what the 3D environment looks like. This works by remembering key features of the scene such as corners and edges. With this information, Tango can improve the Motion Tracking in a way that is not unlike “navigating by stars” for sailors.

Depth Sensing is the ability to know how far things are from the camera at any given time. This is similar to what the Xbox’s Kinect does and relies on infra-red projection to do this in a range of ~0.5-4 yards. Depth Sensing can be combined with Motion Tracking to measure distances between points that are not in the same Z-plane (or frame).

“Into the Wild” virtual tour, technical information by MediaMonks

When it comes to Into the Wild, the augmented reality content is designed in 3D and powered by the Unity engine, which delivers a fluid virtual experience at 60fps in 1080p, which is a good user experience on a mid-range mobile SoC. The vast majority of PC gamers use the 1080p resolution, which represents 2.07 million pixels to move around for the GPU, because when they set the resolution on QHD (3.68 million pixels), the games are not fluid enough, even on PCs. According to Valve’s statistics, only 1.83% of gamers play games in QHD, and 0.22% in 4K.

According to MediaMonks creative agency, the most difficult challenge the team faced while developing the virtual tour, was to correctly map the museum into the virtual rainforest, matching the walls of the ArtScience Museum with trees, and corridors with forest’s paths (see photos above on how the virtual landscape matches the real world). The development team had to make countless trips to the museum to constantly re-adjust perfectly the virtual environment to the real one, the development started in October and involved more than 25 team members.

Qualcomm’s software implementation of Tango on the Snapdragon processor

While in Singapore for the launch event, I met Lauren Thorpe, Senior Director Global Business Development Ecosystem, Qualcomm, and she explained how the San Diego headquartered company implemented Google’s Tango baseline software on its Snapdragon System on Chip (SoC).

The implementation of Google Tango on Snapdragon 652 (and probably other Snapdragon chips as well, like the Snapdragon 835) doesn’t utilize the graphics processor (GPU) at all. “[on Snapdragon] nothing from Tango is running on the GPU, so we leave it completely available for the developers” said Lauren Thorpe.

Although some of the tasks could have been performed on a GPU, it does make sense to free up GPU resources for the actual App 3D graphics. It is also probable that Tango processing done with sub-unit such as the DSP and Neon co-processors are more power-efficient (perf/Watt). GPUs are incredibly powerful, but their massively parallel nature requires equally massive amounts of data to utilize fully. Qualcomm made the right call by distributing these specific tasks on its heterogeneous architecture.

Q&A with
Khoo Hung Chuan, Country General Manager, Malaysia and Singapore, Lenovo

1/ What was the goal in selecting the ArtScience Museum in Singapore to showcase the first Tango experience in Asia, on the commercially available Lenovo Phab 2 Pro?

Lenovo is very happy for the Phab 2 Pro to be the ‘magic window’ for the ArtScience Museum’s visitors. It is, after all, the world’s first Tango-enabled smartphone, and the working model for all Tango app developers. With every new project such as ‘In the Wild’, the developer community gains more experience at implementing immersive mixed-reality experiences, which is great for the AR/VR ecosystem as a whole.

2/ Will you develop more Tango-enabled smartphones with smaller form factors in the near future?

Lenovo is committed to staying at the forefront of Tango devices. As the AR/VR ecosystem grows and consumer needs become more varied, we believe there will be room for a variety of form factors to gain acceptance.

3/ Would you consider to develop a pro tablet with Tango? Is there a professional market for it?

Lenovo started with the mission of taking a concept technology that is Tango, to the mainstream, and this includes reaching both consumer and professional users. With the Phab 2 Pro, which was conceived as an everyday device, we had decided on a 6.4-inch phablet which offers an optimal viewing experience vis-à-vis portability. We will continue innovating to serve the needs of various customer segments, in form factors that make sense to them.

4/ Do you have other projects like “Into the Wild” to promote the Tango Experience on the Lenovo hardware to consumers?

All I can say is, stay tuned for more! From the world’s first Tango augmented reality museum experience in the Detroit Institute of Arts, to the Singapore ArtScience Museum’s latest permanent development ‘Into the Wild’, there is so much visual and informational value that can be added to the world’s existing places of interest, it’s staggering to think about and very exciting indeed.

Disclaimer: Lenovo organized a media tour for the official launch of “into the Wild” on the Lenovo Phab 2 Pro at the ArtScience museum in Singapore – travel expenses including flight, hotel and food were paid by Lenovo.

Eliane Fiolet
Categories: Cellphones
Tags: Augmented Reality (AR), Google, Lenovo, project tango