How NASA Plans to Explore Mars With Microsoft's Holographic Goggles

By Kelsey Campbell-Dollaghan on at

During Microsoft's demo of its fascinating holographic headset HoloLens today, the company barely mentioned the coolest way it's already using it: to develop software with NASA's Jet Propulsion Laboratory that will let scientists explore and work on the Red Planet remotely.

Halfway through Microsoft's promotional video for HoloLens, the company showed a clip of a scientist wearing the headset as he points out an area of interest on the surface of Mars. HoloLens, the video explained, will be used by NASA scientists to work "on" Mars by July of this year. Intrigued by the brief mention, we reached out to NASA for more information, and in a new statement and video it explains a bit more about the project.

Over the past year, JPL and Microsoft have been building a software platform called OnSight that's designed to let scientists carry out their work through realistic holograms of the distant planet they're studying. Using HoloLens, of course.

Imagine being able to put on a headset and walk around another planet, or control a rover as if you were sitting in it, or look out over the yawning rim of a Martian canyon. That's exactly what OnSight does, using data and images taken from Mars itself.

According to JPL, it's the next best thing to actually being there—recreating the experience that earthbound geologists benefit from using augmented reality. That includes not only inspecting the landscape, but communicating with other scientists from around the world through this virtual meeting place—not to mention making notes and creating points of interest that other teams can explore and comment upon as they go.

How NASA Plans to Explore Mars With Microsoft's Holographic Goggles

"By building tools that make us feel more connected to those robots and the environments they're exploring, we can change our experience of exploration in a very fundamental and exciting way," says Jeff Norris, the project manager of OnSight, in a video published on YouTube today. "Not because it's a gimmick and not because it's fun, but because it will help them to reach scientific insight more quickly and more confidently than peering at images on the screen, like they do today."

They'll also be able to actually program the rover from within the HoloLens environment, JPL explains in a statement:

They then can stroll around the rocky surface or crouch down to examine rocky outcrops from different angles. The tool provides access to scientists and engineers looking to interact with Mars in a more natural, human way.

The OnSight tool also will be useful for planning rover operations. For example, scientists can program activities for many of the rover's science instruments by looking at a target and using gestures to select menu commands.

There's been plenty of clamouring about sending human scientists to Mars over the past few years. HoloLens doesn't completely obliterate the reasons we'd need to visit the red planet, but it definitely expands on the ways we can do it.

As rovers get more complex and sensitive, it might be possible to explore using these virtual reality-augmented proxies—much more durable envoys in our ever-expanding reach into space.