Feature

There's augmented reality in space now

It's a very high-tech lifeline to Earth

Astronauts aboard the International Space Station already have humanity's most impressive view of Earth. But a new NASA project is enhancing it even more — through augmented reality.

The program, called Project Sidekick, uses the Microsoft HoloLens headset as a high-tech lifeline to Earth — one that will let technologists on the ground guide astronauts through repairs and experiments. Unlike virtual reality, which immerses users in a lifelike digital world, augmented reality adds to the real world by projecting computer-generated information or images onto what the user sees (think Google Glass). The space station's astronauts received their headsets last December but have only recently begun to test them.

NASA explains that Project Sidekick will help astronauts in two ways. First, it enables them to view an animated holographic illustration on top of the object in front of them — for example, a Sidekick-equipped astronaut could tackle repairs to the space station's notoriously finicky toilet by interacting with a holographic illustration of the toilet complete with specs and instructions from the people who designed it. In that way, the illustration serves as a kind of interactive user's manual that in the future could reduce training and help astronauts with their fixes even when engineers and techs are offline.

But a second mode is even more futuristic: Astronauts can let operators on the ground view exactly what they're seeing in space. The remote technologists then guide the astronauts through repairs and experiments in real time using a Skype uplink. Experts can even draw on and annotate the augmented "environment" astronauts see as they peer through the HoloLens.

"Augmented reality can allow an on-the-ground expert to see through the eyes of someone working in space," Columbia University computer science professor Steven Feiner told The Week. Feiner's Computer Graphics and User Interface Lab has designed and built experimental augmented reality systems for the last 25 years, and he and his colleagues know its uses go beyond mere visual fireworks: They have repeatedly demonstrated that people can reduce both the amount of time it takes to perform a task and their overall error rate when they are assisted by augmented reality units.

Because augmented reality lets people interact both with their environment and with experts and objects that aren't physically there, says Feiner, it gives users a crucial advantage. HoloLens-wearing astronauts can access the deep expertise of the technologists who designed and built instruments and experiments on the International Space Station — even when those engineers are hundreds of miles away. And in the future, Feiner adds, remote experts may even be able to add virtual 3D imagery to augmented reality environments that could explain things even better than the expert would in person.

"I'm somebody who believes in this technology and its potential," he says. But there could be a downside — since augmented reality environments are built by people, they're only as good as the information those humans can provide. Feiner says that it's incumbent on technologists and augmented reality users alike to make sure that people aren't put in technically challenging situations without the ability to problem-solve on their own. "We have to enable people to be resourceful when the technology doesn't work," he continues. "You don't put somebody out in the desert without telling them about how to use the stars to navigate…if the rest of their technology fails, you still want them to be able to know which way is north."

HoloLens augmented reality repairs may be the realm of astronauts for now — Microsoft released its first purchase invitations to select developers in late February — but that doesn't mean the rest of us can't follow along while the space station team tests out the technology. British astronaut Tim Peake seems to be a fan; he tweeted that the AR test was "going so well until we found the Alien Invaders game…." As for astronaut Scott Kelly, who tried the visionary goggles a few weeks before returning to Earth, he tweeted a single opinion: "Wow."

Recommended

Scientists claim ancient human relatives buried their dead, sparking evolution debate
skeleton fossil findings of the Homo Naledi Hominin species
is it true?

Scientists claim ancient human relatives buried their dead, sparking evolution debate

Why some workplaces still use old technology
A hand using a fax machine
Briefing

Why some workplaces still use old technology

The strange science of 'time reflections'
An illustration of purple waves to represent electromagnetism
Briefing

The strange science of 'time reflections'

Experimental drug shown to 'significantly' slow progression of brain cancer
Human brain scan.
Cancer breakthrough

Experimental drug shown to 'significantly' slow progression of brain cancer

Most Popular

Can Chris Christie make a comeback?
A black and white photo of Chris Christie waving
Profile

Can Chris Christie make a comeback?

Is Trump's wall working?
International Border Wall Between Tecate California and Tecate Mexico.
Briefing

Is Trump's wall working?

Orlando's Gay Days draws LGBTQ crowds to Disney World amid DeSantis feud
Gay Days at Walt Disney World in 2023
Come as You Are

Orlando's Gay Days draws LGBTQ crowds to Disney World amid DeSantis feud