Holograms are changing the way we interact with computers
Roula Khalaf, Editor of the FT, selects her favourite stories in this weekly newsletter.
Until recently, delivering hands-on training to aircraft mechanics meant bringing trainees into a hangar to see a plane under maintenance and removing the engine’s cover to allow them to look inside. At Japan Airlines, however, trainees using HoloLens, Microsoft’s “mixed reality” headset, can observe the different parts of an aircraft engine virtually.
Because the training can be conducted at any time from anywhere, it is more efficient and cost-effective, says Greg Sullivan, Microsoft’s director of communications. In addition, technicians can get up close to moving engines safely, with the ability to place their hands on virtual engines and parts. “Mechanics can be inside the digital engine while it’s running and can see how the system interacts,” he says.
Whether applied in an industrial setting, in gaming or by consumers — for example to see how a new kitchen might look in their home — use of holograms in augmented reality (AR) is being pursued by a growing number of companies.
Florida-based Magic Leap, for example, is developing a device for designers and developers that mixes digital graphics with the physical world, while Looking Glass, a start-up based in Brooklyn and Hong Kong, is developing HoloPlayer One, a portable device roughly the size of a laptop that displays interactive 3D images that can be manipulated with hand gestures.
“What we’ve been seeing with Magic Leap and Microsoft is this desire to figure out the next shift in how human beings interact with information,” says Shawn Frayne, co-founder and chief executive of Looking Glass.
Not all the AR devices being developed use holograms, in the technical sense — photographic recordings of a light field.
“Most of what you’re seeing are not actually holograms,” says Michael Bove, principal research scientist at the object-based media Group at MIT Media Lab. “Hologram has a specific meaning to people in the 3D display community, while to people in the marketing community, it means any three-dimensional image.”
Leaving aside such distinctions, the effect of 3D virtual images can be powerful, as demonstrated by the success of Nintendo’s Pokémon Go. The game uses smartphone cameras to superimpose digital imagery across players’ views of the real world, sending them to parks, streets and landmarks to find and capture virtual creatures.
Other forms of AR are based on similar principles. “It combines computer-generated graphics with what’s going on in the real world,” says Paul Merritt, a technology specialist, based in PA Consulting Group’s Cambridge Technology Centre. This is what distinguishes AR from virtual reality (VR). While users of VR wear goggles or headsets to become immersed in a 3D world, AR technologies place 3D virtual objects in real physical space and allow groups of people to interact with them, opening up the potential for collaboration.
Moreover, users of the technology can do more than view the virtual objects. They can also manipulate them, using hand gestures or voice commands. Many AR devices or systems are equipped with an array of cameras and sensors that enable them to understand the physical environment and the people in it.
“The real magic is the algorithms that fuse the data from all those inputs,” says Mr Sullivan.
Applications of AR range from industrial design to architecture. In medicine, it could increase accuracy, speeding up diagnosis and reducing error rates. It could help surgeons look inside a patient’s body, reducing the need for large incisions.
Prof Bove believes there is more work to be done in developing AR applications. “As of now, there are many fascinating demonstrations but no killer app,” he says. “Everyone is throwing everything into the space and seeing what sticks.”
Meanwhile, when it comes to devices, different approaches are emerging. While some are focusing on developing headsets, others are trying to avoid the use of wearable equipment.
At Looking Glass, this drives company’s approach. Its HoloPlayer One device displays interactive 3D images without the need for goggles or headsets.
“We have a philosophical belief that the real world should be prioritised over going into 3D digital space,” says Mr Frayne.
Regardless of the type of device used, the ability to manipulate virtual objects using voice or gesture will change the way people interact with technology, freeing them from the need to use screens, keyboards and mice. “If I can use gesture and touch, it’s not so much a computer any more, it’s a whole system,” says Mr Merritt.
For Mr Sullivan, the ability to interact with holograms in the same way as with other physical objects will bring people closer to their natural instincts for communication.
“Bringing computing into the three-dimensional world that humans have always existed in is the next step in making computing more personal,” he says.
50 ideas to change the world
We asked readers, researchers and FT journalists to submit ideas with the potential to change the world. A panel of judges selected the 50 ideas worth looking at in more detail. This third tranche of 30 ideas (listed below) is about new ways to handle information and education. The next 10 ideas, looking at advances in healthcare, will be published on March 5, 2018.
- Holograms
- Scanners to read our minds
- Quantum computing
- All the world’s data stored on DNA
- The next challenges for AI
- Rethinking our sleep schedule
- Personalised learning
- Neural networks allow us to read faces
- Learning to overcome digital distractions
- Robots in the classroom
Comments