Getting Started with Mixed Reality & Why You Should Care
Author: Ed Schmit, AVP Product Marketing Management, AT&T Developer Program
Ed tracks new technologies for the AT&T Developer Program. His specialties include network technologies, technology enablement, and strategic marketing.
Are you interested in replacing your reality with a mixed reality?
Creating new virtual experiences is becoming more popular and useful.
Maybe it’s time to learn the skills and tools to give you real experience with mixed, augmented and virtual realities?
We wrote this post for you.
Inside this article:
What is Mixed Reality?
It’s what someone experiences when blending the physical world with the digital world.
Where real life experiences meet imagined ones.
Mixed reality combines interactions between humans, computers, and environments. Rather than just imagine in your head, you can experience things with your eyes and ears (who knows, maybe more senses are just ahead?).
All this is made possible with high powered computing- for vision, graphics, and display technologies.
The term mixed reality was originally introduced by Paul Milgram and Fumio Kishino in 1994. Their paper described the virtuality continuum concept, which defines the polar ends of the spectrum (for simplicity, let’s call this the ‘mixed reality spectrum’).
On the left- we have raw reality, where we humans exist.
On the right- 100% virtual reality (VR). It’s all digital.
Between the two- lies a mixture of real + virtual. This is known as augmented reality and augmented virtuality.
Augmented Reality (AR)
AR expands the physical world (reality) by adding layers of digital information onto it.
Unlike VR, AR does not create an entire artificial environment to replace the real with the virtual. More likely, it overlaps an existing environment with sounds and images.
Augmented Virtuality (AV)
AV overlays the digital world with real world elements. Like touching and interacting with real physical objects while looking through ‘smart glasses.’
Here’s the thing.
To differentiate between AR and AV, think about where the user is taking action. If it’s in the real world, it’s AR. If it’s in a virtual space, it’s AV.
In other words, AR occurs in the real world. AV occurs in a virtual environment.
A game played in a park with virtual characters appearing alongside the live view of the park.
An aircraft engineer viewing a real time model of an engine in flight on a computer screen, where real world elements are thousands of miles away.
Why Should I Care About Mixed Reality?
Well… one reason is because many others do.
The global mixed reality market was valued at $123.2 million in 2017, and is projected to reach at $5,362.1 million by 2024, growing at a CAGR of 71.6% from 2018 to 2024.
With Mixed Reality, companies can:
- Market products in innovative ways by using MR to create immersive content for audiences
- Improve brand image by offering unique experiences
- Increase sales with new and entertaining ways to solve customers’ problems
- Be seen as an innovator when using these tools to create new solutions
- Stand out from competitors by offering exciting, unique and innovative MR solutions
- Reach a younger audience that’s more up to date with technology
There’s more, I’m sure. But you get the idea.
And you want to get in on the MR fun, right? Developing and keeping skills in MR will make you more valued, too.
Which leads us to…
Who’s Adopting MR?
And how are they using it?
Let’s look at a few.
Every aspect of entertainment is embracing mixed reality- impacting players, fans and sponsors, with gaming being a leader in the space. Others include:
Stereoscopic music allows you, the listener, to be immersed, smack-dab-and-center on stage with a band or down in an orchestra pit. You hear music from all sides combined with vivid 3D imagery that takes the shape of the music. And, you can thump to the beat along with others in their living space thousands of miles away. That’s what MicrodoseVR does. Wow, huh?
Sports fans should keep their eye on the startups that are gaining traction globally. Take NextVR, which offers a virtual reality platform for delivering live sports and music in VR to fans globally. NextVR has partnered with FOX Sports, the NBA, the NHL and others to give fans immersive experiences while watching a game, match or news.
Search is still mostly 2D. Load and run your map app, type stuff in, see results on a flat screen.
But know Google and Bing have bigger ideas.
Imagine finding jeans in a mall, while being escorted by a popular celebrity, of your choice!
These companies are 3D-mapping every square centimeter of the planet. So that we’ll use smartphones less and smart glasses more. Search will become more visual while language barriers disappear. For instance, The Wall Street Journal recently took a look at the AR future of Google Maps.
Google introduced Expeditions back in 2015, a MR platform built for learning.
Teachers can go beyond the space of the classroom. VR lets you explore the world virtually while AR brings the abstract to life. Teachers can guide students through collections of 360° scenes and 3D objects of interesting things out in the world.
As an example, medical school students at Washington State University (WSU) are using virtual reality to study anatomy and physiology.
Virtual reality therapy can be an effective method for treating anxiety disorders and related problems.
For example, do you get anxious when flying?
The University of Louisville physicians might be able to help. They’re helping patients face their fears using VR to create a controlled environment. Patients can practice coping strategies in-flight, virtually. Put on the glasses, buckle up, and take off. Then, stop everything, talk about it, rewind and try again. Practice feeling good with your feet safely on the ground, but your head in the air.
Let’s rapid fire through a few more industries using or considering mixed reality.
Using 360-degree videos to tell journalistic stories. Imagine standing in the middle of a destruction zone to feel more of what a crisis is really like.
Not sure where to go on your next trip? Take a virtual one to help you decide. Destination, hotels, car rentals: experience it virtually first, then book it.
Same kind of thing. Put on some 3D glasses, walk the floor plan, sit on the deck, reach across counters. All to help you decide if it’s even worth driving over to see the real thing.
Smart helmets on factory floors to help workers perform welding on virtual materials before working on real ones.
Where Do I Start?
Start with these.
Create AR and VR games across most platforms and devices. They claim to power 60% of the world’s AR/VR content. Seems like a good place to start.
AR Foundation is their base technology for AR creators. This ecosystem allows you to create an AR app once, yet have it run on all platforms, without losing any platform-specific features. It’s currently for handheld devices, but will soon extend to wearable AR.
Here’s a post on their site to learn more of the features, which packages to consider, and how to get started.
Soon, Unity will unleash Project MARS, their Mixed and Augmented Reality Studio. This will give creators the power to build apps that intelligently interact with any real-world environment. With little-to-no custom coding.
Unreal Engine is a suite of tools for creating AR and VR experiences. Designed for individuals or an entire team. Create games, film, and other photo realizations.
Use the Unreal Engine right out of the box, with no need for additional plugins or purchases. Everything included to build and ship.
Don’t code? Don’t worry. Use their advanced motion controls to build from within a “what-you-see-is-what-you-get” environment. For PCs, console and mobile platforms.
Now, go make something unreal.
CRYENGINE plans to be known as the most powerful game engine in the industry. No matter what your budget or team size is. Quite a goal.
CRYENGINE strives to make your learning curve slight, not steep. They offer: full source code, tutorials, detailed documentation and a strong development community. And, their marketplace provides a wide range of assets, ready to use in your projects.
Use their editor to create stunning visuals, vivid characters, and awesome animations for all platforms. And of course, add just the right audio effects. See all the features here.
Use Amazon Sumerian to create and run VR, AR and 3D applications quickly and easily, without requiring any specialized programming or 3D graphics expertise.
Lifelike characters, immersive environments, and build once while running on all the popular platforms (Oculus Go, Oculus Rift, HTC Vive, HTC Vive Pro, Google Daydream, and Lenovo Mirage. And, Android and iOS mobile devices). With no VR or AR expertise required.
They lay out a simple process: create a scene, upload assets, add a host, edit, then publish.
Here’s a sample scene (even better if you have a head-mounted display and browser supporting WebVR).
A-Frame is a framework for building VR experiences on WebVR, using HTML.
That’s right. Start building VR in HTML, without installing anything.
Don’t believe me? Fine. See for yourself here. Hit the show and view source buttons to see how fast and easy it is to get started.
Believe me now?
A-Frame supports most VR headsets such as Vive, Rift, Windows Mixed Reality, Daydream, GearVR, Cardboard, Oculus Go. And, can even be used for augmented reality.
A-Frame makes VR simple:
- Use what you already know, HTML. Declarative HTML makes it easy to read, easy to understand, easy to copy and paste. With everything accessible to everyone: web developers, VR enthusiasts, artists, designers, educators, makers, and kids, too.
- Cross-Platform VR. Build VR applications for Vive, Rift, Windows Mixed Reality, Daydream, GearVR, and Cardboard. With support for all respective controllers. Don’t have a headset or controllers? No problem! A-Frame still works on standard desktop and smartphones. Cool, huh?
- It’s visual. A-Frame provides a handy built-in visual 3D inspector. Open up any A-Frame scene to fly around to peek under the hood.
See the rest of the features here. While you’re there, note all the support for learning, creating and sharing. With blogs, documentation, coding playgrounds, tutorials and forums.
Some ideas for you to get up to speed:
- Browse the site links above to get familiar with those tools
- Subscribe to a few newsletters to learn more. Here. And here. Here, too.
- Get started with Unity. Pick a course that interests you.
- Same for Unreal Engine. Lots of tutorials for you there.
- Here are some courses, not related to any particular tool
- With even more here, many of them self-paced
- And of course, let’s not forget about com
- Check out some of these popular MR headsets for Windows
- Register to attend the next AT&T SHAPE event, where you can learn about MR, try it out in exhibits, and network with others in the industry.
So there you have it – info about mixed reality- what it is, where and how it’s being used, some of the top tools, and how to start your MR journey. We can’t wait to see what kinds of MR experiences you’ll create!