How Developers Built Winning Apps at First Magic Leap Hackathon

by Michael Pacholec    01.09.2019 09:16 PM

Magic Leap Hackathon winning team LifeScope

I had an opportunity to speak with Liam Broza, team lead for LifeScope, the winner of our AT&T Mixed Reality Hackathon with Magic Leap on November 9-11 downtown San Francisco.  Liam shared his experiences participating at the hackathon and developing for Magic Leap.

Liam is CEO & Co-Founder at LifeScope & BitScoop Labs and is a graduate of Rochester Institute of Technology who currently lives in Los Angeles.  LifeScope is the project brought to the hackathon and was incubated inside BitScoop Labs.

The Project

LifeScope has been a dream project of his to build an open source digital twin – similar to Aya and Mica in Magic Leap.  To capture a map of everywhere you have been, everything you have done, and be able to revisit the past – like an H.G. Wells time machine.

Over the weekend, Liam’s team brought the LifeScope concept to AR using the Magic Leap One.  The theme was, “a day in the life”, of someone visiting downtown SF.  His team shot 360 video, and then placed the video on a map of downtown SF using LifeScope.   If you dial in a time, it would take you to where you are at that moment.  Like a surrogate digital memory.

The Team

Liam himself has worked in AR for about 10 years, starting with low level eye tracking and marker AR projects. Originally using OpenGL and DirectX in college and continued as hobby.  The LifeScope team decided to start WebVR development in April and made its first WebXR prototype in July. Recently, the LifeScope team has been working on expanding into mapping, photogrammetry, and computer vision in AR.

His team for the hackathon included WebXR and Unity experts, 3D modelers, and professional 360 video creators.  Half the team Liam has worked with previously.  The other half were friends recruited for the hackathon in Los Angeles.   The magic of LA is the number of people familiar with VR and AR skills.  Only four team members were development centric.  Others on the team were content creators and immersive artists.

The Technologies

Unity – “The world’s leading real-time engine.  Unity is used to create half of the world’s games.”

They love using Unity for XR prototyping, asset creation and creating content across platforms. We Connect Unity to the LifeScope backend GraphQL API using the BitScoop C# SDK so we can power the time machine with user data. They are currently working on methods to use Social Sign On (OAuth 2) with LifeScope inside the Magic Leap and other devices.

Magic Leap Lumin SDK – “Lumin Runtime provides a foundational set of APIs and a UI toolkit for delivering high fidelity video, graphics, and audio.”

Lumin SDK features he would like to include in the next build include expanded multiplayer, 3D scanning, photogrammetry, object recognition and facial recognition.

 

LifeScope side viewLifeScope overhead view

Exokit – “Exokit enables developers to build XR experiences using the same code that runs on the web.”

They use Exokit to bring our WebGL, WebVR and WebXR content to the Magic Leap. Using Networked Aframe, they were able to get a 36 person multiplayer environment working with the Magic Leap, VR headsets, and Mobile/Desktop browsers. They have been extremely impressed with Exokit and the fast development of the Open Source XR community.

GraphQL – “GraphQL is a query language for APIs and a runtime for fulfilling those queries with your existing data.”

LifeScope is a personal database and digital passport built for the Metaverse. It is a smart distributed database with a GraphQL API and App interface. The app allows anyone to explore people, places, physical things, digital content, and events in their life. The GraphQL API allows anyone to build apps and machine learning on this data. We even have SDKs and GraphQL Playground for Developers.

Blender – “Blender is the free and open source 3D creation suite.”

They absolutely love Blender even though its interface is complete madness.

BitScoop – “Easily tackle API integrations and get back to delivering awesome products.”

BitScoop is the underlying integration technology in LifeScope allowing us to connect to data across many services. They use the BitScoop API in Unity through the OpenAPI C# SDK and the API in WebXR using the JavaScript SDK to talk to the LifeScope Server and power OAuth Social Sign On.

LifeScope diagram
Other technologies include AWS, Adobe Premiere, GitHub and G Suite.

Advice for Hackathon Attendees

Developers looking to immerse in AR should consider gaining experience with 360 video and emerging hardware.  Liam recommended the Kandao QooCam as an example of consumer tools useful for experimenting in 360, stereo 180.

Study up and coming technologies such as volumetric video and photogrammetry.

Rapidly come up with ideas and weigh them by level of effort, risk, and ROI. Then vote ideas out to hone in on an MVP (minimum viable product) that will help your focus for the weekend.

Asset stores are your friend. Create a public web server to host 2D and 3D media. This lets everyone experiment quickly.

The more team members the better, but you need to ensure nobody is idle.  Soft skills around communication and relationships are important.

With a product like the Magic Leap One, which must map your surroundings, a “bright white clean” area like a hackathon venue may not contain enough furniture or other fixtures in the space for good tracking – especially when it comes time to demo.

About Magic Leap

Liam had no regrets about buying the Magic Leap One.  He intends to bring LifeScope to the Magic Leap store ASAP.

“Was very quick to port over.  Our content will demo better on the ML than any other hardware.  We have been building for WebXR and Unity, because we want to be cross platform.  But the Magic Leap is the most immersive and visually stunning of any platform.”

Liam did find a need to stitch together some USB extension cables and power cables with velcro ties.  He essentially built a 40 foot “umbilical cord” for the hardware to keep it powered during development since multiplayer required a large amount of walking around.

https://lifescope.io/mlvideo

Thank you to Liam for taking the time to speak with us!


Share this post

Comments
by Grimlock on 01.31.2019 10:45 PM

Love the “They absolutely love Blender even though its interface is complete madness.” part. 🙂 so true.
But hey, you can’t beat the price.

Leave a Comment:
Sign in is required to comment on this blog. Not a member? Join now.