At L.E.A.P. Conference in October 2018, I presented our software roadmap and made a commitment to look into Social and UX interactions, improve our perceptions stack, and to enable mapping and XR technology at the platform level to herald the coming of the Magicverse.

Software roadmap presented at L.E.A.P. Conference in October 2018.

Six months have passed, and it is time to assess our progress. What did we deliver? What did we miss? And most importantly, what is coming next?

Since the launch of Magic Leap One, we have made four Over-The-Air (OTA) Lumin OS software updates - 0.92, 0.93, 0,94 and, just yesterday, 0.95.

Social and UX Interaction - Lumin OS 0.92, 0.93 and 0.94

With software update 0.93 (Nov 2018), we deployed the first part of our social and communication suites - Avatar Chat. The application allows for three way calls between friends and comes fully enabled with contacts, 3D emojis, and gesture interactions. Avatar Chat uses Lumin’s sharing and casting services that allow users to share various content and applications across devices (images, web pages, videos, the Gallery app, and casting-enabled Lumin applications) - either remotely or geo-localized.

Still in the application space, we have also greatly improved our Helio browser user experience by enabling key features that consumers have learned to expect - such as easy access to recently browsed sites, drag-and-drop interactions, text select, and scrolling. For developers, we also delivered the ability to call webviews from Lumin Runtime applications.

In terms of UX and key interaction, we have made some great strides in improving our general interaction model. We continued to improve our text input through improved Bluetooth keyboard integration and improvements to our Magic Leap Mobile App (MLA). We also added new mesh visualization graphics and feedback and improved our shutdown sequence. More importantly, we have greatly improved the robustness and accuracy of our 6Dof (6 Degrees of freedom) control solution and we started introducing 6Dof in landscape mode.

On the topic of perception, we kept iterating through the past three updates on our software stack - systematically improving on all our KPIs. A couple of features are worth highlighting: we have enabled prescription support through our initial setup and we have enabled a set of APIs for developers to get polygonal planes.

In terms of multimedia streaming, we delivered the first support of MPEG-DASH in our 0.94 update and continue to iterate on it.

During this period, we have also been working on our developer tool set. Magic Leap Remote has been enhanced with Metal Graphic APIs and improved Vulkan performance. We have added portal icon creation and validation tool to our Lumin editor. The overall user experience of our developer tools were also updated to make it more intuitive and consistent by having the same themes and control schemes across all our tools.

Those are only some of the highlights of versions 0.92, 0.93 and 0.94, but we also included countless amount of user, developer and partner feedback on quantities of subjects from general UX improvement to specific functionality such as sharing, capture, notification, OTAs and more, alongside many bug fixes and performance optimizations.

Enabling the Magicverse - Lumin OS 0.95 & Lumin SDK 0.20

That was the past, now for the present.

As of Friday, March 29, customers who purchased our Professional Developer Package have received early access to Lumin OS 0.95 software. Today through Friday we will start deployment of 0.95 through over-the-air updates. The release is already available for direct download from our Creator Portal as well. The staged deployment process allows us to better monitor the deployment of the release and to make any necessary adjustments.

Social & UX Interactions With this release, we have enabled the ability for up to four people to take part in Avatar Chat calls, as promised at L.E.A.P. Conference, we’ve introduced the ability for users to personalize their avatars. In terms of interaction, 6DoF has been now piped through all the main functionality of our OS and both users and developers can make 6DoF their default mode of interaction through Settings. We have also enabled open palm gesture support.

Lumin SDK and Developer Tools On the SDK and tools front, we now support animations with Lumin Runtime applications and we are making Lumin Animation Studio available to developers to create those. One more noteworthy item is that SDK 0.20 is supporting Unity 2019.1 which provides editor workflow and many performance improvements.

More importantly - and again as promised at L.E.A.P. - we are deploying two key technologies in 0.95 that will become essential building blocks of the Magicverse.

MagicScript The first one is about cross platform development. We are extremely happy to announce the first release of MagicScript. MagicScript will allow developers to build applications with JavaScript and Web API for Magic leap devices and it will also be central in building cross platform experiences for the Magicverse.

This first release of MagicScript offers a complete set of standard JavaScript API compliant with ES7+ specification provided by an industry grade JavaScript engine V8. This release of MagicScript includes Fetch API for accessing and manipulating parts of the HTTP pipeline, File System access API and a complete set of JavaScript API to create Landscape and Immersive applications with Lumin Runtime.

In the next releases, MagicScript will support WebGL API and a Component Library for rapid cross-platform application development. A first subset of WebGL is coming in 0.96 for Immersive. We will also gradually roll out a component Library - most likely starting with software update 0.96. Documentation & samples are available on our Creator Portal and at MagicScript.org.

Device Stream Still related to XR, 0.95 also includes Device Stream - which allows you to view your spatial computing experience directly on your mobile device running Magic Leap Mobile App (MLA). This constitutes the first deliverables for our spectator view modes and will continue to grow in terms of functionality to enable other sources of streaming, point of view, and even interactivity to unleash fully cross platform experiences.

Shared World The second set of technology - Shared World (Beta) - is related to mapping. For the best shared experiences in spatial computing or mixed reality, all users must have a common, 3D digital map of the world. This 3D digital map of the world is also the backbone of the Magicverse, enabling ubiquitous relocalization and persistency of content with pixel perfect registration wherever the user is. This map, that essentially digitizes reality, needs to be accurate, scalable, and constantly evolving - always improving in quality and adapting to the ever-changing nature of the environment (light, geometrical changes, etc).

This common, digital map of the world is built and updated by map merge algorithms. As users walk around the world with their spatial computing devices, they each generate environment mapping data. The data from all users who elect to use Shared World is ingested and produces a single digital map of the world. This common digital map continually updates and expands as new environment mapping data is received, ultimately resulting in a single 3D map that could cover an entire building and will evolve little-by-little to cover city blocks and entire towns. This spatial world map can be used by spatial computers and mixed or augmented reality devices from headsets to mobile phones, thereby enabling large scale, cross-platform mixed reality experiences.

Shared World (Beta) is being made available as an experimental feature in 0.95, and will be rolled out publicly in a future release. If you elect to use Shared World, each spatial map created from your device about a particular space will be saved and contributed to a collective map of that space stored in Magic Leap's cloud ecosystem. Shared World will enable various features including new ways to share content between devices, multi-user experiences, improved content persistence, and location-based experiences. For example, it will make the same PCFs (persistent coordinate frames), which are spatial anchors, accessible across different devices in the same space. The PCF API has been available through our SDK API since the launch of our product. You can learn more about what to expect from the Shared World (Beta) version here.

As you can see, we strived to deliver what we committed but there are also a few things that are yet to make it into our releases: we are still working on two controller support and iris detection login. I will keep you posted.

What is next?

Between now and the end of summer, two other software updates are scheduled. Those OTAs will include our first delivery in terms of enterprise features. We will continue to increase our large scale mapping capacity and take more steps towards the Magicverse. We will also continue iterating on our javascript and web frameworks with MagicScript and towards webXR and cross platform development.

Last, but not least, we will also deliver the first enablers to trigger international sales for our European developers. We know that we have many fans across the world and we can’t wait to get the device to each of you, wherever you reside. More details to come in future posts.

Before ending this post, I would like to take the opportunity to thank the handful of software developers and developer relations engineers who are building Lumin OS, day-in and day-out, at Magic Leap. You are creating magic and setting the bar for spatial computing, all in record time. You are true heroes.

Last but not least, I would like to thank our community of users and developers for all the feedback and input you have sent us - please keep it coming.

Yannick Pellet SVP of Software