Amazon’s AR View is one of the most popular and useful example of AR today. The ability to place products in the real world gives customers a tangible benefit when making purchase decisions. Other companies such as Ikea, Lowes and Magnolia have built similar features for their mobile applications.

Whether you are building a standalone app or adding AR as a feature to your existing app, augmented reality can provide a differentiated experience for your customers.

This guide will show you how to build Amazon’s AR View for ARCore on Android using ViroCore. ViroCore is SceneKit for Android, a 3D framework for developers to build immersive applications using Java. ViroCore combines a high-performance rendering engine with a descriptive API for creating 3D/AR/VR apps. While lower-level APIs like OpenGL require you to learn and precisely implement rendering algorithms, ViroCore requires only high-level scene descriptions and the events and interactions you desire.

Our goal by the end of this tutorial is to:

Create a simple AR Scene and add objects into the scene at real world scale. Show how to integrate 2D UI (Android Views) with AR Scenes.

** If this is your first time as a ViroCore developer, use our Getting Started guide to get setup with ViroCore before continuing with this tutorial. ViroCore is open source and easy to set up! **

PREREQUISITES

Demo Project Overview

The ViroCore AR Retail demo is a simple Android workspace project that you should be able to open easily with Android Studio. Within it, there are 3 main activities:

ProductSelectionActivity — Users select the desired furniture to look at in this Android Gallery / ListView. ProductARActivity — Users use ViroCore AR to place the 3D model of the selected furniture in AR. ProductDetailActivity — User can view detailed information of the selected furniture in a WebView ( For example Amazon.com).

By default, the ProductARActivity.java file, with starting code, is launched when a user clicks a product in the selection activity. The full implementation of the this Activity experience (and more), can be seen in Github, along with the rest of our AR sample code! You can refer to this file, if needed, as you follow the tutorial below.

Lastly, all model data items can be seen under the model directory. Furniture data is mocked in a Product DB that contains a list of mocked product objects, stored in the application layer, shared across activities. As such, the name of a selected product object is passed into the AR Activity through the intent bundle. The product is retried from the DB and stored as mSelectedProduct. In this activity, we will be displaying that in AR.

Creating a Simple AR Scene

To get started with AR, our ProductARActivity extends from ViroActivity, of which contains all the required setup goodness for the Viro Renderer to function properly. When the renderer is ready (onRenderStart), we can then create an AR Scene with all our 3D furnitures. An ARScene acts like a view container, and displays the passthrough Camera on the background, hence, rendering virtual objects in “AR”.

First, let’s start with creating very simple ARScene with an ambient light:

We placed 3 placeholder init functions for creating our AR UI. In the next few sections, we will be going through these 3 functions to essentially:

Pull in your 3D model furniture product into the scene. Create an AR Cursor used for finding surface to place your furniture. Adding an Android View HUD to enhance the AR shopping experience.

Importing your 3D Furniture

Once we have created our scene, we can start placing objects in our scene within the init3DModelProduct() that we have already constructed before. Loading 3D Models with Object3D into Viro is pretty simple:

As shown above, you loadModel simply needs a URI to where your 3D model is stored, the format type of the 3D model, and a listener callback to be notified of errors, if any.

As part of the download, we included a lamp and table assets. These are 3D FBX models that were converted into VRX format to be used with Viro. (Viro support OBJ and FBX formats with other file formats coming soon.) You can read more about 3D Objects and the different formats in our documentation guide -> 3D Objects.

Great! Now, to make our furniture look realistic, we’ll also need to add shadows into our scene, thereby helping the object blend and ‘fit into’ the environment. This can be done by having a base parent node, that contains our 3D model, a surface, and a spot light:

The spotlight effectively shines and projects a shadow off the 3D object onto the surface underneath. Huzzah!

Lamp Shade

After you’ve verified that your furniture is pulled in correctly, let’s hide it by setting mProductModelGroup.setOpacity(0). You can add this at line 39 in the code sample above. In the next section, we’ll focus on making the furniture appear, after the user has decided where to place it with the ARCrosshair cursor.

Placing your Furniture in AR

Customers will be looking around their room through the lens of our AR Application (remember the ARScene has a Camera Texture background). As they look around, an AR Cross hair follows their gaze, and is placed on any flat surface that our application has found. This indicates that furniture may be placed on this surface:

When they tap on the AR Cross hair above, they are essentially placing the furniture at that position.

Thankfully, with deep ARCore-ViroCore integration, we can find feature points in the direction the customer is gazing at: by using the setCameraARHitTestListener API. As the document states:

When an ARHitTestListener is registered, it will be continually notified of hit tests from the camera’s forward vector into the AR environment. This is an efficient way to be notified of real-world objects intersecting the center of the user’s AR screen.

Thus, we can use this listener to be notified of intersection hit points of real-world objects the user is gazing at from the center of his screen, and to place the AR Cross hair at that point. Here’s a quick example of how we implement such a listener, and then register it to our mViroView:

You might be wondering what mCrosshairModel is — this is the fbx AR Crosshair cursor model that represents the user’s gaze. As seen from above, we effectively positioned this model at the hit position of whatever the user is looking at.

Registering the ARHitTest listener for the cross hair above, can only be done when the ARScene has initialized tracking components. As such, we’ll need to register this listener within the onTrackingInitialized callback that is attached to the scene, as shown below.

Next, loading up the AR Crosshair 3D model pretty much the same as creating the product model — we use the Object3D object and finally code the initARCrossHair function:

Huzzah! You should now see an ARCrosshair Cursor follow your every gaze! Finally, when the user decides to tap the crosshair, we can then at last have the furniture placed at the crosshair’s location:

And that’s it! You should now have the basic mechanics of a gazing, and placing the 3D furniture in a desired location.

**In our completed ProductARActivity.java file, we have also introduced additional code for dragging and rotating the furniture model. So definitely check those out, if you are feeling up for a challenge!**

Adding an AR HUD Element

In addition to our 3D objects, it is very easy to integrate existing Android views / controls with ViroCore. Since the ViroRenderer itself is an Android View, you can effectively inflate any other types of Android along side it, thereby creating your FrameLayout with Android-Hud like controls on top of the renderer.

In this demo, we’ll be inflating a simple Android main_hud_layout on top of our AR experience. This layout will contain two Image Views that acts as buttons. We inflate our layout programmatically in Android as usual:

To bind directly to the Android UI Views in our main_hud_layout, we can do so with the initARHud function that we had added in the beginning.

One ImageButton is a “Cross”, so that the developer can exit out of the ARProductActivity. And the second image button is a “details” button, of which pushes the user into a third webview activity with additional details of the product in question.

Now when you start the app, you should see two buttons on the top left and right side of the screen. Tapping on the “X” should take you back, to the screen selection, and tapping on the details button on the top right should take you into the details activity with a webview page with more product details.

That’s it! You now have your own basic Amazon AR View!

Next Steps

In the completed ProductARActivity.java file, there are additional features not covered in this tutorial which will help you make a more robust application:

Additional Android TextView texts for user instructions.

Code for dragging and rotating the furniture models.

Screen capture of the AR Scene and sharing.

So definitely check those out, if you are feeling up for a challenge! Some other things to try as you keep building your scene:

Add more 3D Object — Either create your own or download some from the web. Here are some resources:

Viro Free 3D Assets

SketchFab — just launched an asset store for 3D models

Google Poly — another resource for 3D assets

Fun with Physics — using the ViroCore Physics Engine to add polish and realness to your AR experience. You can have furniture drop from the sky and land with a small bounce or add friction to objects as the user slides them across planes.