How to control a robot with OpenSpace3D

In this tutorial I will show you how to build an interactive Augmented Reality (AR) app using OpenSpace3D and ARToolKit. ARToolKit was included into the latest release of OpenSpace3D.

Components for this tutorial

OpenSpace3D (Windows only) - I used the portable version which is available here -

This images which will be used for tracking

And as usual for an AR app you need a camera

Setup the model

Start OpenSpace3D and load the supplied robot model:

Start OpenSpace3D Add the robot model (Andy) to the scene. Right click on the scene root

Add mesh -> Next

Select the robot mesh: "OS3D_root"\OpenSpace3D\assets\models\common\andy\mesh\andyv2_andy.mesh -> Next

Select Add script and select "OS3D_root"\OpenSpace3D\assets\models\common\andy\material\andyv2.material -> Open -> Import

Now you have Andy in! If you select it in the Scene tree you get a little '+' icon next to the model with which you can explore the model and all it's animations.

You can play the available animations by right click on one of then and select Edit setting. In the following dialog you can see a play icon in the lower left corner. That lets you play the selected animation. You can also select to loop that animation if needed.

For better control over the model we need to add a dummy node:

Add a separate root node to the scene for the Andy model. Right click on the Sceen root

Select Add dummy

Name the dummy ARAndy_dummy Drag the andy model onto the dummy

Activate the material

Good, now we have the basic scene set-up. But, our model is not looking very good. We need to enable the material to make it look good.

Right click the Scene root Set material scheme In the dialog select basic_mat

Now Andy should appear with a white and blue material with the OpenSpace3D logo on his chest.

Light and shadow

The scene is still a bit dark and also it looks much more realistic if we enable shadows. So let's just do that.

Right click on ARAndy_dummy Add light -> OK In the dialog ensure you select Cast shadow Also make your light a direct light

Now we add a plane to which the shadow is projected onto and enable the shadow mode.

Right click on ARAndy_dummy Add Mesh -> Next Select "OS3D_root"\OpenSpace3D\assets\primitives\prim_plane.mash -> Next -> Import Right click on Scene root / Add resources to scene Select "OS3D_root"\OpenSpace3D\assets\materials\transshadow.material -> Open Right click the plane you just added -> Edit materials For material name select ASSET/TransShadow Right click on Scene root -> Set shadow method For Preset method select presetPSSM

That's good Andy now produces a shadow onto the plane which makes the whole scene much more realistic. But uhm, Andy is somehow cut in half by the plane. We need to move him up along the y-axis 0.9 units.

Also the light is still placed at the centre of the scene. Move and rotate it till you are satisfied with the light and the corresponding shadow on the plane.

Augment it

If we run the scene now you can see that Andy is doing some work and that it has a shadow.

You can run the scene using the little play icon at the top left corner of the scene view.

Add the AR camera

Now we want Andy to be augmented onto the robot image.

At the bottom inside the groups tab double click Scene In the scene tab right click/input/AR capture -> OK

You can configure the settings to your liking. For now the default settings will do.

Add the marker

If you play the scene now you see that Andy is rendered in front of the camera stream.

Now we need to place Andy on a marker.

Right click in the scene tab at the bottom Input / Add marker For Bitmap path select the image of the robot. This starts the marker training which may take a bit. Marker size needs to be changed to 1 or bigger otherwise Andy will be really large. Play around with this setting till Andy appears to your liking. (I set it to 2) In Object name select the ARAndy-dummy node.

Alright, now Andy is placed on top of the marker when the marker is visible to the camera. However, it is also rendered when the marker is not shown which is kind of disturbing. So let's fix that.

Right click in the scene tab at the bottom Select object/hide Set Source name to ARAndy_dummy Select Hide on start

Now Andy is no longer rendered when we play the scene. To make Andy appear do the following:

Right click AR marker inst Select found Now a line is shown starting at AR marker inst and following your mouse pointer Right click on hide inst and select show

This makes Andy appear as soon as the marker is detected. Do the same for Lost to hide Andy when the marker is not visible any more.

OpenSpace3D has a video illustrating the above steps which can be viewed here.

Control Andy

Now that you can show and hide Andy based on the robot marker the next step is to show him what to do based on the other markers.

Add three more markers for girlbot, beerbrake and computer. Right click in the scene tab at the bottom Select object/animation switcher Add the animation you would like to be played for each of the marker (girlbot, beerbrake, computer) to the Animations list. Right click on one of the new markers. Select found, right click on the animation switcher and select the animation you would like to play when the selected marker is found. Be aware that one animation is set to default so you won't see the name of this animation in the list. Repeat this step for all of your markers Don't forget to select some animation (or non) when the marker isn't visible any more (lost)

For each chosen animation it might be good to loop it so that the animation still plays while the marker is shown to Andy.

Conclusion

You have learned how you can setup an augmented reality app using OpenSpace 3D. Additionally this tutorial gives you some background how you can use markers to control 3D objects like our Andy and tell a nice story with it. Try to think this example further and make real objects react on images shown to them (or to a camera). Maybe using a Raspberry PI or an Arduino.

I'd love to hear from you and see your stories and apps. Let's keep in contact on Twitter @Thor_Bux or @ARToolKit_. Also for questions and feedback please write to the ARToolKit forum.