This post is about augmented reality in the browser. In a recent post, we saw that it is now possible to have live video in webgl with WebRTC. Due to this, augmented reality is under the spotlight. Recently html5rock published a tutorial by ilmari heikkinen about ”writting augmented reality application using jsartoolkit”. Ilmari is google devrel for webgl and the author of JSARToolKit too. So we are in good hands :) The tutorial even include a part about binding it with three.js. I took this nice tutorial and packaged the code even easier to reuse.

Try this demo. If you got WebRTC is available in your browser, take this marker, and put it in front on your webcam. It is the same you see on the right. It is best to print it on paper. If you can’t, point your phone to this page instead. Phone screens tend to reflect lights from your environment tho. They may reduce the accuracy of the marker detection.

Now let’s see how you can to use all this to build your own demos :)

You Include it

First you include it as usual in your page. The code is available here. Note that JSARToolKit is released under GPL, so some restrictions apply. The threex is under MIT license as usual.

1 2 <script src= "JSARToolKit.js" ></script> <script src= "threex.jsartoolkit.js" ></script>

You Initialize It

Once you get the code, you instanciate the object like this.

1 2 3 4 5 var threexAR = new THREEx . JSARToolKit ({ srcElement : srcElement , threshold : threshold , callback : function ( event ){} });

The srcElement may be a video, an image or a canvas. When the video is shoot in a uncontrolled environement, the marker detection may be less reliable, due to variations of lighting. threshold is a value between 0 and 255 to adapt your detection to those variations. To detect the augmented reality markers in the srcElement , just use this line in your rendering loop.

1 threexAR . update ();

No rocket science here.

You Use It

During .update() , the callback is notified with events. They describe the markers present on the srcElement. Each event got various fields: A .markerId which tell you which marker has been recognized. The one used above is the 64. You can find the whole list in JSARToolKit repository.

The .type field describe what is happening to this marker. It may be create, update or delete. create if the marker has just been detected, update if the marker was present before. and delete if the marker is no more present. Rather obvious :) The .matrix field is a THREE.Matrix4. It is a transform which matches the position of the marker in space.

Conclusion

So now we can do augmented reality in a browser. The code of the demo is here. Up to us to find actual application of this technology. Currently most browsers with webcam are running on desktop/laptop tho. As most their webcam is directly attached to screens, It limits the flexibility of what you could put on front of the camera. As for mobile, only opera 12 is currently the only mobile browser able to read the webcam.

That’s all folks. Have fun :)