I live in Denmark for over a year now and I can tell you that it is not easy to learn Danish language. For everyday life I can use English since everybody here speaks it, but it would be also nice to learn Danish because I have plans to stay here in Denmark. Reading and understanding is kinda easy task, I managed to learn a lot of words so far and I can figure out meaning when I read text, but when it comes to pronunciation and listening - than I'm completely lost.

I had this idea of making augmented reality live translator app for some time now, so when all the puzzles fell into place I decided to give it a try to see am I able to achieve this. I spent some time developing app for iOS and yesterday I took it to work together with VR headset, so I could test it with my colleagues who are native Danish speakers. App is still in early alpha version, but I'm really happy with results since it is able to translate spoken sentences. There is a bit of lag since speech to text conversion and translation is done in cloud, some things are not translated perfectly, but for me it is a great start of this fun project.

I call this app BabelFishAR - Babel fish, as you probably know, is fictional fish from the book The Hitchhiker's Guide to the Galaxy, and book says when you put this fish in your ear you'll be able to understand all languages in universe - that could be really nice, but question still remains would you really stick some fish into your ear. My idea is that one day I create this app for the smart glasses, but since I don't have them at the moment I created it for VR headset for start.

How it works? App records the audio and get chunks of it when speech pause or stops, each of this chunks is sent to the cloud for speech to text recognition and afterwards to the other cloud service which translates recognised Danish text into English. Result is rendered on screen in split-screen mode (since we are using VR headset) and it looks like a subtitles.

Here is demo video of app and please have in mind that this is still an early alpha version:



As I wrote earlier and as you were able to see on video there is a bit of lag, text is not translated perfectly, but app is able to catch some sentences and display translation of them. If you try to Google translate some text in Danish you'll see that it also doesn't translate perfectly, so for me this is huge achievement and I'm really happy with result. In case you wonder why is my colleague Niels saying "Hello Thomas": Goran is not common name in Denmark and speech to text gets confused a bit with my name.

For end of this blog post I would like to say thanks a lot to my colleagues Niels Hanberg and Burhan Köse who helped me to test application and make this video and images.

Don't forget to like and/or share if you found this small project interesting, that keeps me motivated to work on it in my free time. Thanks!