April 24, 2013

Disclaimer: I do not have access to the Google Glass Explorer Program. The code in this blog post is purely speculative, it is not production ready.

Google has recently made the developer documentation for it’s Google Glass product publicly available. Named the Mirror API, it allows developers to create applications called Glassware that interacts with user’s Glass devices via a RESTful interface. Developers authenticate to the API on the users behalf using OAuth v2. Once authorized, Glassware applications can publish timeline notifications to the user’s Glass devices and subscribe to receive notifications of actions preformed on the devices.

Authorization

Since Glassware applications do not get installed directly onto Glass devices the first step to interacting with a user’s device is the authorization process. This web based process has the user accessing your site and authenticating with their Google account by way of OAuth v2.

require 'google/api_client' require 'sinatra' CLIENT_ID = '<YOUR_CLIENT_ID>' CLIENT_SECRET = '<YOUR_CLIENT_SECRET>' SCOPES = [ 'https://www.googleapis.com/auth/glass.timeline' , 'https://www.googleapis.com/auth/userinfo.profile' , ] enable :sessions def api_client ; settings . api_client ; end def mirror_api ; settings . mirror_api ; end def oauth2_api ; settings . oauth2_api ; end def store_credentials ( user_id , credentials ) raise NotImplementedError , 'store_credentials not implemented' end def get_stored_credentials ( user_id ) raise NotImplementedError , 'get_stored_credentials not implemented' end def get_user_info result = api_client . execute ! ( api_method : oauth2_api . userinfo . get , authorization : user_credentials ) user_info = nil if result . status == 200 user_info = result . data else puts "An error occurred: #{ result . data [ 'error' ] [ 'message' ] } " end user_info end def bootstrap_user end def user_credentials @authorization || = ( auth = api_client . authorization . dup auth . redirect_uri = to ( '/oauth2callback' ) auth . update_token ! ( get_stored_credentials ( session [ :user_id ] ) ) auth ) end configure do client = Google : : APIClient . new client . authorization . client_id = CLIENT_ID client . authorization . client_secret = CLIENT_SECRET client . authorization . scope = SCOPES mirror = client . discovered_api ( 'mirror' , 'v1' ) oauth2 = client . discovered_api ( 'oauth2' , 'v2' ) set :api_client , client set :mirror_api , mirror set :oauth2_api , oauth2 end before do unless user_credentials . access_token || request . path_info = ~ /^\/oauth2/ redirect to ( '/oauth2authorize' ) end end get '/oauth2authorize' do redirect user_credentials . authorization_uri . to_s , 303 end get '/oauth2callback' do user_credentials . code = params [ :code ] if params [ :code ] user_credentials . fetch_access_token ! user_info = get_user_info session [ :user_id ] = user_info . id store_credentials user_info . id , user_credentials bootstrap_user redirect to ( '/' ) end get '/' do "<h1>Welcome to my Glassware</h1>" end

The above Sinatra application authenticates the user and stores their authentication for later use. Storing the user’s authentication isn’t required but it could prove important as most of the time you will be pushing timeline updates to a user’s device when they are away from their computer.

Timeline Cards

Timeline cards allow developers to present information to their users. As timeline cards are added through the API they are presented as a queue the that the user can read through as they wish. You may have notice the # TODO comment in the above example, lets use that method to push a timeline card that welcomes the user to our Glassware application.

def insert_timeline_item ( timeline_item , opts ) if opts [ :filename ] media = Google : : APIClient : : UploadIO . new ( opts [ :filename ] , opts [ :content_type ] ) result = api_client . execute ( api_method : mirror_api . timeline . insert , body_object : timeline_item , media : media , parameters : { 'uploadType' = > 'multipart' , 'alt' = > 'json' } authorization : user_credentials ) else result = api_client . execute ( api_method : mirror_api . timeline . insert , body_object : timeline_item , authorization : user_credentials ) end if result . success ? result . data else puts "An error occurred: #{ result . data [ 'error' ] [ 'message' ] } " end end def bootstrap_user text = "Welcome to JasonInCode's Mirror API for Rubyist" timeline_item = mirror_api . timeline . insert . request_schema . new ( { 'text' = > text } ) insert_timeline_item ( timeline_item ) end

Now when a user authenticates to the application they receive a timeline card welcoming them. You may have noticed that the insert_timeline_item method has a set of optional parameters. This implementation of insert_timeline_item allows the developer to upload images and other media to a user’s timeline.

get '/send_image' do text = "Sample image upload" timeline_item = mirror_api . timeline . insert . request_schema . new ( { 'text' = > text } ) insert_timeline_item ( timeline_item , { filename : 'sample.png' , content_type : 'image/png' } ) end

In addition to adding images, audio, and video a developer can also upload a list of actions the user can perform on a timeline card. There are a number of system provided menu items built into Glass devices:

REPLY and REPLY_ALL : Initiates a reply to the timeline item using the voice recording UI.

and : Initiates a reply to the timeline item using the voice recording UI. DELETE : Deletes the timeline item.

: Deletes the timeline item. SHARE : Allows the Glass user to share the timeline item with their contacts.

: Allows the Glass user to share the timeline item with their contacts. READ_ALOUD : Tells the device to read the item’s speakableText to the user, if that is not set it will read the item’s text instead.

: Tells the device to read the item’s to the user, if that is not set it will read the item’s instead. VOICE_CALL : Calls the phone number in the item’s creator.phone_number .

: Calls the phone number in the item’s . NAVIGATE : Starts navigation to the item’s location .

: Starts navigation to the item’s . TOGGLED_PINNED : Toggles the item’s isPinned state.

Developers can also provide custom menu items that are specific to the individual Glassware application. The custom menu items can have custom display text and an icon image. Each custom menu item has an id that is returned to the application via the application’s registered subscription URL.

get '/item-with-actions' do text = "How much wood could a woodchuck chuck?" speakableText = "If a woodchuck could chuck wood." timeline_item = mirror_api . timeline . insert . request_schema . new ( { 'text' = > text , 'speakableText' = > speakableText } ) menuItems = [ ] menuItems < < { 'action' = > 'READ_ALOUD' } menuItems < < { 'action' = > 'CUSTOM' , 'id' = > 'id-for-notification' , 'values' = > [ { 'displayName' = > 'JasonInCode Action' , 'iconUrl' = > 'path/to/icon.png' } ] } timeline_item . menuItems = menuItems insert_timeline_item ( timeline_item ) end

Subscriptions

As mentioned above a Glassware application can register to receive a notifications from a user’s Glass device. When creating a subscription the developer specifies a URL for Google to send a request to about the user’s actions.

def subscribe_to_notification ( collection , callback_url ) subscription = mirror_api . subscriptions . insert . request_schema . new ( { 'collection' = > collection , 'userToken' = > session [ :user_id ] , 'callbackUrl' = > callback_url } ) result = api_client . execute ( api_method : mirror_api . subscription . insert , body_object : subscription , authorization : user_credentials ) if result . success ? result . data else puts "An error occurred: #{ result . data [ 'error' ] [ 'message' ] } " end end def bootstrap_user subscribe_to_notification ( 'timeline' , to ( '/notify' ) ) end

The code above tells Google to let your application know when a user does an UPDATE , INSERT , or DELETE on their timeline. Google will send a POST request to the specified callbackUrl containing a minimal JSON ping that contains the information to retrieve the full information about. It is important to note that Google requires that the callbackUrl be secured via HTTPS.

post '/notify' do request = JSON . parse ( request . body . read ) session [ :user_id ] = request [ 'userToken' ] text = "Got a notification: #{ request } " timeline_item = mirror_api . timeline . insert . request_schema . new ( { 'text' = > text } ) insert_timeline_item ( timeline_item ) end

I hope by now you can see the possiblities that Ruby and the Mirror API make available to developers. There are stil topics I didn’t touch on such as the Contacts and Location APIs, perhaps in a future post.