Building Hello World

Creating the project

First of all, go to Actions On Google Console and create a new project. (Don’t forget to notice how beautiful the new UI is 😍) Pick Games & Fun category since the Interactive Canvas is only available for Games as of today. Pick Conversational at the next step as we’re going to be using Dialogflow. Under the Build Your Action section, choose Add Action(s) and click Add your first action button. Select Custom intent and click Build. At this point, Dialogflow should open. Log in if you haven’t already and click Create at the top right corner. Go back to Actions On Google Console and select Deploy from the top menu. Scroll at the very bottom and check the box with the label Interactive Canvas. Scroll up to the top and click Save.

Voilà! Your project and your Dialogflow agent is ready for development.

If you didn’t pick the right category at this step, you can set it anytime via the dropbox that’s located under Deploy > Directory Information > Additional Information

Setting up the local development environment

In this tutorial, we’ll be using Firebase Cloud Functions for webhook fulfillment and Firebase Hosting to host our web app. If you’ve never used firebase-tools on your computer, first run:

firebase login

If you’re logged in, let’s create a directory on our computer run the following command:

firebase init

Say yes when it asks if you’re ready and select Hosting and Functions features from the list and hit enter.

A new list will pop up, select the project-id of the project you’ve created in the previous steps. Answer the following question in as:

function configuration:

JavaScript

No ESLint

Yes, install the dependencies.

hosting configuration:

Use (public) as our public directory. (Just hit enter)

Yes, configure this app as a single-page app.

There you have it, a local development environment for our Cloud Function and Hosting, ready to deploy on Firebase.

Configuring the Welcome Intent

Our Default Welcome Intent needs to open our web app in the Assistant. That means we have to configure it so that it’s fulfilled by our webhook. Open Dialogflow console and:

Pick Default Welcome Intent from the list and delete all the training phrases for the intent. (We don’t want our users to come back to this intent by using those phrases) Go ahead and delete all the responses, too. Lastly, enable webhook call for the intent by using the button at the bottom of the page. Click Save.

Building the basics: Web App

We need to build and deploy the web app first becuase we’re going to use its URL while building our fulfillment.

Open public > index.html file and change its content with the following code:

Notice the asset that Google provides us with: interactive_canvas.min.js

interactive_canvas.min.js is the tool that’s going to register the state updates that we’ll send from our webhook. Basically, this is the file that makes it possible to communicate between our conversational agent and our web app.

interactive_canvas.min.js also has a method to get the header height of the user’s device.

Now, run the following command to deploy our hosting:

firebase deploy --only hosting

After it’s done uploading, you should see the Hosting URL in the terminal. You might want to copy that URL as we’re going to use it in the next step.

Building the basics: Fulfillment

We have already generated the necessary files for our project with the firebase-tools, we can now go ahead and edit them.

First, we need to set the dependencies for our webhook fulfillment. Open functions> package.json and change its content with the following code:

As you can see, we’re depending on the “preview” version of the actions-on-google client library to use Interactive Canvas.

Then run the following command in the functions directory to install the dependencies:

npm install

When it’s done installing, open functions > index.js and change its content with the following code:

⚠️ Don’t forget to replace YOUR_URL_HERE with your web apps URL ⚠️

You need to return a new HtmlResponse object to use Interactive Canvas. You need to write your web app’s URL, which you’ve copied in the last step, where it says YOUR_URL_HERE.

After you’ve updated the URL, run the following command to deploy your cloud functions:

firebase deploy --only functions

Now in your terminal, you should see the Function URL. Copy that and go to Dialogflow Console. On the left sidebar, click on Fulfillment, enable Webhook and paste your function URL.

Test your Action

Go to Actions On Google Console and open the Simulator by clicking Test on the top menu. When you say “Talk to my test app” you should see Hello, World! written on the display and also hear “hello world”.

How exciting 👀

It’s cool to see our custom web page inside the Assistant but we still can’t interact with it using our voice. Let’s move on to the next section to see how we can do it.