Dialogflow-fulfillment-nodejs: What's the structure needed in dialogflow webhook V2 response to return either List or Carousel?

Created on 24 Apr 2018 · 11Comments · Source: dialogflow/dialogflow-fulfillment-nodejs

I have seen most of the examples related to how to return rich content via webhooks involving just basic response and Card.

What's the structure needed in dialogflow webhook V2 response to return either List or Carousel?

Source

kwizfreak

👍3

Most helpful comment

Unfortunately carousels require some Actions on Google-specific information. Carousels in Actions on Google can be "selected" and information on how to identify what should happen when a user selects a carousel item. This information needs to be provided when the carousel is surfaced to the user. To use carousels in Actions on Google there are two options using the actions-on-google library integration or using a custom payload. Here are two examples:

Using the Actions on Google library integration:

const { Carousel } = require('actions-on-google');
const { WebhookClient } = require('dialogflow-fulfillment');
const agent = new WebhookClient({ request, response });
let conv = agent.conv();
conv.ask(new Carousel({
  items: {
    // Add the first item to the carousel
    [SELECTION_KEY_ONE]: {
      synonyms: [
        'synonym of title 1',
        'synonym of title 2',
        'synonym of title 3',
      ],
      title: 'Title of First Carousel Item',
      description: 'This is a description of a carousel item.',
      image: new Image({
        url: IMG_URL_AOG,
        alt: 'Image alternate text',
      }),
    },
    // Add the second item to the carousel
    [SELECTION_KEY_GOOGLE_HOME]: {
      synonyms: [
        'Google Home Assistant',
        'Assistant on the Google Home',
    ],
      title: 'Google Home',
      description: 'Google Home is a voice-activated speaker powered by ' +
        'the Google Assistant.',
      image: new Image({
        url: IMG_URL_GOOGLE_HOME,
        alt: 'Google Home',
      }),
    },
    // Add third item to the carousel
    [SELECTION_KEY_GOOGLE_PIXEL]: {
      synonyms: [
        'Google Pixel XL',
        'Pixel',
        'Pixel XL',
      ],
      title: 'Google Pixel',
      description: 'Pixel. Phone by Google.',
      image: new Image({
        url: IMG_URL_GOOGLE_PIXEL,
        alt: 'Google Pixel',
      }),
    },
    // Add last item of the carousel
    [SELECTION_KEY_GOOGLE_ALLO]: {
      title: 'Google Allo',
      synonyms: [
        'Allo',
      ],
      description: 'Introducing Google Allo, a smart messaging app that ' +
        'helps you say more and do more.',
      image: new Image({
        url: IMG_URL_GOOGLE_ALLO,
        alt: 'Google Allo Logo',
      }),
    },
  },
}));

agent.add(conv);

Using the a payload response:

const { WebhookClient, Payload } = require('dialogflow-fulfillment');
const agent = new WebhookClient({ request, response });
agent.add(new Payload(agent.ACTIONS_ON_GOOGLE,   {
    "expectUserResponse": true,
    "richResponse": {
      "items": [
        {
          "simpleResponse": {
            "textToSpeech": "Choose a item"
          }
        }
      ]
    },
    "systemIntent": {
      "intent": "actions.intent.OPTION",
      "data": {
        "@type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
        "carouselSelect": {
          "items": [
            {
              "optionInfo": {
                "key": "first title"
              },
              "description": "first description",
              "image": {
                "url": "https://developers.google.com/actions/images/badges/XPM_BADGING_GoogleAssistant_VER.png",
                "accessibilityText": "first alt"
              },
              "title": "first title"
            },
            {
              "optionInfo": {
                "key": "second"
              },
              "description": "second description",
              "image": {
                "url": "https://lh3.googleusercontent.com/Nu3a6F80WfixUqf_ec_vgXy_c0-0r4VLJRXjVFF_X_CIilEu8B9fT35qyTEj_PEsKw",
                "accessibilityText": "second alt"
              },
              "title": "second title"
            }
          ]
        }
      }
    }
  });
);

matthewayne on 23 May 2018

👍9

All 11 comments

I would like to see the same thing. I have have a carousel working for facebook as per something I read here https://dialogflow.com/docs/rich-messages "If you have several cards in the same intent, they will displayed in a horizontal scrollable carousel"

pseudo
agent.add(new Card( ... ))
agent.add(new Card( ... ))
agent.add(new Card( ... ))

This creates the carousel in FBM, but it causes an error in google assistant.
"expected_inputs[0].input_prompt.rich_initial_prompt: only one 'card' or 'structured_response' or 'custom_response' is supported"

Do we need to program separate code for each platform?

McGern on 22 May 2018

👍1

I'm looking for something similar.

@matthewayne , if possible and if time permits, could you please add a sample for this as well?

SiddAjmera on 23 May 2018

Using the Actions on Google library integration:

const { Carousel } = require('actions-on-google');
const { WebhookClient } = require('dialogflow-fulfillment');
const agent = new WebhookClient({ request, response });
let conv = agent.conv();
conv.ask(new Carousel({
  items: {
    // Add the first item to the carousel
    [SELECTION_KEY_ONE]: {
      synonyms: [
        'synonym of title 1',
        'synonym of title 2',
        'synonym of title 3',
      ],
      title: 'Title of First Carousel Item',
      description: 'This is a description of a carousel item.',
      image: new Image({
        url: IMG_URL_AOG,
        alt: 'Image alternate text',
      }),
    },
    // Add the second item to the carousel
    [SELECTION_KEY_GOOGLE_HOME]: {
      synonyms: [
        'Google Home Assistant',
        'Assistant on the Google Home',
    ],
      title: 'Google Home',
      description: 'Google Home is a voice-activated speaker powered by ' +
        'the Google Assistant.',
      image: new Image({
        url: IMG_URL_GOOGLE_HOME,
        alt: 'Google Home',
      }),
    },
    // Add third item to the carousel
    [SELECTION_KEY_GOOGLE_PIXEL]: {
      synonyms: [
        'Google Pixel XL',
        'Pixel',
        'Pixel XL',
      ],
      title: 'Google Pixel',
      description: 'Pixel. Phone by Google.',
      image: new Image({
        url: IMG_URL_GOOGLE_PIXEL,
        alt: 'Google Pixel',
      }),
    },
    // Add last item of the carousel
    [SELECTION_KEY_GOOGLE_ALLO]: {
      title: 'Google Allo',
      synonyms: [
        'Allo',
      ],
      description: 'Introducing Google Allo, a smart messaging app that ' +
        'helps you say more and do more.',
      image: new Image({
        url: IMG_URL_GOOGLE_ALLO,
        alt: 'Google Allo Logo',
      }),
    },
  },
}));

agent.add(conv);

Using the a payload response:

const { WebhookClient, Payload } = require('dialogflow-fulfillment');
const agent = new WebhookClient({ request, response });
agent.add(new Payload(agent.ACTIONS_ON_GOOGLE,   {
    "expectUserResponse": true,
    "richResponse": {
      "items": [
        {
          "simpleResponse": {
            "textToSpeech": "Choose a item"
          }
        }
      ]
    },
    "systemIntent": {
      "intent": "actions.intent.OPTION",
      "data": {
        "@type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
        "carouselSelect": {
          "items": [
            {
              "optionInfo": {
                "key": "first title"
              },
              "description": "first description",
              "image": {
                "url": "https://developers.google.com/actions/images/badges/XPM_BADGING_GoogleAssistant_VER.png",
                "accessibilityText": "first alt"
              },
              "title": "first title"
            },
            {
              "optionInfo": {
                "key": "second"
              },
              "description": "second description",
              "image": {
                "url": "https://lh3.googleusercontent.com/Nu3a6F80WfixUqf_ec_vgXy_c0-0r4VLJRXjVFF_X_CIilEu8B9fT35qyTEj_PEsKw",
                "accessibilityText": "second alt"
              },
              "title": "second title"
            }
          ]
        }
      }
    }
  });
);

matthewayne on 23 May 2018

👍9

@matthewayne , what exactly do these selection key mean here? I'm referring to SELECTION_KEY_ONE, SELECTION_KEY_GOOGLE_HOME, SELECTION_KEY_GOOGLE_PIXEL, and SELECTION_KEY_GOOGLE_ALLO here that you've used in the code above. Where exactly are these used? Could you please share a link to some doc. that could help me understand it?

SiddAjmera on 23 May 2018

https://developers.google.com/actions/assistant/responses#carousel

matthewayne on 24 May 2018

dialogueflow fullfillment.zip

The error I am getting
screen shot 2018-08-27 at 5 07 35 pm

sravyagajavalli on 27 Aug 2018

Using the Actions on Google library integration:

const { Carousel } = require('actions-on-google');
const { WebhookClient } = require('dialogflow-fulfillment');
const agent = new WebhookClient({ request, response });
let conv = agent.conv();
conv.ask(new Carousel({
  items: {
    // Add the first item to the carousel
    [SELECTION_KEY_ONE]: {
      synonyms: [
        'synonym of title 1',
        'synonym of title 2',
        'synonym of title 3',
      ],
      title: 'Title of First Carousel Item',
      description: 'This is a description of a carousel item.',
      image: new Image({
        url: IMG_URL_AOG,
        alt: 'Image alternate text',
      }),
    },
    // Add the second item to the carousel
    [SELECTION_KEY_GOOGLE_HOME]: {
      synonyms: [
        'Google Home Assistant',
        'Assistant on the Google Home',
    ],
      title: 'Google Home',
      description: 'Google Home is a voice-activated speaker powered by ' +
        'the Google Assistant.',
      image: new Image({
        url: IMG_URL_GOOGLE_HOME,
        alt: 'Google Home',
      }),
    },
    // Add third item to the carousel
    [SELECTION_KEY_GOOGLE_PIXEL]: {
      synonyms: [
        'Google Pixel XL',
        'Pixel',
        'Pixel XL',
      ],
      title: 'Google Pixel',
      description: 'Pixel. Phone by Google.',
      image: new Image({
        url: IMG_URL_GOOGLE_PIXEL,
        alt: 'Google Pixel',
      }),
    },
    // Add last item of the carousel
    [SELECTION_KEY_GOOGLE_ALLO]: {
      title: 'Google Allo',
      synonyms: [
        'Allo',
      ],
      description: 'Introducing Google Allo, a smart messaging app that ' +
        'helps you say more and do more.',
      image: new Image({
        url: IMG_URL_GOOGLE_ALLO,
        alt: 'Google Allo Logo',
      }),
    },
  },
}));

agent.add(conv);

Using the a payload response:

const { WebhookClient, Payload } = require('dialogflow-fulfillment');
const agent = new WebhookClient({ request, response });
agent.add(new Payload(agent.ACTIONS_ON_GOOGLE,   {
    "expectUserResponse": true,
    "richResponse": {
      "items": [
        {
          "simpleResponse": {
            "textToSpeech": "Choose a item"
          }
        }
      ]
    },
    "systemIntent": {
      "intent": "actions.intent.OPTION",
      "data": {
        "@type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
        "carouselSelect": {
          "items": [
            {
              "optionInfo": {
                "key": "first title"
              },
              "description": "first description",
              "image": {
                "url": "https://developers.google.com/actions/images/badges/XPM_BADGING_GoogleAssistant_VER.png",
                "accessibilityText": "first alt"
              },
              "title": "first title"
            },
            {
              "optionInfo": {
                "key": "second"
              },
              "description": "second description",
              "image": {
                "url": "https://lh3.googleusercontent.com/Nu3a6F80WfixUqf_ec_vgXy_c0-0r4VLJRXjVFF_X_CIilEu8B9fT35qyTEj_PEsKw",
                "accessibilityText": "second alt"
              },
              "title": "second title"
            }
          ]
        }
      }
    }
  });
);

How to handle the OPTION choosen in WebhookClient ?

vairaselvam on 12 Sep 2018

how to write for viber carousel, can you tell me please?

blessedsangpi on 2 Mar 2019

how to write for viber carousel, can you tell me please?

Nothing different coding for any Integration Channel. Only the Channel has to support Carousel. If Viber supports normal code will work.

vairaselvam on 2 Mar 2019

function fallback(agent) {
var carouselViber = {
"ButtonsGroupColumns": 6,
"ButtonsGroupRows": 2,
"BgColor": "#FFFFFF",
"Buttons": [{
"ActionBody": "http://www.website.com/go_here",
"ActionType": "open-url",
"BgMediaType": "picture",
"Image": "http://www.images.com/img.jpg",
"BgColor": "#000000",
"TextOpacity": 60,
"Rows": 4,
"Columns": 6
}, {
"ActionBody": "http://www.website.com/go_here",
"ActionType": "open-url",
"BgColor": "#85bb65",
"Text": "Buy",
"TextOpacity": 60,
"Rows": 1,
"Columns": 6
}]
};

agent.add(new Payload('VIBER', carouselViber));
}
what's is wrong in the code? can you correct for me, please?

blessedsangpi on 4 Mar 2019

👍1

I can't manage to make ir work on Viber as well. :(

The only workaround I found is to use direct Viber messaging API from Dialogflow fulfillment . See details https://www.npmjs.com/package/messaging-api-viber