How to get images/attachments sent by user from Fa

Question

I have Chabot powered by Dialog Flow (API.AI) which requires user to send images.

I know that when the user clicks on Get Started button in messenger (while starting a conversation with the bot) an intent in Dialog Flow (API.AI) with WELCOME/FACEBOOK_WELCOME event will get triggered.

When the user sends simple text message an intent in Dialog Flow (API.AI) with that text in User Says will get triggered.

My doubt is there any intent in Dialog Flow (API.AI) that gets triggered when a user sends an image as attachment to the bot or is there any methodology to achieve that kind of functionality.

Please help me with this

Thanks in advance

Answer 1

Below is the response I get when I upload an image as an attachment on facebook messenger. I used dialogflow.com and integrated my webhook handler in nodejs using AWS Lambda function, API Gateway and AWS CloudWatch Logs. You can also serve webhook request and print logs. Writing console.log() really helped to print the object sent from facbook messenger to our webhook.

exports.handler = (event, context, callback) => {
  console.log(event);
  console.log(event.originalRequest.data.message.attachments[0].payload.url);
};

You will get event object url printed in the cloudwatch logs: Here in the event object the url is: https://scontent-ort2-2.xx.fbcdn.net/v/t34.18173-12/30776728_1969968496378460_1504397895_n.png?_nc_cat=0&_nc_ad=z-m&_nc_cid=0&oh=4aad83994a5501d1c50f7e2e6c7d50ea&oe=5ADBEF72

{
    "originalRequest": {
        "source": "facebook",
        "data": {
            "sender": {
                "id": "2037600292946778"
            },
            "recipient": {
                "id": "592499574453638"
            },
            "message": {
                "attachments": [
                    {
                        "payload": {
                            "url": "https://scontent-ort2-2.xx.fbcdn.net/v/t34.18173-12/30776728_1969968496378460_1504397895_n.png?_nc_cat=0&_nc_ad=z-m&_nc_cid=0&oh=4aad83994a5501d1c50f7e2e6c7d50ea&oe=5ADBEF72"
                        },
                        "type": "image"
                    }
                ],
                "mid": "mid.$cAAJAFU4_rqppFAhJFli4sL-nvI2y",
                "seq": 274
            },
            "timestamp": 1524222785882
        }
    },
    "id": "323b2069-1fb3-4643-9cab-a36562286069",
    "timestamp": "2018-04-20T11:13:06.117Z",
    "lang": "en",
    "result": {
        "source": "agent",
        "resolvedQuery": "FACEBOOK_MEDIA",
        "speech": "",
        "action": "",
        "actionIncomplete": false,
        "parameters": {},
        "contexts": [
            {
                "name": "facebook_media",
                "parameters": {},
                "lifespan": 0
            },
            {
                "name": "generic",
                "parameters": {
                    "facebook_sender_id": "2037600292946778"
                },
                "lifespan": 4
            }
        ],
        "metadata": {
            "intentId": "52d18e01-1ff2-4e35-af42-bc2de65fa38b",
            "webhookUsed": "true",
            "webhookForSlotFillingUsed": "false",
            "intentName": "attachment intent"
        },
        "fulfillment": {
            "speech": "Received an image",
            "messages": [
                {
                    "type": 0,
                    "speech": "Received an image"
                }
            ]
        },
        "score": 1
    },
    "status": {
        "code": 200,
        "errorType": "success",
        "webhookTimedOut": false
    },
    "sessionId": "d815740b-4f6d-432b-991d-c1125ceb2665"
}

Answer 2

Use FACEBOOK_MEDIA in event section of any intent in DialogFlow (API.AI). Now whenever user uploads an image to the bot, the intent which contains FACEBOOK_MEDIA in its event section will get triggered and you will get a payload which contains the image URL to you WebHook.

Answer 3

My solution:

exports.imageFb = function imageFb (request, response){
    console.log(request.body.originalDetectIntentRequest.payload.data.message.attachments[0].payload.url);
}           

Answer 4

When using WebhookClient in DialogFlow, you can get image url like this:

const agent = new WebhookClient({ request, response });
const imageUrl = agent.request_.body.originalDetectIntentRequest.payload.data.message.attachments[0].payload.url;