Linking with multimedia intent

Available in Classic and VPC

If the user sends multimedia instead of text, then that multimedia can be sent to an external analysis API to be analyzed and an answer can be provided according to the result. Only image intents are supported at this time, and it works most smoothly on the LINE messenger channel.

• NAVER Cloud Platform also offers the CLOVA OCR service which utilizes multimedia (image) intents, and it provides the feature to link the OCR service to the chatbot through multimedia intent linkage.
• The LINE messenger channel provides the existing image processing specifications and push specifications. If linking with the LINE messenger, then the chatbot can receive multimedia input, check the intent with OCR, and send a specific information as a question again. However, if you use the feature in platforms other than the LINE messenger, it will provide the image analysis but the chatbot's answers will not be provided. Due to the characteristics of each channel, there is a limitation of one-way multimedia intent recognition and linkage.

Link with multimedia intent

The following describes how to link image intents.

1. From the NAVER Cloud Platform console, click the Services > CLOVA Chatbot > Domain menus, in that order.
2. Click the [Run builder] button of the domain you want to run the chatbot builder.
3. From the chatbot builder, click Chatbot settings > Multimedia intent tab.
4. Click the [Link] button of the image intent.
5. Select Link external image analysis solution.
   • If you're using the CLOVA OCR, then select Link with CLOVA OCR (However, only OCR template domains are supported)
6. Enter the API URL and secret key for authenticating images.
   • When the user sends an image, the image is sent to the input API URL.

Image delivery format: for LINE messenger

{
  "domainCode": "ai.company.v6",
  "userId": "U77c8fe9743a0af83d4dbd57dc2f39e8a",
  "platform": "line",
  "query": "",
  "multimediaLine": {
    "contentId": 11096934827632,
    "typ": "image"
  }
}

Image delivery format: for custom channel
As the push feature is used, the chatbot's answers are not provided, even though image analysis is available with the use of the feature in platforms other than the LINE messenger.

{
  "domainCode": "ai.company.v6",
  "userId": "U77c8fe9743a0af83d4dbd57dc2f39e8a",
  "platform": "custom",
  "query": "",
  "multimedia": {
    "data": "",
    "typ": "",
    "extension": ""
  }
}

The format that should be used to respond to the chatbot after the image analysis
The name of matchedTemplate is sent to the chatbot again as a question.

{
  "matchedTemplate": {
  "name": "Analyzed template name",
   }
}

Compose conversation using multimedia intent

To utilize multimedia intent, select the question type as "Regular expression question" when registering conversation questions, and enter the CLOVA OCR template name in the question. The template name analyzed in CLOVA OCR will be used by the chatbot to ask as a question.
It can also be used in the same way when it's linked with an external image analysis solution.