Available in Classic and VPC
If a user sends multimedia instead of text, you can forward the received multimedia to an external analysis API for analysis and then use them to generate a response. Currently, only image intents are supported, and they work most smoothly on the LINE messenger.
- NAVER Cloud Platform provides a CLOVA OCR service that utilizes multimedia (image) intents, and through multimedia intent integration, it supports the ability to connect OCR services to chatbots.
- LINE messenger already provides specifications for handling images as well as Push specifications. When integrated with LINE messenger, the chatbot can receive multimedia input, use OCR to identify the intent, and then reframe specific information as a follow-up question to the chatbot. However, when using platforms other than LINE messenger, image analysis is possible, but chatbot response generation is not supported. Due to the characteristics of each channel, there is a limitation in that multimedia intents can only be recognized and integrated in a one-way manner.
Integrate with multimedia intent
To integrate with image intents:
- In the NAVER Cloud Platform console, navigate to
> Services > AI Services > CLOVA Chatbot > Domain. - Click [Run builder] of the domain you want to run the Chatbot Builder.
- From the chatbot Builder, navigate to Chatbot setting > [Multimedia intent].
- Click [Integrate] of the image intent.
- Select Integrate with an external image analysis solution.
- If CLOVA OCR is in use, select Integrate with CLOVA OCR (However, only OCR template domains are supported)
- Enter the API URL and Secret key that can verify images.
- Once the user transfers images, deliver them to the entered API URL.
Image delivery format: If it is LINE messenger
{
"domainCode": "ai.company.v6",
"userId": "U77c8fe9743a0af83d4dbd57dc2f39e8a",
"platform": "line",
"query": "",
"multimediaLine": {
"contentId": 11096934827632,
"typ": "image"
}
}
| Value | Required | Type | Description |
|---|---|---|---|
| multimediaLine | Object | N | Used when it is necessary to pass content ID information such as images. |
| multimediaLine.contentId | Long | Y | Content's id |
| multimediaLine.typ | String | Y | Indicate the type of the content. Currently, only images are supported. |
| multimediaLine | Object | N | Used when it is necessary to pass content ID information such as images. |
Image delivery format: If it is a custom channel
When using platforms other than LINE messenger for the Push feature, image analysis is possible, but chatbot response generation is not supported.
{
"domainCode": "ai.company.v6",
"userId": "U77c8fe9743a0af83d4dbd57dc2f39e8a",
"platform": "custom",
"query": "",
"multimedia": {
"data": "",
"typ": "",
"extension": ""
}
}
| Value | Required | Type | Description |
|---|---|---|---|
| multimedia | Object | N | Used when it is necessary to pass multimedia such as images. |
| multimedia.data | String | Y | Data of the multimedia The data need to be base64-encoded. |
| multimedia.typ | String | Y | Indicate the type of multimedia. Currently, only images are supported. |
| multimedia.extension | String | Y | File extension information of the corresponding multimedia jpeg, png, etc, are also supported. |
Format for responding back to the chatbot after image analysis is completed
The name of the matchedTemplate is be asked back to the chatbot as a question.
{
"matchedTemplate": {
"name": "name of the analyzed template",
}
}
Conversation creation using multimedia intents
To utilize multimedia intents, when registering a conversation question, select the question type as 'Regular Expression Question' and enter the template name of CLOVA OCR in the question. The template name analyzed by CLOVA OCR is asked back to the chatbot as a question.
The same method can be applied even when integrated with an external image analysis solution.
