Skill trainer FAQs

Available in Classic and VPC

The following describes frequently occurring problems and how to troubleshoot them.

Skill set

Q. How many skills can I create in a skill set?
A. You can create up to 10 skills. Please note that depending on the number of characters saved in the skill information (API Spec, Manifest), token excess errors may occur during data collection.

Q. I've set the response format for the skill set, but it's not properly applied to the final response during data collection.
A. Depending on the complexity of the skill and user queries, the response format you set may not be properly applied to the final response. In such cases, try manually editing the final response on the data collection screen and then perform training.
For example, if you set the response format of the skill set to "Format: JSON", but the actual final response generated by the model is in the form of "Final response A", edit it to "Final response B" and save it. To yield improved responses, try collecting multiple instances of this type of data and then start training.

• Final response A

  As of August 9, 2023, the number of purchase conversion users is as follows: 1,003,120 page views and 1,222,334 users.
  

• Final response B

  {
     "20230809":{
        "purchase":[
           {
              "pv":"1,003,120",
              "user":"1,222,334"
           }
        ]
     }
  }
  

Skill

Q. I want to develop APIs for skill creation. Are there any precautions to consider when developing APIs?
A. When developing APIs, try using parameters that provide users with predictable and useful search (filtering) conditions. If there are parameters with similar or overlapping roles (e.g., post author, post editor, tagged users, commenter), the accuracy of the skill may be decreased. When the API response time takes too long, we also recommend designing pagination parameters together in order to prevent number of tokens exceeded errors. Also, to ensure smooth data collection and service application, check the required parameters for API calls in advance.

Q. Can I write a description for API Spec in English?
A. Yes, you can. But we recommend writing in Korean to maximize the performance of the model.

Q. What should I write for a parameter description?
A. Offer examples in a parameter description. If there are examples, the model can create a more accurate action input value based on the examples in the description.

{
    "date": {
        "type": "string",
        "description": "base date. Must be in yyyymmdd format. \<example> 20230101. 20231121. 20210403"
    }
}

Q. Can I configure a URL in the form of paths for API calls?
A. We recommend using arguments to configure API URLs. Since all training data used in the skill model is based on the argument method, performance cannot be guaranteed if you configure the URL using the path method.

Q. Is there a designated port that I need to enter in my skill Endpoint URL?
A. There is no designated port. But errors may occur due to communication issues or model performance issues related to Endpoint. If the problem persists, please send us an inquiry with a screenshot.

Q. Does the Endpoint URL of the skill support only https?
A. The Endpoint URL supports both http and https.

Q. How should I write the Description for model for my multi-path API to work properly?
A. Description for the model is used as information to select an appropriate path for user queries when there are multiple paths within one skill. So, you need to specify the roles for each path and provide keywords to be searched for each path.
The performance of the language model can be improved by specifying path information and parameter information separately as follows:. Here, the path should be specified, not the operationId.

<writing example>
when searching for a beverage, use /sample_api/beverage. This API operates well when you enter a beverage name as a search keyword. When searching for a liquor, use /sample_api/liquor. This API operates well when you enter a liquor name as a search keyword. When searching for daily discounted items, use /sample/event. This API operates when you enter a date as a search keyword, and the search date must be entered in yyyymmdd format.

Data collection

Q. What is the maximum number of skill calls I can call during data collection?
A. When collecting data, there is no limit to the number of skill calls. But note that up to 4 calls can be made via the skill set answer creation API. A final response may not be created for additional calls.

Q. The results of the action input are different from what I expected. What should I do to ensure that the model creates action input as intended?
A. Add an example to the description of the parameters in API Spec. With examples, the language model can understand user queries based on API Spec and generate URLs with correct parameters.
<writing example>

{
    "keyword": {
        "type": "string",
        "description": "a modifier in the form of an adjective or an adverb. \<example> kind, satisfactory, good",
    },
    "review": {
        "type": "string",
        "description": "positive or negative assessment. \<example> either pos or neg"
    }
}

Q. When collecting data, I received the error message "No plannable skills available." But the skills are already marked as "completed."
A. This error occurs when the model fails to understand the API Spec or Manifest of the skills properly. Please rewrite them according to the API Spec and Manifest writing guide to ensure that the model can understand them sufficiently. If the problem persists, please leave an inquiry on the Forum.

Q. When collecting data, I received the error message "Token limit exceeded." Can I check how many tokens have been exceeded?
A. The maximum number of tokens when collecting data is limited to 4096 per step. For the sum of the tokens per step, refer to the following information. To calculate the number of tokens, you can use Explorer > Tokenizer (HCX) in CLOVA Studio.

• Query analysis: system prompt, user query, Manifest, query analysis result
• Skill calls
  • Step 2 (viewing API search result): system prompt, call method, call skill, API query details, API search result, action, action input
  • Step 3 (call result): system prompt, call method, call skill, API query details, API search result, action, action input, call result, thought, and result
• Final response: system prompt, result, final response

Q. Will the call options be initialized every time I collect data?
A. Call options will remain unchanged even after clicking the [Initialize] button for the next data collection. However, if you exit the data collection screen and return later, the call options will be initialized. In such cases, you can use the Load function to retrieve your temporarily saved work and the previously used call options as well.

Can you provide an example of call options for different purposes?
A. You can use the baseOperation field when you want to apply a call option across all APIs registered within the skill set. The following shows how to write it.

{
  "baseOperation": {
    "query": {
      "serviceKey": "value"
    }
  }
}

You can use the operations field when you want to apply a call option only to a specific Operation ID. The following shows how to write it.

{
  "operations": [
    {
      "operationId": "filterBakeryProducts",
      "query": {
        "serviceKey": "value"
      }
    }
  ]
}

It is also possible to apply the baseOperation field and the operations field together. The following shows how to write it.

{
  "baseOperation": {
    "header": {
      "baseHeader": "baseHeaderValue"
    },
    "query": {
      "baseQuery": "baseQueryValue"
    },
    "requestBody": {
      "baseBody": "baseBodyValue"
    }
  },
  "operations": [
    {
      "operationId": "testOperationId1",
      "header": {
        "header1": "headerValue1"
      },
      "query": {
        "query1": "queryValue1"
      },
      "requestBody": {
        "body1": "bodyValue1"
      }
    },
    {
      "operationId": "testOperationId2",
      "header": {
        "header2": "headerValue2"
      },
      "query": {
        "query2": "queryValue2"
      },
      "requestBody": {
        "body2": "bodyValue2"
      }
    }
  ]
}

Q. An error has occurred during data collection. How can I check the log?
A. If an error occurs during a task, send us an inquiry with the following information. We will walk you through the troubleshooting steps.

• Error screen (screenshot)
• Skill set
• Skill information

Q. Can the model create an answer asking again for missing parameters when a required parameter is missing from a user query?
Yes, it can. When a required parameter is missing during API designing, you must set it to not return an error code (e.g., HTTP status code 400) but instead return a normal code (e.g., HTTP code 200). However, we recommend returning a message notifying the missing parameter along with the code. With this, you can have the model create questions requesting for missing parameters in the final response area, and tune/train the model in the form you want.

<Data collection examples>

Data training

Q. What content is the model trained with during data training?
A. Data training includes skill call training and final response training.

• Skill call training: if the model does not accurately generate the API URL and parameters to perform user requests, you can edit them accordingly and store the data for training to improve the model's performance. When editing data, avoid including keywords unrelated to user requests in the fields.
• Final response training: while generating final responses based on skill call results, if the model misses some information or you want to change the style of the answer (format, tone, etc.), you can edit it accordingly and store the data for training to improve the model's performance. However, do not include information in the final response that was not included in the skill call results.

Version management

Q. Up to how many skill set versions can I create?
A. You can save up to 300 versions of skill sets.

Q. What are the criteria for updating a version?
A. The criteria for creating or updating a skill set version are as follows:

• For the initial creation of a skill in a skill set, v1 of the skill set is created.
• When you create a new skill in the skill set, the version is updated.
• When you delete a skill registered in the skill set, the version is updated. The deletion history is recorded in the version comment.
• When you edit and save an existing skill, the version is updated.
• Once the data training is complete, the version is updated.