How to handle invalid tool calls?

[mlynch]

I'm using tool/function calling to modify a data structure (using gpt-4o). My tool has a number of parameters, such as label and description of a new item to add to the structure. Often, the tool will be called with missing fields as required by the zod schema, causing a runtime error. I am unsure what to do in this situation. I would think the model just wouldn't call my tool if it wasn't able to provide the required parameters, yet it does. I'm really at a loss about how to handle this scenario or even how error handling works in this case.

Any advice?

[lgrammel]

@mlynch I've written up a short guide on how to improve tool calling with the AI SDK: https://sdk.vercel.ai/docs/ai-sdk-core/tools-and-tool-calling#prompt-engineering-with-tools

Automatic retries when the parsing fails are planned (no timeline yet)

[mlynch]

Thanks @lgrammel. I wrote a custom implementation that calls chatgpt directly using gpt-4o and the same prompt, and tool calls are working successfully. There seems to be a bug with the AI SDK with tool calling params not being correctly built, which causes validation to fail and it seems impossible to catch this condition as it's happening async somewhere deep in the call stack and not awaited.

[lgrammel]

interesting - we use zod-to-json-schema under the hood. would you mind filing a bug (ideally with a reproduction if you can share and the json schema that works for you)? I'd like to look into this in more detail.

[lgrammel]

also, in a few minutes I'm going to ship the option of having custom fetch functions, so you could wrap fetch at a low level and inspect what we send in

[lgrammel]

@mlynch I've found one issue that may have contributed to tool instability (bc the temperature was not automatically set to 0): #2012 - would you be open to trying out your examples again one this lands (prob in 3.2.0)?

[amirm363]

It happened to me when I set the whole z.object to optional, if you do so, try removing it (or if you have a nested optional on the parameters) and check if it works.