tools NotImplementedError when using streamText

[FreshHillyer]

Hi, I'm building a local llma3 chatbot. However, I found that the tools were not implemented. I am not clear whether the tools still do not support streamText or I do something wrong.

export const maxDuration = 30;
const openaiModel = createOpenAI({
  baseURL: 'http://localhost:11434/v1',
  apiKey: 'ollama',
});
export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = await streamText({
    model: openaiModel('llama3'),
    messages: convertToCoreMessages(messages),
    tools: {
      // server-side tool with execute function:
      getWeatherInformation: {
        description: 'show the weather in a given city to the user',
        parameters: z.object({ city: z.string() }),
        execute: async ({ }: { city: string }) => {
          console.log("tool")
          const weatherOptions = ['sunny', 'cloudy', 'rainy', 'snowy', 'windy'];
          return weatherOptions[
            Math.floor(Math.random() * weatherOptions.length)
          ];
        },
      },
    },
    async onFinish({ text, toolCalls, toolResults, usage, finishReason }) {
      console.log('Tool Calls:', toolCalls);
      console.log('Tool Results:', toolResults);
      console.log('Usage:', usage);
      console.log('Finish Reason:', finishReason);
    },
  });

  return result.toAIStreamResponse();
}

when I input,

Tell me the weather in LA.

It returns:

Tool Calls: []
Tool Results: []

[lgrammel]

There is a dedicated Ollama provider that might help: https://sdk.vercel.ai/providers/community-providers/ollama

[FreshHillyer]

Hi! Thanks for your reply, I have changed my model to Ollama, however, the bot asks me the tools, but does not execute anyway. As

const result = await streamText({
    model: ollama('llama3'),
    messages: convertToCoreMessages(messages),
    tools: {
      // server-side tool with execute function:
      getWeatherInformation: {
        description: 'show the weather in a given city to the user',
        parameters: z.object({ city: z.string() }),
        execute: async ({ }: { city: string }) => {
          console.log("tool")
          const weatherOptions = ['sunny', 'cloudy', 'rainy', 'snowy', 'windy'];
          return weatherOptions[
            Math.floor(Math.random() * weatherOptions.length)
          ];
        },
      },
    },
    toolChoice: "required",
    async onFinish({ text, toolCalls, toolResults, usage, finishReason }) {
      // implement your own logic here, e.g. for storing messages
      // or recording token usage
      console.log('Tool Calls:', toolCalls);
      console.log('Tool Results:', toolResults);
      console.log('Usage:', usage);
      console.log('Finish Reason:', finishReason);
    },
  });

with the

User: Hi!
AI: I'm happy to help! However, I need you to use the correct format to call a tool. To get started, please provide the following JSON object:
[
{
"name": "<tool_name>",
"arguments": {"city": "<city_name>"}
}
]
Replace <tool_name> with the name of the tool you want to use (in this case, getWeatherInformation), and replace <city_name> with the city for which you want to get weather information.
User: [ { "name": "getWeatherInformation", "arguments": {"city": "LA"} } ]
AI: You're using the correct format!
I'll simulate the getWeatherInformation tool and provide the output.

but still

Tool Calls: []
Tool Results: []

[lgrammel]

Required tool choice might not be supported by the Ollama provider. You could file a bug on their repo: https://github.com/sgomez/ollama-ai-provider