Using streamText with OpenAI instead of directly accessing the OpenAI API seems to slow down the requests significantly

[rene-ka]

Here is a an example of how I tested it:

export async function getNotes({ MeetingNotes}) {
  console.log('Starting the stream request with streamText...');

  const startTime1 = performance.now(); // Start the timer for streamText

  const result1 = await streamText({
    model: openai('gpt-4-turbo'),
    messages: [
      {
        role: 'user',
        content: `Please summarize the meeting in 3 sentences "${MeetingNotes}".`
      },
    ]
  });

  let completeText1 = '';
  for await (const textPart of result1.textStream) {
    completeText1 += textPart;
  }
  const endTime1 = performance.now(); // End the timer for streamText

  console.log('Stream request with streamText completed.');
  console.log('Time taken:', endTime1 - startTime1, 'ms');
  console.log('Complete Text from streamText:', completeText1);

  console.log('Starting the stream request with openai2...');

  const startTime2 = performance.now(); // Start the timer for openai2

  const response = await openai2.chat.completions.create({
    model: 'gpt-4o',
    stream: true,
    messages: [
      {
        role: 'user',
        content: `Please summarize the meeting in 3 sentences "${MeetingNotes}".`
      },
    ]
  });

  let completeText2 = '';
  for await (const textPart of response) {
    const choices = textPart.choices || [];
    for (const choice of choices) {
      if (choice.delta && choice.delta.content) {
        completeText2 += choice.delta.content;
      }
    }
  }
  const endTime2 = performance.now(); // End the timer for openai2

  console.log('Stream request with openai2 completed.');
  console.log('Time taken:', endTime2 - startTime2, 'ms');
  console.log('Complete Text from openai2:', completeText2);

  return { completeText1, completeText2 };
}

The first request implementation will take significantly longer to complete, about twice as much time. Any thoughts why and how to fix? I would like to use the VercelAI function to be able to switch LLMs easily down the road.

[lgrammel]

You are comparing gpt-4o to gpt-4-turbo. After changing to the same model, there was no difference for me. In fact, the AI SDK was slightly faster (after moving it to the 2nd position to account for warmup).

[rene-ka]

Thanks a lot and sorry for that oversight! I now implemented the Vercel API to connect to OpenAI. Really like your implementation, super useful!