[Bug]: Async Streaming Mock is different to sync streaming mock

[almajo]

What happened?

When using the streaming mock feature with the completion() function, everything works as expected and described in the docu

However, when using the acompletion function with stream=True, the whole mock_response is put into a delta content, instead of only a delta-token. This is mocking, so not the worst bug, but I guess we would like to keep completion and acompletion interchangeable.

Relevant log output

import litellm
generator = await litellm.acompletion(
            messages=[{"role":"user", "content":"Why is LiteLLM amazing?"}],
            mock_response="LiteLLM is awesome",
            stream=True,
            model="gpt-3.5-turbo",
        )
async for t in generator:
    print(t)

----
ModelResponse(id='chatcmpl-cefbd92a-efbc-47da-913c-217cc9304f24', choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(content='LiteLLM is awesome', role='assistant', function_call=None, tool_calls=None), logprobs=None)], created=1718275107, model='gpt-3.5-turbo', object='chat.completion.chunk', system_fingerprint=None)
ModelResponse(id='chatcmpl-cefbd92a-efbc-47da-913c-217cc9304f24', choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(content='LiteLLM is awesome', role=None, function_call=None, tool_calls=None), logprobs=None)], created=1718275107, model='gpt-3.5-turbo', object='chat.completion.chunk', system_fingerprint=None)
ModelResponse(id='chatcmpl-cefbd92a-efbc-47da-913c-217cc9304f24', choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(content='LiteLLM is awesome', role=None, function_call=None, tool_calls=None), logprobs=None)], created=1718275107, model='gpt-3.5-turbo', object='chat.completion.chunk', system_fingerprint=None)
ModelResponse(id='chatcmpl-cefbd92a-efbc-47da-913c-217cc9304f24', choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(content='LiteLLM is awesome', role=None, function_call=None, tool_calls=None), logprobs=None)], created=1718275107, model='gpt-3.5-turbo', object='chat.completion.chunk', system_fingerprint=None)
ModelResponse(id='chatcmpl-cefbd92a-efbc-47da-913c-217cc9304f24', choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(content='LiteLLM is awesome', role=None, function_call=None, tool_calls=None), logprobs=None)], created=1718275107, model='gpt-3.5-turbo', object='chat.completion.chunk', system_fingerprint=None)
ModelResponse(id='chatcmpl-cefbd92a-efbc-47da-913c-217cc9304f24', choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(content='LiteLLM is awesome', role=None, function_call=None, tool_calls=None), logprobs=None)], created=1718275107, model='gpt-3.5-turbo', object='chat.completion.chunk', system_fingerprint=None)
ModelResponse(id='chatcmpl-cefbd92a-efbc-47da-913c-217cc9304f24', choices=[StreamingChoices(finish_reason='stop', index=0, delta=Delta(content=None, role=None, function_call=None, tool_calls=None), logprobs=None)], created=1718275107, model='gpt-3.5-turbo', object='chat.completion.chunk', system_fingerprint=None)

Twitter / LinkedIn details

No response

[ishaan-jaff]

fixed here @almajo #4268

[ishaan-jaff]

Hi @almajo any chance we can hop on a call ? I'd love to learn how how we can improve litellm for you.

I reached out to you on Linkedin if DMs work.
Sharing a link to my cal for your convenience: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat