Support Message API for chatbot and chatinterface

[freddyaboulton]

Description

Main Changes:

• Adds a msg_format parameter to Chatbot and ChatInterface so that messages can be returned as either the current list of tuples or a dictionary that is a superset of the "OAI" format, e.g. {"role": "user", "content": ""}.
• If msg_format is "messages", then in ChatInterface developers can just yield the next token. They don't have to yield the entire message up to and including that token. I think this makes demos easier to write. And lets developers simply yield from their iterator.
• Consecutive messages with the same "role" are displayed in the same chat bubble. This is needed for the agent case where the assistant streams multiple messages consecutively.
• Added the ability to parametrize e2e tests. If you create a python file that ends with _testcase.py in a demo directory corresponding to an e2e test, that demo will also be loaded to the e2e app. And you can use go_to_testcase to navigate to that testcase.

OAI format overview

The message format is a dict with two required keys role and content. There is an additional metadata key that is not required but it can be used for tools and additional info about the message. Most messages returned from an "openai compatible" client will be compatible with gradio.

The implementation of the message is below:

class Metadata(GradioModel):
    tool_name: Optional[str] = None
    error: bool = False


class Message(GradioModel):
    role: str
    metadata: Metadata = Field(default_factory=Metadata)
    content: str | FileData

Examples

Realistic Inference API Streaming

from huggingface_hub import InferenceClient
import gradio as gr

"""
For more information on `huggingface_hub` Inference API support, please check the docs: https://huggingface.co/docs/huggingface_hub/v0.22.2/en/guides/inference
"""
client = InferenceClient("HuggingFaceH4/zephyr-7b-beta")


def respond(
    prompt: str,
    history,
):
    if not history:
        history = [{"role": "system", "content": "You are a friendly chatbot"}]
    history.append({"role": "user", "content": prompt})

    yield history

    response = {"role": "assistant", "content": ""}
    for message in client.chat_completion(
        history,
        temperature=0.95,
        top_p=0.9,
        max_tokens=512,
        stream=True,
    ):
        response["content"] += message.choices[0].delta.content or ""

        yield history + [response]


with gr.Blocks() as demo:
    gr.Markdown("# Chat with Hugging Face Zephyr 7b 🤗")
    chatbot = gr.Chatbot(
        label="Agent",
        msg_format="openai",
        avatar_images=(
            None,
            "https://em-content.zobj.net/source/twitter/376/hugging-face_1f917.png",
        ),
    )
    prompt = gr.Textbox(lines=1, label="Chat Message")
    prompt.submit(respond, [prompt, chatbot], [chatbot])


if __name__ == "__main__":
    demo.launch()

ChatInterface Streaming

import time
import gradio as gr

def slow_echo(message, history):
    yield f"You typed: "
    for i in range(len(message)):
        time.sleep(0.05)
        yield message[i]


demo = gr.ChatInterface(slow_echo, msg_format="openai").queue()

if __name__ == "__main__":
    demo.launch()

ChatInterface Multimodal

import gradio as gr


def echo(message, history):
    return message["text"]


demo = gr.ChatInterface(
    fn=echo,
    examples=[{"text": "hello"}, {"text": "hola"}, {"text": "merhaba"}],
    title="Echo Bot",
    multimodal=True,
    msg_format="openai",
)
demo.launch()

Chatbot

import gradio as gr
import random
import time

with gr.Blocks() as demo:
    chatbot = gr.Chatbot(msg_format="openai")
    msg = gr.Textbox()
    clear = gr.ClearButton([msg, chatbot])

    def respond(message, chat_history: list):
        bot_message = random.choice(["How are you?", "I love you", "I'm very hungry"])
        chat_history.extend([{"role": "user", "content": message}, {"role": "assistant", "content": bot_message}])
        time.sleep(2)
        return "", chat_history

    msg.submit(respond, [msg, chatbot], [msg, chatbot])

if __name__ == "__main__":
    demo.launch()

Multimodal Chatbot

import gradio as gr
import time

def add_message(history, message):
    for x in message["files"]:
        history.append({"role": "user", "content": {"path": x}})
    if message["text"] is not None:
        history.append({"role": "user", "content": message["text"]})
    return history, gr.MultimodalTextbox(value=None, interactive=False)

def bot(history: list):
    response = "**That's cool!**"
    history.append({"role": "assistant", "content": ""})
    for character in response:
        history[-1]['content'] += character
        time.sleep(0.05)
        yield history

with gr.Blocks() as demo:
    chatbot = gr.Chatbot(
        [],
        elem_id="chatbot",
        bubble_full_width=False,
        msg_format="openai"
    )

    chat_input = gr.MultimodalTextbox(interactive=True, file_types=["image"], placeholder="Enter message or upload file...", show_label=False)

    chat_msg = chat_input.submit(add_message, [chatbot, chat_input], [chatbot, chat_input])
    bot_msg = chat_msg.then(bot, chatbot, chatbot, api_name="bot_response")
    bot_msg.then(lambda: gr.MultimodalTextbox(interactive=True), None, [chat_input])

demo.queue()
if __name__ == "__main__":
    demo.launch()

🎯 PRs Should Target Issues

Before your create a PR, please check to see if there is an existing issue for this change. If not, please create an issue before you create this PR, unless the fix is very small.

Not adhering to this guideline will result in the PR being closed.

Tests

1. PRs will only be merged if tests pass on CI. To run the tests locally, please set up your Gradio environment locally and run the tests: bash scripts/run_all_tests.sh

2. You may need to run the linters: bash scripts/format_backend.sh and bash scripts/format_frontend.sh

[gradio-pr-bot]

🪼 branch checks and previews

•	Name	Status	URL
	Spaces	ready!	Spaces preview
	Website	ready!	Website preview
	Storybook	ready!	Storybook preview
🦄	Changes	detecting...

---

Install Gradio from this PR

pip install https://gradio-builds.s3.amazonaws.com/3c289033bb851b380f0e0cd9b774624aa822e57e/gradio-4.32.2-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@3c289033bb851b380f0e0cd9b774624aa822e57e#subdirectory=client/python"

Install Gradio JS Client from this PR

npm install https://gradio-builds.s3.amazonaws.com/3c289033bb851b380f0e0cd9b774624aa822e57e/gradio-client-0.20.1.tgz

[gradio-pr-bot]

🦄 change detected

This Pull Request includes changes to the following packages.

Package	Version
@gradio/chatbot	minor
@gradio/tootils	minor
gradio	minor

• Maintainers can select this checkbox to manually select packages to update.

With the following changelog entry.

OAI message format for chatbot and chatinterface

Maintainers or the PR author can modify the PR title to modify this entry.

Something isn't right?

• Maintainers can change the version label to modify the version bump.
• If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

[abidlabs]

@freddyaboulton this looks great! Just one quibble: I'd suggest not using "openai" as the name of the format. It might be that they modify their message format in a way that we don't want to track. Also Anthropic and others use the same format as well. What about "tuples" | "dicts"?

So thinking how we could add support for components into this: we could modify the content key to accept Component as well?

class Message(GradioModel):
    role: str
    metadata: Metadata = Field(default_factory=Metadata)
    content: str | FileData | Component

wdyt @dawoodkhan82 @freddyaboulton

[freddyaboulton]

Yes makes sense regarding renaming. What about "messages" (the name used by tgi/transformers and from the looks of it is actually the industry standard name transformers docs anthropic docs )

[abidlabs]

"messages" sounds good, compatibility with transformers/tgi makes more sense 👍

[dawoodkhan82]

This format looks good.

So thinking how we could add support for components into this: we could modify the content key to accept Component as well?

Regrading this, it would have to be another dict ComponentMessage or ComponentContent which stores the component name, the processed value, and the constructor_args.

class ComponentMessage(GradioModel):
    component: str
    value: Any
    constructor_args: List[Dict[str, Any]]

[abidlabs]

imo it would be much nicer DX if the ComponentMessage class was only used internally and the developer could just pass in a Component object, e.g. gr.Gallery([..., ...])

[freddyaboulton]

Yes that is what I had in mind, we can handle the conversion from component instance to internal payload format in pre/postprocess