Skip to content

vllm.entrypoints.openai.chat_completion.message_assembly

Functions:

assemble_chat_message_fields(request, reasoning, content, tool_calls, *, enable_auto_tools, parser_cls)

Pick the assistant message fields for one parsed chat choice.

Shared by OpenAIServingChat and batch chat derender so both surface the same reasoning, content and tool_calls for the same parser output.

Parameters:

  • request

    (ChatCompletionRequest) –

    Supplies tools, tool_choice, include_reasoning and parallel_tool_calls.

  • reasoning

    (str | None) –

    Reasoning from Parser.parse.

  • content

    (str | None) –

    Content from Parser.parse or the raw text without a parser.

  • tool_calls

    (list[FunctionCall] | None) –

    Tool calls from Parser.parse.

  • enable_auto_tools

    (bool) –

    The server's --enable-auto-tool-choice.

  • parser_cls

    (type[Parser] | None) –

    The server's parser class, if any.

Returns:

  • ChatMessageFields –

    The fields for the response ChatMessage.

Source code in vllm/entrypoints/openai/chat_completion/message_assembly.py
def assemble_chat_message_fields(
    request: ChatCompletionRequest,
    reasoning: str | None,
    content: str | None,
    tool_calls: list[FunctionCall] | None,
    *,
    enable_auto_tools: bool,
    parser_cls: type[Parser] | None,
) -> ChatMessageFields:
    """Pick the assistant message fields for one parsed chat choice.

    Shared by `OpenAIServingChat` and batch chat derender so both surface the
    same `reasoning`, `content` and `tool_calls` for the same parser output.

    Args:
        request: Supplies `tools`, `tool_choice`, `include_reasoning` and
            `parallel_tool_calls`.
        reasoning: Reasoning from `Parser.parse`.
        content: Content from `Parser.parse` or the raw text without a parser.
        tool_calls: Tool calls from `Parser.parse`.
        enable_auto_tools: The server's `--enable-auto-tool-choice`.
        parser_cls: The server's parser class, if any.

    Returns:
        The fields for the response `ChatMessage`.

    """
    if not request.include_reasoning:
        reasoning = None

    if not keeps_tool_calls(
        request, enable_auto_tools=enable_auto_tools, parser_cls=parser_cls
    ):
        return ChatMessageFields(reasoning, content, [])
    if _forces_tool_calls(request):
        content = content or ""
    return ChatMessageFields(reasoning, content, _to_tool_calls(tool_calls, request))

get_chat_response_role(request, response_role)

Role of the response message: response_role (the server's --response-role), or the last message's role when the request continues it (add_generation_prompt=False).

Source code in vllm/entrypoints/openai/chat_completion/message_assembly.py
def get_chat_response_role(request: ChatCompletionRequest, response_role: str) -> str:
    """Role of the response message: `response_role` (the server's
    `--response-role`), or the last message's role when the request continues
    it (`add_generation_prompt=False`).
    """
    if request.add_generation_prompt or not request.messages:
        return response_role
    return request.messages[-1]["role"]

get_echo_content(messages, role)

Text that echo prepends to the response content.

Parameters:

  • messages

    (Sequence[Mapping[str, Any] | Message]) –

    The conversation, parsed (Harmony messages for gpt-oss) or as sent in the request.

  • role

    (str) –

    The response role from get_chat_response_role.

Returns:

  • str –

    The last message's text when its role is role, otherwise "". Text

  • str –

    parts of list content are extracted and joined with newlines, as chat

  • str –

    parsing does. Multimodal parts are left out.

Source code in vllm/entrypoints/openai/chat_completion/message_assembly.py
def get_echo_content(
    messages: Sequence[Mapping[str, Any] | HarmonyMessage], role: str
) -> str:
    """Text that `echo` prepends to the response content.

    Args:
        messages: The conversation, parsed (Harmony messages for gpt-oss) or
            as sent in the request.
        role: The response role from `get_chat_response_role`.

    Returns:
        The last message's text when its role is `role`, otherwise "". Text
        parts of list content are extracted and joined with newlines, as chat
        parsing does. Multimodal parts are left out.

    """
    if not messages:
        return ""
    last = messages[-1]
    if isinstance(last, HarmonyMessage):
        last = last.to_dict()
    if last.get("role") != role:
        return ""
    content = last.get("content") or ""
    if isinstance(content, str):
        return content
    texts = (_part_text(part) for part in content)
    return "\n".join(text for text in texts if text is not None)

keeps_tool_calls(request, *, enable_auto_tools, parser_cls)

Whether a chat choice returns the tool calls its parser extracts.

Shared by the non-streaming and streaming chat paths so both apply tool_choice the same way. Parsers don't all gate on it themselves: HarmonyParser ignores "none" and null is parsed like "auto".

Parameters:

  • request

    (ChatCompletionRequest) –

    Supplies tools and tool_choice.

  • enable_auto_tools

    (bool) –

    The server's --enable-auto-tool-choice.

  • parser_cls

    (type[Parser] | None) –

    The server's parser class, if any.

Returns:

  • bool –

    True for "required", a named tool, or "auto" with tools and auto

  • bool –

    tool choice enabled.

Source code in vllm/entrypoints/openai/chat_completion/message_assembly.py
def keeps_tool_calls(
    request: ChatCompletionRequest,
    *,
    enable_auto_tools: bool,
    parser_cls: type[Parser] | None,
) -> bool:
    """Whether a chat choice returns the tool calls its parser extracts.

    Shared by the non-streaming and streaming chat paths so both apply
    `tool_choice` the same way. Parsers don't all gate on it themselves:
    `HarmonyParser` ignores `"none"` and `null` is parsed like `"auto"`.

    Args:
        request: Supplies `tools` and `tool_choice`.
        enable_auto_tools: The server's `--enable-auto-tool-choice`.
        parser_cls: The server's parser class, if any.

    Returns:
        True for `"required"`, a named tool, or `"auto"` with tools and auto
        tool choice enabled.

    """
    if _forces_tool_calls(request):
        return True
    auto_tools = (
        enable_auto_tools
        and parser_cls is not None
        and parser_cls.tool_parser_cls is not None
    )
    if not auto_tools or request.tool_choice != "auto":
        return False
    if not request.tools:
        logger.error(
            "Cannot determine if tools should be extracted. "
            "Returning a standard chat completion."
        )
        return False
    return True