Skip to main content

mellea.helpers.server_type

Utilities for detecting and classifying the target inference server.

Defines the _ServerType enum (LOCALHOST, OPENAI, REMOTE_VLLM, UNKNOWN) and _server_type, which classifies a URL by hostname. Also provides is_vllm_server_with_structured_output, which probes a server's /version endpoint to determine whether it supports the structured_outputs parameter introduced in vLLM ≥ 0.12.0. Used by the OpenAI-compatible backend to choose between guided_json and structured_outputs request formats.

Functions​

FUNC is_vllm_server_with_structured_output ​

is_vllm_server_with_structured_output(base_url: str, headers: Mapping[str, Any]) -> bool

Attempts to determine if the backend is a vllm server with version >= v0.12.0. Defaults to false.

v0.12.0 was the last version to support guided_json params. It's now under structured_outputs.

Args:

  • base_url: Base url for LLM API.
  • headers: Additional headers to pass to the request.

Returns:

  • True if the server is vLLM >= v0.12.0, False otherwise.