Skip to main content
Version: Next

mellea.backends.model_options

Common ModelOptions for Backend Generation.

Classes​

CLASS ModelOption ​

A type that wraps around model options.

Uses sentinel values (wrapped by @@@) to provide backend and model-agnostic keys for common model options.

Create a dictionary containing model options like this:

from mellea.backends import ModelOption
model_options = {{
    ModelOption.TEMPERATURE : 0.0,
    ModelOption.SYSTEM_PROMPT : "You are a helpful assistant"
}}

Attributes:

  • TOOLS: Sentinel key for a list or dict of MelleaTool instances to expose for tool calling.
  • TOOL_CHOICE: Key for tool choice strategy (passed through to the backend).
  • MAX_NEW_TOKENS: Sentinel key for the maximum number of new tokens to generate.
  • SYSTEM_PROMPT: Sentinel key for the system prompt string.
  • TEMPERATURE: Key for the sampling temperature (passed through to the backend).
  • CONTEXT_WINDOW: Sentinel key for the context window size.
  • THINKING: Sentinel key for enabling/configuring reasoning/thinking mode.
  • SEED: Sentinel key for the random seed for reproducible generation.
  • STREAM: Sentinel key for enabling streaming responses.
  • STREAM_TIMEOUT: Sentinel key for the per-chunk streaming timeout in seconds (applied to every chunk, including time-to-first-token). None disables the timeout. Defaults to 120.0 when not set.
  • STOP_SEQUENCES: Sentinel key for a list[str] of strings that, when encountered in the model output, cause generation to halt.
  • LOGITS: Sentinel key for requesting per-token processed logit scores (post-LogitsProcessor).
  • RAW_LOGITS: Sentinel key for requesting per-token raw LM-head logits (pre-LogitsProcessor).

Methods:

FUNC replace_keys ​

replace_keys(options: dict, from_to: dict[str, str]) -> dict[str, Any]

Return a new dict with selected keys in options renamed according to from_to.

Returns a new dict with the keys in options replaced with the corresponding value for that key in from_to.

  • Any key with value == None is treated the same as the key missing.

  • If the destination key already exists in options, the original value is kept in the output.

  • Regardless of the presence of the destination key in options, the source key is always absent in the output.

Example:

>>> options = {{"k1": "v1", "k2": "v2", "M1": "m1"}}
>>> from_to = {{"k1": "M1", "k2": "M2"}}

>>> new_options = replace_keys(options, from_to)
>>> print(new_options)
... {{"M1": "m1", "M2": "v2"}}
  • Notice that "M1" keeps the original value "m1", rather than "v1".
  • Notice that both "k1" and "k2" are absent in the output.

Args:

  • options: The source dictionary whose keys may be renamed.
  • from_to: Mapping of old key names to new key names.

Returns:

  • dict[str, Any]: A new dictionary with the specified keys renamed.

FUNC remove_special_keys ​

remove_special_keys(model_options: dict[str, Any]) -> dict[str, Any]

Return a copy of model_options with all sentinel-valued keys removed.

Sentinel keys are those whose names start with @@@ (e.g. ModelOption.TOOLS). These are Mellea-internal keys that must not be forwarded to backend APIs.

Args:

  • model_options: A model options dictionary that may contain sentinel keys.

Returns:

  • dict[str, Any]: A new dictionary with all @@@-prefixed keys omitted.

FUNC merge_extra_body ​

merge_extra_body(base: dict[str, Any], overwrite: dict[str, Any]) -> dict[str, Any]

Merge two extra_body dicts, deep-merging their chat_template_kwargs.

Every other key is a flat overwrite, matching merge_model_options. chat_template_kwargs is omitted from the output when the merged nested mapping is empty.

Args:

  • base: Lower-precedence extra_body dict.
  • overwrite: Higher-precedence extra_body dict.

Returns:

  • dict[str, Any]: A new merged extra_body dict.

Raises:

  • TypeError: If chat_template_kwargs is present on either side but is not a dict. The message names the offending value and which side ("base" or "overwrite") it came from.

FUNC merge_model_options ​

merge_model_options(persistent_opts: dict[str, Any], overwrite_opts: dict[str, Any] | None) -> dict[str, Any]

Merge two model-options dicts, with overwrite_opts taking precedence on conflicts.

Creates a new dict that contains all keys and values from persistent opts and overwrite opts. If there are duplicate keys, overwrite opts key value pairs will be used, except for extra_body: when both sides have a dict there, their chat_template_kwargs sub-dicts are deep-merged (see merge_extra_body) instead of one replacing the other.

Args:

  • persistent_opts: Base model options (lower precedence).
  • overwrite_opts: Per-call model options that override persistent_opts on key conflicts; None is treated as empty.

Returns:

  • dict[str, Any]: A new merged dictionary.