Skip to main content
A generic OpenAI compatible backend that wraps around the openai python sdk.

​
Classes

​
CLASS OpenAIBackend

A generic OpenAI compatible backend. Args:
  • model_id: OpenAI-compatible model identifier. Defaults to model_ids.OPENAI_GPT_5_1.
  • formatter: Formatter for rendering components. Defaults to [TemplateFormatter](../formatters/template_formatter#class-templateformatter).
  • base_url: Base URL for the API endpoint; defaults to the standard OpenAI endpoint if not set.
  • model_options: Default model options for generation requests.
  • default_to_constraint_checking_alora: If False, deactivates aLoRA constraint checking; primarily for benchmarking and debugging.
  • api_key: API key; falls back to OPENAI_API_KEY env var.
  • kwargs: Additional keyword arguments forwarded to the OpenAI client.
Attributes:
  • to_mellea_model_opts_map_chats: Mapping from chat-endpoint option names to Mellea [ModelOption](model_options#class-modeloption) sentinel keys.
  • from_mellea_model_opts_map_chats: Mapping from Mellea sentinel keys to chat-endpoint option names.
  • to_mellea_model_opts_map_completions: Mapping from completions-endpoint option names to Mellea [ModelOption](model_options#class-modeloption) sentinel keys.
  • from_mellea_model_opts_map_completions: Mapping from Mellea sentinel keys to completions-endpoint option names.
Methods:

​
FUNC filter_openai_client_kwargs 

filter_openai_client_kwargs(**kwargs) -> dict
Filter kwargs to only include valid OpenAI client constructor parameters. Args:
  • kwargs: Arbitrary keyword arguments to filter.
Returns:
  • A dict containing only keys accepted by openai.OpenAI.__init__.

​
FUNC filter_chat_completions_kwargs 

filter_chat_completions_kwargs(self, model_options: dict) -> dict
Filter model options to only include valid OpenAI chat completions parameters. See https://platform.openai.com/docs/api-reference/chat/create for the full list of accepted parameters. Args:
  • model_options: Model options dict that may contain non-chat keys.
Returns:
  • A dict containing only keys accepted by chat.completions.create.

​
FUNC filter_completions_kwargs 

filter_completions_kwargs(self, model_options: dict) -> dict
Filter model options to only include valid OpenAI completions parameters. See https://platform.openai.com/docs/api-reference/completions for the full list of accepted parameters. Args:
  • model_options: Model options dict that may contain non-completions keys.
Returns:
  • A dict containing only keys accepted by completions.create.

​
FUNC generate_from_chat_context 

generate_from_chat_context(self, action: Component[C] | CBlock, ctx: Context) -> tuple[ModelOutputThunk[C], Context]
Generate a new completion from the provided Context using this backend’s [Formatter](../core/formatter#class-formatter). Formats the context and action into OpenAI-compatible chat messages, submits the request asynchronously, and returns a thunk that lazily resolves the output. Args:
  • action: The component or content block to generate a completion for.
  • ctx: The current generation context.
  • _format: Optional Pydantic model class for structured output decoding.
  • model_options: Per-call model options.
  • tool_calls: If True, expose available tools and parse responses.
Returns:
  • tuple[ModelOutputThunk[C], Context]: A thunk holding the (lazy) model output and an updated context that includes action and the new output.

​
FUNC processing 

processing(self, mot: ModelOutputThunk, chunk: ChatCompletion | ChatCompletionChunk)
Accumulate content from a single OpenAI response object into the output thunk. Called for each ChatCompletion (non-streaming) or ChatCompletionChunk (streaming). Tool call parsing is deferred to post_processing. Args:
  • mot: The output thunk being populated.
  • chunk: A single response object or streaming delta from the OpenAI API.

​
FUNC post_processing 

post_processing(self, mot: ModelOutputThunk, tools: dict[str, AbstractMelleaTool], conversation: list[dict], thinking, seed, _format)
Finalize the output thunk after OpenAI generation completes. Reconstructs a merged chat response from streaming chunks if applicable, extracts any tool call requests, records token usage metrics, emits telemetry, and attaches the generate log. Args:
  • mot: The output thunk to finalize.
  • tools: Available tools, keyed by name.
  • conversation: The chat conversation sent to the model, used for logging.
  • thinking: The reasoning effort level passed to the model, or None if reasoning mode was not enabled.
  • seed: The random seed used during generation, or None.
  • _format: The structured output format class used during generation, if any.

​
FUNC generate_from_raw 

generate_from_raw(self, actions: list[Component[C]], ctx: Context) -> list[ModelOutputThunk[C]]

​
FUNC generate_from_raw 

generate_from_raw(self, actions: list[Component[C] | CBlock], ctx: Context) -> list[ModelOutputThunk[C | str]]

​
FUNC generate_from_raw 

generate_from_raw(self, actions: Sequence[Component[C] | CBlock], ctx: Context) -> list[ModelOutputThunk]
Generate completions for multiple actions without chat templating via the OpenAI completions API. Passes formatted prompt strings directly to the completions endpoint. Tool calling is not supported on this endpoint. Args:
  • actions: Actions to generate completions for.
  • ctx: The current generation context.
  • format: Optional Pydantic model for structured output; passed as a guided-decoding parameter.
  • model_options: Per-call model options.
  • tool_calls: Ignored; tool calling is not supported on this endpoint.
Returns:
  • list[ModelOutputThunk]: A list of model output thunks, one per action.
Raises:
  • openai.BadRequestError: If the request is invalid (e.g. when targeting an Ollama server that does not support batched completion requests).

​
FUNC base_model_name 

base_model_name(self)
Returns the base_model_id of the model used by the backend. For example, granite-3.3-8b-instruct for ibm-granite/granite-3.3-8b-instruct.