Skip to main content

mellea.formatters.granite.base.io

Input and output processing for chat completions-like APIs.

Classes and functions that implement common aspects of input and output string processing for chat completions-like APIs.

Classes​

CLASS InputProcessor ​

Interface for generic input processors.

An input processor exposes an API to transform a chat completion request into a string prompt.

Methods:

FUNC transform ​

transform(self, chat_completion: ChatCompletion, add_generation_prompt: bool = True) -> str

Convert the structured representation of the inputs to a completion request.

Converts the structured representation of the inputs to a completion request into the string representation of the tokens that should be sent to the model to implement said request.

Args:

  • chat_completion: Structured representation of the inputs to the chat completion request.
  • add_generation_prompt: If True, the returned prompt string will contain a prefix of the next assistant response for use as a prompt to a generation request. Otherwise, the prompt will only contain the messages and documents in chat_completion. Defaults to True.

Returns:

  • String that can be passed to the model's tokenizer to create a prompt for generation.

CLASS OutputProcessor ​

Base class for generic output processors.

An output processor exposes an API to transform model output into a structured representation of the information.

This interface is generic; see individual classes for more specific arguments.

Methods:

FUNC transform ​

transform(self, model_output: str, chat_completion: ChatCompletion | None = None) -> AssistantMessage

Convert the model output into a structured representation.

Convert the model output generated into a structured representation of the information.

Args:

  • model_output: String output of the generation request, potentially incomplete if it was a streaming request.
  • chat_completion: The chat completion request that produced model_output. Parameters of the request can determine how the output should be decoded. Defaults to None.

Returns:

  • The parsed output so far, as an instance of :class:AssistantMessage possibly with model-specific extension fields.

CLASS ChatCompletionRewriter ​

Base class for objects that rewrite a chat completion request.

Base class for objects that rewrite a chat completion request into another chat completion request.

Methods:

FUNC transform ​

transform(**kwargs) -> ChatCompletion

Rewrite a chat completion request into another one.

Rewrite a chat completion request into another chat completion request. Does not modify the original :class:ChatCompletion object.

Args:

  • chat_completion: Original chat completion request, either as a :class:ChatCompletion dataclass, the JSON string representation, or a plain dictionary.
  • **kwargs: Additional keyword arguments forwarded to the underlying :meth:_transform implementation.

Returns:

  • Rewritten copy of the original chat completion request.

Raises:

  • TypeError: If chat_completion is not a :class:ChatCompletion object, a JSON string, or a dictionary.

CLASS ChatCompletionResultProcessor ​

Base class for chat completion result processors.

Base class for objects that convert the raw json result of a chat completion request into a JSON object with model-specific postprocessing applied.

Methods:

FUNC transform ​

transform(self, chat_completion_response: ChatCompletionResponse | dict | pydantic.BaseModel, chat_completion: ChatCompletion | None = None) -> ChatCompletionResponse

Parse and post-process the result of a chat completion request.

Args:

  • chat_completion_response: Response to a chat completion request, provided as a parsed :class:ChatCompletionResponse dataclass, a raw dictionary, or another Pydantic model.
  • chat_completion: The original chat completion request that produced chat_completion_response. Required by some implementations to decode references back to the original request. Defaults to None.

Returns:

  • Post-processed copy of the chat completion response with model-specific transformations applied.

Raises:

  • TypeError: If chat_completion_response is not a supported type.

CLASS Retriever ​

Base class for document retrievers.

Provides APIs for searching by text snippet and for inserting new documents.

Methods:

FUNC retrieve ​

retrieve(self, query: str, top_k: int = 10) -> list[Document]

Retrieve the top-k matching documents for a query from the corpus.

Args:

  • query: Query string to use for lookup.
  • top_k: Maximum number of results to return. Defaults to 10.

Returns:

  • list[Document]: List of the top-k matching :class:Document objects, each with fields such as text, title, and doc_id.