Skip to main content
Input and output processing for chat completions-like APIs. Classes and functions that implement common aspects of input and output string processing for chat completions-like APIs.

​
Classes

​
CLASS InputProcessor

Interface for generic input processors. An input processor exposes an API to transform a chat completion request into a string prompt.
Methods:

​
FUNC transform 

transform(self, chat_completion: ChatCompletion, add_generation_prompt: bool = True) -> str
Convert the structured representation of the inputs to a completion request. Converts the structured representation of the inputs to a completion request into the string representation of the tokens that should be sent to the model to implement said request. Args:
  • chat_completion: Structured representation of the inputs to the chat completion request.
  • add_generation_prompt: If True, the returned prompt string will contain a prefix of the next assistant response for use as a prompt to a generation request. Otherwise, the prompt will only contain the messages and documents in chat_completion. Defaults to True.
Returns:
  • String that can be passed to the model’s tokenizer to create a prompt for generation.

​
CLASS OutputProcessor

Base class for generic output processors. An output processor exposes an API to transform model output into a structured representation of the information. This interface is generic; see individual classes for more specific arguments.
Methods:

​
FUNC transform 

transform(self, model_output: str, chat_completion: ChatCompletion | None = None) -> AssistantMessage
Convert the model output into a structured representation. Convert the model output generated into a structured representation of the information. Args:
  • model_output: String output of the generation request, potentially incomplete if it was a streaming request.
  • chat_completion: The chat completion request that produced model_output. Parameters of the request can determine how the output should be decoded. Defaults to None.
Returns:
  • The parsed output so far, as an instance of :class:AssistantMessage possibly with model-specific extension fields.

​
CLASS ChatCompletionRewriter

Base class for objects that rewrite a chat completion request. Base class for objects that rewrite a chat completion request into another chat completion request.
Methods:

​
FUNC transform 

transform(**kwargs) -> ChatCompletion
Rewrite a chat completion request into another one. Rewrite a chat completion request into another chat completion request. Does not modify the original :class:ChatCompletion object. Args:
  • chat_completion: Original chat completion request, either as a :class:ChatCompletion dataclass, the JSON string representation, or a plain dictionary.
  • **kwargs: Additional keyword arguments forwarded to the underlying :meth:_transform implementation.
Returns:
  • Rewritten copy of the original chat completion request.
Raises:
  • TypeError: If chat_completion is not a :class:ChatCompletion object, a JSON string, or a dictionary.

​
CLASS ChatCompletionResultProcessor

Base class for chat completion result processors. Base class for objects that convert the raw json result of a chat completion request into a JSON object with model-specific postprocessing applied.
Methods:

​
FUNC transform 

transform(self, chat_completion_response: ChatCompletionResponse | dict | pydantic.BaseModel, chat_completion: ChatCompletion | None = None) -> ChatCompletionResponse
Parse and post-process the result of a chat completion request. Args:
  • chat_completion_response: Response to a chat completion request, provided as a parsed :class:ChatCompletionResponse dataclass, a raw dictionary, or another Pydantic model.
  • chat_completion: The original chat completion request that produced chat_completion_response. Required by some implementations to decode references back to the original request. Defaults to None.
Returns:
  • Post-processed copy of the chat completion response with model-specific transformations applied.
Raises:
  • TypeError: If chat_completion_response is not a supported type.

​
CLASS Retriever

Base class for document retrievers. Provides APIs for searching by text snippet and for inserting new documents.
Methods:

​
FUNC retrieve 

retrieve(self, query: str, top_k: int = 10) -> list[Document]
Retrieve the top-k matching documents for a query from the corpus. Args:
  • query: Query string to use for lookup.
  • top_k: Maximum number of results to return. Defaults to 10.
Returns:
  • list[Document]: List of the top-k matching :class:Document objects, each with fields such as text, title, and doc_id.