Skip to main content
Common Pydantic types shared across the Granite formatter package. Defines reusable Pydantic models and mixins, including NoDefaultsMixin (which suppresses unset default fields from serialized JSON output) and message/request types for Granite model chat completions (ChatMessage, ChatCompletion, VLLMExtraBody, ChatCompletionLogProbs, and related classes). These types are consumed internally by the Granite intrinsic formatters.

​
Classes

​
CLASS NoDefaultsMixin

Avoid filling JSON with default values. Mixin so that we don’t need to copy and paste the code to avoid filling JSON values with a full catalog of the default values of rarely-used fields.

​
CLASS UserMessage

User message for an IBM Granite model chat completion request. Attributes:
  • role: Always "user", identifying the message sender.

​
CLASS DocumentMessage

Document message for Granite model chat completion. Document message for an IBM Granite model (from the Ollama library) chat completion request. Attributes:
  • role: A string matching the pattern "document <name>", identifying this message as a document fragment.

​
CLASS ToolCall

Represents a single tool-call entry produced by an assistant message. Captures the identifier, name, and arguments of a tool invocation returned by the model during a chat completion response. Attributes:
  • id: An optional unique identifier for this tool call, used to correlate calls with their results.
  • name: The name of the tool to invoke.
  • arguments: A mapping of argument names to values, conforming to the parameter schema of the associated tool definition.

​
CLASS AssistantMessage

Assistant message for chat completion. Lowest-common-denominator assistant message for an IBM Granite model chat completion request. Attributes:
  • role: Always "assistant", identifying the message sender.
  • tool_calls: Optional list of tool calls requested by the assistant during this turn.
  • reasoning_content: Optional chain-of-thought or reasoning text produced by the model before the final response.

​
CLASS ToolResultMessage

Tool result message from chat completion. Message containing the result of a tool call in an IBM Granite model chat completion request. Attributes:
  • role: Always "tool", identifying this as a tool-result message.
  • tool_call_id: The identifier of the tool call this message responds to.

​
CLASS SystemMessage

System message for an IBM Granite model chat completion request. Attributes:
  • role: Always "system", identifying this as a system-level instruction.

​
CLASS DeveloperMessage

Developer system message for a chat completion request. Attributes:
  • role: Always "developer", identifying this as a developer-role message.

​
CLASS ToolDefinition

An entry in the tools list in an IBM Granite model chat completion request. Attributes:
  • name: The name used to identify and invoke the tool.
  • description: An optional human-readable description of what the tool does.
  • parameters: An optional JSON Schema object describing the tool’s input parameters.

​
CLASS Document

RAG document for retrieval. RAG documents, which in practice are usually snippets drawn from larger documents. Attributes:
  • text: The textual content of the document snippet.
  • title: An optional title for the document.
  • doc_id: An optional string identifier for the document, required by some backends such as vLLM.

​
CLASS ChatTemplateKwargs

Keyword arguments for chat template. Values that can appear in the chat_template_kwargs portion of a valid chat completion request for a Granite model. Attributes:
  • model_config: Pydantic model configuration allowing arbitrary types and extra fields to be passed through to model-specific I/O processors.

​
CLASS VLLMExtraBody

Extra body parameters for vLLM API. Elements of vllm.entrypoints.openai.protocol.ChatCompletionRequest that are not part of OpenAI’s protocol and need to be stuffed into the “extra_body” parameter of a chat completion request. Attributes:
  • documents: RAG documents made accessible to the model during generation, if the template supports RAG.
  • add_generation_prompt: When True, the generation prompt is appended to the rendered chat template. Defaults to True.
  • chat_template_kwargs: Additional keyword arguments forwarded to the chat template renderer.
  • structured_outputs: Optional JSON schema that constrains the model’s output format.

​
CLASS ChatCompletion

Chat completion request schema. Subset of the schema of a chat completion request in vLLM’s OpenAI-compatible inference API that is exercised by Granite models. See the class vllm.entrypoints.openai.protocol.ChatCompletionRequest for more information. Attributes:
  • messages: The ordered list of chat messages forming the conversation history.
  • model: An optional model identifier specifying which model to use for the completion.
  • tools: An optional list of tool definitions made available to the model during generation.
  • extra_body: Optional vLLM-specific parameters not covered by the OpenAI protocol, such as documents and chat-template kwargs.

​
CLASS GraniteChatCompletion

Granite chat completion request. Lowest-common-denominator inputs to a chat completion request for an IBM Granite model.

​
CLASS Logprob

Prompt log-probability from vLLM API. Subset of the vLLM API passing prompt log-probabilities back from vLLM’s OpenAI-compatible server. Note that this is different from the API for token logprobs. See the class vllm.entrypoints.openai.protocol.Logprob for more information. Attributes:
  • logprob: The log-probability value for this token.
  • rank: The rank of this token among the top candidates, if available.
  • decoded_token: The decoded string representation of the token, if available.

​
CLASS ChatCompletionLogProb

Token log-probability from vLLM API. Subset of the vLLM API passing token log-probabilities back from vLLM’s OpenAI-compatible server. Note that this is different from the API for prompt logprobs. See the class vllm.entrypoints.openai.protocol.ChatCompletionLogProb for more information. Attributes:
  • token: The decoded token string.
  • logprob: The log-probability of the token. Defaults to -9999.0 when not returned by the server.
  • bytes: The UTF-8 byte values of the token, if provided by the server.

​
CLASS ChatCompletionLogProbsContent

Token log-probabilities content from vLLM API. Subset of the vLLM API passing token log-probabilities back from vLLM’s OpenAI-compatible server. See the class vllm.entrypoints.openai.protocol.ChatCompletionLogProbsContent for more information. Attributes:
  • top_logprobs: The list of top-k candidate tokens and their log-probabilities at this position.

​
CLASS ChatCompletionLogProbs

Token logprobs for chat completion choice. Subset of the schema of a token logprobs for a single choice in a chat completion result in vLLM’s OpenAI-compatible inference API. See the class vllm.entrypoints.openai.protocol.ChatCompletionLogProbs for more information. Attributes:
  • content: Per-token log-probability entries for each generated token, or None if logprobs were not requested.

​
CLASS ChatCompletionResponseChoice

Single choice in chat completion result from vLLM API. Subset of the schema of a single choice in a chat completion result in vLLM’s OpenAI-compatible inference API that is exercised by Granite intrinsics. See the class vllm.entrypoints.openai.protocol.ChatCompletionResponseChoice for more information. Attributes:
  • index: The zero-based index of this choice in the response.
  • message: The generated message for this choice.
  • logprobs: Token log-probabilities for this choice, if they were requested.
  • finish_reason: The reason the model stopped generating. Defaults to "stop" per the OpenAI specification.

​
CLASS ChatCompletionResponse

Chat completion result from vLLM API. Subset of the schema of a chat completion result in vLLM’s OpenAI-compatible inference API that is exercised by Granite intrinsics. See the class vllm.entrypoints.openai.protocol.ChatCompletionResponse for more information. Attributes:
  • choices: The list of generated response choices returned by the model.
  • prompt_logprobs: Per-token prompt log-probabilities returned by vLLM, if requested. This field is not part of the OpenAI specification.
  • model_config: Pydantic configuration allowing extra fields to be passed through when transforming data.