Skip to main content
Requirement interface for constrained and validated generation. A Requirement pairs a human-readable description with a validation function that inspects a [Context](base#class-context) (and optionally a backend) to determine whether a model output meets a constraint. ValidationResult carries the pass/fail verdict along with an optional reason, score, and the [ModelOutputThunk](base#class-modeloutputthunk) produced during validation. Helper factories such as default_output_to_bool make it easy to build requirements without boilerplate.

​
Functions

​
FUNC default_output_to_bool 

default_output_to_bool(x: CBlock | str) -> bool
Convert a model output string to a boolean by checking for a “yes” answer. Checks if the output is exactly equal to “yes” or “y” (case-insensitive). If not, also checks if any of the words in the output are “yes” (case-insensitive). Args:
  • x: The model output to evaluate, as a [CBlock](base#class-cblock) or plain string.
Returns:
  • True if the output indicates a “yes” answer, False otherwise.

​
Classes

​
CLASS ValidationResult

ValidationResults store the output of a Requirement’s validation. They can be used to return additional info from validation functions, which is useful for sampling/repairing. Args:
  • result: Boolean indicating whether the requirement passed.
  • reason: Optional human-readable explanation for the verdict.
  • score: Optional numeric score returned by the validator.
  • thunk: The [ModelOutputThunk](base#class-modeloutputthunk) produced during LLM-as-a-Judge validation, if applicable.
  • context: The context associated with the validation backend call, if applicable.
Methods:

​
FUNC reason 

reason(self) -> str | None
Reason for the validation result.

​
FUNC score 

score(self) -> float | None
An optional score for the validation result.

​
FUNC thunk 

thunk(self) -> ModelOutputThunk | None
The ModelOutputThunk associated with the validation func if an llm was used to generate the final result.

​
FUNC context 

context(self) -> Context | None
The context associated with validation if a backend was used to generate the final result.

​
FUNC as_bool 

as_bool(self) -> bool
Return a boolean value based on the validation result. Returns:
  • True if the requirement passed, False otherwise.

​
CLASS Requirement

Requirements are a special type of Component used as input to the Validate step in Instruct/Validate/Repair patterns. Args:
  • description: A natural-language description of the requirement. Sometimes included in Instruction prompts; use check_only=True to suppress this.
  • validation_fn: If provided, this function is executed instead of LLM-as-a-Judge. The bool() of its return value defines pass/fail.
  • output_to_bool: Translates LLM-as-a-Judge output to a boolean. Defaults to a “yes”-detection heuristic.
  • check_only: When True, the requirement description is excluded from Instruction prompts.
Attributes:
  • description: A natural-language description of the requirement.
  • output_to_bool: Function used to convert LLM-as-a-Judge output into a boolean pass/fail result.
  • validation_fn: Optional custom validation function that bypasses the LLM-as-a-Judge strategy entirely.
  • check_only: When True, the requirement description is excluded from Instruction prompts to avoid influencing model output.
Methods:

​
FUNC validate 

validate(self, backend: Backend, ctx: Context) -> ValidationResult
Chooses the appropriate validation strategy and applies it to the given context. Uses validation_fn if one was provided, otherwise falls back to LLM-as-a-Judge by generating a judgement response with the backend. Args:
  • backend: The inference backend used when the LLM-as-a-Judge strategy is selected.
  • ctx: The context to validate, which must contain a [ModelOutputThunk](base#class-modeloutputthunk) as its last output.
  • format: Optional structured output format for the judgement call.
  • model_options: Optional model options to pass to the backend during the judgement call.
Returns:
  • The result of the validation, including a boolean pass/fail and optional metadata.

​
FUNC parts 

parts(self) -> list[Component | CBlock]
Returns all of the constituent parts of a Requirement.

​
FUNC format_for_llm 

format_for_llm(self) -> TemplateRepresentation | str
Returns a [TemplateRepresentation](base#class-templaterepresentation) for LLM-as-a-Judge evaluation of this requirement. Populates the template with the requirement’s description and the stored model _output. Must only be called from within a validate call for this same requirement, after _output has been set. Returns:
  • TemplateRepresentation | str: A [TemplateRepresentation](base#class-templaterepresentation) containing the description
  • and the model output to be judged.