Skip to main content
Abstract interfaces for sampling strategies and their results. SamplingStrategy defines the contract for all sampling algorithms: an async sample method that takes an action, context, backend, and requirements, and returns a SamplingResult. SamplingResult records the chosen generation alongside the full history of intermediate samples, their validation outcomes, and associated contexts — enabling detailed post-hoc inspection of the sampling process.

​
Classes

​
CLASS SamplingResult

Stores the results from a sampling operation. This includes successful and failed samplings. Args:
  • result_index: Index into sample_generations identifying the chosen final output.
  • success: Whether the sampling operation produced a passing result.
  • sample_generations: All output thunks generated during sampling.
  • sample_validations: Per-generation validation results; each inner list contains one tuple per requirement evaluated.
  • sample_actions: The actions used to produce each generation.
  • sample_contexts: The contexts associated with each generation.
Attributes:
  • result_index: Index into sample_generations identifying the chosen final output.
  • success: Whether the sampling operation produced a passing result.
  • sample_generations: All output thunks generated during sampling; always a list (None input is normalised to []).
  • sample_validations: Per-generation validation results; always a list (None input is normalised to []).
  • sample_actions: The actions used to produce each generation; always a list (None input is normalised to []).
  • sample_contexts: The contexts associated with each generation; always a list (None input is normalised to []).
Methods:

​
FUNC result 

result(self) -> ModelOutputThunk[S]
The final output or result from applying the sampling strategy.

​
FUNC result_ctx 

result_ctx(self) -> Context
The context of the final output or result from applying the sampling strategy.

​
FUNC result_action 

result_action(self) -> Component[S]
The action that generated the final output or result from applying the sampling strategy.

​
FUNC result_validations 

result_validations(self) -> list[tuple[Requirement, ValidationResult]]
The validation results associated with the final output or result from applying the sampling strategy.

​
CLASS SamplingStrategy

A SamplingStrategy class defines an abstract base class for implementing various sampling strategies. This class provides a template for creating concrete sampling strategies that can be used to generate model outputs based on given instructions. It allows setting custom validation and generation functions through properties.
Methods:

​
FUNC sample 

sample(self, action: Component[S], context: Context, backend: Backend, requirements: list[Requirement] | None) -> SamplingResult[S]
This method is the abstract method for sampling a given component. It must be implemented by any concrete subclasses to provide specific sampling logic. Args:
  • action : The action object to be sampled.
  • context: The context to be passed to the sampling strategy.
  • backend: The backend used for generating samples.
  • requirements: List of requirements to test against (merged with global requirements).
  • validation_ctx: Optional context to use for validation. If None, validation_ctx = ctx.
  • format: output format for structured outputs.
  • model_options: model options to pass to the backend during generation / validation.
  • tool_calls: True if tool calls should be used during this sampling strategy.
Returns:
  • A result object indicating the success or failure of the sampling process.