Skip to main content
Version: Next

mellea.formatters.granite.intrinsics.output

Classes and functions that implement common aspects of output processing for intrinsics.

Classes​

CLASS TransformationRule ​

Base class for transformation rules to apply to JSON outputs of intrinsics.

Args:

  • config: Configuration of the parent output processor, as parsed YAML.
  • input_path_expr: Path expression matching all instances of the field that this rule transforms. Elements can be strings for object fields, integers for list indices, or None for wildcard matches.

Attributes:

  • YAML_NAME: The name used to identify this rule in YAML configuration files. Subclasses must set this to a non-None string.

Methods:

FUNC rule_name ​

rule_name(self) -> str

Return the YAML name that identifies this transformation rule.

Returns:

  • The value of YAML_NAME for this rule subclass.

Raises:

  • ValueError: If YAML_NAME has not been set by the subclass.

FUNC apply ​

apply(self, parsed_json: Any, reparsed_json: Any, logprobs: ChatCompletionLogProbs | None, chat_completion: ChatCompletion | None) -> Any

Apply this transformation rule to the parsed model output.

Args:

  • parsed_json: Output of running model results through json.loads(), plus applying zero or more prior transformation rules.
  • reparsed_json: Output of running the same model results through _json_util.reparse_json_with_offsets(), preserving position information on literal values.
  • logprobs: Optional logprobs result associated with the original model output string, or None if no logprobs were present.
  • chat_completion: The chat completion request that produced this output. Required by some rules.

Returns:

  • Transformed copy of parsed_json after applying this rule.

CLASS InPlaceTransformation ​

Base class for TransformationRules that replace values in place in JSON.

Base class for TransformationRules that replace values in place in the source JSON. The values replaced can be a scalar, object, or list.

CLASS AddFieldsTransformation ​

Base class for TransformationRules that add values to JSON.

Base class for TransformationRules that add one or more values adjacent to an existing value in the source JSON.

CLASS TokenToFloat ​

Transformation rule that decodes token logprobs to a floating point number.

The floating point number replaces the original categorical value in the JSON.

Args:

  • config: Configuration of the parent output processor, as parsed YAML.
  • input_path_expr: Path expression matching all instances of the field that this rule transforms.
  • categories_to_values: Mapping from categorical labels to floating-point values. Defaults to None.

Attributes:

  • YAML_NAME: YAML configuration key for this rule; always "likelihood".

CLASS DecodeSentences ​

Transformation rule that decodes sentence refs into begin, end, text tuples.

Args:

  • config: Configuration of the parent output processor, as parsed YAML.
  • input_path_expr: Path expression matching all instances of the field that this rule transforms.
  • source: Name (or list of names) of the location(s) to look for sentences; each name can be "last_message", "documents", or "all_but_last_message".
  • output_names: Mapping from output role name ("begin", "end", "text", "document_id") to the name of the new field to add in the result JSON.

Attributes:

  • YAML_NAME: YAML configuration key for this rule; always "decode_sentences".
  • begin_name: Name of the output field that receives the sentence begin offset; extracted from output_names, or None if not configured.
  • end_name: Name of the output field that receives the sentence end offset; extracted from output_names, or None if not configured.
  • text_name: Name of the output field that receives the sentence text; extracted from output_names, or None if not configured.
  • document_id_name: Name of the output field that receives the document ID (only used when source="documents"); extracted from output_names, or None if not configured.

CLASS Explode ​

Expand list-valued attributes in a list of records.

Turn each row in a list of records into zero or more rows by expanding the elements of a list-valued attribute.

Args:

  • config: Parsed YAML config for IO processing.
  • input_path_expr: Path expression for the list of records to explode.
  • target_field: Name of list-valued field within each record to expand.

Attributes:

  • YAML_NAME: YAML configuration key for this rule; always "explode".
  • target_field: Name of the list-valued field within each record to expand.

CLASS DropDuplicates ​

Remove duplicate records from a list of records.

Args:

  • config: Parsed YAML config for IO processing.
  • input_path_expr: Path expression for the list of records to deduplicate.
  • target_fields: Names of fields to use for determining whether two records are duplicates.

Attributes:

  • YAML_NAME: YAML configuration key for this rule; always "drop_duplicates".
  • target_fields: Names of fields used to determine whether two records are considered duplicates.

CLASS Project ​

Project records down to a specified set of fields.

Project records down to a specified set of fields. Can also rename the retained fields.

Args:

  • config: Parsed YAML config for IO processing.
  • input_path_expr: Path expression for the list of records to project.
  • retained_fields: Names of fields that remain after the projection. Can be either a list of field names (identity mapping) or a mapping from original field name to the (possibly renamed) output field name.

Attributes:

  • YAML_NAME: YAML configuration key for this rule; always "project".
  • retained_fields: Mapping from original field name to the (possibly renamed) output field name. Initialized from either a list of field names (identity mapping) or an explicit mapping.

CLASS Nest ​

Convert a value within a JSON structure into a record with a single field.

Args:

  • config: Parsed YAML config for IO processing.
  • input_path_expr: Path expression for the values to wrap.
  • field_name: Name of the single field in the JSON object that this rule will wrap around each matching value.

Attributes:

  • YAML_NAME: YAML configuration key for this rule; always "nest".
  • field_name: Name of the single field in the output JSON object that wraps each matching value.

CLASS MergeSpans ​

Merge adjacent spans into larger spans.

Args:

  • config: Parsed YAML config for IO processing.
  • input_path_expr: Path expression for the list of records to merge.
  • group_fields: List of fields used for grouping prior to merging spans.
  • begin_field: Name of the field that holds the begin offset of spans.
  • end_field: Name of the field that holds the end offset of spans.
  • text_field: Optional field containing covered text strings that should be concatenated when spans are merged. Defaults to None.

Attributes:

  • YAML_NAME: YAML configuration key for this rule; always "merge_spans".

CLASS IntrinsicsResultProcessor ​

General-purpose chat completion result processor for intrinsics.

General-purpose chat completion result processor for use with the models that implement intrinsics. Reads parameters of the model's input and output formats from a YAML configuration file and edits the input chat completion appropriately.

Attributes:

  • config: Parsed YAML configuration file for the target intrinsic.
  • rules: Ordered list of transformation rules that this processor applies to each choice in a chat completion response.

Args:

  • config_file: Optional path to a YAML configuration file. Exactly one of config_file and config_dict must be provided.
  • config_dict: Optional pre-parsed YAML configuration dict. Exactly one of config_file and config_dict must be provided.