Skip to main content

mellea.formatters.granite.base.util

Common utility functions for the library and tests.

Functions​

FUNC import_optional ​

import_optional(extra_name: str)

Handle optional imports.

Args:

  • extra_name: Package extra to suggest in the install hint (e.g. pip install mellea[extra_name]).

FUNC find_substring_in_text ​

find_substring_in_text(substring: str, text: str) -> list[dict]

Find all substring matches in text.

Given two strings - substring and text - find and return all matches of substring within text. For each match return its begin and end index.

Args:

  • substring: The string to search for.
  • text: The string to search within.

Returns:

  • List of dicts with begin_idx and end_idx for each match found.

FUNC random_uuid ​

random_uuid() -> str

Generate a random UUID string.

Returns:

  • Hexadecimal UUID string suitable for use as a unique identifier.

FUNC load_transformers_lora ​

load_transformers_lora(local_or_remote_path: str) -> tuple

Load transformers LoRA model.

AutoModelForCausalLM.from_pretrained() is supposed to auto-load base models if you pass it a LoRA adapter's config, but that auto-loading is very broken as of 8/2025. Workaround powers activate!

Only works if transformers and peft are installed.

Args:

  • local_or_remote_path: Local directory path of the LoRA adapter.

Returns:

  • Tuple of (model, tokenizer) where model is the loaded LoRA model and
  • tokenizer is the corresponding HuggingFace tokenizer.

Raises:

  • ImportError: If peft or transformers packages are not installed.
  • NotImplementedError: If local_or_remote_path does not exist locally (remote loading from the Hugging Face Hub is not yet implemented).

FUNC chat_completion_request_to_transformers_inputs ​

chat_completion_request_to_transformers_inputs(request: dict, tokenizer: PreTrainedTokenizerBase, model: PreTrainedModel, constrained_decoding_prefix: str | None = None, ll_tokenizer: llguidance.LLTokenizer | None = None) -> tuple[dict, dict]

Translate an OpenAI-style chat completion request.

Translate an OpenAI-style chat completion request into an input for a Transformers generate() call.

Args:

  • request: Request as parsed JSON or equivalent dataclass.
  • tokenizer: HuggingFace tokenizer.
  • model: HuggingFace model object. Used for model.device placement and when constrained_decoding_prefix is set.
  • constrained_decoding_prefix: Optional generation prefix to append to the prompt.
  • ll_tokenizer: Pre-built llguidance.LLTokenizer. Only used when the request uses constrained decoding; if not provided, one is constructed from tokenizer. Pass an existing instance to avoid the construction cost.

Returns:

  • Tuple of (generate_input, other_input) where generate_input contains
  • kwargs to pass directly to generate() and other_input contains
  • additional parameters for generate_with_transformers.

Raises:

  • ImportError: If torch, transformers, or llguidance packages are not installed (the latter only when constrained decoding is used). TypeError: If tokenizer.apply_chat_template() returns an unexpected type. ValueError: If padding or end-of-sequence token IDs cannot be determined from the tokenizer.

FUNC generate_with_transformers ​

generate_with_transformers(tokenizer: PreTrainedTokenizerBase, model: PreTrainedModel, generate_input: dict, other_input: dict) -> ChatCompletionResponse

Call Transformers generate and get usable results.

All the extra steps necessary to call the :func:generate() method of a Transformers model and get back usable results, rolled into a single function.

There are quite a few extra steps.

Args:

  • tokenizer: HuggingFace tokenizer for the model, required at several stages of generation.
  • model: Initialized HuggingFace model object.
  • generate_input: Parameters to pass to the generate() method, usually produced by chat_completion_request_to_transformers_inputs().
  • other_input: Additional kwargs produced by chat_completion_request_to_transformers_inputs() for aspects of the original request that Transformers APIs don't handle natively.

Returns:

  • A chat completion response in OpenAI format.