Skip to main content
Version: Next

mellea.formatters.granite.base.util

Common utility functions for the library and tests.

Functions​

FUNC import_optional ​

import_optional(extra_name: str)

Handle optional imports.

Args:

  • extra_name: Package extra to suggest in the install hint (e.g. pip install mellea[extra_name]).

FUNC random_uuid ​

random_uuid() -> str

Generate a random UUID string.

Returns:

  • Hexadecimal UUID string suitable for use as a unique identifier.

FUNC load_transformers_lora ​

load_transformers_lora(local_or_remote_path: str) -> tuple

Load transformers LoRA model placed on the best available device.

AutoModelForCausalLM.from_pretrained() is supposed to auto-load base models if you pass it a LoRA adapter's config, but that auto-loading is very broken as of 8/2025. Workaround powers activate!

Device selection mirrors LocalHFBackend: CUDA → MPS → CPU. The returned model is already on that device; callers do not need to move it.

Only works if transformers and peft are installed.

Args:

  • local_or_remote_path: Local directory path of the LoRA adapter.

Returns:

  • Tuple of (model, tokenizer) where model is the loaded LoRA model placed on
  • the best available device, and tokenizer is the corresponding Hugging Face
  • tokenizer.

Raises:

  • ImportError: If peft or transformers packages are not installed.
  • NotImplementedError: If local_or_remote_path does not exist locally (remote loading from the Hugging Face Hub is not yet implemented).

FUNC chat_completion_request_to_transformers_inputs ​

chat_completion_request_to_transformers_inputs(request: dict, tokenizer: PreTrainedTokenizerBase, model: PreTrainedModel, constrained_decoding_prefix: str | None = None, ll_tokenizer: llguidance.LLTokenizer | None = None) -> tuple[dict, dict]

Translate an OpenAI-style chat completion request.

Translate an OpenAI-style chat completion request into an input for a Transformers generate() call.

Args:

  • request: Request as parsed JSON or equivalent dataclass.
  • tokenizer: Hugging Face tokenizer.
  • model: Hugging Face model object. Used for model.device placement and when constrained_decoding_prefix is set.
  • constrained_decoding_prefix: Optional generation prefix to append to the prompt.
  • ll_tokenizer: Pre-built llguidance.LLTokenizer. Only used when the request uses constrained decoding; if not provided, one is constructed from tokenizer. Pass an existing instance to avoid the construction cost.

Returns:

  • Tuple of (generate_input, other_input) where generate_input contains
  • kwargs to pass directly to generate() and other_input contains
  • additional parameters for generate_with_transformers.

Raises:

  • ImportError: If torch, transformers, or llguidance packages are not installed (the latter only when constrained decoding is used). TypeError: If tokenizer.apply_chat_template() returns an unexpected type. ValueError: If padding or end-of-sequence token IDs cannot be determined from the tokenizer.

FUNC generate_with_transformers ​

generate_with_transformers(tokenizer: PreTrainedTokenizerBase, model: PreTrainedModel, generate_input: dict, other_input: dict) -> ChatCompletionResponse

Call Transformers generate and get usable results.

All the extra steps necessary to call the :func:generate() method of a Transformers model and get back usable results, rolled into a single function.

There are quite a few extra steps.

Args:

  • tokenizer: Hugging Face tokenizer for the model, required at several stages of generation.
  • model: Initialized Hugging Face model object.
  • generate_input: Parameters to pass to the generate() method, usually produced by chat_completion_request_to_transformers_inputs().
  • other_input: Additional kwargs produced by chat_completion_request_to_transformers_inputs() for aspects of the original request that Transformers APIs don't handle natively.

Returns:

  • A chat completion response in OpenAI format.