Skip to main content

mellea.stdlib.requirements.rag

Requirements for RAG (Retrieval-Augmented Generation) workflows.

Classes​

CLASS GroundednessRequirement ​

Requirement that validates LLM responses are grounded by citations.

This requirement implements a sophisticated 4-step validation pipeline to ensure that LLM responses are fully grounded by citations to provided documents:

  1. Citation Generation: Generate citations for the response using the find_citations intrinsic.
  2. Citation Necessity: Identify which response spans require citations (vs. conversational/inference text that doesn't need citations).
  3. Citation Support: For spans that need citations, assess the level of citation support: fully, partially, or not supported.
  4. Groundedness Output: Declare response grounded iff all spans needing citations are fully supported.

Important: This requirement requires a HuggingFace backend (LocalHFBackend) as it uses both the find_citations intrinsic and LLM-as-Judge for assessment.

Args:

  • documents: Optional documents to validate against. Can be Document objects or strings (will be converted to Documents). If provided, these documents will be used instead of documents attached to messages in the context. Default is None (use context documents).
  • allow_partial_support: Whether to accept partially supported spans as grounded. If False (default), response is grounded iff all spans needing citations are FULLY supported. If True, response is grounded if spans are fully or partially supported.
  • max_new_tokens: Maximum tokens for LLM judgment outputs. Increase this if LLM outputs are being truncated (particularly for complex responses with many spans). Default is 500.
  • description: Custom description for the requirement. If None, generates a default description.

Methods:

FUNC validate ​

validate(self, backend: Backend, ctx: Context) -> ValidationResult

Validate groundedness of the response using the 4-step pipeline.

Args:

  • backend: Backend to use for citation detection and LLM judgment. Must be LocalHFBackend as this uses find_citations intrinsic.
  • ctx: Context containing the conversation history
  • format: Unused for this requirement
  • model_options: Unused for this requirement

Returns:

  • ValidationResult with pass/fail status and detailed reason