mellea.stdlib.context.compactor

Generic Compactor protocol for shrinking a Context.

A Compactor returns a fresh, compacted copy of a context. Implementations must never mutate the input — by convention, every alteration must produce a new Context instance (the base class enforces this via from_previous).

Two usage patterns are supported:

• Pattern 1 (in Context.add): A subclass of Context holds a Compactor and applies it whenever a new component is appended.
• Pattern 2 (manual): The caller invokes compactor.compact(ctx) directly between turns, e.g. when compaction is exposed to the model as a tool.

See docs/examples/context/ for full usage examples.

Functions

FUNC pin_nothing

pin_nothing(components: list[Component | CBlock | ModelOutputThunk]) -> int

A :class:PinPredicate that pins nothing — pure body, no protected prefix.

Args:

• components: The full ordered list of context components. Unused; the argument exists to satisfy the :class:PinPredicate signature.

Returns:

• Always 0, meaning the entire context is body and nothing is pinned.

FUNC pin_system

pin_system(components: list[Component | CBlock | ModelOutputThunk]) -> int

Pin contiguous leading Message(role="system") components.

Stops at the first non-system component. A system message that appears later in the conversation is not pinned.

Args:

• components: The full ordered list of context components.

Returns:

• Index after the contiguous leading system messages — i.e. the
• length of the system-message prefix. 0 if the first component
• is not a system message.

FUNC pin_system_and_initial_user

pin_system_and_initial_user(components: list[Component | CBlock | ModelOutputThunk]) -> int

Pin leading system messages PLUS the first user message that follows.

Useful when the initial user prompt encodes the goal of the conversation and should survive compaction along with any system instructions.

Args:

• components: The full ordered list of context components.

Returns:

• Index after the system-message prefix plus the first user message
• that immediately follows it. Equal to :func:pin_system's result
• when no user message follows the system prefix.

Classes

CLASS Compactor

Protocol for objects that compact a Context into a smaller copy.

A compactor receives a context and returns a new context that retains only the data the strategy considers worth keeping. Implementations MUST NOT mutate the input context; they must return a fresh instance and copy over any data that should be preserved.

compact() is generic in T (a Context subtype) so concrete compactors can narrow their input/output type — for example a chat-only compactor overrides the method as def compact(self, ctx: ChatContext, *, backend=None) -> ChatContext.

The protocol is sync. Compactors that need to perform a backend call (e.g. :class:LLMSummarizeCompactor) hide the async work behind the sync method internally — see that class for the strategy used.

Methods:

FUNC compact

compact(self, ctx: T) -> T

Return a compacted copy of ctx.

Args:

• ctx: The context to compact. Must be left unchanged.
• backend: Optional backend. Generic compactors that only filter components can ignore it.

Returns:

• A new context of the same type as ctx containing only the
• retained data.

CLASS InlineCompactor

Marker base for compactors safe to attach directly to ChatContext.

A compactor is "inline-safe" when its compact() does not call a backend on every add(). ChatContext.add() invokes compact() without a backend argument, so any compactor wired into ChatContext(compactor=...) must either avoid backend calls (e.g. :class:WindowCompactor) or gate them sparsely (e.g. :class:ThresholdCompactor). Compactors that would invoke the backend on every add() (e.g. :class:LLMSummarizeCompactor) must NOT inherit this marker — use them via react(compactor=...) or by calling compact(ctx, backend=...) manually instead.

The marker is purely nominal: opt in by inheriting, opt out by not. Pure structural :class:Compactor Protocol satisfaction is not enough.

Subclasses must override :meth:compact; the base implementation raises :class:NotImplementedError. Carrying the method signature here lets InlineCompactor be used as a static type (ChatContext parameters, _compactor attribute) without losing the Compactor contract.

Methods:

FUNC compact

compact(self, ctx: ChatContext) -> ChatContext

Subclasses must override this with their concrete strategy.

Args:

• ctx: The chat context to compact.
• backend: Optional backend for compactors that need one. Inline compactors typically ignore this argument.

Returns:

• A compacted ChatContext. The base implementation never
• returns — see Raises.

Raises:

• NotImplementedError: Always — InlineCompactor is a marker base class; concrete subclasses are expected to provide their own compact() implementation.

CLASS WindowCompactor

Retains the last size body components of a ChatContext.

Uses pin_predicate to decide which leading components to preserve as a protected prefix; the size limit is then applied to the body that remains. The total context length after compaction is len(prefix) + min(size, body_len). size counts only body components.

When the body is already at or below size, ctx is returned unchanged so the original linked-list and previous_node chain are preserved. The result carries the same Compactor as the input so subsequent add() calls keep compacting.

Args:

• size: Maximum number of most-recent body components to retain. Pinned prefix components do NOT count against this budget. size=0 is a special case that drops the body entirely, keeping only the pinned prefix. Negative values raise :class:ValueError.
• pin_predicate: Function that decides the prefix boundary. Defaults to :func:pin_system, which pins contiguous leading Message(role="system") components. Pass :func:pin_nothing for pure last-N behaviour or any other PinPredicate (e.g. :func:pin_system_and_initial_user).

Methods:

FUNC compact

compact(self, ctx: ChatContext) -> ChatContext

Return a copy of ctx truncated to the last size body components.

Args:

• ctx: The chat context to compact.
• backend: Unused by this strategy; accepted for protocol compatibility.

Returns:

• A new ChatContext whose history is the pinned prefix plus the
• last size body components, carrying ctx's compactor.
• Returns ctx itself if no truncation is required.

CLASS ThresholdCompactor

Wraps an inner Compactor, gating it on the conversation's token size.

Despite the suffix, this class does not compact directly — it forwards to inner.compact only when the conversation has grown larger than threshold tokens; otherwise the input is returned unchanged.

The token measurement is read off the most recent ModelOutputThunk's generation.usage (via :func:_last_usage_tokens). Because chat backends report prompt_tokens as the size of the full history they were given as input, total_tokens = prompt_tokens + completion_tokens on the latest thunk effectively measures the size of the conversation after that turn, not just one isolated call. So the gate fires once cumulative context size crosses threshold.

Caveats:

• Components appended after the last thunk (e.g. a tool response in the same turn) are not yet reflected in the reading — there is a one-turn lag, negligible unless a single tool call adds a very large payload.
• When the inner compactor shrinks the context, the next model call will produce a smaller prompt_tokens, so the gate will close again. The threshold is not a high-water mark.
• Returns the input unchanged if no thunk with usage is found yet (typical before the first model call completes).

Args:

• inner: The compactor to invoke once the threshold is exceeded.
• threshold: Trigger the inner compactor when the conversation's measured token size (most recent thunk's total_tokens) exceeds this value. 0 or negative disables the gate (the inner is never invoked).

Methods:

FUNC compact

compact(self, ctx: ChatContext) -> ChatContext

Forward to inner.compact only when ctx exceeds the threshold.

Args:

• ctx: The context to potentially compact.
• backend: Forwarded to the inner compactor.

Returns:

• inner.compact(ctx, backend=backend) when the recovered token
• count exceeds self.threshold, otherwise ctx unchanged.

CLASS LLMSummarizeCompactor

Replace old body components with an LLM-generated summary, keep last keep_n verbatim.

Implements the sync :class:Compactor protocol. The compactor's body needs to call the (async) backend; that async work is hidden inside the sync compact method via :func:_run_coro_blocking. The pinned prefix (chosen by pin_predicate) is preserved unchanged; body components older than the last keep_n are flattened into a single Message(role="user") whose content is a structured summary; the last keep_n body components are kept verbatim.

Default pin_predicate is :func:pin_nothing, which means the entire conversation participates in summarisation. For react workflows pass :func:mellea.stdlib.components.react.pin_react_initiator so the goal and tool registration survive untouched.

Args:

• default_backend: Backend used by compact() when the caller does not supply one. Required: LLMSummarizeCompactor cannot do its job without a backend at compaction time. A backend= kwarg passed to compact() overrides this default for that call only.
• keep_n: Number of recent body components to keep verbatim. 0 summarises everything below the prefix.
• pin_predicate: Function that decides the prefix boundary. Defaults to :func:pin_nothing.
• prompt_template: Custom summary prompt. Must contain the literal \{conversation\} placeholder, which is filled in with a textual rendering of the body to summarise. Defaults to a generic conversation-summary template.
• model_options: Forwarded to mfuncs.aact for the summarisation call. Use this to set a real max_tokens budget (most local backends default to 256-512, which silently truncates long summaries) or any other backend-specific knob. Note: :func:react_summary_prompt's max_tokens_hint adds only a soft prompt-side nudge; pair it with model_options=\{"max_tokens"\: N\} for hard enforcement.

Methods:

FUNC compact

compact(self, ctx: ChatContext) -> ChatContext

Return a context with the prefix, an LLM summary, and recent body components.

Args:

• ctx: The chat context to compact.
• backend: Backend used to generate the summary. When None the default_backend set at construction is used instead.

Returns:

• A new ChatContext containing the prefix, a single summary
• Message produced by the backend, and the most-recent
• keep_n body components verbatim. Returns ctx unchanged
• when the body is already at or below keep_n in length, or
• when the backend call fails (see Note).