LLMON (LLM Object Notation)
LLMON (LLM Object Notation, arXiv:2603.22519)
is an LLM-native markup for conveying the structure and semantic metadata of
text to a language model — for example, distinguishing instructions from data to
reduce prompt-injection risk. Mellea can talk to a model that has been trained on
LLMON without the developer hand-building any markup: LLMONFormatter sends each
part of a Mellea component in the message fields the model's chat template turns
into LLMON.
Where the LLMONizing happens
The granite LLMON models (ibm-granite/granite-4.1-3b-llmon-preview and
ibm-granite/granite-4.1-8b-llmon-preview) LLMONize in their chat template. It
writes the blocks, their ids, and the exec block that tells the model what to act
on, all from the fields of each chat message. LLMONFormatter fills those fields
and writes no LLMON itself.
The fields reach the template intact only on LocalHFBackend, which hands it each
message as Mellea built it, so LLMON is supported on that backend alone.
OpenAI-compatible servers such as vLLM, and Ollama, rebuild every message from its
standard keys before applying the template, which drops the fields. So every other
backend raises ValueError when it is given an LLMONFormatter or a registered
LLMON model, rather than silently sending prompts without their LLMON structure.
Using LLMONFormatter
Pass the formatter to the backend:
from mellea.backends.huggingface import LocalHFBackend
from mellea.formatters import LLMONFormatter
from mellea.stdlib.context import ChatContext
from mellea.stdlib.sampling import MultiTurnStrategy
from mellea.stdlib.session import MelleaSession
model_id = "ibm-granite/granite-4.1-3b-llmon-preview"
m = MelleaSession(
LocalHFBackend(model_id=model_id, formatter=LLMONFormatter(model_id)),
ctx=ChatContext(),
)
result = m.instruct(
"What is the warehouse code for Lyon?",
grounding_context={"codes": "The Lyon warehouse code is WH-7Q4Z-LY."},
requirements=["Answer with the code only."],
strategy=MultiTurnStrategy(loop_budget=3), # See "Requirement validation".
)
print(result.value) # Plain text, never LLMON markup.
# Output will vary — LLM responses depend on model and temperature.
LocalHFBackend chooses LLMONFormatter automatically for a registered LLMON
model, which both granite LLMON models are, so formatter= above can be
left out. A model that is neither registered nor given an LLMONFormatter is
unaffected: it gets the usual TemplateFormatter prose.
What the model receives
LLMONFormatter builds each message from the same format_for_llm() args that
TemplateFormatter renders through Jinja2 templates, putting each part in the
field the chat template expects for it. The template turns a user message's content
and instruction items into instr blocks, its data items into data blocks, its
requirement items into req blocks, and its documents into doc blocks, and it
wires every item into the turn's exec block (each is sent with "exec": True).
- Instruction: the prefix and description, joined, are the content, which the
template takes as the instruction. A repair (the feedback
RepairTemplateStrategyadds after a failed check) is a furtherinstructionitem, each requirement arequirementitem, and each in-context example and grounding-context value adataitem. ADocumentin the grounding context is adataitem too, sent as its text alone, without its title. Each grounding-context value is labeled with its key, askey: value, so the model can tell the values apart. An instruction with anoutput_prefixraisesValueError, since the template cannot start the reply with one. - Message: its content, plus a user message's documents as
documentsitems. - Content block: its text, as the instruction.
- Requirement check: the LLM-as-a-judge turn that requirement validation sends
is an instruction to judge, worded as in Mellea's
Requirementtemplate, withOutput: …andRequirement: …asdataitems. In arequirementitem, the requirement would constrain the judge's own reply instead.
Any other component raises TypeError, and so does a component inside an
instruction's args, other than a Document or a model output, which is sent as its
text.
The fields travel in each message's provider_fields, declared for the HuggingFace
backend. print returns a turn's text alone, so it raises for a turn with fields:
raw generation, which sends one string per action, cannot carry them.
System, assistant, and tool turns are sent as their plain text, because the
model's chat template wraps each of them in its own block. A model output in the
history is sent as the model's reply text. An assistant's tool calls travel as the
message's tool_calls, which the chat template writes out and wires into the next
exec block; calls the model wrote in its reply text, as the HuggingFace backend
leaves them, are moved there once the backend has parsed every one. A tool result
is sent as its text for the template to mark as a tool response.
For example, this instruction:
from mellea.stdlib.components import Instruction
Instruction(
"Summarise the review in one sentence.",
requirements=["Mention the shipping."],
grounding_context={"review": "Loved it, but shipping was slow."},
)
becomes a user message whose content is Summarise the review in one sentence.,
with these fields:
{
"data": [{"value": "review: Loved it, but shipping was slow.", "exec": True}],
"requirement": [{"value": "Mention the shipping.", "exec": True}],
}
The granite LLMON models' chat template renders that message as follows
(the template draws each ID at random; newlines added for readability):
<|open|>instr<|:|>ID<|close|>Summarise the review in one sentence.<|open_end|>
<|open|>data<|:|>ID<|close|>review: Loved it, but shipping was slow.<|open_end|>
<|open|>req<|:|>ID<|close|>Mention the shipping.<|open_end|>
<|open|>exec<|:|>ID<|close|>
<|open|>exec<|.|>instruction<|close|>instr<|:|>ID<|open_end|>
<|open|>exec<|.|>input<|close|>data<|:|>ID<|open_end|>
<|open|>exec<|.|>requirement<|close|>req<|:|>ID<|open_end|>
<|open|>exec<|.|>output<|close|>assistant<|:|>ID<|open_end|>
<|open_end|>
<|open|>assistant<|:|>ID<|close|>
Special tokens in content
Content can contain LLMON's special tokens itself: a retrieved web page might
include <|open_end|>, for example. The chat template inserts content as it is, so
that token would close the page's data block, and whatever followed it could open
an instr block of its own. So LLMONFormatter sanitizes content with the llmon
library's LLMONConverters.sanitize, which replaces each LLMON special token with a
look-alike that is not one (<|open_end|> becomes <!open_end!>). That covers the
text of every turn and of every field item, so the model reads the text but never as
structure.
Only what passes through the formatter is sanitized, and only for LLMON's tokens. A
system prompt set with ModelOption.SYSTEM_PROMPT, the tool definitions, and the
arguments of tool calls reach the chat template without passing through it, and
other special tokens, such as end-of-text, are left as they are.
What comes back
The chat template's prompt ends by opening the assistant block, so the model's
reply is its text, then the block's closer and end-of-text. LocalHFBackend decodes
a generation without special tokens, LLMON's among them, so the output's value is
the reply text itself, with no markup to extract. A tool call the model wrote stays
part of the reply, where the backend finds it.
Structured output works as usual: format= is passed to the backend unchanged. A
@generative function is a component of its own, so LLMONFormatter raises for it.
Requirement validation
An instruction's requirements are checked with the LLM-as-a-judge turn described
above, and a sampling strategy repairs an output that fails a check. The strategy
matters on LocalHFBackend, which decodes greedily:
RejectionSamplingStrategy, the default form.instruct, sends the same prompt again, which the model answers the same way, so it cannot fix a miss.MultiTurnStrategykeeps the failed answer in the conversation and adds a message naming the requirements it missed.RepairTemplateStrategysends the instruction again with the failed requirements as a furtherinstructionitem. A second try that fails the same requirements sends the same prompt again.
Use MultiTurnStrategy or RepairTemplateStrategy for an LLMON model. The judge is
unreliable on requirements it has to count (an exact number of words, how often a
word appears), so check those with a validation function instead.
The llmon library dependency
LLMONFormatter uses the llmon converter library (llmon.lang) to sanitize
content. It is an optional dependency:
pip install "mellea[llmon]"
Constructing an LLMONFormatter without it raises an ImportError that names the
extra. Models that are not LLMON models never import it.
See also: Requirements System — how requirements are
validated; Context and Sessions — how m.instruct
reaches the backend that carries the message to the LLMON model.