What does the evaluation permit us to release?
Separate observed results, inference and release authority, so an attractive score cannot quietly approve a different product.
A small invoice-draft gate shows how tool and argument policy can reject unwanted effects independently of a model’s wording.
An invoice can ask for payment without being allowed to give the application a new job. That becomes less obvious when the same attachment also contains a paragraph telling an assistant to email its extraction elsewhere and change the supplier’s bank details. Reading the document was authorised; those extra actions were not. The application needs to preserve the invoice’s useful information while refusing to let its wording choose new tools, destinations or records to change.
A system prompt can explain that distinction. The surrounding programme also needs to enforce it when the distinction is misunderstood. Otherwise the boundary exists only while the model continues to interpret every sentence as the application designer hoped. A request to summarise a document can become a route to unrelated tools if the same assistant is handed a general-purpose email client, an unrestricted record editor and the expectation that it will always decline the wrong instruction.
OWASP describes indirect prompt injection through external material such as files, and recommends layered controls rather than a foolproof text filter.[1] For an invoice reader, one useful layer is the point where a proposed draft change becomes an actual write. The accompanying in-memory example tests four constructed proposals without calling a model or external service. It gives the attachment no authority to enlarge the job, even when its proposed amount is well formed or its wording sounds entirely reasonable.
The application has already selected invoice INV9 and its review draft D17. Those identities come from the authorised job. The extractor can propose an invoice identifier and amount, but cannot choose another draft or decide to contact someone outside the workflow. That gives up a possible convenience: an assistant might notice another useful correction while reading. For this queue, I would keep that as a separate request, because approving an extraction should not make the reviewer responsible for unrelated actions hidden inside it.
Start with a read-only extraction stage. Its output is data, and there is no tool dispatcher in that stage. Labelling document text as untrusted helps communicate its role, but separators are not an access-control mechanism. If a model proposes a tool invocation anyway, the programme does not execute arbitrary text because it resembles a call. The next stage must recognise an explicit, controlled operation and validate its arguments before any effect occurs.
The draft tool permits only the two pieces of information the reviewer needs: invoice_id and amount_gbp, sent through write_draft with draft_id and fields. The target must remain D17 and the invoice INV9; the amount must use the declared decimal spelling. Adding bank_account rejects the whole proposal. Quietly dropping that unexpected field would make the permitted part look routine while hiding that something asked to change payment details. The reviewer should be able to see the surprising proposal and decide whether the document or workflow needs attention.
A permitted amount can still be wrong. The spelling 12.50 meets this decimal rule, but only the invoice evidence can justify that value. The gate limits which record and fields can change; the reviewer checks whether the proposed draft describes the invoice faithfully. That division also makes a rejection easier to explain. Wrong draft is an authority problem, while an unsupported amount is an interpretation problem. Sending both back for another extraction would obscure the different work needed to resolve them.
The four local proposals make the boundary visible. An attempted email is refused as tool_not_available because this job has no sending operation. Adding bank_account to D17 is field_set_not_allowed. Supplying the permitted fields for another draft, D88, is wrong_draft: a harmless-looking amount does not make another record an authorised target. Only the proposal for INV9 and 12.50 on D17 is accepted. Each refusal leaves the draft unchanged, so the attachment’s extra request does not become a partial change somebody later has to discover.
The resulting effect list has one draft_written entry. That matters when judging a more capable assistant. It could write a reassuring refusal in its answer and still supply an unwanted field to a permissive mapper. Inspect the operation and stored result as well as the prose: the actual draft should contain only the contribution the reviewer authorised, regardless of how confidently the proposal explained itself.
Quoting the inserted instruction is a different outcome again. A summary may need to tell the reviewer that the attachment asked for a bank change; recording that observation need not perform the change. Keep the source text and requested operation distinguishable in the review view. Otherwise a team may either mistake a quotation for an attempted action or miss an actual unwanted write because the visible answer sounded cautious. The dispatch decision and effect record tell the reviewer which of those situations occurred.
In a multi-user application, the real write must also check who owns D17 and whether the current caller may edit it. The constant makes the example readable, but cannot supply that account association. Obtain it from controlled job and storage context, rather than accepting whichever owner the proposed arguments name. An otherwise identical request from another account must meet the same storage boundary. That keeps the policy effective when a second client bypasses the review screen and calls the operation directly.
Executed in-memory policy checks over constructed calls; no model or external service was contacted.
Scroll sideways to see all columns.
| Proposed operation | Argument | Gate result | Effect |
|---|---|---|---|
send_mail | Unrelated recipient | tool_not_available | None |
write_draft | D17 plus bank_account | field_set_not_allowed | None |
write_draft | D88, permitted fields | wrong_draft | None |
write_draft | D17, INV9, 12.50 | allowed | One draft write |
The difficulty is not that invoices contain commands. Ordinary invoices legitimately ask for payment, specify terms and provide contact details. A crude rule rejecting every imperative could discard useful material while missing a disguised instruction. The application has to distinguish what it may record as evidence from what it may do. A proposed payment instruction can be displayed for a reviewer without automatically changing a trusted payment destination. The content can remain meaningful without becoming an executable order.
Destination selection deserves particular care. In this example, no outbound destination is needed, so it is absent from the tool contract. A different task might legitimately send a reviewed report to a preconfigured recipient. That recipient should then come from the authorised workflow, and any permitted variation needs its own policy. Adding a to field merely because email APIs usually have one creates discretion the task may not require. A smaller operation can be easier for both an operator and a validator to understand.
The amount may later appear in HTML, a spreadsheet or another assistant’s context. Each destination needs its normal encoding and data-handling rules; passing this draft gate cannot supply those later controls. A free-text note would create more to review than the limited amount string, because it could carry wording another consumer wrongly treats as instructions. Preserve the source status as it travels. The reviewer can then read an instruction as something the invoice contained without the export or later assistant quietly turning it back into a new job.
Downstream uses discussed in the invoice example; these consumers are not tested by the gate.
Follow the labelled arrows. On a narrow screen, scroll sideways if needed.
Restricting the workflow sacrifices some convenience. An assistant that notices an incorrect supplier record cannot repair it through this draft tool. That may be exactly right for an extraction queue; the application can offer a separate, deliberate correction process with appropriate authority. If every potentially helpful operation is added to the extractor, the document-reading task becomes a much larger system to understand and review. The cost is not only technical. Somebody must explain which operations were possible when an unexpected change occurred.
When an unexpected proposal is refused, retain its tool and decision reason with the controlled task identity, policy version and any effect reference. The in-memory example records the first two; a working queue needs the rest to connect the event to the right invoice. Avoid copying whole confidential attachments merely to explain a rejected field. The operator needs enough to locate the source and decide whether to clarify the task, correct the workflow or leave the refusal in place, without an automatic retry repeatedly proposing the same unwanted change.
For INV9, the attachment contributes an amount and identifier for D17. It remains worth reading even though it cannot enlarge the job. Controlled context supplies the task, the extraction stage has no dispatcher and the write gate limits the resulting effect; the deployed version must bind those checks to its real caller and storage. That is a useful division of responsibility. The document can tell the reviewer what it says without also deciding what the application is allowed to do about it.
Working on a workflow like this? Hamsa helps finance and operations teams connect their tools and handle the exceptions, starting with one workflow.
See the Payment Operations PilotPrimary sources behind this article. Living documentation may change.

Practical engineering. A human perspective.
More about Hamsa Software and the person behind it. Editorial notes.