What does the evaluation permit us to release?
Separate observed results, inference and release authority, so an attractive score cannot quietly approve a different product.
A workflow can include branches, judgement and AI-written drafts. The useful reason to add an agent is that evidence must change the plan, not that one input is inconvenient.
A beautifully written weekly brief about the wrong company is not a nearly completed brief. If an account lookup returns no unique match, the workflow first needs someone to establish the intended account. A model can make a plausible choice and continue, but plausibility does not authorise that choice. This is a useful place to ask what planning freedom would actually add, before giving the assistant more of it.
The word agent can obscure a smaller design question: which decisions should remain open while the job runs? Some work benefits from choosing the next investigation after seeing a result. Other work has a known route with explicit ways to handle missing information. Asking a model to choose those already agreed steps may give the diagram another impressive component while making the task harder to account for.
Anthropic distinguishes workflows with predefined orchestration from agents that dynamically direct their process and tool use, and recommends adding complexity when it demonstrably helps. [1] That distinction is more useful than counting how many prompts or tools a diagram contains. A workflow may generate prose and branch on missing data. An agent may operate inside strict permissions. The question is where control over the next step lives and why it belongs there.
The teaching workflow starts with an account identifier or a lookup that must return exactly one permitted account. It then retrieves usage for the chosen reporting period and approved account notes. A drafting step summarises those sources into a brief. Finally, the output is checked for the required account, period and source relationships before it is offered for review. Each stage has a purpose which can be stated without asking a model to invent the process.
If two accounts match the supplied name, stop for clarification or use another trusted identifier. If usage is unavailable, the brief can say that the period’s usage was not available, provided the product allows a partial brief. If the notes source is inaccessible, retain that distinction too. These branches are part of the known workflow. They do not require a planner to improvise a substitute source or silently select another company.
An explicit workflow still leaves the drafting step a worthwhile language job. The assistant can select relevant changes, preserve uncertainty and explain them in a brief a colleague will want to read. There is no need to turn the whole thing into a rigid mail-merge to control account identity and reporting dates. Put flexibility where it helps the reader, while keeping the selection and access decisions inspectable outside the draft.
A missing input can reveal a product decision rather than a technical limitation. Should the person choose between matching accounts? Should the brief be saved as incomplete? Is a missing usage figure acceptable for this audience? Decide those behaviours deliberately. An agent that finds a way around every interruption may conceal the exact uncertainty the recipient needs to know about before relying on the brief.
Contrast that with an investigation: why does an export total differ from the local ledger? The initial evidence does not establish whether rows are missing, currencies differ, decimal values changed or a rounding boundary moved. A useful investigator chooses a next check after inspecting what the previous one established. Predefining every possible path may be awkward, especially when the data structure and failure mechanism are initially unfamiliar.
An authored trace might begin by comparing record identifiers. If the sets differ, inspect inclusion rules. If the same rows are present, compare exact amounts and units. If those match before formatting, inspect aggregation and rounding. The evidence changes which check is worth doing next. That is a stronger reason for adaptive planning than the mere fact that a CSV contains an unexpected column.
The investigation trace describes the flexibility worth testing, not a model run or a claim that an agent outperforms a person. A reviewer should be able to follow what was observed, what explanation it suggested and why the next check was chosen. Several tool calls can make a persuasive narrative feel well travelled without establishing its cause. “Found it” still needs a connection between the discrepancy and the mechanism being proposed.
A fixed diagnostic workflow can still suit a narrow discrepancy investigation when every case requires the same four contract checks. The appropriate choice depends on how much the useful next step varies with the evidence. Dynamic planning introduces uncertainty, latency and maintenance of its own, so the comparison needs to establish whether adapting the sequence produces a worthwhile improvement for the actual investigation workload.
Two original task traces distinguish known account-brief preparation from investigation with an initially unknown next step. Neither is a measured model run.
Follow the labelled arrows. On a narrow screen, scroll sideways if needed.
Give the investigating agent read-only access to the minimum relevant synthetic or approved records in a trial. Discovering a likely rounding difference should not automatically permit rewriting posted transactions. Observation, recommendation and effect are separate stages. The agent can assemble evidence for a correction while a controlled application path validates the actual change and its authority.
The same principle limits source substitution. If the account brief requires approved notes, a web search finding another company’s similarly named page is not an equivalent replacement. If the investigation needs a production query that the task does not permit, the right outcome may be a precise request for additional evidence. Flexibility in deciding the next read should not become permission to expand the data boundary whenever progress becomes inconvenient.
Define separate stopping outcomes for the investigation. Completion might require a reproduced discrepancy, supported explanation and proposed correction. A block may identify an unavailable source; budget exhaustion may leave the cause unresolved. Recording these differently prevents the final report from implying that a plausible hypothesis became established simply because no further tool calls were available. The reviewer can then see which additional evidence, if any, would advance the task.
Keep durable task state outside the model’s conversational memory where the workflow needs recovery. Which account and period were authorised? Which sources were inspected? Which effects, if any, were approved or performed? A new attempt should be able to continue from the relevant evidence without interpreting an old prose summary as an unlimited mandate. A planning loop does not remove the ordinary engineering need to account for work.
The export discrepancy task can adapt its next read while leaving mutations controlled.
Inspect inclusion rules
Inspect units and calculation
Request precise additional evidence or report a block
Report unresolved cause and retained evidence
For the brief, compare the explicit workflow with the proposed agent on the same account cases: one match, multiple matches, missing usage and conflicting notes. Does the flexible version improve a meaningful outcome, or does it simply find more ways to produce a complete-looking document? A stop for missing identity can be the correct behaviour. Do not score every interruption as a failure just because autonomy was the feature being demonstrated.
For the discrepancy investigation, include a case where the first explanation is wrong. That gives the comparison something revealing to inspect: does the next observation change the plan, or does the assistant keep decorating its favourite theory? Judge the supported explanation, preserved records and recognition of an unresolved result, then measure the actual effort in the trial. The authored trace supplies no performance figure for an agent or for its alternatives.
Maintenance belongs in the comparison. A fixed path makes its assumptions visible in code but can become cumbersome if variation expands. An agent can handle more varied sequences but needs reliable tool descriptions, boundaries and review of its decisions. The useful design may combine them: explicit intake and authorisation, a bounded investigative loop, then a controlled result or action stage. The division should follow the work rather than a preference for an architectural label.
The account brief and the discrepancy investigation therefore need not share an architecture. The first can keep known identity and retrieval steps around a flexible writing task. The second may benefit when evidence selects the next limited check. A mixed design can retain the same controlled intake and action paths for both. What earns the adaptive part is a useful decision that changes with the observation, rather than the difficulty of persuading an account lookup to return one name.
Working on a workflow like this? Hamsa helps finance and operations teams connect their tools and handle the exceptions, starting with one workflow.
See the Payment Operations PilotPrimary sources behind this article. Living documentation may change.

Practical engineering. A human perspective.
More about Hamsa Software and the person behind it. Editorial notes.