hamsa software
← All articles
Automation7 min read

The difficult part of doing it manually

A manual fallback needs a transfer of control. A paused worker, a stale claim and an external booking show why.

Two offset rectangles are joined by several short parallel stitches, a geometric metaphor for the boundary between automatic work and manual takeover.

The switch that changes very little

An operator sees a stalled courier job and books the collection on the courier’s website. Later, the automatic worker wakes up and submits its original request. Both routes have completed the task, which is precisely the problem: the business wanted one collection. Before offering a manual fallback, I would want to know what stops the worker already in progress from joining in after the operator takes over.

The courier integration here is hypothetical. Its worker can hold a claimed collection while a network problem pauses progress, and the operator has a separate route through the courier website. Blocking new claims on takeover deals with workers that have not started; it leaves the earlier claimant to account for. The fallback needs to identify that attempt and its possible external result before treating the manual route as the only sender left.

The fallback needs to transfer control. A second screen for doing the same task does not arrange that transfer, and a button labelled manual cannot instruct a paused worker that has not read the new state. The screen, database and operating instructions need to agree on when the operator may act. Until then, the manual route is another possible sender rather than a safe replacement for the first one.

I would start the design with one question: what evidence allows the operator to conclude that it is safe to act? The answer may be a confirmed cancellation, an enforced ownership boundary or a reconciled external outcome. “The worker has probably given up” is an observation worth recording, but an awkward foundation for another booking.

Follow the worker that was already running

Consider an illustrative record for collection C-81. Its control version is 7, its owner is automation and worker W-4 has claimed the work. At 10:00 the worker prepares a request, then pauses. At 10:01 an operator requests manual takeover. The application changes the record to pause_requested and version 8. New workers refuse to claim it. So far, this is useful progress.

At 10:02 W-4 resumes. If it sends before checking the new record, the courier can accept the booking. The database may then reject the worker’s version 7 update, correctly protecting the local row from stale information. That is useful, but it happens after the external action we needed to control. The rejected write has not recalled the collection request. Keep that limit visible when deciding what the version check has actually protected.

Checking the version immediately before sending narrows the gap, although takeover can still occur between that check and the external action. A database row lock has a similarly defined reach: PostgreSQL describes competing database operations being blocked until the transaction ends. It does not extend that transaction to the courier API. Holding it open while an operator reviews the screen would also keep those competing database operations waiting without establishing what the courier has accepted.

The handover record therefore needs separate evidence that new automatic claims are blocked and that earlier attempts have been accounted for. Those conditions may become true at different times. Showing both lets the operator understand why a takeover request has been accepted but a new external booking is not yet permitted, rather than inferring readiness from a single changed status.

A takeover request does not stop an earlier send

An illustrative interleaving for collection C-81. Local version checks and external effects have different boundaries.

  1. 10:00 · version 7Automatic

    W-4 claims C-81 and pauses before its external send.

  2. 10:01 · version 8Pause requested

    Takeover blocks new claims; W-4 remains unresolved.

  3. 10:02 · stale workerExternal risk

    W-4 sends before observing the new authority. A rejected local update cannot retract the booking.

  4. Before manual actionReconcile first

    Account for the earlier attempt and establish the permitted next operation.

Synthetic sequence. A receiver-enforced fence can reject stale future writes; it cannot undo a booking already accepted.

Authority has a number; the booking has an identity

A version number can support a conditional local transition. For example, move C-81 from automatic version 7 to pause_requested version 8 only if the current owner and version still match. If no row changes, refresh the evidence rather than assuming the takeover succeeded. This makes competing local decisions visible and gives one decision a clear place in the record.

For an external receiver, a fencing token can provide a stronger boundary: a monotonically increasing authority number that the receiver checks to reject older requests. Martin Kleppmann's discussion of distributed locking explains why an expired lease alone cannot stop a delayed process from writing. This requires receiver enforcement. Merely including an epoch in the request leaves the delayed worker able to act if the courier ignores it.

Even proper fencing needs careful interpretation. Suppose the courier accepted a request bearing authority 7 before it saw authority 8. Rejecting subsequent requests from 7 does not remove the booking already created. Nor should authority 8 automatically create another booking. Authority answers who may act now; the business operation identity answers which booking this action belongs to. These are related questions, but they are not interchangeable identifiers.

Control of C-81 needs both an ownership rule and a way to recognise the collection it represents. Use a provider’s documented deduplication or stable-reference lookup where available. If neither exists, the operator may need courier support or an explicit wait before making a replacement booking. That is inconvenient, but the missing acknowledgement is real. The queue being awkward does not give the application any new information about whether the van is already booked.

New authority does not erase an earlier booking

The fencing example separates permission to send from the identity of the collection already created.

Follow the labelled arrows. On a narrow screen, scroll sideways if needed.

New authority does not erase an earlier bookingRequest under authority 7. A booking may already exist. Retain its business-operation identity. Receiver later sees authority 8. Receiver-enforced fencing can reject subsequent older requests. The earlier booking still needs reconciliation. Operator acts under authority 8. Current authority does not permit automatically creating a second booking. Establish the earlier outcome first. From Request under authority 7 to Receiver later sees authority 8: Earlier effect remains. From Receiver later sees authority 8 to Operator acts under authority 8: Reconcile before replacement.Earlier effectremainsReconcilebeforereplacementRequest under authority7A booking may already exist.Retain its business-operationidentity.Receiver later seesauthority 8Receiver-enforced fencing canreject subsequent olderrequests. The earlier bookingstill needs reconciliation.Operator acts underauthority 8Current authority does notpermit automatically creatinga second booking. Establishthe earlier outcome first.
This requires receiver-enforced fencing. A local version field alone cannot supply the external guarantee.

Make the pause an observable process

A useful state model might contain automatic, pause_requested, reconciling and manual_ready. These are proposed application states, not a universal protocol. Their value lies in the evidence required to move between them. pause_requested blocks new claims; reconciling accounts for existing attempts; manual_ready grants an operator a specific action against a specific revision of the record.

For C-81, the reconciliation view should show the worker claim, the last attempted external reference and any acknowledgement already received. An operator may discover a valid courier booking and attach its reference rather than reproduce it. Alternatively, the integration may establish that no request left the system. Those outcomes support different next actions, so a single green “takeover complete” tick would discard useful information.

If the external request remains unresolved, manual_ready must not quietly grant permission to create another collection. Offer the investigation the operator can actually perform: check status later, contact support or inspect the existing attempt. Explain which uncertainty still prevents the booking. A delayed collection is undesirable; adding an unrecorded duplicate merely gives the team a second customer obligation to sort out alongside the first.

The request to take over should also survive a page refresh. Otherwise a colleague can mistake a disappearing spinner for a rejected request and start again from another screen. Persist the decision before presenting it as accepted.

A durable handover note can record the operation reference, control version, known external result, permitted action and person taking responsibility. Those fields explain the particular decision the operator is making and let another colleague continue later. The note is especially useful when the investigation spans several interfaces, because it preserves which evidence established the next action rather than requiring the next person to reconstruct it.

Returning control is another handover

After making the manual booking, the operator should attach its external reference to C-81. Automation can then reconcile that result and continue from the appropriate stage. Returning the original booking instruction to the queue would disregard the completed manual action. Resumption therefore needs the same attention to operation identity as the initial takeover, even if the worker ownership has already changed cleanly.

Abandoning a takeover also requires an outcome record. The courier may have accepted a form even if its confirmation page failed, so cancelling the local handover cannot establish that no booking exists. Ask for the evidence available and retain an unresolved result where needed. The control state must represent that uncertainty until investigation supports either continuing the original operation or recording its completion.

There is a legitimate trade-off here. For a low-impact, reversible task, a simpler exclusive local claim and an obvious duplicate check may be sufficient. For an irreversible or costly external action, delayed manual readiness can be the more honest design. The decision belongs to the actual consequence of overlap, the receiver’s capabilities and the effort of recovery. More state is not automatically better; unexplained state is especially unhelpful.

Pause C-81 before the send, after acceptance but before acknowledgement, and after the manual booking but before automation resumes. Each interruption leaves a different person able to act on different information. A useful fallback explains those differences and offers a supported continuation at each one. The operator should be able to tell whether they are taking responsibility for a new collection or recording one that already exists, without having to guess whether the automatic worker has finally stopped.

Working on a workflow like this? Hamsa helps finance and operations teams connect their tools and handle the exceptions, starting with one workflow.

See the Payment Operations Pilot

References & further reading

Primary sources behind this article. Living documentation may change.

  1. Martin Kleppmann: How to do distributed locking Published 8 February 2016
  2. PostgreSQL: Explicit locking
Hamsa Jama

Practical engineering. A human perspective.

More about Hamsa Software and the person behind it. Editorial notes.

A few more things
worth a read.

Back to the reading room
Automation6 min read

A budget that three workers can share

Enforce task limits through atomic reservations and repeat-safe settlement, including the uncomfortable case where a timed-out operation may still be spending.

Automation7 min read

What exactly did that approval authorise?

Bind an assistant’s approval to the action, target and material revision, then enforce that binding when the write happens.