A pause needs more than one meaning

What exactly did the pause button stop?

Pressing Pause can leave the website accepting requests, an earlier booking call waiting for an answer and staff beginning to make bookings by hand. None is necessarily wrong. Together, though, they need an agreement about what stopped and what did not. Before someone confirms a request manually, they must know whether the application merely received it or may already have created a booking elsewhere. One pause button is being asked to answer several different questions.

Manual fallback begins with that initial boundary. Decide separately whether new intake remains open, whether automatic workers may start external writes, and whether authorised people may handle particular requests manually. These controls affect different commitments. Stopping intake can prevent new obligations from accumulating; stopping writes can prevent an unreliable connector from creating more uncertain work. Neither choice retroactively cancels an operation already issued to another system.

The illustrative workflow uses three modes. Normal accepts intake and permits automatic starts. Degraded accepts intake but blocks new automatic external starts; staff may record controlled manual handling. Suspended closes intake and blocks automatic starts while preserving access to record and resolve existing work. These are proposed meanings for this example, not standard settings supplied by every automation vendor. A real pause control needs to be checked against the platform's actual behaviour before these names are attached to it.

Three requests make the boundary visible

One booking was confirmed before the pause. Another request had already gone to the supplier but still has no confirmed result. A third arrives during degraded working and has not been sent automatically. The table identifies them as J10, J11 and J12. Staff need those different histories before choosing what to do: the completed booking needs no repeat, the uncertain one needs investigation, and the untouched arrival can follow the remaining manual service.

In the synchronous Python example, the new arrival is accepted as received while automatic starts are blocked. The earlier issued request remains unresolved. Suspended mode instead rejects a further arrival without creating an intake record. Those local outcomes distinguish receiving work from permission to write. In the deployed service, a worker and colleague also need a coordinated takeover: a mode check is insufficient if manual handling begins before that worker starts its external call.

Staff then book the untouched request manually and record its confirmation against the original intake. The illustrative record keeps the business operation, requested service, acting role, action, external confirmation and resulting status together. A later colleague can distinguish completed manual work from a note asking somebody to look at it. Putting the confirmation in a separate spreadsheet with no intake link would force them to reconstruct whether this request had already been handled.

The earlier issued request stays under investigation. Pausing did not establish whether the supplier accepted it, so making another booking by hand would guess at the unresolved result. The already-confirmed booking needs no repetition either. Give staff a visible distinction between confirmed, issued-unconfirmed and not started before they take over. Restart cannot reconstruct what happened at the supplier if the handover has erased the original attempt and its identity.

Keep the promise at intake honest

A degraded intake response should describe the service currently available. Request received, confirmation to follow is different from booking confirmed. That distinction belongs where the person submits the request, in subsequent messages and in the staff view. A status banner at the top of an internal dashboard does little for somebody who has already received an automatic confirmation email. Review the whole promise, including templates triggered by intake, rather than only the worker that calls the booking API.

Keeping intake open lets people express what they need while the connection is repaired. It also leaves the organisation responsible for following up. If staff cannot provide that service, suspended intake may be the more honest choice. A time-sensitive booking can lose its value before the connector returns, even when the request was stored correctly. Give someone responsibility for deciding whether the remaining response is acceptable and for changing the customer-facing promise. That judgement needs the service owner as well as the engineer who can stop a worker.

Staff handling requests manually need the original request, its current external status and a way to record action against the same identity. Raw webhook logs require too much reconstruction for this role, while unrestricted database editing gives broader access than the task calls for. A small record form or controlled worksheet can be sufficient if entries have explicit meanings and a defined route back into the operational record.

Check that the fallback can actually stand apart from the fault. If exporting pending requests requires the broken automation, or the only manual account belongs to someone off duty, the procedure has inherited the dependency it was meant to avoid. Google’s SRE workbook discusses procedures, escalation and practice for responders.[1] For this small service, that translates into a concrete rehearsal: use the expected people and permissions to retrieve the three booking histories and record a manual confirmation. Borrowing a large organisation’s staffing assumptions would not make those actions available.

A queue state is not the external state

A message waiting in a queue can be useful evidence of local work, but it cannot fully describe whether an external booking exists. For example, Amazon SQS documents messages remaining in the queue while temporarily invisible after receipt, and becoming visible again if not deleted before the visibility timeout. It also warns that standard-queue delivery can still be duplicated.[2] A change to message visibility therefore should not be described as a cancellation of a remote business effect.

The unconfirmed request is where queue controls become particularly misleading. Redelivering its message may create another attempt while the earlier booking remains unresolved at the supplier. Deleting it to clear the queue can instead remove the prompt to investigate. Use the business history beside those delivery observations. The queue says where another instruction may wait; the booking record says what is still owed and what the service actually knows about the result.

A distributed worker can pass a mode check just before another process changes the mode, then carry on writing. The synchronous model does not prevent that race. It specifies the desired decisions, leaving a real design to establish who owns the operation before someone handles it manually. This is why a shared boolean alone cannot prove safe takeover. The initial record and classification are worth doing now, but exclusive ownership and coordinated restart remain work to finish before making that stronger operational promise.

When the connection returns, the manually booked arrival must remain distinguishable from untouched work. The issued request still needs its supplier outcome established. A healthy connector cannot answer either question. Preserve the handover so normal mode continues from those histories, rather than treating every unfinished-looking queue row as permission to book again. That is how the colleague’s manual effort survives recovery instead of becoming the cause of another customer confirmation.

Make the fallback a smaller service people can operate

At low volume, a controlled manual-only service during an outage may be easier to operate than partial automation. It sacrifices speed but can retain a clear record and responsibility. At higher volume, keeping intake open without adequate handling may create commitments staff cannot meet. The decision should reflect the service still available, with requests and confirmations labelled accordingly. Continuing to accept work is useful only when there is a credible way to carry it through.

Revisit the three histories when a template, external write or account model changes. A familiar pause button can interrupt a service that now makes another promise. Keep the completed booking complete, retain ownership of the unconfirmed request and carry the manual confirmation with its original intake. Staff can then resume from work already done instead of repeating it to satisfy the automation’s stale view of the queue.