Describe the exact uncertainty
Suppose a workflow sends an approved request to create order PO-614. The target accepts it, but the connection drops before the response reaches the caller. The local system sees a timeout. Sending the same payload with a newly generated request identifier could create another order.
Amazon’s Builders’ Library explains why caller-provided request identifiers are useful: two identical payloads can represent either a retry or two intentionally separate operations. It also discusses coupling token recording with the relevant mutations and rejecting changed intent under the same identifier. Read the idempotent API design. Verify the particular target’s contract before relying on these properties.
Define the operation record
| Field | Design requirement |
|---|---|
| Operation ID | Generated once for this authorized creation; reused by every attempt. |
| Scope | Organization, target account and operation type. |
| Intent | Accepted request revision and a canonical payload fingerprint. |
| State | Prepared, attempting, uncertain, confirmed or rejected. |
| Result | Target record reference and confirmation evidence. |
Scope prevents one tenant’s identifier from colliding with another’s operation. The fingerprint detects changed intent; it does not replace the operation ID. Two deliberate purchases of the same item need separate IDs even when their payloads match.
Resolve uncertain writes
- Confirmed
- Target reference exists. Use recorded result
- Uncertain
- Response was lost. Reconcile before retry
- Different intent
- Payload changed. Review a new operation
| Evidence | Meaning |
|---|---|
| Confirmed | Target reference exists. Use recorded result |
| Uncertain | Response was lost. Reconcile before retry |
| Different intent | Payload changed. Review a new operation |
Changed intent requires a new decision.
Specify the fingerprint’s normalization rules. Decide how field order, decimals, absent optional fields and units are represented. Keep the original approved payload too. A hash tells you that something differs; readable evidence tells an operator what changed.
Make the local transition concurrency-safe
Persist the prepared operation before attempting the external call. Claim it through a transaction or equivalent atomic conditional update so two workers cannot both act as its owner. Record an attempt identifier separately from the logical operation identifier.
If using an outbox, create the business decision and queued operation together in the local transaction. That closes one local gap: a committed approval without a queued action, or an action without its approval. It does not create an atomic transaction across your database and the external service.
A worker lease expiring does not prove the external effect never occurred. The replacement worker must examine the operation state and target evidence. Do not let a generic retry scheduler turn every abandoned attempt into another creation.
Use the target contract deliberately
Where the target supports an idempotency key, reuse the exact key and approved payload. Record the documented scope, retention window and duplicate-response behavior. A key that expired yesterday may no longer protect a retry today.
If the target permits a client-assigned record reference, use a permitted format and reconcile that reference after uncertainty. Check what the service guarantees about concurrent creation and uniqueness. A searchable memo field is weaker than an enforced unique key.
Where neither capability exists, look up the intended record using strong evidence and route unresolved cases to an operator. A matching amount and date alone may identify several legitimate orders. Do not claim exactly-once execution when the target cannot support or verify it.
Handle changed intent and late results
The buyer changes quantity from 12 to 15 after the original attempt. That is a new business decision. First establish what happened to the 12-unit request, then choose an authorized amendment or cancellation path. Reusing its idempotency key with 15 units would confuse correction with retry.
When a delayed confirmation arrives after manual reconciliation, link it to the same logical operation. Compare target references and flag a contradiction. It should enrich existing evidence rather than create another completed order in the local register.
Prove the behavior with fault injection
Test a lost response after acceptance, duplicate event delivery, concurrent workers, an expired lease, changed payload under the same key and a delayed confirmation after recovery. For each case, inspect both the target records and the operation ledger.
Acceptance tests for a Trion integration should show one intended effect or an owned unresolved operation in both systems.