An automation sends a request to create a customer record. The connection times out before a response arrives. Did the creation fail, or did it succeed while the confirmation was lost?

If the workflow simply tries again, the team may end up with two records. The same ambiguity can produce repeated emails or repeated downstream actions. This matters whether the workflow contains AI or only conventional integration code.

Reliable automation needs a plan for uncertain outcomes, not just a retry button. The examples below are illustrative design patterns rather than claims about a deployed system.

Separate failure from an unknown outcome

A rejected request and a missing response are different situations. Invalid input usually needs correction. A temporary service interruption may justify another attempt. A timeout after submitting a business action may require checking what already happened.

Represent that distinction in the workflow. Useful states include pending, in progress, succeeded, failed, and needs investigation. Avoid treating every non-success response as proof that the destination made no change.

Give the business operation a stable identity

Create an operation identifier when a task is accepted, and keep it across attempts. This is different from an attempt identifier, which changes each time the worker tries.

For example, “create the CRM inquiry for accepted submission 847” is one business operation. Its first and second attempts should not become two unrelated instructions. Avoid putting personal information into identifiers that appear in logs or third-party systems.

Store the operation's status somewhere the next worker can read. A status held only in one process's memory disappears when that process restarts.

Use destination support where it exists

Some APIs support idempotency: repeated attempts can refer to the same operation rather than creating it again. Stripe's idempotent request documentation provides a concrete example of retrying supported operations with an idempotency key.

Do not assume another service has the same behavior. Check its supported endpoints, key lifetime, parameter rules, and response semantics. Generating a new key for every retry defeats the purpose. Reusing an old key for a genuinely different action is wrong too.

An idempotency mechanism at one destination does not automatically protect an entire multi-step workflow.

Coordinate workers, not only requests

Two workers can see the same pending task at once. Both might check that no result exists, then both perform the action. A simple “look before writing” check is not enough when those steps can overlap.

Use an appropriate atomic claim, uniqueness constraint, or destination-supported mechanism for the system involved. Plan for a worker crashing after an external action succeeds but before the local status is saved. That gap is where a reconciliation step becomes important.

Do not describe a workflow as “exactly once” merely because one layer filters duplicate events.

Put limits around retries

For errors the destination documents as retryable, use a bounded policy: a maximum number of attempts, increasing delays where appropriate, and a total deadline. Respect the service's rate-limit and retry guidance.

After the limit, move the task to an exception queue with enough context to investigate. Do not keep trying indefinitely. Record the operation identifier, attempt count, safe error details, and known destination references without copying secrets or unnecessary customer data into logs.

Handle messages with particular care

Sending an email is not the same as updating a field. If a provider accepts a message but the acknowledgement is lost, repeating the send may produce another message. A provider lookup or supported deduplication feature may help, but availability and guarantees vary.

When the outcome cannot be established safely, escalate rather than guessing. The operating team needs to know whether it is investigating a delivery status or authorizing a new send.

Test the awkward moments

  • Submit the same event twice.
  • Let two workers start the same operation.
  • Simulate a timeout after the destination accepts a request.
  • Restart a worker before it records success.
  • Exhaust the retry limit and inspect the exception queue.
  • Resume a failed batch without replaying completed operations.

The acceptance question is practical: can the team explain which business actions occurred and which still require a decision?

Before expanding an automation, pair this test with a clear operating handover. For help turning a brittle prototype into a maintainable workflow, explore prototyping and delivery or email us.