API retry safety route: idempotency, webhooks, and 429 handling

Webhook retry is the policy for what a receiver or provider does when a webhook delivery fails, times out, or returns an uncertain response.

It looks like a small transport detail until the first duplicate charge, duplicate email, missing invoice update, or silent integration gap appears in production. The hard part is not "retry or do not retry". The hard part is naming what can safely happen twice.

Typical failure scene

A payment provider sends an event. The receiving app starts processing it, writes one row, then times out before returning a clean 2xx response. The provider retries. The app sees the same event again. If the handler is not idempotent, it might create a duplicate order, send a second email, decrement stock twice, or mark an internal job as failed even though the first attempt partially worked.

The opposite failure also happens. A team disables retries because duplicates scared them once. Now a short outage drops important events and nobody notices until reconciliation days later.

Fields a webhook retry policy should name

- event ID or delivery ID used for deduplication
- idempotency key or unique constraint
- retry window and maximum attempts
- backoff schedule
- timeout budget
- safe response code for accepted work
- dead-letter or manual review path
- alert threshold
- replay mechanism
- whether order matters for this event type

The key distinction

Idempotency and retries are not the same thing. Retry says "try delivery again". Idempotency says "the same event can be handled again without changing the result twice." A webhook system needs both words because either one alone can be misleading.

Safe handler shape

The receiver should store the event identity before doing side effects, or make the side effect itself conditional on a unique event identity. It should return success only after the app has either processed the event or safely accepted it for later processing. If the app uses a queue, the queue record should carry the provider event ID, not just an internal job ID.

Common edge cases

Some providers retry for minutes; others retry for days. Some preserve event order; others do not. Some expose manual replay; others only send live deliveries. Some event types are harmless to repeat, while others create money, access, or inventory changes. A useful answer cannot say "just retry"; it has to name the event type and the side effect.

What to log

Log delivery ID, provider event ID, received timestamp, handler result, response code, attempt number, and dedupe decision. Without those fields, the team cannot tell whether it skipped a duplicate, lost an event, or processed the same action twice.

Reusable rule

Do not design webhook retry before naming the side effect. If repeating the event can change money, access, inventory, notification state, or support records, build idempotency first and then choose the retry window.

Webhook retry policy

// COMMENTS

ON THIS PAGE