Idempotency key: stop duplicate API retries from becoming duplicate work

An idempotency key is a client-provided identifier that lets a server recognize the same intended operation when a request is retried. It is most useful when the client cannot tell whether the first attempt succeeded.

The classic example is payment creation. The client sends a request, the network times out, and nobody knows whether the payment was created. Retrying without a stable key may create a second payment. Retrying with the same idempotency key gives the server a way to answer, "I have already seen this operation; here is the original result."

## What problem it solves

Retries are necessary in real systems. Networks drop. Browsers close. Workers crash after sending a request but before storing the response. A rate limit may ask the client to wait and try again. A webhook sender may deliver the same event twice. Those are normal failures, not rare edge cases.

The dangerous part is treating every retry as a new command.

An idempotency key separates the user's intent from transport attempts. The intent is "create this order payment once." The transport attempts may be one request, three requests after timeouts, or a replay from a job queue. The server should perform the operation once and make later matching attempts return the same result or a clear conflict.

## Where the key belongs

The key is usually sent in a header such as Idempotency-Key, or sometimes as a request field. It should be generated by the client for one logical operation. It should not be reused across unrelated operations.

Good keys are:

- unique enough to avoid accidental collisions;
- stable across retries of the same operation;
- scoped to the account, API key, or endpoint;
- stored with the resulting operation state;
- retained for a documented time window.

The server should not rely on the key alone. It should also compare the shape of the request. If the same key arrives with a different amount, currency, destination, or order ID, the server should reject it as a conflict rather than silently returning an unrelated prior result.

## What the server stores

A useful idempotency record usually includes:

- key;
- account or API key scope;
- endpoint or operation name;
- normalized request fingerprint;
- result status and response body or result reference;
- creation time and expiry time;
- in-progress marker when the first request is still running.

The in-progress case matters. If two identical requests arrive at nearly the same time, the second should not race ahead and create duplicate work. It should wait, return a safe pending response, or return a conflict/retry response depending on the API contract.

## Idempotency is not the same as deduplication

Deduplication often happens after the fact: "we received two events, let us merge them." Idempotency is a request contract: "if you retry this same command with the same key, we will not perform it twice."

Webhook event IDs are related but different. A webhook receiver stores provider event IDs to avoid processing the same event twice. An API server stores idempotency keys to avoid executing the same client command twice. Both protect against repeated delivery, but they sit on opposite sides of the integration.

## Client rule

Generate one key per user intent. Reuse that key for retries caused by timeout, connection reset, worker crash, or a Retry-After delay. Do not generate a new key for each retry attempt.

If the user changes the amount, destination, item list, or other meaningful input, create a new intent and a new key.

## Server rule

When the key is new, process the operation and store the result. When the key is repeated with the same request fingerprint, return the stored result or the current in-progress state. When the key is repeated with a different request fingerprint, return a conflict.

## Common mistakes

Generating the key on every retry. This makes the header decorative and does not prevent duplicate work.

Using the database primary key as the idempotency key after the object is created. The key must exist before the risky operation.

Keeping keys forever. Long retention may create storage and privacy problems. Short retention may fail slow retries. The retention window should match the operation risk.

Not scoping the key. The same random value from two accounts should not collide globally if account scope is part of the contract.

## Reusable test

If the client sends the same logical command twice because it did not receive the first response, should the server do the work once or twice? If the answer is once, the operation needs an idempotency story before retry logic is considered complete.

Idempotency key: stop duplicate API retries from becoming duplicate work

// COMMENTS

ON THIS PAGE