System Design

Idempotency in APIs

Why retries create duplicate orders and double charges, which HTTP methods are safe to repeat, and how idempotency keys make any request safe to retry.

Intermediate·10 min read·Updated Sep 27, 2026

An operation is idempotent when doing it once or doing it five times leaves the system in the same state. Networks fail halfway through requests, clients retry, queues redeliver. If your write endpoints are not idempotent, every one of those retries is a duplicate order, a second charge or a double email. The fix is a small, well-understood pattern: an idempotency key plus a stored result.

Why it matters

The network gives you no way to tell "the server never got my request" apart from "the server did the work and the response got lost". Both look like a timeout. A client that never retries loses data on transient failures; a client that retries blindly duplicates side effects. Idempotency is what lets you retry safely, and it is the reason at-least-once delivery, the only kind distributed systems can actually promise, is workable in practice.

The ambiguous failure. From the client's point of view cases 1 and 2 are identical, so a retry must be safe in both.

What HTTP already promises

HTTP defines idempotency per method. This is a contract about the server's state, not about the response: a second DELETE may return 404 instead of 204, and it is still idempotent because the resource is gone either way.

Method	Safe	Idempotent	Why
GET, HEAD, OPTIONS	yes	yes	Read-only by definition. Proxies and browsers retry these freely.
PUT	no	yes	Replaces the whole resource with the payload. Same payload, same end state.
DELETE	no	yes	Resource is absent afterwards no matter how many times you call it.
PATCH	no	depends	"Set status=paid" is idempotent. "Increment balance by 10" is not.
POST	no	no	Creates or triggers. Nothing in the method says a repeat is harmless, so you must add it.

The idempotency key pattern

For POST and non-idempotent PATCH, the client generates a unique key per logical operation (a UUID is fine) and sends it with every attempt of that operation. The server records the key before doing the work, and replays the stored response for any repeat.

ClientIdempotency-Key: 7f3a…

Lookup keyseen before?

No → do workstore key + result atomically

Yes → replayreturn stored response

1
Client creates the key when the user intends the action (clicks Pay), not per HTTP attempt. Every retry of that click reuses the same key.
2
Server tries to insert (key, scope) into an idempotency table with a unique constraint, with status in_progress. A constraint violation means a previous attempt exists.
3
First attempt: perform the operation and save the response (status code and body) against the key, ideally in the same transaction as the side effect.
4
Repeat attempt: if the stored status is completed, return the saved response. If it is still in_progress, return 409 so the client backs off instead of racing the original.
5
Expire keys after a window (Stripe uses 24 hours) so the table does not grow forever.

A minimal implementation

idempotency.sql

CREATE TABLE idempotency_keys (
  key            text        NOT NULL,
  scope          text        NOT NULL,  -- user or account id
  request_hash   text        NOT NULL,  -- to reject key reuse with a different body
  status         text        NOT NULL,  -- 'in_progress' | 'completed'
  response_code  int,
  response_body  jsonb,
  created_at     timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (scope, key)
);

createPayment.ts (Express + Postgres)

app.post('/payments', async (req, res) => {
  const key = req.header('Idempotency-Key');
  if (!key) return res.status(400).json({error: 'Idempotency-Key required'});

  const scope = req.user.id;
  const hash = sha256(JSON.stringify(req.body));

  // 1. Claim the key. The unique constraint is the lock.
  const claimed = await db.query(
    `INSERT INTO idempotency_keys (key, scope, request_hash, status)
     VALUES ($1, $2, $3, 'in_progress')
     ON CONFLICT (scope, key) DO NOTHING
     RETURNING key`,
    [key, scope, hash],
  );

  if (claimed.rowCount === 0) {
    // 2. Someone got here first — replay or reject.
    const {rows: [prev]} = await db.query(
      'SELECT * FROM idempotency_keys WHERE scope = $1 AND key = $2',
      [scope, key],
    );
    if (prev.request_hash !== hash) {
      return res.status(422).json({error: 'Idempotency-Key reused with a different payload'});
    }
    if (prev.status === 'in_progress') {
      return res.status(409).json({error: 'Original request still processing, retry later'});
    }
    return res.status(prev.response_code).json(prev.response_body);
  }

  // 3. First time: do the work and store the result in ONE transaction.
  const result = await db.transaction(async tx => {
    const payment = await chargeCard(tx, req.user, req.body);
    await tx.query(
      `UPDATE idempotency_keys
       SET status = 'completed', response_code = 201, response_body = $3
       WHERE scope = $1 AND key = $2`,
      [scope, key, payment],
    );
    return payment;
  });

  res.status(201).json(result);
});

Three details carry the whole design. The insert is the lock: two concurrent attempts with the same key cannot both claim it, so you never need a distributed mutex. The request hash catches a client bug that reuses a key for a different operation. And the result is committed with the side effect, so a crash between charging and recording cannot leave a key that says "in progress" forever while the money already moved. If your side effect lives in another system (a payment provider), pass the same key downstream so it is idempotent there too, and reconcile stale in_progress rows with a background job.

Design choices you will be asked about

Where to store keys

The same database as the side effect is the safest option, because it gives you the atomic commit above. Redis with SET key value NX EX 86400 is popular for high-volume endpoints and costs one round trip, but then the claim and the side effect are two systems, so you need to handle "claimed in Redis, crashed before the DB write" explicitly.

Scope

Keys are scoped per caller (user, account, API key). Without scope, one tenant could replay another tenant's response by guessing a key, and two independent clients that both picked "1" would collide.

Natural idempotency instead of keys

Sometimes you can make the operation idempotent by shape and skip the header entirely. Modelling "create order" as PUT /orders/{clientGeneratedId} makes the identifier itself the key. Setting a value (status = shipped) instead of applying a delta (stock -= 1) is idempotent by construction. Prefer these when the domain allows; use keys when it does not.

not idempotent

UPDATE products
SET stock = stock - 1
WHERE id = $1;
-- retry twice → stock off by one

idempotent by construction

INSERT INTO reservations (order_id, product_id)
VALUES ($1, $2)
ON CONFLICT DO NOTHING;
-- stock derived from reservations

Pitfalls

Generating the key per HTTP attempt
If the retry loop creates a fresh UUID each time, every attempt looks new and the server dutifully duplicates the work. The key belongs to the user intent, created once and reused until the operation is confirmed.
Check-then-act without a constraint
SELECT to see if the key exists, then INSERT is a race: two concurrent attempts both see nothing. Let the unique constraint or an atomic SET NX do the check.
Storing the key after the side effect
A crash between charging the card and recording the key means the retry charges again. Claim first, then act, then complete, and keep the completion in the same transaction as the effect whenever possible.
Returning a different response on replay
A replay should return the original status and body, including the original resource id. Returning 200 with a new payload, or 409 "already exists" with no id, forces every client to write special-case code and defeats the purpose.
Confusing idempotency with deduplication of identical payloads
Two users legitimately ordering the same product with the same quantity are two orders. Hashing the body to detect duplicates blocks real traffic; the key expresses intent, the hash only guards against key misuse.

Interview questions

Q1What does it mean for an API operation to be idempotent, and why does it matter in distributed systems?

Repeating the operation leaves the system in the same state as doing it once. It matters because a client cannot distinguish a lost request from a lost response, so safe retries are the only way to get reliability without duplicating side effects. At-least-once delivery plus idempotent handlers is how you approximate exactly-once.

Q2Which HTTP methods are idempotent? Is DELETE idempotent if the second call returns 404?

GET, HEAD, OPTIONS, PUT and DELETE are idempotent; POST is not, and PATCH depends on the semantics of the patch. Yes: idempotency is about server state, not the status code. After either call the resource is gone, so the operation is idempotent even though the responses differ.

Q3Walk me through implementing an idempotency key for a POST /payments endpoint.

Client sends a UUID per payment intent in an Idempotency-Key header. The server atomically claims (scope, key) with a unique-constraint insert marked in_progress. On conflict it compares the request hash, then replays the stored response if completed or returns 409 if still in progress. On the first attempt it performs the charge and stores the response in the same transaction, then expires keys after a window.

Q4Two identical requests with the same key arrive at the same millisecond on two servers. What happens?

Both try to insert the key; the database serialises them, one succeeds and one hits the unique constraint. The loser reads the row, sees in_progress and returns 409 (or waits and polls), so the work happens exactly once. This is why the claim must be a single atomic write, not a read followed by a write.

Q5How would you make a message queue consumer idempotent?

Use the message id (or a business key like order id) as the idempotency key and record processed ids in the same transaction as the effect, or design the handler so the effect is naturally idempotent, for example an upsert keyed on the business id or setting absolute values rather than applying deltas. Then redelivery is harmless.

Q6When would you not use idempotency keys?

When the operation is already idempotent by shape (PUT to a client-chosen id, absolute updates, upserts) the header adds storage and complexity for nothing. Also for low-stakes, non-financial actions where a rare duplicate is cheaper than the extra round trip, such as analytics events that are deduplicated downstream anyway.

Key takeaways

A timeout tells you nothing about whether the server did the work, so writes must be safe to retry.
GET, PUT and DELETE are idempotent by contract; POST is not, and infrastructure retries accordingly.
An idempotency key identifies the user's intent, is created once, and is reused across every retry.
Claim the key with an atomic unique insert, do the work, store the response, all in one transaction where possible.
Replay must return the original response, including the original resource id.
Prefer operations that are idempotent by shape (client-chosen ids, absolute updates, upserts) and use keys when the domain will not allow it.

Sources

#Why it matters

#What HTTP already promises

#The idempotency key pattern