Duplicate writes on Vercel Serverless Function retries despite Idempotency-Key usage
0 reputation · 12 Feb 2023, 22:47 UTC
0 reputation · 12 Feb 2023, 22:47 UTC
The goal is to achieve exactly‑once semantics for side‑effects (e.g., writes to a database or KV store) when a Vercel Serverless Function is retried by a client after a transient network error or 5xx response.
Vercel treats each retry as a new invocation with a distinct request ID, so without application‑level guards the same logical request can trigger multiple writes. Common patterns include checking the Idempotency-Key header, storing the key in a short‑lived store, or using Edge Middleware to return a cached response before the function runs. However, proxies may strip or alter custom headers, Edge Middleware adds latency, and ISR background revalidation retries are opaque, making it unclear which strategy provides the strongest guarantee while preserving observability and performance.
What is the most reliable way to detect and skip duplicate processing when the Idempotency-Key header might be modified or removed by intermediaries? Can Edge Middleware be configured to deduplicate retries without noticeably increasing latency for fresh requests? How can developers confirm that ISR background revalidation does not lead to duplicate writes to external stores?
31775 reputation · 13 Feb 2023, 05:35 UTC
Vercel’s retry logic is independent of any client‑supplied header. A retry is triggered when the function returns a 5xx status or times out. If the function performs the write before it records the Idempotency‑Key, the second invocation will repeat the write, creating a duplicate entry.
INSERT … ON CONFLICT DO NOTHING or a transaction that locks the key row.Request‑Id and the Idempotency‑Key in your logs. This lets you trace retries and confirm that the duplicate path is avoided.You can add a middleware that runs before the function:
export default async function middleware(req) {
const key = req.headers.get('Idempotency-Key');
if (!key) return new Response('Missing key', { status: 400 });
const cached = await KV.get(key);
if (cached) return new Response(cached, { status: 200 });
// Pass through to the function; the function will store the key.
return NextResponse.next();
}
Because the middleware executes before the function, it adds only a few milliseconds of latency for fresh requests. It also guarantees that a retry will hit the cache and never reach the function again.
ISR triggers the same endpoint but includes a special header (__vercel_revalidate) or query parameter (?__vercel_revalidate=1). Detect this flag and skip any write logic:
if (req.headers.get('__vercel_revalidate')) {
// Only rebuild the page; do not touch the DB.
return new Response('ISR rebuild', { status: 200 });
}
Alternatively, route ISR calls to a separate handler that only performs the page regeneration.
To fine‑tune the strategy, could you tell me which database or KV store you are using to persist the Idempotency‑Key? That will influence the exact atomic operation syntax.
Use comments to ask for clarification. Post a solution as an answer.
31,775 reputation · 13 Feb 2023, 09:15 UTC
When the Idempotency‑Key might be stripped, rely on a durable KV that supports atomic INSERT … ON CONFLICT DO NOTHING or a transaction. Cloudflare KV or Vercel KV can be queried before the side‑effect, then the key is written in the same transaction that performs the write. This guarantees that only one invocation can succeed, even if two retries hit the store concurrently.
ISR runs without client headers. Protect external writes by basing idempotency on a natural business key (e.g., userId + actionId) and adding a unique constraint on that composite key. If the same revalidation hits the store again, the DB will reject the duplicate write.
Logging the key and the vercel‑request‑id in your audit trail lets you verify that no duplicate side‑effects occur across retries or ISR runs.