Idempotency by Design: APIs That Survive Duplicate Requests

Your payment API just processed a transaction. The database commit succeeded. But the response never reached the client—network blip. The client retries. Now you've charged the customer twice, and you're fielding support tickets instead of shipping features.

This scenario plays out across production systems constantly. The fix isn't complex, but it requires deliberate design. Idempotency—the property that repeated identical requests produce the same result—isn't optional for reliable APIs. It's foundational.

Why Idempotency Matters

Network unreliability is a given, not an exception. Client libraries retry on timeout. Load balancers retry on connection resets. Users refresh pages. Your API will receive duplicate requests. The question is whether you're prepared.

Without idempotency guarantees, you get:

• Duplicate transactions: Multiple charges, duplicate orders, corrupted balances
• Inconsistent state: Different clients see different results for the same operation
• Silent failures: Retries that look successful but create phantom records

Idempotency is the contract that says: "Run this request once or a hundred times—the outcome is identical."

The Idempotency Key Pattern

The standard solution is the idempotency key: a unique identifier provided by the client that you store server-side. If the same key arrives twice, you return the cached result instead of reprocessing.

Implementation

typescriptCopy
import { v4 as uuidv4 } from 'uuid';

interface IdempotencyStore {
  get(key: string): Promise<CachedResponse | null>;
  set(key: string, response: CachedResponse, ttl: number): Promise<void>;
}

interface CachedResponse {
  statusCode: number;
  body: any;
  timestamp: number;
}

async function handlePayment(
  req: Request,
  idempotencyStore: IdempotencyStore
): Promise<Response> {
  const idempotencyKey = req.headers.get('Idempotency-Key');
  
  if (!idempotencyKey) {
    return new Response('Missing Idempotency-Key header', { status: 400 });
  }

  // Check if we've seen this request before
  const cached = await idempotencyStore.get(idempotencyKey);
  if (cached) {
    return new Response(JSON.stringify(cached.body), {
      status: cached.statusCode,
      headers: { 'X-Cached-Response': 'true' },
    });
  }

  // Process the payment
  const result = await processPayment(req.body);
  
  // Cache the response
  await idempotencyStore.set(idempotencyKey, {
    statusCode: 200,
    body: result,
    timestamp: Date.now(),
  }, 86400); // 24-hour TTL

  return new Response(JSON.stringify(result), { status: 200 });
}

Client-Side Example

Clients should generate and include the idempotency key:

pythonCopy
import requests
import uuid

def make_payment(amount, recipient):
    idempotency_key = str(uuid.uuid4())
    
    response = requests.post(
        'https://api.example.com/payments',
        json={'amount': amount, 'recipient': recipient},
        headers={'Idempotency-Key': idempotency_key},
        timeout=5
    )
    
    return response.json()

The client reuses the same key on retry, so the server knows it's a duplicate.

Design Considerations

Storage Backend

Choose carefully. Redis works well for short-lived idempotency windows (hours to days). For longer retention, a database table is safer.

sqlCopy
CREATE TABLE idempotency_cache (
  key VARCHAR(255) PRIMARY KEY,
  response_body TEXT NOT NULL,
  status_code INT NOT NULL,
  created_at TIMESTAMP DEFAULT NOW(),
  expires_at TIMESTAMP NOT NULL,
  INDEX(expires_at)
);

Scope and Lifecycle

• Per-user or global? Payment APIs typically use global keys; session-scoped operations can use user-scoped keys.
• How long to retain? Retain keys at least as long as clients might retry. 24 hours is common; financial systems might use 30 days.
• Cleanup: Implement garbage collection on expired entries to prevent unbounded growth.

Error Handling

If the original request failed, what do you cache?

typescriptCopy
// Option 1: Cache both success and error responses
await idempotencyStore.set(idempotencyKey, {
  statusCode: 400,
  body: { error: 'Invalid amount' },
  timestamp: Date.now(),
}, ttl);

// Option 2: Only cache successful responses, retry on errors
if (result.success) {
  await idempotencyStore.set(idempotencyKey, result, ttl);
}

Most systems cache both—users shouldn't retry forever on validation errors.

When You're Building at Scale

If you're architecting systems handling millions of transactions, teams at LavaPi can help integrate idempotency patterns into your infrastructure from day one. It's easier to design for it than retrofit it.

The Takeaway

Idempotency isn't advanced—it's fundamental. Build it in from your first API endpoint. Provide clear guidance for clients to include the header. Choose a reasonable TTL and storage backend. Test your retry behavior. The upfront effort pays for itself the first time a network blip hits production and your system handles it silently, correctly, and without customer impact.