# Idempotency

Idempotency means:

Performing the same operation multiple times produces the same final result as performing it once.

It is one of the most important concepts in distributed systems, payments, APIs, retries, and event-driven architectures.

# Simple Real-Life Example

# Non-idempotent

Imagine:

Transfer ₹1000 to Bindu

If request retries 3 times accidentally:

₹1000 × 3 = ₹3000 transferred

Bad.

# Idempotent

Same request has an idempotency_key.

Even if retried 3 times:

Only ONE transfer happens

Server recognizes:

"This request was already processed."

# Why Idempotency Is Needed

In distributed systems, retries happen constantly because of:

  • Network timeout

  • Client retry

  • Gateway retry

  • Mobile internet instability

  • Server restart

  • Queue reprocessing

  • Kafka duplicate delivery

Without idempotency:

  • Double payments

  • Duplicate orders

  • Multiple subscriptions

  • Duplicate emails

  • Incorrect API hit counts

# Basic Flow

# Without Idempotency

Client
  │
  ├── POST /payment
  │
  ▼
Server processes payment
  │
  ▼
Payment completed
  │
  ✖ Response lost due to timeout
  │
Client retries
  │
  ▼
Server processes AGAIN
  │
  ▼
DOUBLE PAYMENT ❌

# With Idempotency

Client
  │
  ├── POST /payment
  │      Idempotency-Key: abc123
  │
  ▼
Server checks key
  │
  ├── Not found
  │
  ▼
Process payment
  │
  ▼
Store:
abc123 → success response
  │
  ▼
Return response

Now retry:

Client retries same request
  │
  ├── Idempotency-Key: abc123
  │
  ▼
Server checks key
  │
  ├── Found existing result
  │
  ▼
Return OLD response
  │
  ▼
NO duplicate payment ✅

# Core Components

# 1. Idempotency Key

Usually:

Idempotency-Key: 550e8400-e29b

Generated by client.

Often UUID.

# 2. Request Fingerprint

Sometimes system also stores:

  • request body hash

  • user ID

  • endpoint

To ensure same key is not reused maliciously.

# Production Database Table

Example:

idempotency_records
-------------------------
id
idempotency_key
user_id
request_hash
response_body
status_code
created_at
expires_at

# Backend Flow Chart

                ┌─────────────────┐
                │ Incoming Request│
                └────────┬────────┘
                         │
                         ▼
          ┌──────────────────────────┐
          │ Read Idempotency-Key     │
          └──────────┬───────────────┘
                     │
          Key Exists in DB/Redis?
             ┌───────┴────────┐
             │                │
            YES              NO
             │                │
             ▼                ▼
 Return Stored Response   Process Request
             │                │
             │                ▼
             │        Store Response
             │        against key
             │                │
             ▼                ▼
         Client Gets Same Consistent Response

# Where It Is Used

# Payments

Stripe heavily uses idempotency.

Example:

POST /charges
Idempotency-Key: txn_123

# Order Creation

Prevent:

Double order creation

# Queue Consumers

Kafka/RabbitMQ may redeliver messages.

Idempotent consumers prevent duplicates.

# API Retries

Mobile apps retry requests automatically.

# HTTP Methods and Idempotency

Method Idempotent?
GET Yes
PUT Usually Yes
DELETE Usually Yes
POST Usually No

POST usually needs explicit idempotency handling.

# Idempotency vs Retry

Important interview question.

# Retry

Means:

Try operation again

# Idempotency

Means:

Retries do NOT create duplicate effects

Retries NEED idempotency.

# Redis-Based Fast Implementation

Very common in large systems.

# Flow

API Request
   │
   ▼
SETNX idempotency_key
   │
   ├── Success → process request
   │
   └── Already exists → return stored response

# Redis Example

SETNX payment:abc123 processing

If success:

Process payment

Then:

SET payment:abc123 response_json

# Kafka + Idempotency

Very important for interviews.

Problem:

Kafka provides:

At-least-once delivery

Meaning duplicates can occur.

# Consumer Flow

Kafka Message
      │
      ▼
Check Processed Event Table
      │
 ┌────┴────┐
 │         │
YES        NO
 │         │
Skip     Process Event
 │         │
 │         ▼
 │     Save Event ID
 │
 ▼
Avoid Duplicate Processing

# Exactly Once vs Idempotency

Another senior-level topic.

# Exactly Once

Very hard and expensive.

# Idempotent Processing

Practical industry solution.

Most systems use:

At-least-once delivery
+
Idempotent consumers

instead of true exactly-once.

# Common Interview Follow-Up Questions

# Q1: Where store idempotency keys?

Answer:

  • Redis → fast temporary storage

  • Postgres → durable storage

  • DynamoDB/Cassandra → huge scale

# Q2: How long keep keys?

Depends on business need.

Examples:

Use Case TTL
Payment 24–48 hrs
Orders Few hours
API retries Minutes

# Q3: What if request body changes with same key?

Store request hash.

If mismatch:

409 Conflict

# Example Architecture

Client
  │
  ▼
API Gateway
  │
  ▼
Application Service
  │
  ├── Redis/Postgres Idempotency Store
  │
  ├── Payment Service
  │
  └── Kafka Event Bus