Thundering herd problem

You do whatever it takes to keep the critical path alive.

And when a service does crash and come back online, you face the final resilience challenge the guide mentions: the thundering herd.

It prescribes retry with exponential backoff and jitter to survive it.

The thundering herd is a terrifying phenomenon to witness in production.

Imagine a critical authorization service drops offline for just five seconds.

During those five seconds, fifty thousand client applications attempt to authenticate and fail.

They will all wait exactly one second, then hit the server simultaneously.

Then they will all wait exactly two seconds and hit it simultaneously.