Systems, Scale, and SRE with Vlad Leyberov

Every day, around 95 million photos and videos are shared on Instagram.

Billions of messages and notifications are sent every single day.

So every single one of these notifications, like Sarah liked your photo or Mike commented on your post, had to reach the right device at the right time without delay.

Now imagine one configuration change, one retry bug, one cascading failure, Suddenly, billions of daily notifications don't arrive.

Revenue stops.

Engagement drops.

Users wonder, what's going on? Is this app broken? So someone has to keep that running.

Someone has to catch these failures before they cascade.

Someone has to make the decision like, do we intentionally take the system down 100% to fix it faster? Or

do we let

30% of our users suffer

for 12

hours? That's the job of a site reliability engineer.

And today's guest, Vlad Liborov, has spent his career doing exactly that at Google Meta and Amazon, systems operating at billions of requests per second.

These systems are the ones we use every single day, systems where even a small mistake can have massive global consequences.

It's quite difficult to comprehend, especially at large scale, how cascading failures could propagate throughout the system.

What I've observed over the years of working in large distributed systems, that their behavior is unpredictable.

Even a little change, if we don't fully understand how cascading failures propagate, will have absolutely catastrophic consequences.

This episode is a conversation about what it actually takes to keep the internet running when failure just simply is not an option.

Every day, around 95 million photos and videos are shared on Instagram.

Billions of messages and notifications are sent every single day.

So every single one of these notifications, like Sarah liked your photo or Mike commented on your post, had to reach the right device at the right time without delay.

Now imagine one configuration change, one retry bug, one cascading failure, Suddenly, billions of daily notifications don't arrive.

Revenue stops.

Engagement drops.

Users wonder, what's going on? Is this app broken? So someone has to keep that running.

Someone has to catch these failures before they cascade.

Someone has to make the decision like, do we intentionally take the system down 100% to fix it faster? Or

do we let

30% of our users suffer

for 12

hours? That's the job of a site reliability engineer.

And today's guest, Vlad Liborov, has spent his career doing exactly that at Google Meta and Amazon, systems operating at billions of requests per second.

These systems are the ones we use every single day, systems where even a small mistake can have massive global consequences.

It's quite difficult to comprehend, especially at large scale, how cascading failures could propagate throughout the system.

What I've observed over the years of working in large distributed systems, that their behavior is unpredictable.

Even a little change, if we don't fully understand how cascading failures propagate, will have absolutely catastrophic consequences.

This episode is a conversation about what it actually takes to keep the internet running when failure just simply is not an option.