IOPS

You're never going to feel limited by the data store.

It's really just about your ability to kind of figure out, hey, how do I construct the problem and how do I kind of almost push the laws of physics or my NVMe drives or my IOPS, right? I

agree.

I should call it click house brain like like you have to you have to like get your click house brain going but anyway sorry

So even in my mileage, the costs have risen three times.

I mean, if you want to buy four terabytes of NVMe storage this moment, one single one, you're going to bust out $1,000 at least, maybe more for a decent NVMe drive that's worth the IOPS and worth the speed.

And

yeah, you can throw more at it.

And recovery prefetch converts those operations into 8 kilobyte random reads.

but this can amplify IO per second as much as 16 times on the replicas, which exhausts the IOPS and causes replicas to significantly lag after writer activity.

And this makes it worse if you're using synchronous replication.

because that essentially brings the whole system down because it has to wait for those changes to be applied, and they're not being applied due to all the IOPS that's happening on the replicas.

And he shows an example of this in action.

So basically, if you have a large sequential I.O. and you're using synchronous replication, you may want to consider disabling recovery prefetch.

They balance price and performance for most workloads.

Then there's provisioned IOPS SSD, the IO1 and the IO2 types.

They are for high performance databases needing maximum IOPS.

And then there is the HDD types, throughput optimized and cold for throughput heavy and infrequently accessed data.

EBS snapshots backup volumes to S3 incrementally.

Rarely accessed but instant is Glacier Instant Retrieval.

On to question number three.

A high-performance transactional database running on EC2 requires storage with consistently high IOPS and low latency, well beyond what general purpose volumes guarantee.

Write a purchase order for more enterprise flash, spin up an expansion shelf, and eat the six-figure budget hit.

The rationale sounds defensible, fast workloads need fast IOPS and near-zero latency.

But if you actually inspect what lives inside those blazing fast arrays, you will find something disturbing.

You are running an absurdly expensive digital museum.

The dilemma? Here lies the catch-22 of systems administration.

Updating on disk every time a file is read forces a disk-write operation during an otherwise read-only sequence.

On high-throughput systems, this turns pure reads into a heavy stream of small metadata IOPS, chewing up IOPs and SSD endurance.

Because of this, many production systems mount file systems with no ATIME, which disables access time tracking entirely, or RELATIME, which only updates if it is older than the or hasn't been updated in 24 hours.

Conceptually, EBS is identical to a physical hard drive connected directly to the motherboard of your server via a SATA or NVMe cable.

It provides high IOPS, input-output operations per second, and extremely low latency.

However, because it acts like a physical cable, it can only be attached to one specific EC2 instance at a time.

Next is object storage, which we touched on earlier with the registry, Simple Storage Service, or S3.

It is highly useful for something like a clustered content management system, where 50 different web servers all need real-time access to the same pool of uploaded user images.

And finally, we have local ephemeral storage, commonly called the instance store.

The text notes it provides ultra-high IOPS, but crucially, the data is completely lost if the instance is stopped.

It is intended for scratch space and temporary local caches.