Nvidia RTX

They are not universal compatibility guarantees, so the exact model, container and settings should always be tested before committing to a production configuration.

VRAM suitable starting point GP is listed on NASANA 8-10 GB transcription, embeddings, classification, compact quantized LLMs and basic image generation RTX 3060, RTX 3070, RTX 4060.

RTX 300 8012 GB quantized 7B class LLMs, smaller RAG applications and moderate image workflows RTX 4070.

RTX 5070 16 GB 7B-8B models with more context headroom and larger image pipelines RTX 4080.

RTX 5080.

RTX 4000, A400 24 GB 7B8 BFP 16 inference.

larger quantized LLMs and advanced image generation RTX 3090, RTX 4090, RTX 5000, A500-032GB AI video, complex comfy UI pipelines and medium-sized quantized LLMs RTX 509040-48GB larger model inference, long contexts, bigger batches and demanding media workflows A140GB, A6000, RTX 6000 ADA, a 408096GB large model inference, high concurrency, multimodal systems and enterprise workloads a 180GB H100, RTX Pro 6000 the GPUs shown in the NASANA interface and their availability can change AS capacity joins or leaves the network.

The useful approach is to identify the likely memory tier first and then compare the price and expected performance of the GPUs currently available within it.

I don't want them knowing what my weaknesses are, where, where my valuable customers are, my strategies, et cetera." 'Cause everything that's out there on the internet is being sucked into the AI, and it's used for training.

And I think there's certainly a trend, and you can certainly tell it from when you go to the computer store and you try and buy a DGX Spark or you try and buy the RTX graphics cards.

RTX 6000s are pretty good for running very, very high performance.

Two of those cards cost you about $50,000, and you can run something like GLM at 1,000 tokens a second and run a whole engineering team off them.

There's a big scramble for this hardware.

Yeah.

It's not a you turn it on on, on the RTX card.

It's the developer wants to use it, they use it.

Just like any other feature of the DLSS suite It's up to the developer to decide to use those features if they wanna use them, right? So, uh, I don't know.

People say, well, you know what, I can't afford the hardware, I can't afford the infrastructure.

You can get a pair of RTX Pro 6000 GPUs, they're PCIe devices, they're air-cooled, they're like 15K a piece.

So like you're telling me, like these companies are spending sometimes over a hundred thousand dollars a week.

You can't find $30,000 to do a pilot, right? That's going to allow you to see the effect of running a model on premises and how much it's going to save you.

And this is just taking it in isolation as its own thing.

The result, as I saw it in this game, and they allowed us to play around with it before it was out, including on like a laptop with a much less powerful RTX GPU in it.

And it was running really, really smooth on there.

That was kind of my feeling on that.

and likely those with 16 gigabytes of RAM.

And there was even speculation from us, for me, that the sort of 50 series super line with its 50% increase to VRAM might have been, you know, basically to accommodate DLSS5, but we've been proven completely wrong, I think it's fair to say.

This is super, super heavy, right? But it's also running on the entire RTX 50 series stack, including laptops, which I didn't anticipate.

Eight gigabyte cards.

Oh, yeah.

DLSS 5 now working on all RTX GPUs.

[laughs]

Yeah.

Without going through and painstakingly buying youtube premium on them Like Its unhinged

i constantly get PISSED Because ill open a video on my work account For work reasons then Have to go UGH Copy the URL Go to personal browser Open it There Then Watch it What happening ughhh Man

Ummm Yknow what Im Gonna Just Kibosh this one Lets talk About DLSS5 HEY Do wanna Get EVEN LESS Performance Outta Older RTX Cards *Snap* Your IN LUCK Modders Now HAVE dlss5 Working On rtx20 AND 30 SERIES GPUs Running FP16 This new implementation is Said To Be An Improvement Over Early FP8 Version Saw frame rates drop From As High AS 138 To JUST Four FPS In Deep Rock Galactic nice Modder Also Demonstrated Dlss5 In Directx 11 Games With No Native DLSS Implementation Like Batman Arkham Knight And Even DX9 And Vulkan Titles And Even Console Emulators Our discussion Question Was Anyone Asking Me For This Who Is This FOR UM I ALSO Wanna Jump In Clarify Something Real Quick Iv seen DLSS5 Used For Webcam Outputs Thats Not DLSS5 It Hasnt Been Tuned Game Developer Should Be Involved In THAT Process And Also DLSS5 In ITS Properly Functioning Form Needs Vector Data SOOOO Theres Ain't NO Way Your webcam Output HAS vector data sooo yeahh

someones gonna try to build Some Junk To Link IT TO [laughing] BIG WHEEL anyways