AMD Instinct

Unfortunately, older GPUs do not have them, so that's why it's slow.

So for example, you can guide-- For example, like I have an AMD Instinct MI60, it has thirty-two gigs of RAM.

So people say, "Oh my God, you have thirty-two gigs of RAM." Yes, that's true, but remember, it doesn't support BF16, and it doesn't support FP8.

Most of the models released, uh, today, usually, uh, in the past, even just last year, most of them were floating point 16, so that wasn't a problem.

So if I pick a used one, I'm gonna pick, uh, which one? Uh, let's pick the, uh...

Which one here? You know what? I'll pick my GPU.

AMD Instinct MI50.

MI50, MI60, they're the same thing, uh, really.

Data center card or recent Nvidia, use VLLM, run batch for OpenAI format, JSONL, or the offline class, or SGLang if your workload is all shared prefixes.

AMD Instinct, use VLLM from AMD's ROCM Docker image.

Consumer Radeon, a mixed box or low VRAM Llama.cpp with a few parallel slots and VLLM if you're on RX 7900 or 9000 series.

Ollama is for development.

Para conocer lo último sobre la carrera de infraestructura de IA en Europa, le damos la palabra a Amber.

La empresa francesa de nube Scaleway anunció esta mañana que construirá la primera supercomputadora comercial de IA basada en AMD en Europa, en Suecia, operando con los chips AMD Instinct MI355X.

La respuesta de AMD a la línea Blackwell de Nvidia.

[ríe] La propuesta: el stack de software de AMD es de código abierto y funciona con las mismas herramientas que los desarrolladores ya usan en hardware de Nvidia, por lo que las empresas no quedan atadas a un solo proveedor.