Cleared for Discussion Podcast | Episode 59 | Model Weights Are the New Crown Jewel

Model weights and checkpoints, not just the final weights, every intermediate checkpoint, every compressed and distilled variant, every adapter file.

Each checkpoint is the whole asset at an earlier moment, and they are frequently stored like any other large data set.

with engineering-grade access because they were treated as a byproduct of training instead of the crown jewel.

Training and fine-tuning data.

This is where poisoning lives, and poisoning is worse than theft in one important way.

Steal the weights and you have a copy.

Poison the corpus and the damage carries into every model trained on it afterward, including the retrain that was supposed to fix the problem, the pipelines, orchestration code, data loaders, build systems, container images.

a path into the model that never touches the weights directly.

And in most organizations, it is treated as plumbing rather than as a security boundary.

The compute, accelerators, servers, firmware, and out-of-band management, which is the blind spot I see most.

Out-of-band management is a complete second network into the asset with its own credentials and its own firmware and very often its own forgotten default settings.

In physical security terms, it is a second door that nobody put on the floor plan.

Evaluation results, red team findings, capability tests, known weaknesses.

Think about that as a counterintelligence officer would.

Somebody who holds your evaluation results holds a map of how to defeat your system, and that material is routinely sitting in a general collaboration tool that half the company can open.

And people, researchers, engineers, data curators, contractors, the facilities crew in the building after hours, the cheapest path to every asset I just named.

In most of these companies, vetting means one commercial background check at hire never revisited.

I hear the objection in every room.

This is a commercial company.

Why would a nation state care? So let me answer with research rather than opinion because the threat is not theoretical.

Rand published a study on securing AI model weights that every physical security professional should read because it is unusually concrete.

The researchers identified 38 distinct attack vectors across nine categories, from ordinary social engineering all the way up to physical interdiction.

They defined five levels of attacker capability, from a low-budget amateur up to the handful of operations The most capable states prioritized most, and they estimated how feasible each attack is for each level.

The spread tells the story.

On the same kind of machine learning software vulnerability, an amateur has something under a 20% chance.

Model weights and checkpoints, not just the final weights, every intermediate checkpoint, every compressed and distilled variant, every adapter file.

Each checkpoint is the whole asset at an earlier moment, and they are frequently stored like any other large data set.

with engineering-grade access because they were treated as a byproduct of training instead of the crown jewel.

Training and fine-tuning data.

This is where poisoning lives, and poisoning is worse than theft in one important way.

Steal the weights and you have a copy.

Poison the corpus and the damage carries into every model trained on it afterward, including the retrain that was supposed to fix the problem, the pipelines, orchestration code, data loaders, build systems, container images.

a path into the model that never touches the weights directly.

And in most organizations, it is treated as plumbing rather than as a security boundary.

The compute, accelerators, servers, firmware, and out-of-band management, which is the blind spot I see most.

Out-of-band management is a complete second network into the asset with its own credentials and its own firmware and very often its own forgotten default settings.

In physical security terms, it is a second door that nobody put on the floor plan.

Evaluation results, red team findings, capability tests, known weaknesses.

Think about that as a counterintelligence officer would.

Somebody who holds your evaluation results holds a map of how to defeat your system, and that material is routinely sitting in a general collaboration tool that half the company can open.

And people, researchers, engineers, data curators, contractors, the facilities crew in the building after hours, the cheapest path to every asset I just named.

In most of these companies, vetting means one commercial background check at hire never revisited.

I hear the objection in every room.

This is a commercial company.

Why would a nation state care? So let me answer with research rather than opinion because the threat is not theoretical.

Rand published a study on securing AI model weights that every physical security professional should read because it is unusually concrete.

The researchers identified 38 distinct attack vectors across nine categories, from ordinary social engineering all the way up to physical interdiction.

They defined five levels of attacker capability, from a low-budget amateur up to the handful of operations The most capable states prioritized most, and they estimated how feasible each attack is for each level.

The spread tells the story.

On the same kind of machine learning software vulnerability, an amateur has something under a 20% chance.