Analyst Chat #314: Can You Trust AI to Verify AI Code?

And is this also the same angle that you see, Guillaume, when it comes to the general perspective? Because I think the security part actually comes to my mind, but we more think of what is the outcome? What is the code? How do we understand the code that has been written? But is this something that you see in general or that you maybe have seen when you were on the different side of the the barrier that you've mentioned, is this something that has arrived also in the development offices of large vendors right now?

Yeah.

What I see is that now organizations produce code faster than they can verify it.

So they are producing code that light speed or electronic speed, if I may say.

And the problem is that this code has to be verified because you don't ship code, you don't deploy code without verification.

And that's, I think, the core of the topic.

We shift from...

Securing code at the production to securing the prod at the validation time, if I may say.

And the core question is, do you trust AI to verify AI-produced code? Or do you put the verification, accountability, and liability still in human hands? And that's the core of the topic.

Because if you produce 10,000 lines of code, but you don't verify them, then you are in trouble.

Okay, so I think the default state of AI in enterprise software development is unmanaged velocity.

So something we saw in our conference EIC, but in many other places, is that people can tell you that AI is in the pipeline, but not how much, not from which tools, or under whose accountability.

So adoption is measured and by a degree is accurate, but provenance is not.

Of course, I'm coming to this out of the recently published Software Supply and Security Leadership Compass.

But what I observe is that our security control set, and this is nowhere more true than in application security, is designed for human authored code moving at human speed.

Neither assumption is now true.

So if I was to sum up one sentence, we have industrialized code production.

We have put in place AI code delivery, augmenting production times significantly.

What we have not done is industrialized code assurance.

So we are still clinging on to the traditional ways of static dynamic code testing assurance.

And of course, we are falling behind and getting further behind.

But if I understand you correctly, that is not necessarily a new challenge.

It's just a change in the dimension and the velocity and the amount of code that is created.

I think we were not necessarily always that brilliant when it came to code checking anyways.

And now we are in the situation that code is produced at a high level and a high, as I said, high volume.