explainable AI

This audio is presented by Hacker Noon, where anyone can learn anything about any technology.

The Explainability Problem in AI Payment Systems, by Sergey Magas.

Nvidia shares jumped nearly 5% in after-hours trading after the company reported quarterly revenue that more than doubled year-over-year, with data center revenue surging 117%.

NVIDIA's outlook reinforced expectations that iInfrastructure spending remains robust, with the company projecting revenue growth of approximately 70% for the 2028 fiscal year.

Welcome to the September 10th edition of the Applied Clinical Trials Brief.

In the latest Beyond Compliance episode, Dr. Joseph Geraci, co-founder and chief scientific and technical officer at Netromark, explained why black box AI models are poorly suited to clinical trial data where regulators and clinicians need to understand how conclusions were reached.

Explainable AI capable of recognizing when it lacks sufficient info surfaces clinically meaningful patient subgroups characterized by variables clinicians can recognize and act on, allowing insights from earlier phases to inform later trials.

And finally, the FDA announced four permanent senior leadership appointments, confirming Dr. Michael Davis as Director of CEDAR and Karim Mikhail as Director of CBER, while also creating the agency's first-ever Deputy Commissioner for Technology and AI role.

Placing AI governance at the deputy commissioner level signals the administration intends AI oversight to be a cross-agency priority, while the center appointments address structural uncertainty that has constrained biotech deal flow and IPO activity throughout the year.

And that's all for today's ACT Brief.

Join us right back here tomorrow for more updates shaping ClinOps and drug development.

So in many ways, that's the conscience of this season.

Explainability, robustness, privacy, reproducibility, sustainability, and the plain right to know and to choose.

A few years ago, every one of these would have been the limitation slide of somebody's talk, the part you skip if you're running late.

But that time is over.

So the way I remember all these different things is this little and you don't have to this it goes a little bit deeper on the slide of valid uh uh valid and reliability safety secure resilient accountability transparency fair and uh harmful bias management so it does cover like the basics of of security on AI.

Explainability, interpretability, privacy enhancements, all of these characteristics are things you'll find in the NIST AI risk management framework.

But what you really need to understand is this right here, this little map.

This little map where you have governance as a centerpiece, and then you have a map that has the context of identifying the risks, the measurements where you're using analysis and assessments to figure out what the risks are and track them, using numbers, and then managing, taking priority and taking action on figuring out what risks to take care of first.

So it has to have a causal chain of reasoning.

Explainable AI is an abomination.

It's an invitation to... to relitigation that we already have way too many alignment meetings to litigate what it is we think we meant, right? When we agreed to do something.

And so causal chain of reasoning is why causal exists in the first place.

I've seen some good examples in financial services and in business functions like customer service and so on.

But to your point, I think this next part is really about building this trust architecture, having also a record of what was done by whom, human agent, at what point in time, based on what information.

So you can go back and you can actually trace and say, okay, this is what happened and why it happened, which hasn't always been that easy thinking back even a couple of years ago with machine learning and looking for things like SHAP, some of the explainable AI methods.

So early, But we know where this needs to go if something like this, agente commerce, is to take off and is to expand from consumer into B2B as well.

transparency the the first question should be what is our transparency because if if you know where ai is being used and how it's being used and you can explain that you're way further ahead than hey we got a policy right that's the mistake oh do we have an ai policy well if you don't have any visibility transparency and they all ai policy isn't going to save you All it's going to be is what everybody compares you against that you didn't do.

So that's the question I think boards get hung up on is going for governance before they have that transparency and understanding of where is it being used? How is it being used? Why is it being used? Can we explain? Explainability is the next piece of transparency, right? Once you have transparency, can you explain why it was done a certain way and why it produces outputs? If you have transparency and explainability, you've got something way more important than a bunch of policy documents.

And you have something that you can actually build your policy documents around and get to that assess, correct, justify component that you need for compliance.

Yeah.

Accountability is important.

Explainability and transparency, again, fundamental.

Uh, be able to explain to customer why a certain decision was made and how that decision was made in plain language, in a way that they can understand, again, is fundamental for that trustworthy AI piece.

And finally, human oversight, which I love to say to this audience, human oversight is seen as the human in the loop, right? So if a high-risk use case, uh, make sure there is a human reviewing the decision that AI is making.

And there were also other example, Nationwide, uh, Unilever.

Again, there are a few companies that are doing things, you know, well when it comes to the implementation of that responsible AI.

So you can see how- Explainability, transparency, accountability, those kind of things come to life through those examples of allowing customers to say, "Actually, no, that wasn't my car," or, "It wasn't parked there," or demonstrate why the AI has made those decisions without bias.

So Enza, this, this keynote was a big hit, but, you know, limited audience.

But if they get a structured output, issue, rule, evidence, recommendation, they can move fast and apply their judgment where it actually counts.

Explainability isn't about making the machine look smart.

It's about making oversight fast and effective.

There's also the question of what happens when something goes wrong.