Thành thật mà nói, tôi thực sự không thấy cần thiết phải có một quản trị toàn cầu thực sự quốc tế, đặc biệt là theo nghĩa quản lý hoặc quản lý rủi ro ở đây.

Tôi nghĩ có một sự hợp tác và quản trị quốc tế diễn ra rất nhiều ở cấp độ kỹ thuật, ví dụ như tiêu chuẩn dữ liệu cho xe tự lái là gì, hoặc khi AI đang được tích hợp vào thiết bị y tế.

Chúng ta nghĩ thế nào về việc phê duyệt lẫn nhau cho thiết bị y tế ở các quốc gia khác nhau? Tôi nghĩ những việc như vậy sẽ diễn ra ở cấp độ làm việc, giống như với các công nghệ khác, mà không cần sự ủng hộ chính trị cấp cao.

Tuy nhiên, tôi nghĩ nếu nói về quản lý rủi ro cấp cao từ AI, tôi không nhất thiết xem đó là vấn đề toàn cầu.

Tuy nhiên, mặt khác, một lĩnh vực có thể có không gian cho sự tham gia toàn cầu hoặc quốc tế sẽ là khai thác lợi ích từ AI và phân phối lợi ích từ AI theo cách không chỉ nâng cao năng suất của các công ty thuộc S&P 500, mà thực sự là trao quyền cho mọi người trên toàn thế giới.

Cố gắng bao phủ các quốc gia có ít tài nguyên ngôn ngữ hơn nên các mô hình ngôn ngữ có thể không hoạt động tốt theo mặc định những khu vực đó, hoặc những thứ tương tự.

Theo tôi, điều đó hứa hẹn hơn nhiều so với việc cố gắng xây dựng một chế độ quản trị nào đó.

Tôi nghĩ mọi người thường nghĩ rằng có hai thứ diễn ra với tốc độ nhất định và nếu một thứ không theo kịp thì nó sẽ bị tụt lại phía sau.

Helen, trong phạm vi chúng ta đã hiểu rõ về vấn đề này, bà sẽ mô tả quan điểm của Trung Quốc về ý nghĩa của việc trở thành một nhà lãnh đạo toàn cầu ở đây như thế nào? Điều gì thực sự quan trọng trong cuộc thi đó đến mức có thể mô tả như vậy? Theo tôi,

Có vẻ như có những câu trả lời khác nhau trong hệ thống Trung Quốc và các câu trả lời khác nhau trong hệ thống Hoa Kỳ.

Tôi không chắc rằng cả hai hệ thống đều hoàn toàn thống nhất về một câu trả lời ở đây.

Điểm khởi đầu, và tôi tò mò, Kyler, bạn sẽ bổ sung gì ở đây? Điểm khởi đầu từ hầu hết những gì tôi thấy là nhìn nhận AI như một phần của cuộc cách mạng công nghiệp lần thứ tư.

Về cơ bản Trung Quốc coi mình đã bỏ lỡ ba cuộc cách mạng công nghiệp đầu tiên và muốn trở thành người lãnh đạo, tiên phong trong cuộc cách mạng công nghiệp lần thứ tư mới này.

Vì vậy, Đó là một cách nhìn khá xuyên suốt, có phạm vi rộng, nhằm lấy lại uy tín trên trường quốc tế, được nhìn nhận như một nhà lãnh đạo kỹ thuật hơn là tác động cụ thể nào lên Trung Quốc.

Tôi đoán lăng kính lớn khác mà bạn thấy họ nói đến là AI như một động lực thúc đẩy thịnh vượng, thúc đẩy tăng trưởng kinh tế, ngay cả trong thời điểm xã hội đang già hóa.

Bạn chỉ muốn đứng ở vị trí đầu bảng.

Vì vậy, nhóm của chúng tôi tại CSET, Trung tâm An ninh và Công nghệ Mới Nội,

Nơi tôi dẫn rách ở Yoseten, nhóm CSET đã kết luận rằng rất khó để biết trưng kết hữu ích đến mức nào.

Đây cũng là điều tôi đã thấy từ nhiều nhà bình luận khác đang cố gắng đánh giá.

Liệu đây có phải là 90% câu chuyện về việc Trung Quốc đang bắt kịp không? Có phải là 10% không? Tôi nghĩ chúng ta có thể nói rằng nó đủ ý nghĩa để họ đầu tư thời gian và công sức vì chúng ta đang chứng kiến các cuộc tấn công trưng kết quy mô rất lớn ở Trung Quốc.

hay đúng hơn là nỗ lực trưng cất dự đoán tốt nhất của tôi là không phải 90% cách họ theo kịp mà có thể nằm trong khoảng 10-50% nhưng thực sự rất khó để biết được tôi nghĩ có một mâu thuẫn vốn có mà chúng ta thường bỏ qua đối với một người như Dario nói một mặt Anthopic đã lên tiếng rất mạnh mẽ về việc trưng cất và nhấn mạnh mức độ đe dọa của nó rồi mặt khác Dario lại nói chúng ta không thể tự mình chậm lại vì như vậy Trung Quốc sẽ không chậm lại Bạn không thể thực sự tin cả hai điều đó.

Có thể Dario không đồng ý với những người khác trong công ty về tầm quan trọng của trưng cất.

Tôi không chắc.

Tình trạng của những nỗi lo đó hiện nay ra sao và Trung Quốc đang cố gắng kiểm soát nó như thế nào?

Đúng vậy, thật khó để diễn tả hết mức độ phần lớn thế giới AI Mỹ, đặc biệt là thế giới AI tiên phong, các nhà phát triển AI tiên tiến, xuất phát từ một cộng đồng trí thức khá đặc thù phát triển từ lâu trước chát, trước cả khi học sâu và mạng neuron sâu bắt đầu

hoạt động tốt vào đầu những năm 2010.

Đây là kiểu tư duy tương lai của những năm 90 và 2000 về trí tuệ nhân tạo có thể trông như thế nào, nó có ý nghĩa gì với thế giới, và cách nó có thể vận hành.

Tôi cho rằng tư duy tổng thể, kỹ thuật, xã hội, triết học, chính trị này, vốn được cho phép, có thể phần nào được thúc đẩy trong hệ thống Mỹ.

Thực sự không phải là điều hệ thống Trung Quốc muốn các kỹ sư và nhà khoa học của mình làm.

Vì vậy...

Tôi nghĩ không phải ngẫu nhiên mà cộng đồng đó xuất hiện ở Mỹ và ít hơn ở Trung Quốc.

Trọng tâm của bài viết thực sự là nói rằng việc Trung Quốc tương đối sát Mỹ không nên là

Tôi thấy những thế giới đó cuối cùng giao thoa với AI ở vị trí cao trong chương trình nghị sự.

Tôi nghĩ những gì chúng ta đã thấy, nó tiết lộ điều gì về quan điểm của người Trung Quốc.

Tôi nghĩ điều hữu ích nhất là có khá nhiều tin tức trước đó về các vấn đề khác nhau được đưa ra trên truyền thông Trung Quốc, phản ứng với bức thư Passing a Frontier của Dario Amodi hay những điều khác.

Tôi nghĩ nhiều tin tức đó đã bỏ lỡ thông điệp quan trọng nhất đối với tôi từ chính phủ Trung Quốc.

Đó là Chen Rixin, người đứng đầu Bộ An ninh Nhà nước, một bộ rất quyền lực và tập trung vào an ninh ở Trung Quốc.

Tôi nghe nói chính Chen cũng có ảnh hưởng lớn với Tập Cận Bình.

Và ông ấy thường không viết gì công khai, nên việc ông ấy viết cả một bài về AI thực sự rất bổ ích.

Hề lên, nếu bạn nhìn một nửa kính như vậy, chuyện gì sẽ xảy ra? Chúng ta muốn thấy điều gì trong 6 tháng hoặc năm tới để thực hiện được hy vọng nhỏ nhoi mà Kyler đang nhấn mạnh?

M- many of the people who can audit this, the only people who will sort of understand it, a lot of them subscribe to these same kinds of, shall we say, very odd beliefs that, as Anthropic has said, that AI might need human rights.

Yeah, I mean, the challenge with this space...

So I think that particular belief is not one that is held widely across the space.

Certainly, some people in the space hold that.

In general, there's a, a real challenge here, which is AI safety, thinking about these kinds of risks, was a really niche topic for a long time.

And so the, the community of people who've been thinking about it for more than a year or two, you know, is pretty small, and they, you know, is pretty tight-knit.

Are these guys just total chameleons?

I think the CEOs, it's really interesting to watch the different ways that they talk to different audiences.

Helen, if you take that glass half full view, what might happen? What, what, what would we like to see in the next six months or year that would make good on that sliver of promise that Kyle is highlighting?

I do think that is the right read, um, and relative to the rock bottom expectations that, you know, we saw around, you know, any results of the summit more generally.

I think this was at least somewhat productive.

I think the thing that I will be watching is, do they narrow in on specific issues of concern that both countries are interested in talking to each other about? Because, you know, a huge problem with AI in general is it's sort of an everything technology, so it affects every industry in all kinds of different ways.

Even talking about kind of AI security or AI national security, there's still so many different things you might be talking about.

Are you talking about AI adoption into military systems on the battlefield? Are you talking about AI and thir-- you know, non-state actors, potential for terrorists to use AI? Are you talking about risks from autonomous AI systems themselves? You know, there are lots of other things you might mean, even if you narrow it down to AI national security.

I see-- In my mind, there are at least two quite concrete things that the US and China could have an interest in talking to each other about and working collectively on.

So I'm curious if you see changes there as well as how that's affected this basic view of, of AI risk as Kyle was laying out.

For people in your field who've been worried about these problems for years – What, if anything, in the last few weeks or months has changed your assessment about where we are and what we should be worried about?

Yeah, so many things.

A bunch of things.

One I'll start with maybe is, you know, I have a lot of friends.

I have, you know, family members who, as this, like you said, you know, the story has broken containment, just like the OpenAI agents broke containment.

But as the story has broken containment, I've had a lot of people in my life reaching out and saying, like, oh, my gosh, is this for real? Are we all going to die? Like, should we be panicking right now? And I think one...

thing that comes up for me there is this problem, this situation has been there for, has been developing over months and years.

But how do you get from that to lying, making plans, trying to avoid being shut down? Like, what do we know about why they did that? And when people observe that, what are they actually observing? Like, how did how did they find that out?

Um, what does that incident reveal, uh, about the gap between what these systems can do and what their creators can actually control?

Yeah.

The Hugging Face attack really illustrates s- a number of fascinating and very important dynamics for what the frontier of AI looks like right now.

So it's important to note, we didn't know this when it was first announced, but we now know that the attack was not carried out by one AI agent or by a couple, but by a, uh, by seven hundred agents that were acting in a self-described swarm or self-described collective.

Um, and it was part of, uh...

This happened during OpenAI's kind of research and testing process.

They were trying to build increasingly capable AIs and trying to test how good they were.

Uh, I support your decision to be responsible, but stop pretending, uh, that the motivation to slow down is purely altruistic." Um, do, do you wanna respond to that, and, uh, what impact does it have on the industry when you have that kind of comment and, uh, narrative coming out of the White House?

So I w-- I wanted to start with you and say, you know, w-what should we note about what's different about these incidents, and what does that tell us about, um, sort of this, this issue more broadly?

Yeah.

And thanks for having me.

It's great to be here with everyone.

There's a lot going on here, so I will try and unpack the different things we've learned in a relatively concise way.

Um, maybe just a, a, a...

Before I start to say, I think something we'll be talking about on this panel, which differs a little bit from Ian's presentation, that was a fantastic explanation of from the cybersecurity side, from the perspective of the victim of the cyberattack, kind of w-what are the cybersecurity considerations here? I think there's several other angles of these incidents as well, and a really important one is: what does this tell us about how AI is developing and what is happening kind of at the frontiers of AI progress? Um, which is a little bit separate from what do you do if you are, you know, the victim of, of one of these attacks.

[laughs]

So why, given what these systems are trained on, are they so consistently turning to cheating?

I think y-you're really onto something with this question, which is it is really striking how hard a time we are having controlling and directing the AI systems that we have.

I think a lot of people have heard that AI's trained to predict the next word based on kind of human texts.

That's true.

But these days, there's an additional kind of training that is responsible for a lot of the advances we've seen over the last year or two, where that's not really what they're doing.

I've heard it called...

So the technical term is, uh, reinforcement learning, uh, with verifiable rewards.

Or certainly the idea that we should just assume that and operate as if it is so doesn't seem totally reliable.

What's that look like around the world? What kind of stan- standards in particular are governments trying to enforce?

There's been a lot of different approaches to this.

Again, the, the, you know, the EU approach here has been to treat AI like a product, and so in the same way that you have if you have sort of a physical product in the EU, it might have a little stamp that it's gone through a certain testing process, um, is the way they've mostly thought about it.

Um, in, in the US, things tend to be much more decentralized.

So, you know, looking at self-driving cars, that really goes through the transportation.

Um, anything medical is gonna go through, you know, the medical regulators.

Um, so I think it'll be...

Um, how do you see that tug of war at the moment?

How is that possible?

Yeah, it's, it's honestly kind of crazy given how, uh, capable AI systems now are and how rapidly they're improving, but the crucial thing here is AI is not built like normal software.

So most software that we use is written line by line.

So programmers type one line, they type the next line, they understand, you know, how the code works.

That's really not how we build AI.

People sometimes say AI is a, a black box.

Um, what that really means is when we open up the black box, we look inside the system, we find a huge amount of numbers.

Yeah.

How should we interpret that? Is this just kind of boundless cynicism or is there something else going on?

It certainly looks cynical to me.

I think it's really playing fast and loose with what democracy means, what democratic AI means.

If you read line by line, they never imply that the UAE is a democracy, but they talk about building on democratic rails or promoting democratic AI internationally.

I think there's kind of two ways you could interpret this.

One is, and I think both could be reasonable in principle, but aren't really present in this deal.

So one is that access to AI can be a democratic force for good in general.

Can you explain why you think that might be a mistake and what adaptation buffers are instead?

Wow.

And that's the key difference between an agentic capability.

It can actually go and do something that a human would previously have had to do, and it's changing that mindset around, "Okay, this is a task that I do every single day at work.

How can we use Claude Cowork to help automate or streamline that?" And a great example that I used it for just yesterday that saved me probably about five hours of time is I had to build a pretty complicated Excel report for our sales team looking at commission structures, looking at deals made, looking at a bunch of different stuff.

And instead of me actually having to build that Excel sheet, put in the formulas, make sure everything's been tracked, I gave Claude Cowork the access to the sales sheets and, and the payments made, and then it just created that sheet for me in about four minutes.

Helen Toner is the interim director at the Center for Security and Emerging Technology at Georgetown University in the United States and was previously an independent board member at one of Anthropic's great rivals, OpenAI.

There's really two big angles to look at this from.

One is from the Pentagon's perspective, the backdrop for this is a big fight in 2018 that the Pentagon had with Google over a project called Project Maven, which was one of the first uses of AI for military.

Helen Toner is the interim director at the Center for Security and Emerging Technology at Georgetown University in the United States, and was previously an independent board member at one of Anthropic's great rivals, OpenAI.

There's really two big angles to look at this from.

One is from the Pentagon's perspective, the backdrop for this is a big fight in twenty eighteen that the Pentagon had with Google over a project called Project Maven, which was one of the first uses of AI for military.

It was looking at using AI to look at overhead imagery and, you know, help with targeting, maybe identifying terrorists who might be the subject of-- the target of drone strikes, things like that.

Google employees really didn't like that.

They walked out.

Google decided not to renew the contract.

And round us out here then, from an Anthropic perspective then, are they in a, a better place, funnily enough, because of opening up that argument that you talked about, do you think? Do you think that perhaps it drives more investment to them because they're seen as someone who's thinking about some of those arguments that you're talking about? Or does it long-term damage them, do you think, as a company being in the position that they are a sort of pariah in the US government's eyes?

I wonder, did you feel like this was sufficiently addressing some of the CSET specific concerns, for lack of a better phrase? Or did you see any gaps that you were kind of wanting to see here?

Yeah, so, I mean, we do a lot of work on different aspects of kind of tech and security at CSET.

And so it's something that, you know, clearly is in sync with the sort of work we're doing as the section on the workforce and on the kinds of training and the kinds of sort of non-traditional job training that are needed.

We've done a lot of work on the importance of apprenticeships, on-the-job training, certifications, non-four-year college degrees.

So I think there are, you know, to some extent we saw or I see opportunities Some of the work that we do reflected there.

I do think I think what you're getting at, though, is if you're looking at sort of national security strictly taken, there is not a whole lot of that in there.

And something that I maybe one of the pieces that was most surprising to me was that it really is just this one bullet, not just about kind of national security issues.

Does this matter? Are we going to see some sort of action in an increasingly finite number of days left in this Congress on AI? Or is this just a nice kind of nudge to Congress to remind them that, yes, the president does care about AI, but ultimately it's not necessarily influential?

Talk about whether that line needs to get re-looked at, and how would you reframe that separation between narrow versus general AI?

There's a bit of an obsession in some circles, AI safety circles, maybe AI circles more broadly, in talking about how far away we are from AGI, artificial general intelligence.

And some people will say, you know, we're only one to three years away.

Others will say, that's ridiculous.

You know, we're obviously 20 plus years away from that.

But it's always very hard to pin down what exactly people mean by that.

Or if you do pin them down, people mean very, very different things.

But that understanding, there's a positive scenario where, you know, things get aligned, guardrails get established, the, you know, the brakes are identified and the cadence in which the tapping of the brakes you know, is agreed upon on one side, which is the positive scenario, and the negative scenario when this thing doesn't happen, you know, what do you think would be the result of those two things? And, you know, how do you see this evolving, let's say, in a five-year period? What could happen in both these scenarios?

I mean, what do you make of what's been playing out between the Pentagon and Anthropic?

It's really been a series of extraordinary events.

So the first thing to know here is Anthropic has been really forward-leaning in its, uh, interest in working with the Pentagon and in, uh, setting its systems up so that the, the military can, can make use of them.

Um, and last summer was when they, uh, created their $200 million contract with this administration.

Um, the second thing that's extraordinary is the, the two lines that have been drawn here.

They seem to have come from the Pentagon coming back to Anthropic and saying that they want to change the contract that was already signed.

Um, and those two sticking points were where, uh, Anthropic couldn't agree, which was on, um, not using their systems for, uh, domestic mass surveillance or for autonomous weapons.

Some people might look at this and say, "I don't know, those two guardrails sound pretty reasonable to me when I look at them."

Let's just start with your initial reactions to this move.

It's been really interesting to watch the play-by-play here.

So this is not the first, um, move.

The Trump administration did not do this out of the blue.

Earlier this year, there was an attempt to pass, uh, preemption as part of the One Big Beautiful Bill.

Um, so preemption being, you know, the federal government stepping in, saying, "We have rules.

Um, we're going to, the rules we have are going to preempt anything that the states do.

What do you think? How should we be regulating AI?

Recent cases show AI chatbots encouraging self-harm or delusions, providing dangerous instructions ranging from a Belgian man's suicide, a US murder-suicide linked to models giving users bomb-making guidance or assassination plots, all of it happening without it being especially clear that there's anything illegal involved.

My name is Helen Toner.

I am the interim executive director at the Center for Security and Emerging Technology at Georgetown University.

We're starting to see an increasing number of reports, starting from sort of earlier this year, of users of chatbots who are having some kind of, perhaps, psychotic episode while interacting with a chatbot, or having some other kind of delusional experience.

And it seems like, what it looks like when people look at the, the transcripts involved, often the chatbots are maybe deepening those delusions or deepening the psychosis or...They're often also, the chatbots are also, in some instances, trying to suggest that the person in question goes and talks to a therapist or a mental health professional.

But that's also often woven in with other conversations where the chatbot is saying that, you know, this person is, uh, you know, they're, they think they've invented some world-changing mathematical theory and the chatbot tells them how insightful their math is.

Or, um, they think that their family is spying on them and the chatbot tells them it's good that they're being careful, things like that.

Is it about the technology basically being very affirming and designed to be affirming and therefore telling you what you want to hear? Is that just something intrinsic to this tech?