Law firms are also in the same situation, right? Which ones would you think will still be available even if AI advances as seems likely over the next five or 10 years?

Yeah, to be clear, I do think current college students are dealing with much more uncertainty than they have, well, maybe ever before in the job market.

So this is definitely not an easy task to give this advice or to... for them to figure out what to do.

Yeah, I mean, one big uncertainty we face is just how quickly AI will advance.

It could be possible to get to AI that can do pretty much all white collar jobs remotely, say in 2030, which would be before a high school student would even graduate college, and that could mean the job market has already, like, completely changed several times over before they graduate.

It's also possible that AI will advance m- more slowly, and it'll be, like, long into the 2030s before they get to that point.

And yeah, one argument you could make is that in these very short timelines, maybe you can't do that much if you're already a high school student, so you should just focus on the scenarios in which AI does take a bit longer and optimize your career around those.

But how are we actually going to have an enforceable global agreement to stop the things that would lead to AI dominating us in that way?

So that's worrying but that's, that's, that's power seeking AI systems is kind of the most pressing problem that you list and you kind of, you already touched on a few minutes ago in this episode the second most pressing problem according to 80,000 Hours which is extreme power concentration.

Yeah.

So as AI alignment has become less neglected we've kind of added a list of these other AI risks that I'd say are still way more neglected.

So we talked about like AI alignment has maybe 1,000 people working on it full time.

Something like that.

But I'd say the concentration of power risks, it's maybe still only, like, 20 people, depending on, depending on exactly how you'd count it, but in terms of researchers specializing in it.

And yeah, the idea here is, like, there's a bunch of ways that AI could make it possible to concentrate power much more than has been possible in history, and there's, like, actually a bunch of different routes by which this could happen.

So maybe can we... can you give us like maybe like kind of your top heuristics that you think are the most useful for people to be selecting the right career for them?