Welcome to the Tech Humanist Show, a multimedia format program exploring how data and technology shape the human experience.

I'm your host, Kate

O'Neill.

Hello humans.

I'm Kate O'Neill.

And today I'm speaking with Kate O'Neill.

So why is this dangerous? What's the downside of this much trust with AI?

there's a couple of things one is obviously you know what you're talking about with uploading a bunch of financial data or company secrets or whatever there's some cyber security issues there for sure there's a matter of you know how secure that data is how protected it is from any kind of breach or leak or anything like that and how it might get trained into the system and then you know be accidentally regurgitated at some point for somebody else like down the road somebody's you know kind of saying uh come up with some sort of business model for me where theoretically I might be renting DVDs or something like that.

And suddenly you've got like secrets from blockbusters old playbook.

to circle us full back all the way.

But that kind of thing does happen.

It has been known to happen.

But the more sort of human implication here is that what happens is that we start using the way that we're programmed, which is to treat everything that we interact with as if it's human.

I'm like, well, why didn't you tell me that in the first place? And why didn't you correct yourself? So how do we know when AI is wrong? And how should we be using it? What is the lesson to be learned here?

But actually change only happens when you've got the whole spine of the organization lined up, excited about the change, but also being clear about What are they going to get out of it? What's their role in contributing to it? I think, for me, that people element is as if not more important than the technology.

I think this word, enablement, comes up a lot, but it is one that I think requires a multifaceted consideration.

You have to be thinking about, what does it look like to enable our organization to operate at different AI-assisted levels all the time? And it requires a lot of different lenses.

I think it's why it matters so much to build the trust and the transparency for all of the people around the entire organization to sort of pipe up and say, what I recognize is that we're not doing this in a way that's consistent with this vision we're trying to cast.

For example, maybe we're having these stand-up meetings every day and they need to be in person.

Do we really need to have them in person? Do they really need to be every day? Could we maybe have a certain portion of the people who are not contributing that day only send their transcription bots to the meeting? What are we going to do about that? putting a policy in place that's respectful, but also recognizes not everybody needs to be in every meeting all the time, yet they might need to get the transcript of that meeting.

And can we start adapting some of those processes and adopting some of those processes into a more future ready approach?

If you are honest and transparent around what you're doing, if you're clear about what the risks are and you're clear you're striving to manage them, they'll accept a few bumps in the road within reason if they know that you are striving to manage it.

What do you think of that?

Yeah, it's I think that we're.

in a very young, immature stage of using these tools.

You know, they only came out for public consumption.

They've been around for a while, various versions of them have been around for a while, but the ChatGPT era of, you know, public use of these tools is only since November, 2022, you know, that's only almost four years ago.

And that's just not long enough that we have our feet under us in terms of how to use them well.

But already we've been through several generations, I think, of people knowing what common tells are in AI-generated writing.

really maximize your presence by using these tools? How do you get out there?

Where do you see the biggest kind of gaps in alignment? And what risks, you know, does that create at a global level? It's quite a big question.

Yeah, it is a big question, but it is a fairly, it's fair that it's such a big question because I think it does have those kind of sprawling edges.

There's a sense that I think these tools create an algorithmic model that is a new kind of diplomacy, right? I think at the international geopolitical level, we're talking about new ways that AI has enabled and algorithmic intelligence has enabled a different set of narratives across borders and across decision making.

It's rarely autonomous in More often, it's human intent that has been laundered through systems.

And that's one of the factors that changes the way that AI and algorithms show up at the most sort of geopolitically macro kind of picture and the way that that influences our lives.

And then I think that leaders of companies and organizations who are kind of a level down from that level of abstraction are feeling the pinch of needing to scale and needing to feel like we just talked about competitive with this kind of race perception.

And that is something that is leading to these kind of outsourced consequences or outscale consequences that are coming back to bite in some cases.

So how it's really difficult to measure, isn't it?

And I'm just curious to understand, again, through your lens, what you think some of those risks really look like.

Yeah.

So the way I play this out is in my work with leaders at all levels, one of the models that I often talk about is there's a three-question series that's really part of a tech humanist approach to making these decisions.

And it's, you start with, can we? You ask, then, should we? Yeah.

And then you ask, how should we? And if you skip the should we in there, the can we to how should we becomes more of instructional guidelines on operations.

So just asking can we is a capabilities question.

And then how should we becomes about deployment.

So I'm curious to see, is that something that you've seen in organizations and kind of what are your thoughts around that?

Mm-hmm.

So you got to understand what your business is and does f- primarily, foundationally, and you got to understand what it is as a, as a lever in culture.

You got to understand what it is in terms of brand and experience.

You got to understand all these things, then you can start talking about what data are we collecting at every touch point in order to make those decisions more effectively.

And then we can start talking about what technologies should we be rolling out that allow us to accelerate along all of those different dimensions.

Now, that doesn't mean that we don't also have an innovation, experimentation mindset, and that we're not sort of playing with the tools and technologies to understand them.

Hmm

...

But is that more or less humanist than a MAGA person?

[laughs] Uh, I, I would think so, actually, because a MAGA person's viewpoint isn't necessarily centering the human experience for the good of the human experience.

It's for the good of the ideology.

I think what, what Dr. Tressie's viewpoint is, and I don't mean to speak for her, but, uh, where, where it seems to square with mine, is we're thinking more about the collective of human experience, more about what's good for the majority of people that doesn't short sell some human experience for other human experience.

That's looking to try to create the best futures for the most people instead of it being about, you know, how do I profit the most from this moment? How do I, you know, uh, gather as much as I can for myself and, and restrict the f- the rights of other people and, and use this to sort of trample down on other people? So those things are, don't run, to my mind, those don't run consistent with humanist values i- in a, in a purely humanist sense.

And in a tech humanist sense, they can't, because technology is the thing that scales.

And when you called me out on sort of- Uh, consultant language earlier, talking about at scale, the reason that it is a relevant term to use in this consideration is because you're talking about technology bringing to scale the decisions that are being made.

... tech, tech people? Uh, business people?

all the way through systemically to everybody flipping a hamburger that's not easy to do right now introduce a world of technology and all of a sudden it's a robot or a machine flipping the hamburger or animatronics is the new disney uh the new mickey mouse or whatever um is that gonna now take a take that as an example or some other corporate example and say does that does that put that purpose at risk does it make it harder for them to navigate

No, and I think what makes it difficult is extrapolating from that core purpose, that organizational strategic purpose articulation that you're saying that make people happier or the way that I often talk about it with Disney theme parks and the Disney organization is create magical experiences, which is another way to get at that, right? And so to your point, you can have people who are answering phones related to booking vacations in the Disney world, or you can have somebody who's sweeping Main Street in a theme park.

And they still both have that understanding of go above and beyond to make sure that people have what they need to have that magical experience.

But then when you bring it to the technology level, what's fascinating about it is that same articulation of purpose, create magical experiences or create happiness, can allow you to do things like invest in the My Magic Band wearable that encodes a lot of information about preferences and your purchase information, your sort of your credit card, your reservation at your hotel, what kind of restaurant reservations you have or dietary preferences you have, all kinds of information can be encoded in that wearable.

And all of a sudden you have a more magical experience as you move through the theme park or through the Disney experience.

And that I think is a very clear example at a multiple billion dollar investment level of how technology can be part of amplifying this organizational strategic purpose at the corporate level, at that level of how do we as a company kind of use technology to advance this practice, this purpose as practice.

And yeah, to your point, if you start thinking about all of the animatronics and things like that, you do start running into, um, governance issues and considerations that you want to be careful about.

I am really profound and

However, do you think offloading an awful lot of the jobs that we're meant to be doing ends up creating Output that is either good enough or at a minimum viable level, or at worst, just very synthetic and inauthentic.

Yeah, I think it's all of those things, right? We are all experiencing the glut of generative AI slap, right? We're all getting that everywhere, especially LinkedIn is probably one of the best places to go and experience lots and lots of slap.

And I think companies recognize that that's a risk and there's all kinds of programs different companies have to try to, you know, create education programs for their employees to adopt the use of AI.

Like we've got an enterprise co-pilot and we want everyone using it and we want them to be using it in these particular ways and with this particular structured data set, but the Problem tends to be that not everyone has the same level of skill at using the tools or kind of prompting well and making sure that they're getting the right kind of output.

Not everyone understands what data sets they should use or shouldn't use.

This is something that's kind of been an ongoing evergreen problem throughout my career.

I remember working 20 years ago on an intranet for a hospital corporation.

You know, what kind of skills are people looking for and how do you sort of point them in the direction of thinking, well, maybe you have to look at your workforce in a more holistic fashion as opposed to being quite, you know, granular and task oriented? Yeah.

H- what's your thought, what's your guidance, Kate, around how leaders should be approaching decision-making in this kind of environment? Which I don't think is going to change in the, in the foreseeable future.

No, I, I think every time I ask an audience, you know, to raise their hands if they think that the future is looking very uncertain and that e- everything is moving so fast, you know, a- and raise your hand if you think that's going to be any different or better [laughs] in the years to come, and, like, no hands go up after almost every hand goes up.

Uh, everybody is sort of in agreement on this.

Like, I think we all sort of anticipate a very continuous sense of acceleration.

And the problem with that is, as you say, that what we're dealing, what we're accelerating toward and what's causing that acceleration is this, this increasingly powerful set of technologies that bring capacity and scale and consequence to the outcomes of those decisions.

So the way we use them matters, and we need to be thoughtful about how we implement them, not just because we're missing out on a lot of their capability if we don't implement them well, although that's true too.

But because if we implement them poorly, there will be actual, uh, there, there will be suffering on the other end of that, whether it's because of human job loss or because of the, um, the people in our user and customer communities who won't get ideal outcomes from what we are trying to serve them, uh, and so on.

So there's a, there's a lot going on at the same time, isn't there?

And can you talk about that? When did that idea come to you? Where did you see that as a problem in a lack of clarity in what people were trying to do?

Yeah, I think that I was seeing that in the questions and answers and interactions I was having with executives and business leaders after keynotes.

You know, having these sessions where in many of – and this probably happens for you too – where in many of the conferences I speak at, one of the sort of perks that an organizer might set up is – You get 45 minutes alone with the VIP attendees and you get to sort of field their questions in a much smaller group.

And those kinds of things are just, they're gold.

They're so useful.

And I think the attendees really enjoy that too.

So note to the event planners, that is a really nice feature.

Actually, this is I'm going to change this tool because it's going to be more useful for them.

Like, what are you seeing as trends of where it's working and where it isn't in terms of deploying it on workflows?

Yeah, I saw a great webinar that Gartner produced the other day on agentic AI, and I recommend this.

They have it on demand for folks.

I would even suggest the URL if I had it at hand, so perhaps I can provide it and you can provide it in show notes.

There is a webinar that they offer on agentic AI, and they offer this really nice model that sort of distinguishes between AI agents and agentic AI.

I think that's a really important construct that is very easy to overlook.

They're talking about agentic AI as one area under the umbrella of automation broadly, right? And then you think about agentic as the area in which you can sort of think about some mechanism of automation, some decision process that's going beyond just, I'm prompting, I'm getting a response, I'm prompting, I'm getting a response, right? At this point, I'm saying, I'm prompting.

How are you thinking about that dynamic?

How do you think about it in the context of a smaller org?

Yeah, I think it's probably in some ways the same where you want to be thinking not about replacing the actual person who's doing the job, but about thinking about which tasks are overburdened by the manualness of them in a sense, right? Like which tasks could we augment By bringing AI or by bringing agentic AI or generative AI or any of these kinds of tools into the operation.

And how do we make people more effective? How do we make people able to bring more of their humanness to the interaction, more creativity, more creativity? context awareness more uh emotional intelligence you know more of their good judgment kind of skills to the interaction as opposed to you know trying to find some kind of generic average that we water everything down to i think that is very much not the goal for almost any organization it's really about trying to figure out how do we take what works very very well and then differentiate the pieces that could actually be served more by automation and by augmented AI.

One of the pieces I'll spell out here even more clearly is that one of the pieces I think we've gotten wrong for the last few years is that when we talk about the future of work, we often conflate the idea of the future of work and the future of jobs together with the future of tasks and the future of productivity and all of those things are very different from one another and if we stop and break those out and really think about them in a more disciplined way and and think about how that applies inside of our organizations even an organization as small as mine it makes a big difference to be able to think about, I'm not trying to automate people away.

I have human researchers that I work with, yet I use Perplexity and Cloud and all of these generative AI tools every single day, all the time.

It's just a very different type of workflow now where I feel more productive going into my meetings with my human research team, being able to say, I'm not going to waste your time with this low level stuff.

I've got, I'm briefed, I'm up to speed, but where you add tremendous value with your incredible PhD level research is that you're able to come and bring me like really, really rigorous studies and anecdotes and examples from industry, those kinds of things that are going to be much, much harder for me to generate with generative AI.

Is there a particular framework that you apply to help organizations through this? How do you put that into context so business leaders can think in those terms? Yeah.