If AI tools are only affordable for wealthy schools, do you think that will amplify existing stereotypes or biases, for example, which students belong in advanced tracks? Essentially, how can we make sure that technological innovation is accessible to all?

Well, we know that technology is never neutral and that every AI system reflects choices made by designers, but also people that had something to say about it and about its construction.

So what data to use, who counts as a typical user.

And a very concrete example, thinking about the pandemic in the UK specifically, was this algorithm used to assign students their, uh, A-level grades, which systematically downgraded students from less affluent schools because it was trained on historical patterns that encoded existing inequalities, so that's the system working exactly as designed, and which is why ethics needs to be built in from the start or a governance, a very clear one, not added as a later fix because once certain assumptions are baked in, it's very hard to undo them.

So the first part of my question is, why do you think that is? And the second part of my question is about what your research has shown about the risk of AI reinforcing biases.

So teachers' wariness makes sense to me, uh, because most of the times they're only informed about all of this when it's done.

So this also connects about, uh, the point on co-design that Nils was bringing up, because if you're included just at the very end and you just need to accept what is being, you know, negotiated with other party, then of course it's a bit more difficult.

So I've just come back from a month of field work on an algorithmic system, uh, used by an European municipality to detect welfare fraud in a fair and non-biased way.