What Do We Want AI to Be Good For?

A question beneath the question

I attended a seminar titled Rethinking AI: Entrepreneurship, Technology & Social Good. Hosted by Fast Forward 2030 in collaboration with UCL Institute for Global Prosperity. I expected another evening of breathless prophecy. Instead, I left with a quieter and more unsettling thought.

We spend a great deal of energy asking what AI can do. We spend far less asking what we want it to do. The first is a question of capability; the second is a question of purpose. Philosophers since Aristotle have insisted that we cannot judge a tool well until we understand what it is for.

The room was deliberately mixed: enthusiasts, sceptics and a good many people suspended somewhere in between. That tension turned out to be the most honest thing about the evening. What follows are the threads I have not been able to put down since.

1. The tool does not change us until we change ourselves

One speaker, Mathew De la Fuente, recalled the Solow paradox. When computers entered the workplace, everyone expected a productivity revolution. For years, the measurable gain was almost nothing.

The reason is instructive. A technology does not arrive as productivity; it arrives as potential. Turning potential into practice demands something slower and more human:

– training people;

– redesigning processes;

– integrating systems;

– checking what comes out;

– managing the transition;

– and, above all, changing how people actually work.

We were told that a large share of AI use in one major organisation is simply converting PDFs into editable documents. It is funny, and it is revealing. We hold something that resembles a new kind of intelligence, and we ask it to do our formatting.

I do not read this as failure. I read it as a portrait of every new tool in its infancy. We first use the unfamiliar to do the familiar, and only later discover what it was really for.

2. Tasks dissolve before jobs do
The speakers drew a useful line between disrupting jobs and disrupting tasks. Copying, pasting, formatting, summarising and routine reporting are already yielding to automation. That does not mean the role that contained them disappears.

In fact, a new layer of work may grow around AI itself: managing it, checking it, integrating it and supervising it. The comparison offered was the move to cloud computing, which created a great deal of work simply in making the transition happen.

So the anxious question, “Will AI take my job?”, may be the wrong one. A more searching question is this: which parts of my work should a human stop doing, and what should that human do instead? It asks us to know what our work is actually made of, and which parts of it carry meaning.

3. To be known is not the same as to have consented
Healthcare offered the most hopeful examples of the evening. AI can sift vast quantities of medical data to support earlier diagnosis and prediction. Earlier detection means earlier intervention, and earlier intervention means better lives.

Yet every gift of foresight casts a shadow. If a system can predict that I am likely to develop an illness, who is entitled to that knowledge? Could an insurer raise my premium on the strength of it? Could something I never chose to disclose be inferred from what I did?

This reshapes an old ethical idea. Consent used to be about what I handed over. Now it must also concern what can be deduced about me. The question is no longer only “What data did I give you?” but “What can you know about me from it?”

4. A machine will pursue exactly what we ask of it
This was, for me, the heart of the evening. Tell a system to optimise for efficiency and it will. Tell it to maximise profit and it will. Tell it to maximise human wellbeing, and we must first agree what wellbeing is, a question humanity has argued over for two and a half thousand years.

There is also a profound difference between saving individual people and optimising for humanity as a whole. It is the old tension between the individual and the collective, now written into code. The two aims can lead to very different decisions.

Which brings us to the political question hiding inside the technical one: who decides what AI is for? Companies? Governments? Scientists? the model builders? or society as a whole?

A machine has no purposes of its own. It inherits ours, including the ones we never once examined fully.

5. The quiet loss of the unexpected
Trinity Stenhouse (an Astro particle physicist and AI researcher) raised a subtler danger. Imagine a personal assistant that knows your tastes, habits and past choices. It plans your holiday, chooses your restaurant and arranges your weekend. It sounds wonderful.

But such a system may become very good at giving you more of the person you already are. And we do not become ourselves that way. We grow through the unplanned: the wrong turning, the book picked up by chance, the stranger who changes our mind. A life perfectly predicted is a life with fewer doors in it.

There is a second, gentler loss too. If we grow used to asking a machine for advice, we may slowly stop asking one another. Counsel sought from a friend is never only about the answer; it is also how friendship is made.

6. Who benefits, and what it stands on
Technologies tend to reward those already equipped to use them. The people and organisations with money, education, data, infrastructure and skilled staff are best placed to gain from AI. Communities without them risk falling further behind.

AI may therefore narrow inequality in some places, through education, healthcare and access to knowledge, while widening it in others. Access to AI could become as fundamental as access to any other public infrastructure.

And infrastructure is the right word. AI feels weightless, a matter of words on a screen. It is not. Behind every effortless click stand data centres that consume land, electricity, cooling, water and chips. As people who shape the physical world, we should be the last to forget that the digital always rests on the material

7. Something new, or something familiar?
One speaker went further than the rest. He argued that we may be building systems that behave less like software and more like a new form of intelligence altogether. If we create something more capable than ourselves, how do we keep it aligned with human interests?

There was lively disagreement, which felt entirely healthy. Then came a welcome challenge from the audience. A man who has worked in technology for decades reminded us that we have been here before. Computers, the internet and automation all arrived wrapped in fear, and none of them destroyed society.

His worry lay elsewhere. The algorithms already among us, in social media and recommendation engines built to capture attention, are shaping behaviour now, particularly among the young. Perhaps the wiser posture is not fear of what AI might become, but attention to what technology is already doing to us.

Don’t sleepwalk into it
The message I carried home was neither “embrace AI” nor “fear AI”. It was simpler and more demanding: don’t sleepwalk into it. We need to know enough to choose deliberately:

– what we automate;

– what we delegate;

– what data we give away;

– which decisions must stay human;

– which skills we need to grow;

– and what kind of workplace we actually want.

Beneath all of these sits one old question: what are human beings uniquely good at, and how do we protect it? Empathy. Judgement. Creativity. Relationships. Experience. Leadership. Curiosity. The instinct that something does not feel right.

And, in our own profession, perhaps most of all: understanding the consequences of a decision beyond the task in front of us. A building outlives the drawing that describes it. Our work has always asked us to think past the immediate, and that habit of mind may be exactly what this moment needs.

AI is not only a technology story. It is a story about people, work, economics, the environment and society, and its next chapters are being written now, partly by us. So rather than asking whether AI is good or bad, I would like to ask a different question, and to ask it with you:

What do we want AI to be good for?

Thinking Piece by Nadine Barake