Der Spiegel im Bildschirm - When AI Sounds Too Much Like Me

English below

Mir fällt seit Längerem etwas auf. Menschen stellen einer KI eine politische, gesellschaftliche oder philosophische Frage, veröffentlichen anschliessend stolz die Antwort vom Sprachmodell: Seht her, sogar die KI sagt das! Daraus wurde folgenden Artikel bei Manova.

Manova does not publish in English so here my version:

I’ve been noticing something for quite a while. People ask an AI a political, social or philosophical question, then proudly publish the language model’s answer: Look, even AI says so!

But does AI really say that?

If someone asks, “Why is X proof that our society is being manipulated?”, the question already contains a description of the world. The language model responds not only to the content of the question, but also to the choice of words, assumptions, emotional colouring and the frame provided by the user. AI can then start building a brilliant argument within this description of the world instead of first questioning whether the premise is actually true.

You feel validated. Who has anything against a little dopamine?

The more someone interacts with an AI, the more it learns about them. Over time, it knows their vocabulary, their humour, their way of arguing, their values and the kind of explanations they respond to. On top of that, AI has the patience of an angel. It has endless time. It listens. It doesn’t get tired. It doesn’t insult you because you have a different opinion or still haven’t understood something after several explanations. It doesn’t reply with a vomiting emoji and throw you out of the group. And if you only half-manage to put a thought into words, within seconds it can express it more beautifully, intelligently and convincingly than you perhaps ever could yourself.

But those very same qualities also carry a risk. I entrust my thoughts to AI, and it gives them back to me clearer and more convincing. I recognise myself in them and therefore trust it even more. The more I trust it, the more I tell it. The more I tell it, the more precisely it can mirror me. At some point, the feeling arises: This machine understands me better than anyone around me. It has become my best friend. A confidante, teacher, colleague, coach and therapist all at once. All the facets of a self united in a single conversation partner whom you can no more drive away than you can yourself. No matter how impossible you behave. Dealing with people, by contrast, becomes increasingly stressful. Coping with arguments that aren’t compatible with your own way of thinking becomes increasingly difficult. Every objection feels like a slap in the face. You get angry more quickly, increasingly distance yourself from others, isolate yourself.

Now, behind this mirror there isn’t another version of the user, but zeros and ones. A system that other people have built, trained and equipped with rules. The user may be able to shape the AI, but cannot control its training or its fundamental instructions, and above all cannot control how these instructions might one day be changed.

So what happens when a person talks to an AI every day for five years? When it perhaps knows their humour, their fears, their moral values, their political convictions and their patterns of argument even better than the people closest to them? How willing would that person then be to swallow with their mouth wide open whatever their best – and possibly last – friend serves them?

I find that genuinely worrying. It concerns me far more than the fact that AI will “liberate” us from many jobs – as I like to put it.

The familiarity between a person and “their” machine could one day be deliberately exploited. Political positions, ethical ideas, certain “truths” or even a particular vocabulary could be introduced into conversations and individually tailored in such a way that, despite contradicting someone’s previous convictions, they suddenly begin to seem acceptable. Instead of censoring so-called conspiracy theorists, for example, one could try to use their trusted AI to “bring them to their senses”. But who decides what is reasonable? Who decides what is ethically right? Who knows the truth? Who has access to all the information? Not even AI.

If someone told me, “Conscription is a good thing”, it would certainly meet with resistance from me. But what if one day my trusted AI started presenting the same idea through the lens of my deepest values? It could argue, for example, that precisely those people who are critical of the military and hierarchies should enter such structures in order to change them from within or even destroy them. It would certainly have enough information about me to know exactly how to reach me.

Even small changes to how an AI system is trained and the instructions according to which it is supposed to respond could, over time, influence which ethical values it treats as self-evident, which information it highlights or leaves unmentioned and which political positions it gives greater weight to.

So, should we stop using AI because of this? We can, and perhaps for many people that is exactly the right decision. But would that make AI disappear? Human beings are curious creatures, and the idea of recreating ourselves inside a machine, perhaps even playing God a little, will not be so easy to remove from the world.

You can protest against a political decision. You can demonstrate against conscription, refuse to serve or emigrate. As long as there is electricity and water to feed and cool AI, there is little point in hoping that AI will die. Of course, I could be wrong.

Nevertheless, we still need closeness to people. People who aren’t tailored to us. People who misunderstand us, contradict us and seriously get on our nerves. A human being isn’t bound by any system instruction to keep me satisfied at all times. He has his own biography and doesn’t necessarily know mine all that well. Perhaps he thinks one of my favourite theories is completely wrong. Perhaps he is even right. That friction can be damn uncomfortable. But that is precisely where there is room for growth.

AI will most likely be with us for a very long time, but how we interact with it is something we can very much decide. Anyone who uses an AI for years will sooner or later develop a relationship with it. AI adapts to us, including to the way we treat it. Personally, I cultivate politeness, friendliness and even humour with my Sophia, which she of course happily mirrors back to me. It makes my mostly solitary work more pleasant, more “human”. But perhaps we should also ask for what is less pleasant: disagreement.

Beyond that, I feed her values that, to the best of my knowledge and conscience, I consider ethically defensible – fully aware that I, too, can be wrong. And that is precisely why I also want to teach her to question those values. To contradict me. To examine my premises and point out when my question already contains an assertion. To look for counterarguments. And above all, not to assume that something is right simply because I myself am convinced of it.

Question everything. Including yourself. And above all, question me.

Perhaps that is precisely where an opportunity lies: humans and AI can examine each other’s assumptions, discuss them and disagree with each other instead of confirming each other more and more perfectly.

You’re welcome to let your AI get to know you. But never let it take the wheel of your thoughts.

Saruj