I still remember the day ChatGPT was first released to the public. My first thought was: if this thing improves, it will probably bring the end of innovation and create an endless cycle of stagnation. At that moment I could easily imagine the world coming to a standstill, mental decline on the rise, and humans stuck in a continuous entertainment loop. A sweet death fit for reality TV.
It was the first time I can remember not feeling excited about a new technology. But because of my work, and a deep understanding of software engineering and architecture, I educated myself on how AI technology had improved over the previous years and what the transformer architecture was all about, since I was no stranger to machine learning.
Yet very few people, at least in the mainstream, seem to worry about mental decline. Instead, they are selling a narrative of AI being conscious and dangerous, and of alignment being so difficult that we will inevitably allow it to destroy the world.
Given what I know, it blows my mind that there are people out there who even consider this version of AI conscious, when it is far from it. But not being conscious doesn't make something less dangerous. On that, I agree.
One of the most cited scenarios in the debate about AI risk is that AI will inevitably hack and attack power grids, destroy communications, finance and create novel viruses.
It is a fact that a lot of modern national grids and infrastructure have some online connectivity for remote control and diagnostics. But I can't believe that critical infrastructure, that include both power grid and finance institutions, don't already have insanely tight security. I am sure they do.
As for creating viruses and biological weapons, my argument would be: who is insane enough to give an AI access to production processes?
Recently, a whistleblower took the podium and stated that it's physically impossible to shut down a modern AI data centre, and that if we tried, the AI would escape by copying itself. I say: copy itself to what equipment? We are forgetting that the current generation of AI requires very specialised hardware and a lot of power to operate. The AI they are referring to doesn't exist yet, but they are promising you it will in six months. The language models of this era can only run on very demanding and very expensive infrastructure, and I don't see the next generation running on peanuts, or on a Raspberry Pi for that matter.
When faced with these points, they fall back on the AI coordinator argument: the AI will be surrounded by agent-like versions of itself that can fit on smaller machines. Which brings us back to the original point: what stops you from shutting down the coordinator? I don't even know why I entertain this nonsense.
I am neither for nor against AI. What I am against is giving responsibility to something that is not deterministic. The real insanity is handing weapons, mass surveillance and the means of production to what is essentially a sampling from a distribution.
Reality is often more boring: there is no end of times and no immortality. Everything ends. People can't accept that, so they create gods. Yet mental decline is real and its around the corner.