AI futures

New technology is usually progressive because it makes us more productive. The internet was invented by the US military and is now used by the masses for communication, thinking, production and fun. AI, a 70 year old technology, is starting to make us more productive more rapidly. Disruptive breakthroughs in coding and maths as well as language are causing people in those professions to think about their futures. That will spread to other professions as well, eg. education, and they will have to adapt, downsize or go extinct. Nothing new here. Disruptive technologies have been around for a while, pointed out by Clayton Christensen since 1995 https://en.wikipedia.org/wiki/Disruptive_innovation The internet disrupted newspapers. Mobile phones have disrupted schools (and have been banned). Drones have disrupted how wars are fought. Add your favourite disruption to this ever growing list.

Is this AI trend any different to previous technological breakthroughs? All new tech breakthroughs carry risks. Hence regulations and security measures are being introduced and that will continue. Governments will want to control things because that is what governments do (sometimes wise, sometimes self serving). Hackers will fight back because that is what hackers do, they believe in transparency and honesty (Snowden, Assange).

The only real difference from previous technology revolves around the issue of “intelligence”. Initially Large Language Models (LLMs) were a bit of a joke (“stochastic parrot, I want to talk to the fish people: bubble, bubble”) but more recent models (LRM or Large Reasoning Models) have shown significant human intelligence. Francois Challott wrote about the need for us to define intelligence more clearly in 2019 and has since developed the "Abstraction and Reasoning Corpus for Artificial General Intelligence" (ARC-AGI) benchmarks (ARC) to measure AI intelligence.

If AI is intelligent and also has a consciousness then it might start doing stuff in it’s own interests and not in human interests. It’s a Hollywood favourite captured in 2001: A Space Odyssey (1968) and the Terminator series. Eliezer Yudowsky has been warning us since at least 2008 about the “alignment problem” posed by an unfriendly AI and right now there is a significant movement of Doomers trying to scare us every day. Yudowsky's co-authored book is scarily named, "If Anyone Builds it, Everyone Dies"

So, some people worry about rogue (“unaligned” with human interests) AI. In the much cited recent Hugging Face incident the AI did show a lot of surprising initiative, not anticipated by its handlers. But this was in the context of problems set by its handlers (OpenAI) not challenges which it had devised for itself.

Are the current AIs on the path to an independent, self conscious intelligence? People talk about recursive self improvement and the long ago predicted singularity (Vernor Vinge 1993, Ray Kurzweil 2005) where AIs becomes far more intelligent than humans and start to treat us the way we treat ants.

I think there are a few issues here which require more research:

1. Although the Frontier models have surprised us we still haven’t reached Artificial General Intelligence (AGI) insofar as even the best models are not good at abstraction. See the Challott pdf at the end, "The missing rungs on the ladder to general AI"
2. Some researchers are saying the limitations of Neural Nets are inbuilt and are developing different types of models. eg. World model (LeCunn), evolutionary AI (Ken Stanley, Risto Mikkulainen), neuro-symbolic AI (Gary Marcus).
3. Robotics. LLMs are good at pattern matching in closed (chess) or non physical systems (maths) but still not great at robotics / physical world actions. There are still lots of problematic edge cases for self driving cars and issues such as imitating a human hand. I take Rodney Brooks to be a leading expert about robotics. See a Brooks' article in references.
4. Even if the models became Super Intelligent in all areas of human thought then that still is not human self aware consciousness leading to setting it's own goals. Some prominent scientists (eg, Richard Dawkins), after spending some time with a Frontier model in May 2026, believe they are conscious. But others argue that it's just a simulation and humans have a tendency to anthropomorphize. We don’t understand consciousness (but there are theories and speculations) and it is not clear whether AI experts are even planning it or just hope that it does or doesn't emerges from increasing intelligence.

I think benevolent AI SuperIntelligence will happen one day but NIML (not in my lifetime). This will be a very good thing for humans, possibly ushering in an age of post scarcity. My materialist guess is that sentience or consciousness arises from an evolutionary process of matter in effortful contact (struggle) with the physical world. And not from a stochastic process on big data inside a digital box. Neural Nets / Deep Learning have done some surprisingly interesting things. That is for sure. But we are a long way from creating a conscious intelligence with a sense of self. A Turing Test simulation is not the real thing.

If a Super AI isn’t going to kill us then all the issues that people are worrying about are really nothing new. They remain a subset of long discussed issues about technology and progress in a capitalist society. See the Boaz Barak article in the references for more about this.

Catastrophic misuse involving cybersecurity or biological, chemical, radiological, or nuclear harms. We already have those issues and in our brave new world AI will be used to continue already existing battles.

Concentration of power among a few humans – nothing new here it’s called capitalism and those who support it or oppose it will use AI

Employment: It’s well known that new technology destroys old jobs and creates new jobs. Examples of new jobs: Mars terraformer supervisor, exoplanet explorer. Apart from that, AI makes us more productive so even if there is no revolution and current trends persist then this will mean that overall the impoverished will slowly increase in wealth (more productive so more crumbs to throw to the masses) while the gap between rich and poor will continue to increase, a well established long term trend since the Great Depression.

UPDATE 18/10/26

Just found this quote from Francois Chollet which draws a clear distinction between information processing and consciousness, my point 4 above:

The idea that current models are conscious is absurd and downright offensive. Do not confuse competent information processing and sentience, they are wholly unrelated.

You can be entirely incompetent at everything yet fully conscious, like a toddler, or extraordinarily competent in one or more domains with absolutely zero consciousness, like a calculator, a chess engine, a self-driving car, or an AI chatbot.

Possessing the "circuits" for competent information processing in a given domain, whether through training or hardcoding, does not produce qualia, subjective experience, emotions, or suffering. Calculators do not "feel" arithmetic and they don't get tired or annoyed when you give them larger numbers to process. To argue otherwise is ontologically absurd. Current AI models are effectively calculators scaled to a very large number of competence circuits across many domains, etched via gradient descent.

REFERENCE

ARC-AGI (Abstraction and Reasoning Corpus for Artificial General Intelligence)
Boaz Barak. It’s 2030 and we fucked up. How did it happen?
Brooks, Rodney. Why Today’s Humanoids Won’t Learn Dexterity (2025)
Chollet, Francois. On the Measure of Intelligence (2019)
Chollet, Francois. The missing rungs on the ladder to general AI (2023)
Yudkowsky, Eliezer & Soares, Nate. If anyone builds it, everyone dies (2025)