AI futures

New technology is usually progressive because it makes us more productive. The internet was invented by the US military and is now used by the masses for communication, thinking, production and fun.

AI, a 70 year old technology, is starting to make us more productive more rapidly. Disruptive breakthroughs in coding and maths as well as language are causing people in those professions to think about their futures. That will spread to other professions as well, eg. education, and they will have to adapt, downsize or go extinct.

Nothing new here. Disruptive technologies have been around for a while, pointed out by Clayton Christensen since 1995. (https://en.wikipedia.org/wiki/Disruptive_innovation) The internet disrupted newspapers. Mobile phones disrupted schools (and have been banned). Drones are disrupting how wars are fought. Add your favourite disruption to this ever growing list.

Is this AI trend any different to previous technological breakthroughs? All new tech breakthroughs carry risks. Hence regulations and security measures are being introduced and that will continue. Governments will want to control things because that is what governments do (sometimes justified, sometimes self serving). Hackers will fight back because that is what hackers do, they believe in transparency and honesty (eg. Snowden, Assange).

In the case of the current AI (Deep Learning neural nets) the main significant difference from previous technology revolves around the issue of “intelligence”. Initially Large Language Models (LLMs) were a bit of a joke (“stochastic parrot, I want to talk to the fish people: bubble, bubble”) but more recent models (LRM or Large Reasoning Models) have shown significant human intelligence.

Francois Challott wrote about the need for us to define intelligence more clearly in 2019 and has since developed the "Abstraction and Reasoning Corpus for Artificial General Intelligence" (ARC-AGI) benchmarks (ARC) to measure AI intelligence. See the references for more detail.

If AI is intelligent and also has a consciousness then it might start doing stuff in it’s own interests and not in human interests. It’s a Hollywood favourite captured in 2001: A Space Odyssey (1968) and the Terminator series.

Eliezer Yudowsky has been warning us since at least 2008 about the “alignment problem” posed by an unfriendly AI and right now there is a significant movement of Doomers trying to scare us every day. Yudowsky's co-authored book is scarily named, "If Anyone Builds it, Everyone Dies (2025)"

So, some people worry about rogue or unfriendly (“unaligned” with human interests) AI. In the much cited recent Hugging Face incident the AI did show a lot of surprising initiative, not anticipated by its handlers. But this was in the context of problems set by its handlers (OpenAI) not challenges which it had devised for itself.

Are the current AIs on the path to an independent, self conscious intelligence? People talk about recursive self improvement and the long ago predicted singularity (Vernor Vinge 1993, Ray Kurzweil 2005) when AIs become far more intelligent than humans and start to treat us the way we treat ants.

I think there are a few issues here which require more research:

1. Although the Frontier models have surprised us we still haven’t reached Artificial General Intelligence (AGI) insofar as even the best models are not good at abstraction. See the Challott pdf at the end, "The missing rungs on the ladder to general AI"
2. Some researchers are saying the limitations of Neural Nets are inbuilt and are developing different types of models. eg. World model (LeCunn), evolutionary AI (Ken Stanley, Risto Mikkulainen), neuro-symbolic AI (Gary Marcus).
3. Robotics. LLMs are good at pattern matching in closed (chess) or non physical systems (maths) but still not great at robotics / physical world actions. There are still lots of problematic edge cases for self driving cars and issues such as imitating a human hand. I take Rodney Brooks to be a leading expert about robotics. See a Brooks' article in references.
4. Even if the models became Super Intelligent in all areas of human thought then that still is not human self aware consciousness leading to setting it's own goals. Some prominent scientists (eg, Richard Dawkins), after spending some time with an Anthropic Frontier model, Claude, in May 2026, believe they are conscious. But others argue that it's just a simulation and humans have a tendency to anthropomorphize. We don’t understand consciousness (but there are theories and speculations, eg. Dan Dennett, Anil Seth) and it is not clear whether AI experts are even planning it or just hope that it does or doesn't emerge from increasing intelligence.

I think benevolent AI SuperIntelligence will happen one day but NIML (Not In My Lifetime). This will be a very good thing for humans, possibly ushering in an age of post scarcity. My admittedly hand-wavy, insufficiently researched materialist guess is that sentience or consciousness arises from an evolutionary process of life in effortful contact (struggle) with the physical world. And not from a stochastic process on big data inside a digital box. Neural Nets / Deep Learning have done some surprisingly interesting things. That is for sure. But we are a long way from creating a conscious intelligence with a sense of self. A Turing Test simulation (from his 1950 article) is not the real thing.

If a Super AI isn’t going to kill us then all the issues that people are worrying about are really nothing new. They remain a subset of long discussed issues about technology and progress in a capitalist society. See the Boaz Barak article in the references for more about this.

Catastrophic misuse involving cybersecurity or biological, chemical, radiological, or nuclear harms. We already have those issues and in our brave new world AI will be used to continue already existing battles.

Concentration of power among a few humans – nothing new here it’s called capitalism and those who support it or oppose it will use AI

Employment: It’s well known that new technology destroys old jobs and creates new jobs. Examples of new jobs: Mars terraformer supervisor, exoplanet explorer, yes I've been reading too much scifi. Apart from that, AI makes us more productive so even if there is no revolution and current trends persist then this will mean that overall the impoverished will slowly increase in wealth (more productivity so more crumbs to throw to the masses), while the gap between rich and poor will continue to increase, a well established long term trend since the Great Depression.

UPDATE 18/10/26

Just found this quote from Francois Chollet which draws a clear distinction between information processing and consciousness, my point 4 above:

The idea that current models are conscious is absurd and downright offensive. Do not confuse competent information processing and sentience, they are wholly unrelated.

You can be entirely incompetent at everything yet fully conscious, like a toddler, or extraordinarily competent in one or more domains with absolutely zero consciousness, like a calculator, a chess engine, a self-driving car, or an AI chatbot.

Possessing the "circuits" for competent information processing in a given domain, whether through training or hardcoding, does not produce qualia, subjective experience, emotions, or suffering. Calculators do not "feel" arithmetic and they don't get tired or annoyed when you give them larger numbers to process. To argue otherwise is ontologically absurd. Current AI models are effectively calculators scaled to a very large number of competence circuits across many domains, etched via gradient descent.

REFERENCE

ARC-AGI (Abstraction and Reasoning Corpus for Artificial General Intelligence)
Boaz Barak. It’s 2030 and we fucked up. How did it happen?
Brooks, Rodney. Why Today’s Humanoids Won’t Learn Dexterity (2025)
Chollet, Francois. On the Measure of Intelligence (2019)
Chollet, Francois. The missing rungs on the ladder to general AI (2023)
Dennett, Dan. From Bacteria to Bach and Back: The Evolution of Minds (2017)
Seth, Anil. The Mythology Of Conscious AI (2026)
Turing Test
Yudkowsky, Eliezer & Soares, Nate. If anyone builds it, everyone dies (2025)

Bag of Words model

What is and why make a “Bag of Words” model?

Like everyone, I’ve become interested in the epistemic nature of Large Language Models (LLM).

So, how to proceed? When you focus directly at LLMs they are complicated and difficult to grok.

I’ve opted for an evolutionary approach. I think of this as stepping stones to the LLM revolution. What significant artefacts came before? And then how, who, what, why and when did improvements or leaps occur from those earlier tries to use computers to understand and imitate language, semantic language?

There is excellent information out there on the internet. Step by step I can uncover the path. As well as history, this involves concepts, code (python code) and maths.

First cab off the rank is “Bag of Words” (BOW) or properly speaking the Bag of Words model.

Bag of Words means take all the words from a document and count them. The order of the words is ignored. This turns out to be useful, more about that below.

I’ve imitated and adapted a BOW model for a Dr Seuss book:

Let’s suppose our documents have a small vocabulary. For instance, Dr. Seuss’ book Green Eggs and Ham has only fifty unique words. In alphabetical order, they are: a, am, and, anywhere, are, be, boat, box, car, could, dark, do, eat, eggs, fox, goat, good, green, ham, here, house, I, if, in, let, like, may, me, mouse, not, on, or, rain, Sam, say, see, so, thank, that, the, them, there, they, train, tree, try, will, with, would and you.

If we treat each page of the book as a single document, we can embed each of them as a 50-dimensional vector. Consider the page that reads:

I would not like them here or there.

I would not like them anywhere.

I do not like green eggs and ham.

I do not like them, Sam-I-am.

Summary of the process of making a BOW model:

• preprocessing: tokenize the sentences, removing punctuation and unnecessary spaces
• tokenize the words
• count the word frequencies
• filter out stop words if required (not done for the Dr Seuss case since not really necessary)
• build the BOW model: a binary matrix where each row corresponds to a sentence and each column represents one of the top N frequent words
• visualise the BOW model, in this case I made a heatmap. Other options are a frequency graph and a word cloud.

PYTHON KNOWLEDGE

One of my current goals is to improve my python programming. So, I’ll document that as well. Usually I still feel like a python beginner, but am slowly, very slowly improving. My current goal is to become more fluent with list and dictionary comprehensions, lambda and a few others (RE, heapq, sorting, zip). I have learnt some new things about tokenization, REs, numpy, matplotlib and seaborn, the latter two for visualisation. But, I do need to learn more about seaborn since I’ve yet to figure out how to rotate the xticklabels and yticklabels.

Still not happy with my python coding skills but as I said, slowly improving.

CODE

Here’s my python code, with comments for the BOW model. This particular code has been adapted from a couple of helpful sources, which are acknowledged in the doc string. I’ve left out the ordered frequency graph and Word Cloud. They are useful features but the BOW model is the main game.

# -*- coding: utf-8 -*-
"""
Created on Thu Oct  8 09:43:12 2026

@author: billk
Dr Seuss story
https://builtin.com/machine-learning/bag-of-words
modified to create BOW visual with the help of
https://www.geeksforgeeks.org/nlp/bag-of-words-bow-model-in-nlp/

"""
import re   # regular expressions
import nltk # natural language tool kit
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt


# Assumes that 'doc' is a list of strings and 'vocab' is some iterable of vocab
# words (e.g., a list or set)
def get_bag_of_words(doc, vocab):
   # Create initial dictionary which maps each vocabulary word to a count of 0
   word_count_dict = dict.fromkeys(vocab, 0)
   # For each word in the doc, increment its count
   for word in doc:
       word_count_dict[word] += 1
   # Now, initialize a vector to a list of zeros
   bag = [0] * len(vocab)
   # For every vocab word, set its index equal to its count
   for i, word in enumerate(vocab):
       bag[i] = word_count_dict[word]
   return bag

# Define the vocabulary of the whole document
vocab = ['a', 'am', 'and', 'anywhere', 'are', 'be', 'boat', 'box', 'car',\
        'could', 'dark', 'do', 'eat', 'eggs', 'fox', 'goat', 'good', 'green',\
        'ham', 'here', 'house', 'i', 'if', 'in', 'let', 'like', 'may', 'me',\
        'mouse', 'not', 'on', 'or', 'rain', 'sam', 'say', 'see', 'so', 'thank',\
        'that', 'the', 'them', 'there', 'they', 'train', 'tree', 'try', 'will',\
        'with', 'would', 'you']
# Define one page of the document
doc = ("I would not like them here or there.\n"
      "I would not like them anywhere.\n"
      "I do not like green eggs and ham.\n"
      "I do not like them, Sam-I-am.")

# convert the doc string to list of sentences
dataset = nltk.sent_tokenize(doc) 
#print(dataset) check it's working

for i in range(len(dataset)):
    dataset[i] = dataset[i].lower()  # eliminate capitals
    dataset[i] = re.sub(r'\W', ' ', dataset[i]) # eliminate non letters
    dataset[i] = re.sub(r'\s+', ' ', dataset[i]) # eliminate extra spaces

# run next two lines to check it's working
# for i, sentence in enumerate(dataset):
#    print(f"Sentence {i+1}: {sentence}") 

# initialise and make a word2count dictionary
word2count = {} 

for data in dataset:
    words = nltk.word_tokenize(data) # tokenize each sentence
    # count the words
    for word in words:
        if word not in word2count:
            word2count[word] = 1 # add dictionary item
        else:
            word2count[word] += 1 # count the words

print(word2count) # print the dictionary {'word' : num}

# create BOW matrix graph
BOW = []

for data in dataset:
    vector = []
    for word in word2count:
        if word in nltk.word_tokenize(data):
            vector.append(1)
        else:
            vector.append(0)
    BOW.append(vector)

# BOW is a list of lists
BOW = np.asarray(BOW) # convert to numpy array

# make a heatmap with seaborn
plt.figure(figsize=(10, 6))
sns.heatmap(BOW, cmap='RdYlGn', cbar=False, annot=True, fmt="d", 
            xticklabels=word2count, yticklabels=[f"Sentence {i+1}" 
            for i in range(len(dataset))])

plt.title('Bag of Words Matrix')
plt.xlabel('Frequent Words', rotation=0, ha="right") 
plt.ylabel('Sentences',rotation=0, ha="right")
plt.tight_layout()
plt.show()

Dr Seuss Bag of Words heat map (click to see more clearly):

USEFULNESS

This article summarises the surprising (given its simplicity) uses of BOW:

Although BoW is a very simple technique that turns documents into vectors it can be used by some machine learning algorithms for following use cases:

1. Plagiarism detection

If two documents produce the same or very similar BoW vectors, they likely contain the same words in similar proportions.

2. Spam detection

Emails containing words like “free”, “win”, “lottery” in high frequency can be flagged.Simple ML algorithms trained on BoW vectors still perform well for this use case.

3. Document classification

For example, news categories. Sports articles will have high counts for words like “team”, “win”, “match”. Politics articles will have high counts for words like “election”, “policy”, “minister”.

Issues arising from this research, to be elaborated on in future blogs:

• Other more efficient pathways: scikit-learn, there is always an improved way, TF-IDF to cull out trivial words
• Language classification algorithms, such as the naive Bayes classifier; how do they work? See the third reference below.

REFERENCE
https://builtin.com/machine-learning/bag-of-words
Provides the code, which I adapted, for the Dr Seuss Green Eggs and Ham

https://www.geeksforgeeks.org/nlp/bag-of-words-bow-model-in-nlp/
Provides NLTK code for Bag of Words plus a few visualisation techniques (Frequency table and Word Cloud)

naive Bayes classifiers link
https://scikit-learn.org/stable/modules/naive_bayes.html

In spite of their apparently over-simplified assumptions, naive Bayes classifiers have worked quite well in many real-world situations, famously document classification and spam filtering. They require a small amount of training data to estimate the necessary parameters

NLTK book
https://www.nltk.org/book/
Comprehensive account (concepts and python code) of Natural Language Processing (NLP) up until roughly 2010. Source of the NLTK python library.