machine learning explained

Below is a summary of an article by Rodney Brooks, Machine Learning Explained (2017).

This inspiried me to take on the development of my own version of hexapawn (referenced below) in python. After reading the Rodney Brooks article I did search around and found a hexapawn version written in Scratch by puttering. The rules are outlined there. The machine improves their play as it plays. I played 30 games with it. For the first 10 I won 6 and the machine 4. For the next 10 I won 4 and the machine 6. For the final 10 I won 2 and the machine 8. I hasten to add that in this game black has the advantage and with perfect play should win 0 to 10.

Anyway Machine Learning is all the rage now so I'll summarise large section of the Brooks article. Of course, you should read the whole thing:

Machine Learning

• is what has enabled the new assistants in our houses such as the Amazon Echo (Alexa) and Google Home by allowing them to reliably understand as we speak to them.
• is how Google chooses what advertisements to place, how it saves enormous amounts of electricity at its data centers, and how it labels images so that we can search for them with key words.
• is how DeepMind (a Google company) was able to build a program called Alpha Go which beat the world Go champion.
• is how Amazon knows what recommendations to make to you whenever you are at its web site.
• is how PayPal detects fraudulent transactions.
• is how Facebook is able to translate between languages. And the list goes on!

Machine Learning is not magic.

Every successful application of ML is hard won by researchers or engineers carefully analyzing the problem that is at hand. They select one or many different ML algorithms, and custom design how to connect them together and to the data. In some cases there is an extensive period of training on very large sets of data before the algorithm can be run on the problem that is being solved. In that case there may be months of work to do in collecting the right sort of data from which ML will actually learn. In other cases the learning algorithm will be integrated in to the application and will learn while doing the task that is desired–it might require some training wheels in the early stages, and they too must be designed. In any case there is always a big design project about how, when the ultimate system is operational, the data that comes in will be organized, processed and mapped before it reaches the ML component of the system.

Alan Turing was assisted by Donald Michie in developing the code breaking Colossus computer Bletchley Park during WW2 which helped shorten the war against fascism

After the war Arthur Samuel developed a machine that could play draughts from 1952-56, the first AI in the USA.

Samuel wondered whether the improvements he was making to the program by hand could be made by the machine itself.

What Samuel had realized, demonstrated, and exploited, was that digital computers were by 1959 fast enough to take over some of the fine tuning that a person might do for a program, as he had been doing since the first version of his program in 1952, and ultimately eliminate the need for much of that effort by human programmers by letting the computer do some Machine Learning on appropriate parts of the problem. This is exactly what has lead, almost 60 years later to the great influence that ML is now having on the world.

Samuel explored two (machine) learning techniques:

1. Memoization or memoisation is an optimization technique used primarily to speed up computer programs by storing the results of expensive function calls and returning the cached result when the same inputs occur again
2. The other learning technique that he investigated involved adjusting numerical weights on how much the program should believe each of over thirty measures of how good or bad a particular board position was for the program or its human opponent. This is closer to how ML works today.

Donald Michie himself built a machine that could learn to play the game of tic-tac-toe (Noughts and Crosses in British English) from 304 matchboxes, small rectangular boxes which were the containers for matches, and which had an outer cover and a sliding inner box to hold the matches. He put a label on one end of each of these sliding boxes, and carefully filled them with precise numbers of colored beads. With the help of a human operator, mindlessly following some simple rules, he had a machine that could not only play tic-tac-toe but could learn to get better at it. He called his machine MENACE, for Matchbox Educable Noughts And Crosses Engine, and published⁠5 a report on it in 1961. A matchbox computer is still a computer!

In 1962 Martin Gardner⁠ reported on it in his regular Mathematical Games column in Scientific American, but illustrated it with a slightly simpler version to play hexapawn, three chess pawns against three chess pawns on a three by three chessboard. … Gardner suggested that people try building a matchbox computer to play simplified checkers with two pieces for each player on a four by four board.

Rodney Brooks recounts how he eventually came to realise that this was a wonderful way for explaining how machine learning works.

Today people generally recognize three different classes of Machine Learning, supervised, unsupervised, and reinforcement learning, all three very actively researched, and all being used for real applications. Donald Michie’s MENACE introduced the idea of ML reinforcement learning, and he explicitly refers to reinforcement as a key concept in how it works

Footnote:
I explain in this article, an AI taxonomy, the difference between Machine Learning (ML) and AI. You can have forms of AI that don’t learn over time. Symbolic AI, Traditional robotics and Behaviour based robotics could all fit this category. They are programmed, they do some human like stuff but don’t change or improve over time without human reprogramming. They are still important but sidelined at the moment due to the LLM (Large Language Models) hype.

an AI taxonomy

When I dig below the hype and both positive and critical evaluations of ChatGPT what I discover is that AI thought leaders disagree and argue with each other a lot. As well as the disagreements about the reliability of ChatGPT there is also the issue of different types of AI. The ascendancy of Deep Learning in the public consciousness is a relatively recent phenomenon in the 60 plus years history of AI. As part of my research I found the need to clarify a broad bunch of terms that refer to very different approaches. A fundamental part of understanding AI is knowing the different types of AI.

Aside for future article: Moreover, underlying these different approaches originally were different ideas about how the human mind and / or brain works.

What is AI?

Artificial Intelligence is the field of developing computers and robots that are capable of behaving in ways that both mimic and go beyond human capabilities. An AI can do discovery, inference and reasoning just like a human. I’ve underlined robots because they, too, have been very much sidelined in the current ChatGPT hype.

The difference between AI and AGI, which also introduces the terms narrow and weak AI

The phrase “Artificial Intelligence” originated from John McCarthy at the Dartmouth Conference in 1956. In the beginning there was only AI, the goal to build a machine from which we couldn’t tell the difference from a human. This was Alan Turing’s imitation game, aka the Turing test. In 1950 Alan Turing published “Computer Machinery and Intelligence” which presented his Imitation Game challenge which later became known as the Turing Test. The Turing Test has now become outdated but that is a side issue to the present article.

Imitating humans in all or most respects turned out to be very hard to do. Over time, as researchers built machines that could perform a particular sub task that humans could do, such as playing chess or correctly labelling images or self driving cars then those sub goals retained the AI label. So, new labels, AGI (Artificial General Intelligence) and ASI (Artificial Super Intelligence) were invented to renew the original goal of imitating or surpassing all of human intelligence.

An alternative could have been using the terms narrow AI or weak AI. Narrow AI is a good descriptor for one narrow task. Weak AI is a similar term, an AI that implements a limited part of the mind.

What is Machine Learning (ML)?

Some experts refer to their work as Machine Learning. For me this needs clarification because there is overlap in the usage of ML and AI. ML is a subset of AI but in what way?

ML means that the machine is Learning (duh!). It is learning by example. In some cases lots of examples. It improves its performance with training over time. This is a different type of coding to the one I am used to where you write code to do something and that something is predetermined by the code and doesn’t change over time. With machine learning, we can use algorithms that have the ability to learn.

There are lots of these algorithms. See this geeks for geeks page for a comprehensive list. One simple example is linear regression. The machine can take an input of a series of points on a graph and map a straight line of best fit to those points. To do this you would need the data (the points), the linear regression algorithm and some python code to do the work. I provide links to a couple of beginner's hands on AI courses in the reference section below that take you through this process.

Phrases that go with ML: data, big data (especially for Deep Learning which is a subset of ML), self learning, statistical models, self correction, can only use structured and semi-structured data

I've been searching for a relatively simple example of making Machine Learning (making rather than doing ML) suitable for middle school students. Most people are current excitedly focused on the doing but my belief is that to understand it deeply you need to make it.

Rodney Brooks illustration of the early machine learning device by Donald Michie provides an entertaining introduction to the topic. Michie built a machine that could learn to play the game of tic-tac-toe (Noughts and Crosses in British English) from 304 matchboxes, small rectangular boxes which were the containers for matches, and which had an outer cover and a sliding inner box to hold the matches. He put a label on one end of each of these sliding boxes, and carefully filled them with precise numbers of colored beads. With the help of a human operator, mindlessly following some simple rules, he had a machine that could not only play tic-tac-toe but could learn to get better at it. He called his machine MENACE, for Matchbox Educable Noughts And Crosses Engine, and published⁠ a report on it in 1961

Brooks references a 1962 Scientific American article by Martin Gardner which illustrated the concepts with a simpler version to play hexapawn, three white chess pawns against three black chess pawns on a three by three chessboard. As in chess, a pawn may move forward one space into an empty square, or capture an enemy pawn by moving diagonally forward one space. If you get a pawn to the last row, you win. You also win if you capture all the enemy's pawns, or if the enemy cannot move.

Impressively, TheGamer has coded MENACE in Scratch, here and puttering has coded hexapawn in Scratch, here.

What AI isn’t Machine Learning?

You can have forms of AI that don’t learn over time. Symbolic AI, Traditional robotics and Behaviour based robotics could all fit this category. They are programmed, they do some human like stuff but don’t change or improve over time without human reprogramming. They are still important but sidelined at the moment due to the LLM (Large Language Models) hype. A little more on this below.

This diagram, found on the web, is missing the two robotic forms of AI and the Neuro-Symbolic hybrids

What is Deep Learning?

To repeat, Machine learning (ML) is a subfield of AI that uses algorithms trained on data to produce adaptable models that can perform a variety of complex tasks.

Deep learning is a subset of machine learning that uses several layers within neural networks to do some of the most complex ML tasks sometimes without any human intervention. But sometimes there is human intervention, in the case of Reinforcement Learning. I won't go into detail about Deep Learning here because it is long as well as smelly. I do provide a link to a detailed version in the Reference.

Four main types of AI

Following Rodney Brooks, there are at least four types of AI. They are, along with approximate start dates:

1. Symbolic (1956) aka Good Old Fashioned AI (GOFAI). Acccording to Herb Simon symbols represent the external world; thought consists of expanding, breaking up, and reforming symbol structures, and intelligence is nothing more than the ability to process symbols. This was the originally dominant of AI (from the 1956 Dartmouth Conference) but the enormous effort over decades by Doug Lenant’s Cyc Project failed so now it has become marginalised due to the successes of Deep Learning.
2. Neural networks (1954, 1960, 1969, 1986, 2006) aka Connectionism, which evolved into Deep Learning…lots of different start dates here since it has died and then returned from the dead a few times). Deep Learning gets all the media attention these days.
3. Traditional robotics (1968)
4. Behaviour-based robotics (1985) aka embodied or situated AI or insect inspired AI! (my term)

One more important thing. Some authors, notably Gary Marcus, say that Neuro-Symbolic hybrids are the way forward to robust, reliable AI.

Here's my crude Venn diagram of the different types of AI:

ML = Machine Learning; DL = Deep Learning; S = Symbolic AI;
H = neuro-symbolic hybrid; TR = Traditional Robotics;
BR = Behavioural Robotics

To understand the current deficiencies of the AI debate / hype it’s necessary to look at the strengths and weaknesses of these different types. Rodney Brooks does evaluate them, in his 2018 blog referenced below, against these criteria: Composition, Grounding, Spatial, Sentience and Ambiguity

AI development has had a tortured zig, zag history. Another fascinating way to view it is from the influences and underlying belief systems / philosophies of the AI founders and developers. If we build our machines in our own image then what is that image?

REFERENCE

Brooks, Rodney. Steps Towards Super Intelligence, 1. How we got here (2018)
Brooks, Rodney. Machine Learning Explained (2017)
Marcus, Gary. The next decade in AI: Four Steps Towards Robust Artificial Intelligence (2020)
Deep Learning (DL) vs Machine Learning (ML): A Comparative Guide
Excellent, wide ranging explanations of ML and DL
The 10 Best Machine Learning Algorithms for Data Science Beginners
Linear Regression to fit points to a straight line is their number one
Your first machine learning project in python step by step
Free introductory hands on course to Deep Learning