Below is a summary of an article by Rodney Brooks, Machine Learning Explained (2017).
This inspiried me to take on the development of my own version of hexapawn (referenced below) in python. After reading the Rodney Brooks article I did search around and found a hexapawn version written in Scratch by puttering. The rules are outlined there. The machine improves their play as it plays. I played 30 games with it. For the first 10 I won 6 and the machine 4. For the next 10 I won 4 and the machine 6. For the final 10 I won 2 and the machine 8. I hasten to add that in this game black has the advantage and with perfect play should win 0 to 10.
Anyway Machine Learning is all the rage now so I'll summarise large section of the Brooks article. Of course, you should read the whole thing:
Machine Learning
- is what has enabled the new assistants in our houses such as the Amazon Echo (Alexa) and Google Home by allowing them to reliably understand as we speak to them.
- is how Google chooses what advertisements to place, how it saves enormous amounts of electricity at its data centers, and how it labels images so that we can search for them with key words.
- is how DeepMind (a Google company) was able to build a program called Alpha Go which beat the world Go champion.
- is how Amazon knows what recommendations to make to you whenever you are at its web site.
- is how PayPal detects fraudulent transactions.
- is how Facebook is able to translate between languages. And the list goes on!
Machine Learning is not magic.
Every successful application of ML is hard won by researchers or engineers carefully analyzing the problem that is at hand. They select one or many different ML algorithms, and custom design how to connect them together and to the data. In some cases there is an extensive period of training on very large sets of data before the algorithm can be run on the problem that is being solved. In that case there may be months of work to do in collecting the right sort of data from which ML will actually learn. In other cases the learning algorithm will be integrated in to the application and will learn while doing the task that is desired–it might require some training wheels in the early stages, and they too must be designed. In any case there is always a big design project about how, when the ultimate system is operational, the data that comes in will be organized, processed and mapped before it reaches the ML component of the system.
Alan Turing was assisted by Donald Michie in developing the code breaking Colossus computer Bletchley Park during WW2 which helped shorten the war against fascism
After the war Arthur Samuel developed a machine that could play draughts from 1952-56, the first AI in the USA.
Samuel wondered whether the improvements he was making to the program by hand could be made by the machine itself.
What Samuel had realized, demonstrated, and exploited, was that digital computers were by 1959 fast enough to take over some of the fine tuning that a person might do for a program, as he had been doing since the first version of his program in 1952, and ultimately eliminate the need for much of that effort by human programmers by letting the computer do some Machine Learning on appropriate parts of the problem. This is exactly what has lead, almost 60 years later to the great influence that ML is now having on the world.
Samuel explored two (machine) learning techniques:
- Memoization or memoisation is an optimization technique used primarily to speed up computer programs by storing the results of expensive function calls and returning the cached result when the same inputs occur again
- The other learning technique that he investigated involved adjusting numerical weights on how much the program should believe each of over thirty measures of how good or bad a particular board position was for the program or its human opponent. This is closer to how ML works today.
Donald Michie himself built a machine that could learn to play the game of tic-tac-toe (Noughts and Crosses in British English) from 304 matchboxes, small rectangular boxes which were the containers for matches, and which had an outer cover and a sliding inner box to hold the matches. He put a label on one end of each of these sliding boxes, and carefully filled them with precise numbers of colored beads. With the help of a human operator, mindlessly following some simple rules, he had a machine that could not only play tic-tac-toe but could learn to get better at it. He called his machine MENACE, for Matchbox Educable Noughts And Crosses Engine, and published5 a report on it in 1961. A matchbox computer is still a computer!
In 1962 Martin Gardner reported on it in his regular Mathematical Games column in Scientific American, but illustrated it with a slightly simpler version to play hexapawn, three chess pawns against three chess pawns on a three by three chessboard. … Gardner suggested that people try building a matchbox computer to play simplified checkers with two pieces for each player on a four by four board.
Rodney Brooks recounts how he eventually came to realise that this was a wonderful way for explaining how machine learning works.
Today people generally recognize three different classes of Machine Learning, supervised, unsupervised, and reinforcement learning, all three very actively researched, and all being used for real applications. Donald Michie’s MENACE introduced the idea of ML reinforcement learning, and he explicitly refers to reinforcement as a key concept in how it works
Footnote:
I explain in this article, an AI taxonomy, the difference between Machine Learning (ML) and AI. You can have forms of AI that don’t learn over time. Symbolic AI, Traditional robotics and Behaviour based robotics could all fit this category. They are programmed, they do some human like stuff but don’t change or improve over time without human reprogramming. They are still important but sidelined at the moment due to the LLM (Large Language Models) hype.
