Monday, October 06, 2008

minsky 6: common sense

Overview of Chapter 6 of The Emotion Machine (summary, online draft, buy)

Comparing what computers can do (play chess) with what humans can do which computers can't yet do (make a bed, read a book or babysit) provides us with insights about humans. Computer programs don't have commonsense knowledge, eg. when someone says, "a package is tied up with string", this includes "obvious" facts about the nature of string and packages (eg. with string you can pull but not push a thing)

Computer programs are not self aware of their goals - whether they are achieved or at what quality or cost. Computer programs are not as resourceful as humans, when "stuck", eg. they don't reason by analogies

What Do We Mean by Common Sense?

Minsky provides an extensive account of the common sense knowledge involved in answering a phone call. We aren't normally aware of how much we know

He introduces a new word, panalogy, which means parallel analogy

"Charles gave Joan the book"
  • Physical Realm - book moves from Charles to Joan
  • Social Realm - is Charles generous or hoping to ingratiate himself?
  • Dominion Realm - Joan now controls the book

Here we have three meanings of the word "give". Our brains may structurally connect analogous items of knowledge from different realms (points of view). This might explain how we can easily switch without even being conscious of it between these different meanings

Multiple meanings are sometimes seen as a defect because of their ambiguities. The panalogy concept reframes them as a strength

It is hard to categorise commonsense knowledge. The scheme favoured by Minsky is along the lines of the kinds of thinking that can be applied to the categories:
  • Positive expertise
  • Negative expertise
  • Debugging skills - knowing alternatives when usual methods fail
  • Adaptive skills - how to adapt old knowledge to new situations
Could we build a "baby-machine", a machine that will gradually learn more by itself? Such programs have been tried but have failed to develop good new ways to represent knowledge. A machine will fail to learn the right things from most of its experiences. Minsky argues that learning requires selectivity and appropriate "credit assignments". You cannot learn things that you can't represent

Building intelligent machines becomes stuck due to not having ways to overcome problems like:
The Optimisation Paradox: difficult to improve once you work well
The Investment Principle: reliance on existing processes makes it hard to develop alternatives
The Complexity Barrier: changing complex systems has unexpected side effects

At any rate, good new ways to represent knowledge are usually not quickly and widely adopted:
  • you need new skills to work with them efficiently
  • such skills take time and performance will probably worsen during the changeover period
Evolution is more about rejecting bad changes than selecting beneficial changes. Most species evolve to occupy narrow, specialised niches. Evolution can learn to avoid common mistakes but usually not uncommon mistakes - except by evolving language systems.

We first need to evolve ways to protect against changes that cause bad side effects. Excellent method here is to split system into parts that can evolve more independently. eg. organs


Our Amnesia of Infancy leads us to develop simplistic views of what memories are and how they work. Minsky argues for goal based organisation (accomplishment) rather than descriptive organisation (data base and matches):

  • What kinds of goals might this item serve?
  • In which situations might it be relevant?
  • How has it been applied in the past?

  • What are its most likely side effects?
  • How much will it cost to use it?
  • What are its common exceptions and bugs?

  • Was it learned from a reliable source?
  • Is it likely to be outdated soon?
  • Which other people are likely to know it?

Intentions and Goals

What is "self control", responsibility or intention? Moralists, Psychiatrists and Jurists argue about this.

Sometimes a goal can seem like a physical force, hard to resist, even though part of us does not want to do it. The goal may conflict with our high level values. There is no reason to expect that all our goals should be consistent.

Difference Engines

Psychology words don't meaningfully describe goals, they just pass the meaning onto another word that needs to be explained (want, motive, desire, purpose, aim, hope, aspire, yearn, crave). We need to talk about the underlying machinery:
"A system will seem to have a goal when it persists at applying different techniques until the present situation changes into a certain other condition"
Motives and goals could be explained as consisting of these three things:
Aim - description of a possible future situation
Resourcefulness - methods to reduce the difference between the present situation and the future situation
Persistence - keep applying those methods

When you hear a story you react most to how it differs from what you expected. Some names for this include accommodation, adaptation, acclimatization, habituation, becoming accustomed
Our eyes normally make small motions which helps maintain an image. Our systems mainly react to change.

Roger Schank, Tell Me a Story (1990) has conjectured that representing events as stories may be one of our principal ways to learn and remember

Making Decisions

When people say, "I used my free will to make that decision," this is roughly the same as saying, "some process stopped my deliberations and made me adopt what seemed best at the moment". "Free will" is not a process we use to make a decision, but one that we use to stop other processes! "My decison was free" is similar to "I don't want to know what decided me"

Reasoning by Analogy

Minsky identifies reasoning by analogy as one of the main methods by which we solve new problems. Why does analogy work so well? According to Douglas Lenet it is because there is a lot of common causality in the world.

Positive vs. Negative Expertise

We see things as positive because we censor or suppress other processes that would see them as unpleasant.

Minsky simulates a dialogue with a teacher who believes in positive reinforcement and small steps. Such an approach is not bad in itself but limited:
  • difficult tasks almost always involve episodes of distress and discomfort
  • reinforcement can lead to rigidity, lack of adaption
  • other processes may fail when the "normal way" is abandoned
  • the development of higher level managerial resources is put on hold
We must learn to "enjoy" some suffering when learning new things that need large scale changes in how we think. It's a mistake to make education too pleasant.

1 comment:

Mark Miller said...

We must learn to "enjoy" some suffering when learning new things that need large scale changes in how we think. It's a mistake to make education too pleasant.

Alan Kay and Seymour Papert have talked about "hard fun" in education, though I think they'd say that whatever is taught needs to be presented in a way that's most compatible with the way a student learns. We should expect that learning in an appropriate environment can be hard, but it should not be so frustrating that the student becomes disgusted, "turned off" with it. Kay has said that math and science in particular need to be "presented in their best clothes", because too often they're not and students get the notion that "math is hard", "science is hard", etc., and they end up hating these subjects.

Despite the fact that I don't like discomfort in learning something new, sometimes it's the only way I learn. When confronted with this discomfort I feel like I can handle it for a stretch, but then I need to take a break from it, and then take it on again. When confronted with the hard stuff for too long, I tend to get lost in it, and my frustration level increases. Taking a break to relax my mind gives me insights that I never would have gotten by "slogging it out". I don't know why it happens. My mind just sorts it out and the answer appears without much effort.

I remember Kay saying in one of his speeches that anytime we're learning something challenging and big we feel anxious. And anytime we're not learning something new but just working with what we know, we feel bored. So we spend a lot of our life either feeling anxious or bored. I thought that was a good insight, because I know what both situations feel like, and it's just as he described.