I’m making steady progress through Marvin Minsky and Seymour Papert’s book Perceptrons, about a particular computational model of the mind. It’s a neat book, proving some basic — so far — logical results about the limitations of this particular model. And its goal also seems to be to re-establish some connections between logic, geometry and topology.
A perceptron, unless I misunderstand, is a collection of functions φi, each evaluating the truth of a predicate, each returning true or false, and each drawn from some class Φ, together with some aggregating function Ψ that is a linear combination of the φi. Finally there is a threshold θ; if Ψ > θ, the predicate evaluates to true. Additional structure is imposed on the φi, but they’re supposed to be simple and independent enough that they can each tackle part of a problem in parallel and feed their responses back into the main computational engine Ψ.
So for instance, it may be that each small square within your retina only looks at one portion of an object to decide whether it’s “dog-like,” then tells the aggregator. The aggregator decides, on the basis of a weighted sum of the sub-computers, whether what you’re examining is a dog.
Perceptrons shows that many of these pattern-matching tasks are impossible for a perceptron to perform. Within the first few pages is an absolutely brilliant little proof that a certain class of perceptrons (those carrying out pattern matching in which the φi are constrained to examine only a fixed diameter of a pattern) is incapable of deciding whether a shape is connected (i.e., that given any points A and B within the shape, and any path P between them, P lies entirely within the shape. A doughnut is not connected.)
The book is just filled with really good math, really good explanations, solid intuitions, and a lot of motivations. One of my pet peeves about math books is that they often portray their subject as polished and complete, which may work for the researcher but absolutely doesn’t fit the manner in which the math was likely compiled. Mathematicians take lots of false starts, proceed down blind alleys, and only realize later that they could have simplified their work in many places. Minsky and Papert are determined to preserve a lot of these mistakes in their book, but only to the extent that it helps the reader learn. Wouldn’t you love it if a math book said, “We might like to try this . . . but let’s not, and let’s tell you why that won’t work.” Or, conversely, “Here’s the path we took — and we’ll explain why, because this obviously didn’t land in our laps fully formed.” That’s what Minsky and Papert do, and god bless them for it.
They’re also establishing some connections between logic and linear algebra that I hadn’t thought of before. Imagine you have n propositions xi (i = 1, . . . , n) and you want to compute x1 OR x2 OR . . . . Letting 1 stand for true and 0 for false, the conjunction will be true precisely when x1 + x2 + . . . + xn ≥ 1; it’ll be false when that sum equals zero. Joining all those propositions with AND will be true when and only when the sum equals n, false otherwise. Each of these inequalities defines a plane in n-space. (Actually, the set of all sums of unweighted xi defines a collection of points on the n-dimensional lattice. To fill in the hyperplane, you can weight each of the xi by a real number — which is precisely what a perceptron does.)
I had thought about the 1-true/0-false connection before (you could also state that the conjunction of n statements by AND is true when the product of their 0-1 mappings is 1, false when it’s 0), but hadn’t thought about it in terms of geometry. I’m really intrigued by where Minsky and Papert take this. I’m only on page 30, and already I’m hooked.