Peter J. Denning
wrote an excellent article titled
"The Locality Principle"
in the July 2005 issue of the Communications of the ACM.
The article explained the story behind the
locality of reference, a fundamental principle of computing with many applications.
In a
comment
that appeared in the October issue of the same magazine I commented:
Peter J. Denning's "The Profession of IT" column ("The Locality Principle," July 2005) invoked an anthropomorphic explanation for the prevalence of the locality principle in computational systems, observing that humans gather the most useful objects close around them to minimize the time and work required for their use, and that we've transferred these behaviors into the computational systems we design.
A more intellectually satisfying explanation might be that we are dealing with two parallel and independent evolutionary design paths. Trading some expensive high-quality space (fast memory) in order to gain time performance is a sound engineering decision. It is therefore likely that evolution first adapted the human brain by endowing it with limited but versatile short-term memory and large long-term memory structure that exhibits behavior similar to caching.
Millennia later, we make similar design decisions when building computing systems.
The comment triggered an email exchange with
Phillip G. Armour.
It was one of the most intellectually satisfying email exchanges I've ever had, and I am reproducing it here, with his kind permission.
From: "Phillip G. Armour"
To: "Diomidis Spinellis"
Subject: Reply to Denning
Hi Diomidis,
I read your reply to Peter Demming's article in CACM, and I had a couple of comments, if I may...
I liked the analogy. I like anything that directly and practically links the brain operation with the design of systems, particularly computer systems. With physical systems we do appear to be constrained by the behavior of the material world in which we are working, but with computer systems there is much more conventional (as in "by convention") behavior. An evolutionary view of consciousness is something that people are starting to consider. Did people always think the way they think now? Or has our reasoning and consciousness evolved along with the physical attributes of humanity?
To take a purely evolutionary pressure perspective (which I don't necessarily agree with), our thinking processes have evolved because there was a benefit to them. I do have an issue with any teleological argument, since it is neither provable, nor refutable, but that's a different issue and I will go off on a 270 degree tangent (trigonometry joke) if I pursue that.
Starting from the evolutionary perspective, it would seem that thinking must relate to our ability to interface with the world. With all due respect to your countryman Socrates, there doesn't seem to be much general evolutionary advantage to, say, philosophy. The ability to introspect, to postulate and prove theorems, does not seem to have a direct relationship to our ability to put food on the table (there is a joke in the US: Question: "What's the difference between a philosophy major and a 14 inch pizza?" Answer: "A 14 inch pizza can feed a family of four"). That said, there is evidence from places like the caves at Lasceaux that the hunter-gatherers of 40,000 years ago did visualize and plan the hunt, and were therefore more successful. However, the Cro Magnon man who painted the hunt scenes was a fairly late evolutionary chap, pretty much where we are now. We don't have evidence that Australopithecus did much in the way of meditating. So the early advanced brain functions, on which the later ones were built, were probably related to improving our ability to deal with the world. This means through the senses. It's not a big step to consider the core thinking operations to be sensory metaphors that extend our understanding of the sensed world and our ability to function in it. George Lakoff (in for instance "Where Mathematics Comes From") makes this assertion and it rings true with me.
Therefore, our concept of location is primarily a sensory one. We can't "sense" things which are not close to us, and we can't easily deal with different things unless they are close together. so it makes sense (no pun) that proximity is one the core building blocks of conscious reasoning. Building on this, we've been able to extend the concept of proximity a lot; much of mathematics consists of defining realms (like Riemann space) in which disparate things are somehow linked with things that are "like" (read "near") them. The mechanisms we use to link things to other things that are not obviously (in a sensory way) "close together", are themselves extensions of the same sensory metaphor. While we've been enormously successful at doing this, I think there is evidence that this approach is breaking down. Perhaps more correctly, we are starting to appreciate the limitations of it as an understanding mechanism. I think the sudden realization that systems do not necessarily behave in a way that is empirically predictable from the behavior of their components is a good example.
The trouble is, we can only think the way we can think, and if thinking per se has limitations that do not allow us to fully understand the limitations of, well, thinking, we are kind of stuck. Sounds like a nice metaphysical kind of 3OI to me.
Best,
Phil
____________________________________________
Phillip G. Armour
CORVUS INTERNATIONAL INC
Systems, Psychology and Software
Web:
http://www.corvusintl.com
The basic economic resource... is and will be knowledge... Peter F. Drucker
____________________________________________
From: "Diomidis Spinellis"
To: "Phillip G. Armour"
Subject: Re: Reply to Denning
Hi Phillip,
Your email poses a really interesting question: is our brain's design a result of evolutionary engineering, or does the design mirror properties (locality in our discussion) of the world around us? Given that currently a) we don't have a detailed understanding of how the brain works, b) we don't have access to other intelligent organisms that have evolved in a world with different properties, and c) we've been unable to artificially create deep intelligent behaviour through evolutionary algorithms, this is a question we can't answer directly.
One could argue that our thinking does not appear to be constrained by our sensory limitations: we investigate the atom's particles, black holes, and the big bang. However, one could also argue that the difficulties we face in obtaining a unified view of physics could well be limitations of our thinking processes, and end-up in the 3OI limitation you mention. Furthermore, there is no reason to believe that the brain's low-level design (short term memory in our case) should influence its high level processes: the high level behavior of our computing systems does not appear to be directly linked to the logic gates that compose them.
You also ask whether conciousness offers an evolutionary advantage. I'm really outside my depth here, but I can offer one advantage here. Conciousness allows us humans to form complex social relationships, and these *do* offer an evolutionary advantage.
All the best,
Diomidis
From: "Phillip G. Armour"
To: "Diomidis Spinellis"
Subject: Re: Reply to Denning
Hi Diomidis,
See comments below...
Best,
Phil
-----Original Message-----
Your email poses a really interesting question: is our brain's design a
result of evolutionary engineering, or does the design mirror properties
(locality in our discussion) of the world around us?
---------------------------------------------------------------------------
[Phil:] Mostly likely both. The forces of evolution are primarily a result
of the interaction between the organism and its environment (with some
random mutation thrown in). The interface is the senses. Without the
senses, any organism will be "unaware" of its environment. There are
chemicals around us so we (and other organisms down to the level of a virus)
have a sense of "smell" and "taste"--which is simply a chemical reaction to
a chemical. We appear to be surrounded by physical entities that we can
interact with, generating signals that we interpret as touch. I say "appear
to" because we can reasonably infer that their physical properties are not
the same as those given to us by the sense of touch. There are vibrations
and energy around us that give us the senses of hearing and sight. For
probably a variety of reasons, we have not developed sensitivity to
electromagnetic radiation except in a quite narrow band from around 400nm to
around 700nm wavelength. Radiation outside of these limits is there, but we
can't "see" it.
Locality is a tricky one, since there is some evidence that it is not quite
what we think it is (eg. Alain Aspect's experiment that rather refuted the
EPR concept of locality). That said, it seems that only at the quantum
level is it reasonable to debate locality.
We always end up in an anthropomorphic loop of the "if a tree falls in the
forest..." variety. I wrestle a lot with the argument "...if that's the way
we perceive it, then is it worth trying to perceive it any other way?..."
----------------------------------------------------------------------------
Given that
currently a) we don't have a detailed understanding of how the brain
works, b) we don't have access to other intelligent organisms that have
evolved in a world with different properties, and c) we've been unable
to artificially create deep intelligent behaviour through evolutionary
algorithms, this is a question we can't answer directly.
[Phil:] --------------------------------------------------------------------
I agree. I also think there are quite a few unresolved evolutionary issues.
It is clear that we can go from one variety of a living form to another
variety of a living form using evolutionary pressures or experiments, but it
is not at all clear how to go from non-living to living. People will
trotting out Stanley Miller's experiments even though they didn't result in
anything other than slightly more complicated organic molecules, and their
chemical basis is rather suspect anyway. It's still a REALLY big step from
a long chain organic chemical to something that reproduces itself and
manages itself within its environment.
----------------------------------------------------------------------------
One could argue that our thinking does not appear to be constrained by
our sensory limitations: we investigate the atom's particles, black
holes, and the big bang. However, one could also argue that the
difficulties we face in obtaining a unified view of physics could well
be limitations of our thinking processes, and end-up in the 3OI
limitation you mention.
[Phil:] --------------------------------------------------------------------
But I'm not sure we are really investigating the atom's particles in a
direct sense. We have established mental models (starting with Bohr)
against which we match highly processed information that ultimately hits the
senses. To interpret the output of an electron scanning microscope the
"actual" data is filtered and filtered and filtered usually against a
variety of other mental models and the devices we've constructed based upon
them until the processed output ultimately becomes visible. The event still
has its basis in a sensory interface.
It is an age-old debate whether an event exists or simply we interpret it as
existing. While this is a highly tautological and somewhat fruitless
deliberation, there is an underlying important consideration, namely that
the essence of what we *observe* is anthropomorphic. That being the case,
it is useful to keep in mind that it is not so much a law of the universe we
are observing as a law of our own creation. The history of science is
dotted with people, professions, and even cultures and civilizations which
genuinely believed that their codification of their observations of the
universe was fundamental. Such hubris is invariably punished when the next
generation of thinking comes along, even though the next generation usually
falls into exactly the same trap. There are some really simple examples of
which my favorite is our system of numbers. "Real numbers" are not real,
they are an artificial construct that allow humans to count. The universe
does not need numbers, we need numbers. It is likely that the Romans
considered their numbering system to be "real" even though it didn't contain
provision for nothing, partial things, or negative things, let alone
exponents and logarithms. Numbers, like all of our thinking and all of our
perception of reality, are simply a human created model. And as George Box
said "...all models are wrong, some models are useful."
So for me the issue is not correctness, it is usefulness. The wave-particle
duality issue in sub-nuclear physics is another good example. Is an
electron a wave, a particle, or a probability? Does it exist in one place
or everywhere simultaneously? The ultimate answer is "yes". But we could
give a good argument that it is also, and at the same time, "no". An
electron is not a wave (in what medium?) or a particle (where is its
boundary?) or a probability (which means, what?). It happens that each of
these models is useful for certain human activities involving electrons.
But that's not what an electron *is*. I (along with Edward DeBono) also
think that the "yes"/"no" duality is also anthropomorphic and, while useful,
is not necessarily true (but then maybe it is?)
There is an important point (that I'm long-windedly trying to get to) which
is that our acknowledgement of our ignorance is one of the most important
considerations, because it leaves us open to considering alternative models.
While knowledge is power, it may be that ignorance is more powerful :-).
----------------------------------------------------------------------------
Furthermore, there is no reason to believe that
the brain's low-level design (short term memory in our case) should
influence its high level processes: the high level behavior of our
computing systems does not appear to be directly linked to the logic
gates that compose them.
[Phil:] -------------------------------------------------------------------
While the short term memory acts like a cache, it is probably not
contiguously ordered as such. The "best" model (that is it works best for
me) of thought is that it is a fractal pattern. When one thinks of a purple
cow, it does not generate the intersection of signals from the purple neuron
and the cow neuron. There is no purple neuron. The concept and application
of purple and of cow (and of everything else) is not located in a particular
neuron, it is located all over the brain (though granted there are certain
parts of the brain that specialize in certain kinds of information). The
idea of purple, of cow, and of purple cows is retained along with everything
thing else we are thinking of, have thought of (that has not been
forgotten), or to some extent can think of, in a self-sustaining dynamic
fractal pattern at every moment of every minute of every day. Should that
pattern ever "stop" we would be dead.
Some harmonics of the pattern are "stronger" than others and those represent
our most conscious and intentional thoughts. Some harmonics are so weak
they are almost gone. When they are finally so immersed in other patterns,
we will have "forgotten" that fact or idea or experience. Some patterns are
so similar to others that they coalesce and the memories become "blurred".
It is very common that people overlay what they actually observe with a
generalized pattern based on experience, and that pattern becomes reality.
This has been duplicated in countless psychology experiments.
Some patterns are "retrieval patterns". That is, they are modalities of
thought, or perhaps "search patterns" whose continually self-replicating
function is to process other thought patterns to see how similar they are
and perhaps to perform some optimization on them. My guess is that the
optimization consists of (to use a model):
(a) Combination of patterns, where one is subsumed into another or two are
simply merged. This is the "like" construct.
(b) The establishment of "meta-patterns" that allow relatively independent
processing of the similarity of the two patterns and the difference (whence
come ideas like the Linnean classification schemata in biology and, in OO,
our inheritance ideas)
(c) "Indexes" which are probably a simple form of the meta-pattern, but
which just point us to the relevant patterns but don't perform much
processing. This is where, well, indexes come from.
(d) Meta-meta patterns which guide the construction of the meta patterns
(this list is one such). Meta-meta patterns may be governed by
meta-meta-meta patterns, or (more likely) themselves. This is where we get
the "stack" model from.
(e) hybrid patterns, which are combinations of (say) combinations,
meta-patterns, indexes, meta-meta patterns and, of course, hybrid patterns.
Don't you just love recursion?
----------------------------------------------------------------------------
You also ask whether conciousness offers an evolutionary advantage. I'm
really outside my depth here, but I can offer one advantage here.
Conciousness allows us humans to form complex social relationships, and
these *do* offer an evolutionary advantage.
[Phil:] --------------------------------------------------------------------
Consciousness is another tricky issue. The most useful view of
consciousness for me is that it is that part of the fractal pattern that is
self-aware. Consciousness is arguably the thing that separates humans from
other animals. It is clear other animals think and also form societies.
Interestingly in some animals the bulk of the thinking occurs in the
individuals, but in some highly communal creatures such as ants, the
"thinking" appears to be spread across the society.
It is a useful consideration to contemplate if human societies are
"conscious". I think they are; though perhaps some societies are more
conscious than others. There is no doubt that thought-like fractal patterns
occur in societies and civilizations (think: fashion). They are born,
evolve, and die, or more correctly are subsumed back into the overall
pattern. There are patterns which are life-giving (see Christopher
Alexander) and life-destroying (see Jared Diamond's "Collapse" for a few
good (?) examples).
Given the propensity of the human race to generate destructive patterns, one
has to wonder about the viability of this trait from a strictly evolutionary
perspective.
To sum up, I would expect ALL of the design of computer systems to be
analogues of thought processes. In fact, I cannot think of any way in which
they would not. We think the way we think, and our thought process output,
languages, systems designs, systems models, computer architecture, all the
way up to the models we construct to try to understand the universe MUST be
subject to the same restrictions. But hey, perhaps that's just the way I
think.
Whew!
Comments
Toot!
Share