Merging Two Cognitive Science Theories with Affordances

There are two freaky theories of Perception which are very interesting to me not just for artificial intelligence, but also from a point of view of interfaces, affordances and philosophy of mind:

1. Alva Noë’s enactive approach to perception.
2. Donald D. Hoffman’s interface theory of perception.

What are these and could they overlap?

Enactive Perception

The key element of the enactive approach to perception—aka enactivism—is that sensorimotor knowledge and skills are a required part of perception.¹

Enactivism diverges from tradition, such as in the case of vision—the norm is to keep vision separate from the other senses and sensorimotor abilities and also treat it as a reconstruction program (inverse optics).

The enactive approach suggests that visual perception is not simply a transformation of 2D pictures into a 3D representation. And that vision is dependent on sensorimotor skills.

Computer vision and robotics have occasionally dipped into this realm, such as with behavioral AI and structure-from-motion. But mostly the old paradigm reigns, lately in a new coating of big data and deep learning.²

Aside from vision, enactivism claims that all perceptual representation is dependent on sensorimotor skills.

My interpretation of the enactive approach proposes that perception co-evolved with motor skills such as how our bodies move and how our sensors—for instance, eyes—move.

A static 2D image can not tell you what color blobs are objects and what are merely artifacts of the sensor or environment (e.g. light effects). But if you walk around this scene, and take into account how you are moving, you get a lot more data to figure out what is stable and what is not.

We’ve evolved to have constant motion in our eyes via saccades, so even without walking around or moving our heads, we’re getting motion data for our visual perception system.

The enactive approach says that you should be careful about making assumptions that perception (and consciousness) can be isolated on one side of an arbitrary interface.

For instance, it may not be alright to study perception by looking just at the brain. It may be necessary to include much more of the mind-environment system—a system which is not limited to one side of the arbitrary interface of the skull.

Perception as a User Interface

The Interface Theory of Perception says that “our perceptions constitute a species-specific user interface that guides behavior in a niche.”³

Evolution has provided us with icons and widgets to hide the true complexity of reality. This reality user interface allows organisms to survive better in particular environments, hence the selection for it.

Or as Hoffman et al summarize the conceptual link from computer interfaces:⁴

An interface promotes efficient interaction with the computer by hiding its structural and causal complexity, i.e., by hiding the truth. As a strategy for perception, an interface can dramatically trim the requirements for information and its concomitant costs in time and energy, thus leading to greater fitness. But the key advantage of an interface strategy is that it is not required to model aspects of objective reality; as a result it has more flexibility to model utility, and utility is all that matters in evolution.

Besides supporting the theory with simulations, Hoffman³ uses a colorful real world example:

Male jewel beetles use a reality user interface to find females. This perceptual interface is composed of simple rules involving the color and shininess of female wing cases. Unfortunately, it evolved for a niche which could not have predicted the trash dropped by humans that lead to false positives. This resulted in male jewel beetles humping empty beer bottles known as “stubbies” in Australia.

A biological report of the beetle said that the stubbies were apparently “acting as ‘supernormal releasers’ for male copulation attempts in that they resembled large females.”⁵. That report also noted:

Schlaepfer et al. (2002) point out that organisms often rely on environmental cues to make behavioural and life-history decisions. However, in environments which have been altered suddenly by humans, formerly reliable cues might no longer be associated with adaptive outcomes. In such cases, organisms can become trapped by their evolutionary responses to the cues and experience reduced survival or reproduction (Schlaepfer et al., 2002).

All perception, including of humans, evolved for adaptation to niches. It might make one consider what supernormal stimuli us humans engage with.

There is no reason or evidence to suspect that our reality interfaces provide “faithful depictions” of the objective world. Fitness trumps truth. Hoffman says that Noë supports a version of faithful depiction within enactive perception, although I don’t see how that is necessary for enactive perception.

Interactions

One might think of perception as interactions within a system. This system contains the blobs of matter we typically refer to as an “organism” and its “environment.”

You’ll notice that in the diagram in the previous section, “environment” and “organism” are in separate boxes. But that can be misleading. Really the organism is part of the environment:

True Perception is Right Out the Window

How do we know what we know about reality?

There seems to be consistency at our macroscopic scale of operation.

One consistency is due to natural genetic programs—and programs they in turn cause—which result in humans having shared knowledge bases and shared kinds of experience. If you’ve ever had a disagreement about something seemingly fundamental / obvious with somebody before, then you can imagine how it would be like if us humans didn’t have anything in common conceptually. Communication would be very difficult. For every other entity you want to communicate with, you’d have to establish communication interfaces, translators, interpreters, etc. And how would you even know who to communicate with in the first place? Maybe you wouldn’t have even evolved communication.

So humans (and probably many other related animals) have experiences and concepts that are similar enough that we can communicate with each other via speech, writing, physical contact, gestures, art, etc.

But for all that shared experience and ability to generate interfaces, do we have an inkling of reality?

Since the interface theory of perception says that our perception is not necessarily realistic, and is most likely not even close to being realistic, does this conflict with the enactive theory?

Noë chants the mantra that the world makes itself available to us—echoing some of the 1980s / 1990s era Rod Brooks / behavioral AI approach of “world as its own model”. If representation is distributed in a human-environment system, does it have to be a veridical (truthful) representation? No. I don’t see why that has to be the case. So it seems that the non-veridical nature of perception should not prevent us from combining these two theories.

Affordances

Another link that might assist synthesizing these two theories is that of J.J. Gibson’s affordances.

Affordances are “actionable properties between the world and an actor (a person or animal).”⁶

The connection of affordances to enactivism is provided by Noë (here he’s using an example of flatness):

To see something is flat is precisely to see it as giving rise to certain possibilities of sensorimotor contingency…Gibson’s theory, and this is plausible, is that we don’t see the flatness and then interpret it as suitable for climbing upon. To see it as flat is to see it as making available possibilities for movement. To see it as flat is to see it, directly, as affording certain possibilities.

Noë also states that there is a sense in which all objects of perception are affordances. I think this implies that if there is no affordance relationship between you and a particular part of the environment, then you will not perceive that part—it doesn’t exist to you.

The concept of affordances is also used, in a modified form, for interaction design as well. For those who are designers or understand design, you can perhaps understand how affordances in nature have to be perceived by animals so that they can survive.

It is perhaps the inverse of the design problem—instead of making the artifact afford action for the user, the animal had to make itself comprehend certain affordances through evo-devo, where evo-devo means “that part of biology concerned with how changes in embryonic development during single generations relate to the evolutionary changes that occur between generations.”⁷

Design writer Don Norman makes the point to distinguish between “real” and “perceived” affordances.⁶ That makes sense in the context of his examples such as human-computer interfaces.

But are any affordances in our lives, regardless of computers, actually real?

And that gets back into the interface theory of perception—animals perceive affordances, but there’s no guarantee those affordances, as useful as they are, exist truthfully in the way we perceive them.

We survive with interfaces to reality, but we may not actually have a good grasp of what reality is “behind” the interfaces.