Numerous objections have been raised against the interface theory of perception. We conclude by canvassing some objections and offering replies.

Objection 1

What’s new here? Of course perception is adaptive. We can go back to Gibson and see the same point. But how could it be anything else?

Reply

Indeed, Gibson and others recognized that perception is adaptive. But Gibson’s theory differs from the interface theory on three key points. First, Gibson got evolution wrong: He claimed that evolution shapes veridical perceptions of those aspects of the world that have adaptive significance for us. Thus Gibson proposed naïve realism, not the interface theory. Koenderink (2014) takes Gibson to task for this, noting that he “…holds that a stone of the right size has the affordance of being throwable, even in the absence of any observer. His affordance is like a property of the stone, much like its weight, or shape. This is quite unlike von Uexküll, who holds that a stone can indeed appear throwable—namely, to a person looking for something to throw. Here, the affordance is not a property of the stone but of an observer in a certain state. Gibson’s notion derives from his reliance on the All Seeing Eye delusion.…”

Second, Gibson denied that perception involves information processing. The interface theory does not. Evidence for information processing is now overwhelming.

Third, in place of information processing Gibson proposed direct perception: We directly perceive, for instance, that something is edible; we do not use information processing to infer from visual and tactile cues that it is edible. But this raises a problem for Gibson: Are illusions direct misperceptions? What could one possibly mean by direct misperception? How could a theory of direct perception explain illusions? Gibson never solved this problem (Fodor and Pylyshyn 1981). Instead, as Gregory (1997) notes: “To maintain that perception is direct, without need of inference or knowledge, Gibson generally denied the phenomena of illusion.” The interface theory does not deny the phenomena of illusion. Instead, one of its strengths is that it offers a new theory of illusions that seems far more plausible than the textbook account.

Objection 2

The interface theory of perception makes science impossible. If our perceptions are not veridical, then we can never have reliable data to build our theories.

Reply

The interface theory poses no problem for science. It claims that our perceptions are not veridical reports of reality. If this claim is correct, then we can discard a particularly simple theory of perception. But that is not to discard the methodology of science. We can continue in the normal fashion to propose scientific theories and make falsifiable predictions about what we will observe. If our theory attributes some structure to the world W, and posits some functional relation P : W → X between the world and our perceptions that is not veridical, we can still deduce from W and P what measurement results we should expect to find in X. The methodology of science is not so fragile that it fails entirely if P happens not to be some simple function, such as an isomorphism.

Objection 3

You use the theory of evolution to show that our cognitive faculties are not reliable guides to the true nature of objective reality. But if our faculties are not reliable, then the theories we create are not reliable, including the theory of evolution. Thus, you are caught in a paradox.

Reply

We use evolutionary games to show that natural selection does not favor veridical perceptions. This does not entail that all cognitive faculties are not reliable. Each faculty must be examined on its own to determine how it might be shaped by natural selection.

Perhaps, for instance, selection pressures favor accurate math; one who accurately predicts that the payoff for eating an apple today when hungry, combined with the payoff for eating an apple yesterday when equally hungry, is roughly twice the payoff obtained on either day, might have a selective advantage over his math challenged neighbor. Perhaps selection favors accurate logic; one who combines estimates of payoff in accord with probabilistic logic might avoid having nature and competitors make fitness Dutch books against him.Footnote 6 This is not to predict that natural selection should make us all math whizzes for whom statistical inference is quick and intuitive. To the contrary, there is ample evidence that we have systematic weaknesses and rely on fallible heuristics and biases (Kahneman 2011). Whereas in perception the selection pressures are almost uniformly away from veridicality, perhaps in math and logic the pressures are not so univocal, and partial accuracy is allowed. The point is that we don’t know until we study the implications of natural selection for these specific mental faculties.

Objection 4

You say in the abstract, and elsewhere, that our perceptions have been shaped to hide the truth. This is a fallacy. Adaptation doesn’t work that way. Our perceptions have been shaped to improve fitness wherever possible—where “fitness” could be nonveridical or veridical—so there is no “hiding” which implies an intentionality that evolution can’t and doesn’t have.

Reply

Yes, of course. We use “hide,” because it powerfully and succinctly makes an important point, and we’re not terribly worried that readers might be taken in by any connotations of intentionality.

Objection 5

Isn’t the interface theory of perception just the utilitarian theory of perception proposed earlier by Braunstein (1983) and Ramachandran (1985; 1990)?

Reply

Not at all. The utilitarian theory of perception claims that evolution has shaped perception to employ a set of heuristics or “bag of tricks”, rather than sophisticated general principles. It claims that these tricks are employed to recover useful information about an objective physical world (a claim which the interface theory explicitly denies). Accordingly, when these tricks are sufficiently successful (which sometimes they’re not), our perceptions are thus veridical about useful aspects of reality. The utilitarian theory is a naïve realist theory, not an interface theory.

Objection 6

The interface theory says that our perceptions of objects in space-time are not veridical, but are just species-specific icons. Doesn’t it follow that (1) no object has a position, or any other physical property, when it is not perceived, and (2) no object has any causal powers? If so, isn’t this a reductio of the interface theory? It entails, for instance, that neurons, which are objects in space-time, have no causal powers and thus cause none of our behaviors.

Reply

The interface theory indeed makes both predictions. If either proves false, then the interface theory is false. No one can claim that the interface theory makes no falsifiable predictions.

But neither prediction has yet proven false. Moreover, both predictions are made by the standard “Copenhagen” interpretation of quantum theory and by more recent interpretations, such as quantum Bayesianism (Allday 2009; Fuchs 2010). According to these interpretations an electron, for instance, has no position when it is not observed and the state of the electron does not, in general, allow one to predict the specific position one will find when making a position measurement, i.e., no causal account can be given for the precise measurement obtained. Thus, both predictions of the interface theory are compatible with current physical theory and experimental data.

Both predictions are, of course, deeply counterintuitive. Our intuitions here are the result of evolutionary pressures to interact successfully with the world, e.g., tracking objects behind occluders and predicting where they’re likely to reappear. Hence, they are innate as far as an individual child is considered. Belief in “object permanence,” the belief, e.g., that a doll still exists and has a position even when it’s hidden behind a pillow, begins as early as 3 months postpartum and is well-ensconced by age 18 months (Bower 1974; Baillargeon and DeVos 1991; Piaget 1954). Rich causal interpretations of physical objects are evident in children by age 6 months (Carey 2009; Keil 2011). We have been shaped by evolution to believe early on that objects exist unperceived and have causal powers.

The interface theory predicts that these beliefs are adaptive fictions.

Objection 7

The interface theory is nothing but the old sense-datum theory of perception—which claims that we see curious objects called sense data and do not see the world itself—that philosophers rightly discarded long ago.

Reply

The short reply is: No, the interface theory is not a sense-datum theory and does not entail the existence of the sense data, or sensibilia, posited by such theories.

The longer reply is: “Sense-datum theory” covers a diverse set of philosophical ideas about perception. Precursors to these ideas can be found in the notion of sensory impressions or ideas proposed by the British empiricists, Locke, Berkeley and Hume. The origin of the modern conceptions of sense data can be traced to the writings of Moore (1903) and Russell (1912; 1918).

According to the act-object theory of sense data originated by Moore, each sense datumis a real concrete object with which an observer has a primitive relation in an act of perceptual awareness, but which nevertheless is distinct from that act of awareness. The act of perceptual awareness is a kind of knowing, and the sense datum thus known has exactly the properties it appears to have. Moreover, some philosophers propose that sense data have exact and discernible properties (if a sense datum is speckled, the sense datum has a precise number of speckles), and that they are objects that are private to each subject and distinct from physical objects.

The sense datum theory has been criticized by philosophers for, inter alia, conflating nonconceptual phenomenal consciousness with the physical events that are perceived (Coates 2007; Sellars 1956), for getting wrong the phenomenology of ordinary perceptual experience (Austin 1962; Firth 1949; Merleau-Ponty 1945), for requiring determinate phenomenal properties (Barnes 1944), and for breeding epistemological issues, such as skepticism or idealism. Logical positivists and logical empiricists conscripted sense data into service as the incorrigible foundation for a verificationist program of knowledge, and when this program was discredited, e.g., by Quine’s (1951) attack on the analytic/synthetic division and Hanson’s (1958) attack on the theory neutrality of observation data, the theory of sense data suffered similar decline.

Sense data also run afoul of current theory and empirical data in vision science. The shapes, lightness, colors, and textures of sense data were claimed to be seen directly and without intervening inferences. It is now clear that these visual properties are the end products of computations of such sophistication that they are still not fully understood (Frisby and Stone 2010; Knill and Richards 1996; Marr 1982; Palmer 1999; Pizlo et al. 2014).

The interface theory does not entail that perception is an act whose objects are sense data or that sense data are an incorrigible foundation for an edifice of verified knowledge. The interface theory is metaphysically neutral, in that it does not posit anything about the world W other than measurability (in the probability-theory sense, rather than the scientific measurement sense). In particular, in addition to not entailing the existence of sense data, the interface theory does not entail idealism. However, it can be embedded in a mathematically rigorous theory of idealism (Hoffman 2008; Hoffman and Prakash 2014).

The interface theory is a general, but mathematically precise, theory of perception and action. It says that in a world represented by the probability space \( \left(W,\;\mathcal{W},\;\mu \right) \), a perceiving agent, \( \mathcal{A} \), is a six-tuple \( \mathcal{A}=\left(X,\;G,\;P,\;D,\;A,\;N\right) \), where X and G are measurable spaces, \( P:W\times \mathcal{X}\to \left[0,1\right] \), \( D:X\times \mathcal{G}\to \left[0,1\right] \) and \( A:G\times \mathcal{W}\to \left[0,1\right] \) are Markovian kernels, and N is an integer. X denotes the agent’s possible perceptions, G its possible actions, P its perceptual mapping, D its decision process, A its action on the world, and N its counter of perceptions (as described more fully in the section on the PDA Loop). Perceiving agents can be combined, in several mathematically precise ways, to create new perceiving agents that are not reducible to the original agents (Hoffman and Prakash 2014).

When the evolution of a perceiving agent is shaped by a (suitably normalized) fitness function f : W → ℝ+, then that agent is shaped towards an X and P that maximize the mutual information I(μf; μf P) and not the mutual information I(μ; μP); this is the formal way to state that perception is tuned to fitness rather than to veridicality.

This, in a nutshell, is the mathematical structure of the interface theory. The proper philosophical interpretation of this structure is a separate and interesting question. In response to this question we, as the authors of the theory, can opine but are not final authorities. When Schrodinger, for instance, first proposed his famous equation, he mistakenly interpreted its wave functions as waves of matter; Born later corrected that interpretation to waves of probability amplitudes.

With this proviso, we interpret X as the possible phenomenal states of the observer, and we interpret a specific x ∈ X not in terms of an act-object relation as proposed by the sense-datum theory, but as a specific phenomenal aspect or constituent of the observer’s mind; ours is a one-place account rather than the two-place account of the sense datum theory. In this regard, our interpretation is much like the critical realist interpretation of Coates (2007). Also like Coates, we take phenomenal qualities to carry information about the environment that normally triggers them. However, whereas Coates takes this information to be about mind-independent physical objects, we take it to be information about fitness and the fitness consequences of possible actions; there is a mind-independent world, but it almost surely does not consist of physical objects in space-time that are the targets of intentional content proposed by Coates.

Objection 8

The interface theory entails that there are no public physical objects. But this is absurd. Even our legal system knows this is absurd. My car is a public object, and if you steal it you break the law.

Reply

The interface theory denies that there are public physical objects, but it does not deny that there is an objective reality that exists even if not perceived by a specific observer. When you and I both look at your car, the car I experience is not numerically identical to the car you experience. We both interact with the same objective reality, and we both represent our interaction with a species-specific set of experiences that we refer to as a car. But the objective reality is not a car and doesn’t remotely resemble a car; moreover, the car of your experience is distinct from the car of my experience.

This might seem puzzling or logic chopping, but it’s quite straightforward. Consider, for instance, the Necker cube of Fig. 4. Sometimes you see a cube with corner A pointing forward (call it “cube A”), and other times a cube with corner B pointing forward (“cube B”). Your cube A experience is not numerically identical to your cube B experience. If you and a friend are both looking at Fig. 4, and she experiences cube A while you experience cube B, then clearly your cube experiences are not numerically identical. Even if you both see cube A at the same time, your cube A experiences are not numerically identical. And yet we have no problem talking about “the cube,” because we both assume that the experience of the other, although numerically distinct from our own experience, is nevertheless similar enough to permit communication. In the same way, we can discuss our migraine headaches, even though there are no public headaches; we assume that the headaches of others are similar enough to our own to make communication possible.

When I see your car, I interact with an objective reality, but my experience of that reality as a car is not an insight into that reality, but is merely a species-specific description shaped by natural selection to guide adaptive behaviors. The adaptive behaviors might include complimenting you on your car or offering to wash it, but not stealing it. If I do steal it, I’ve changed objective reality in a way that injures you and rightly puts the law on your side, but the reality that I’ve changed doesn’t resemble a car.

Similarly, if I’m in California and you’re in New York and we’re competing in an online video game trying to steal cars, I might find “the Porsche” before you do and steal it. But the Porsche on my screen is not numerically identical to the Porsche on your screen. What is behind my screen that triggers it to display a Porsche is a complex tangle of code and transistors that does not resemble a Porsche. I assume that the Porsche on my screen is similar to the Porsche on yours, so that we can discuss genuinely and compete for the Porsche. But there is no public Porsche.

We understand that our denial of public physical objects—our claim that physical objects are simply icons of one’s perceptual interface—appears, to almost everyone, as not just counterintuitive but prima facie false. To many it’s not worth dignifying with a response. That’s how deeply H. sapiens assumes the existence of public physical objects. This assumption is an adaptive fiction shaped by natural selection, because it’s helpful in the practical endeavors required to survive and reproduce. This fiction becomes an impediment when we turn to scientific endeavors, such as solving the mind-body problem. Here the assumption that neurons are public physical objects that exist unperceived and have causal powers is the starting point for almost all theories, and is, we propose, the reason for the (widely acknowledged) failure of all such theories to solve the mind-body problem.

Objection 9

You say that evolution drives veridicality to extinction only when it conflicts with fitness. In general, truth is useful and indeed optimal within the everyday human scheme: e.g., my chances of rendezvousing with you are better if I know the truth about where you are.

Reply

Yes, my chances of rendezvousing with you are better if I know the truth about where you are, just as my chances of deleting a text file are better if I know the truth about where the icon of the text file is on my desktop interface. However, a truth about the state of the interface is not ipso facto a truth about objective reality. Knowing that the icon is in the center of the desktop does not entail that the file itself is in the center of the computer. Similarly, knowing where in space-time to rendezvous with you does not entail any knowledge of objective reality; indeed it does not even entail that space-time itself is an aspect of objective reality (as we proved above in the Invention of Space-Time Theorem). An interface can be an accurate guide to behavior without being an accurate guide to the nature of objective reality.